From f85067d6a744829afbc3263acb67e1afcf184cdc Mon Sep 17 00:00:00 2001 From: Zheng Xi Zhou Date: Sat, 11 Sep 2021 20:03:38 +0800 Subject: [PATCH] Develop a ComponentDefinition for a cloud resource (#261) * Develop a ComponentDefinition for a cloud resource - add a doc on developing a ComponentDefinition for a cloud resource - add an application sample only to provison cloud resource * add docs in Chinese * Update docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md Co-authored-by: Hongchao Deng * Update docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md Co-authored-by: Hongchao Deng * Update docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md Co-authored-by: Hongchao Deng * Update docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md Co-authored-by: Hongchao Deng * address comments * fix ci * remove how to develop a Terraform module/source Co-authored-by: Hongchao Deng --- .../provider-and-consume-cloud-services.md | 44 +++- .../components/component-terraform.md | 123 ++++++++- .../provider-and-consume-cloud-services.md | 43 +++- .../components/component-terraform.md | 238 +++++++----------- 4 files changed, 300 insertions(+), 148 deletions(-) diff --git a/docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md b/docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md index f505206b..8e6b0d6e 100644 --- a/docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md +++ b/docs/end-user/components/cloud-services/provider-and-consume-cloud-services.md @@ -29,6 +29,48 @@ Terraform | Alibaba Cloud | [ACK](./terraform/alibaba-ack) | Terraform configura All supported Terraform cloud resources can be seen in the list above. You can also filter them by command by `vela components --label type=terraform`. +### Provision cloud resources + +Use the following Application to provision an OSS bucket: + +```yaml +apiVersion: core.oam.dev/v1beta1 +kind: Application +metadata: + name: provision-cloud-resource-sample +spec: + components: + - name: sample-oss + type: alibaba-oss + properties: + bucket: vela-website-0911 + acl: private + writeConnectionSecretToRef: + name: oss-conn +``` + +The above `alibaba-oss` component will create an OSS bucket named `vela-website-0911`, with private acl, with connection information stored in a secreted named `oss-conn`. +description, whether it's compulsory, and default value. + +Apply the above application, then check the status: + +```shell +$ vela ls +APP COMPONENT TYPE TRAITS PHASE HEALTHY STATUS CREATED-TIME +provision-cloud-resource-sample sample-oss alibaba-oss running healthy Cloud resources are deployed and ready to use 2021-09-11 12:55:57 +0800 CST +``` + +After the phase becomes `running` and `healthy`, you can then check the OSS bucket in Alibaba Cloud console or by [ossutil](https://partners-intl.aliyun.com/help/doc-detail/50452.htm) +command. + +```shell +$ ossutil ls oss:// +CreationTime Region StorageClass BucketName +2021-09-11 12:56:17 +0800 CST oss-cn-beijing Standard oss://vela-website-0911 +``` + +### Consume cloud resources + Let's deploy the [application](https://github.com/oam-dev/kubevela/tree/master/docs/examples/terraform/cloud-resource-provision-and-consume/application.yaml) below to provision Alibaba Cloud OSS and RDS cloud resources, and consume them by the web component. @@ -74,7 +116,7 @@ spec: - name: sample-oss type: alibaba-oss properties: - bucket: vela-website + bucket: vela-website-0911 acl: private writeConnectionSecretToRef: name: oss-conn diff --git a/docs/platform-engineers/components/component-terraform.md b/docs/platform-engineers/components/component-terraform.md index c5b62e16..1e96c626 100644 --- a/docs/platform-engineers/components/component-terraform.md +++ b/docs/platform-engineers/components/component-terraform.md @@ -2,11 +2,14 @@ title: Terraform Component --- +To enable end users to [provision and consume cloud resources](../../end-user/components/cloud-services/provider-and-consume-cloud-services), +platform engineers need to 1) apply the credentials for a cloud provider, and 2) prepare ComponentDefinitions for cloud resources +if end users' requirements are beyond the [built-in capabilities](../../end-user/components/cloud-services/provider-and-consume-cloud-services#supported-cloud-resource-list). + +## Apply the credential for a Cloud Provider + To enable the ability to provision cloud resources by Terraform, the credential for a cloud provider needs to be applied. - -### Apply the credential for a Cloud Provider - Taking Alibaba Cloud as an example, for other cloud providers, please refer to [Terraform controller getting started](https://github.com/oam-dev/terraform-controller/blob/master/getting-started.md). ```shell @@ -27,4 +30,116 @@ alibaba-account-creds Opaque $ kubectl apply -f https://raw.githubusercontent.com/oam-dev/terraform-controller/master/examples/alibaba/provider.yaml provider.terraform.core.oam.dev/default created -``` \ No newline at end of file +``` + +## Develop a ComponentDefinition for a cloud resource + +### Alibaba Cloud + +Take [Elastic IP](https://www.alibabacloud.com/help/doc-detail/36016.htm) as an example. + +#### Develop a Terraform resource or module for the cloud resource + +We recommend you to search the required cloud resource module in [Terraform official module registry](https://registry.terraform.io/browse/modules). +If no luck, you can create a Terraform resource or module yourself per +[Terraform Alibaba Cloud Provider specifications](https://registry.terraform.io/providers/aliyun/alicloud/latest/docs). + +```terraform +module "eip" { + source = "github.com/zzxwill/terraform-alicloud-eip" + name = var.name + bandwidth = var.bandwidth +} + +variable "name" { + description = "Name to be used on all resources as prefix. Default to 'TF-Module-EIP'." + default = "TF-Module-EIP" + type = string +} + +variable "bandwidth" { + description = "Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second)." + type = number + default = 5 +} + +output "EIP_ADDRESS" { + description = "The elastic ip address." + value = module.eip.this_eip_address.0 +} +``` + +For Alibaba Cloud EIP, here is the complete ComponentDefinition. You are warmly welcome to contribute this extended cloud +resource ComponentDefinition to [oam-dev/kubevela](https://github.com/oam-dev/kubevela/tree/master/charts/vela-core/templates/definitions). + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: ComponentDefinition +metadata: + name: alibaba-eip + namespace: {{.Values.systemDefinitionNamespace}} + annotations: + definition.oam.dev/description: Terraform configuration for Alibaba Cloud Elastic IP + labels: + type: terraform +spec: + workload: + definition: + apiVersion: terraform.core.oam.dev/v1beta1 + kind: Configuration + schematic: + terraform: + configuration: | + module "eip" { + source = "github.com/zzxwill/terraform-alicloud-eip" + name = var.name + bandwidth = var.bandwidth + } + + variable "name" { + description = "Name to be used on all resources as prefix. Default to 'TF-Module-EIP'." + default = "TF-Module-EIP" + type = string + } + + variable "bandwidth" { + description = "Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second)." + type = number + default = 5 + } + + output "EIP_ADDRESS" { + description = "The elastic ip address." + value = module.eip.this_eip_address.0 + } + +``` + +#### Verify + +You can quickly verify the ComponentDefinition by command `vela show`. + +```shell +$ vela show alibaba-eip +# Properties ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ +| NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT | ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ +| name | Name to be used on all resources as prefix. Default to 'TF-Module-EIP'. | string | true | | +| bandwidth | Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second). | number | true | | +| writeConnectionSecretToRef | The secret which the cloud resource connection will be written to | [writeConnectionSecretToRef](#writeConnectionSecretToRef) | false | | ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ + + +## writeConnectionSecretToRef ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +| NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT | ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +| name | The secret name which the cloud resource connection will be written to | string | true | | +| namespace | The secret namespace which the cloud resource connection will be written to | string | false | | ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +``` + +If the tables display, the ComponentDefinition should work. To take a step further, you can verify it by provision an actual EIP instance per +the doc [Provision cloud resources](../../end-user/components/cloud-services/provider-and-consume-cloud-services#provision-cloud-resources +). diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/end-user/components/cloud-services/provider-and-consume-cloud-services.md b/i18n/zh/docusaurus-plugin-content-docs/current/end-user/components/cloud-services/provider-and-consume-cloud-services.md index e275e32b..31a9e669 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/end-user/components/cloud-services/provider-and-consume-cloud-services.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/end-user/components/cloud-services/provider-and-consume-cloud-services.md @@ -24,6 +24,47 @@ Terraform | Alibaba Cloud | [ACK](./terraform/alibaba-ack) | 用于部署阿里 KubeVela 支持的所有由 Terraform 编排的云资源如上所示，你也可以通过命令 `vela components --label type=terraform` 查看。 +### 部署云资源 + +我们以 OSS bucket 为例展示如何部署云资源。 + +```yaml +apiVersion: core.oam.dev/v1beta1 +kind: Application +metadata: + name: provision-cloud-resource-sample +spec: + components: + - name: sample-oss + type: alibaba-oss + properties: + bucket: vela-website-0911 + acl: private + writeConnectionSecretToRef: + name: oss-conn +``` + +`alibaba-oss` 类型的组件的 properties 在上面文档有清晰的描述，包括每一个 property 的名字、类型、描述、是否必填和默认值。 + +部署应用程序并检查应用程序的状态。 + +```shell +$ vela ls +APP COMPONENT TYPE TRAITS PHASE HEALTHY STATUS CREATED-TIME +provision-cloud-resource-sample sample-oss alibaba-oss running healthy Cloud resources are deployed and ready to use 2021-09-11 12:55:57 +0800 CST +``` + +当应用程序处于 `running` 和 `healthy`状态。我们可以在阿里云控制台或通过 [ossutil](https://partners-intl.aliyun.com/help/doc-detail/50452.htm) +检查OSS bucket 是否被创建。 + +```shell +$ ossutil ls oss:// +CreationTime Region StorageClass BucketName +2021-09-11 12:56:17 +0800 CST oss-cn-beijing Standard oss://vela-website-0911 +``` + +### 消费云资源 + 下面我们以阿里云关系型数据库（RDS）的例子，作为示例进行讲解。 首先请直接复制一个编写好的应用部署计划，在命令行中执行： @@ -112,7 +153,7 @@ EOF ## 自定义云资源 -如果我们提供的开箱即用云资源没有覆盖你的研发需求，你依然可以通过灵活的[Terraform 组件](../../../platform-engineers/components/component-terraform.md)去自定义业务所需要的云资源。 +如果我们提供的开箱即用云资源没有覆盖你的研发需求，你依然可以通过灵活的[Terraform 组件](../../../platform-engineers/components/component-terraform)去自定义业务所需要的云资源。 ## 下一步 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/platform-engineers/components/component-terraform.md b/i18n/zh/docusaurus-plugin-content-docs/current/platform-engineers/components/component-terraform.md index 3e26ef63..7453add7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/platform-engineers/components/component-terraform.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/platform-engineers/components/component-terraform.md @@ -6,157 +6,57 @@ title: Terraform 组件 Terraform 是目前业内支持云资源最广泛也最受欢迎的组件，KubeVela 对 Terraform 进行了额外的支持，使得用户可以通过 Kubernetes CRD 的方式配合 Terraform 使用任意的云资源。 -目前支持的云资源如下： +为了使最终用户能够[部署和消费云资源](../../end-user/components/cloud-services/provider-and-consume-cloud-services)，管理员需要： +1）配置云提供商的鉴权信息；2）当用户的要求超出了 [内置云资源的能力](../../end-user/components/cloud-services/provider-and-consume-cloud-services) +，要为云资源准备 ComponentDefinitions。 + +### 配置云服务商的鉴权 + +为了使 Terraform 能够部署云资源，需要配置云服务商的鉴权信息。 + +以下示例以阿里云为例，对于其他云供应商，请参考 [Terraform controller getting started](https://github.com/oam-dev/terraform-controller/blob/master/getting-started.md)。 ```shell -$ vela components --label type=terraform -NAME NAMESPACE WORKLOAD DESCRIPTION -alibaba-ack vela-system configurations.terraform.core.oam.dev Terraform configuration for Alibaba Cloud ACK cluster -alibaba-oss vela-system configurations.terraform.core.oam.dev Terraform configuration for Alibaba Cloud OSS object -alibaba-rds vela-system configurations.terraform.core.oam.dev Terraform configuration for Alibaba Cloud RDS object -``` - -本章节，将专门为你介绍如何通过 KubeVela 以及 Terraform 来实现自定义的云资源组件。我们以阿里云的对象存储(OSS)，作为例子来进行讲解。 - -它的大致流程如下： -1. 熟悉各云服务商的鉴权机制，准备相关的 `secret` 和 `token` 等必要信息。 -2. Terraform 使用 `Provider` 对象，完成对应云服务商的权限校验过程。 -3. KubeVela 通过 Terraform 控制器来拉起对应的云资源。 - - -### 配置阿里云鉴权 -以下是阿里云的鉴权相关步骤，其它云服务商同理： - -``` $ export ALICLOUD_ACCESS_KEY=xxx; export ALICLOUD_SECRET_KEY=yyy -If you'd like to use Alicloud Security Token Service, also export ALICLOUD_SECURITY_TOKEN. +``` +If you'd like to use Alicloud Security Token Service, also export `ALICLOUD_SECURITY_TOKEN`. +```shell $ export ALICLOUD_SECURITY_TOKEN=zzz -$ sh hack/prepare-alibaba-credentials.sh +``` + +``` +$ sh https://raw.githubusercontent.com/oam-dev/terraform-controller/master/hack/prepare-alibaba-credentials.sh $ kubectl get secret -n vela-system NAME TYPE DATA AGE alibaba-account-creds Opaque 1 11s -$ kubectl apply -f examples/alibaba/provider.yaml +$ kubectl apply -f https://raw.githubusercontent.com/oam-dev/terraform-controller/master/examples/alibaba/provider.yaml provider.terraform.core.oam.dev/default created ``` -`provider.yaml` 请使用示例： +## 为云资源开发 ComponentDefinition -``` -apiVersion: terraform.core.oam.dev/v1beta1 -kind: Provider -metadata: - name: default -spec: - provider: alibaba - region: cn-beijing - credentials: - source: Secret - secretRef: - namespace: vela-system - name: alibaba-account-creds - key: credentials -``` +### 阿里云 -### 云资源关联 Provider +以 [弹性 IP](https://help.aliyun.com/document_detail/120192.html)为例。 -Terraform 可以编写、应用一个 configuration_hcl_oss.yaml 来配置阿里云的 OSS 存储 Bukcet。它会去自动关联 Kubernetes 集群里的默认 `Provider`，即上一步的 `name: default` YMAL 文件。 +#### 为云资源开发一个 ComponentDefinition -``` -apiVersion: terraform.core.oam.dev/v1beta1 -kind: Configuration -metadata: - name: alibaba-oss -spec: - hcl: | - resource "alicloud_oss_bucket" "bucket-acl" { - bucket = var.bucket - acl = var.acl - } +这是 Terraform ComponentDefinition 的脚手架。你只需要修改三个字段：`metadata.name`，`metadata.annotations.definition.oam.dev/description` +和 `spec.schematic.terraform.configuration`。 - output "BUCKET_NAME" { - value = "${alicloud_oss_bucket.bucket-acl.bucket}.${alicloud_oss_bucket.bucket-acl.extranet_endpoint}" - } - - variable "bucket" { - description = "OSS bucket name" - default = "vela-website" - type = string - } - - variable "acl" { - description = "OSS bucket ACL, supported 'private', 'public-read', 'public-read-write'" - default = "private" - type = string - } - - variable: - bucket: "vela-website" - acl: "private" - - writeConnectionSecretToRef: - name: oss-conn - namespace: default -``` -然后部署： - -``` -$ kubectl apply -f configuration_hcl_oss.yaml - -$ kubectl get configuration.terraform.core.oam.dev -NAME AGE -alibaba-oss 1h - -$ kubectl get configuration.terraform.core.oam.dev alibaba-oss -o yaml -apiVersion: terraform.core.oam.dev/v1beta1 -kind: Configuration -metadata: - annotations: - kubectl.kubernetes.io/last-applied-configuration: | - {"apiVersion":"terraform.core.oam.dev/v1beta1","kind":"Configuration","metadata":{"annotations":{},"name":"alibaba-oss","namespace":"default"},"spec": - ... - \"private\"\n }\n }\n}\n","variable":{"acl":"private","bucket":"vela-website"},"writeConnectionSecretToRef":{"name":"oss-conn","namespace":"default"}}} - creationTimestamp: "2021-04-02T08:17:08Z" - generation: 2 -spec: - ... - variable: - acl: private - bucket: vela-website - writeConnectionSecretToRef: - name: oss-conn - namespace: default -status: - outputs: - BUCKET_NAME: - type: string - value: vela-website.oss-cn-beijing.aliyuncs.com - state: provisioned -``` - -利用 [ossutil](https://help.aliyun.com/document_detail/50452.html) 可以看到 OSS bucket 已经被创建出来。 - -``` -$ ossutil ls oss:// -CreationTime Region StorageClass BucketName -2021-04-10 00:42:09 +0800 CST oss-cn-beijing Standard oss://vela-website -Bucket Number is: 1 - -0.146789(s) elapsed -``` - -### 自定义 OSS 云资源组件 -当前面的步骤全部完成之后，最后我们只需要继续使用组件定义，将这个 `alibaba-oss` 引入，作为后续应用部署计划的内置能力即可。 ```yaml apiVersion: core.oam.dev/v1alpha2 kind: ComponentDefinition metadata: - name: alibaba-oss + name: # 1. ComponentDefinition name, like `alibaba-oss` + namespace: {{.Values.systemDefinitionNamespace}} annotations: - definition.oam.dev/description: Terraform configuration for Alibaba Cloud OSS object + definition.oam.dev/description: # 2. description, like `Terraform configuration for Alibaba Cloud OSS object` + labels: type: terraform spec: workload: @@ -166,24 +66,78 @@ spec: schematic: terraform: configuration: | - resource "alicloud_oss_bucket" "bucket-acl" { - bucket = var.bucket - acl = var.acl + # 3. The developed Terraform HCL +``` + +这里阿里云 EIP 的完整的 ComponentDefinition，我们热烈欢迎你将扩展的云资源的 ComponentDefinition 贡献到 [oam-dev/kubevela](https://github.com/oam-dev/kubevela/tree/master/charts/vela-core/templates/definitions)。 + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: ComponentDefinition +metadata: + name: alibaba-eip + namespace: {{.Values.systemDefinitionNamespace}} + annotations: + definition.oam.dev/description: Terraform configuration for Alibaba Cloud Elastic IP + labels: + type: terraform +spec: + workload: + definition: + apiVersion: terraform.core.oam.dev/v1beta1 + kind: Configuration + schematic: + terraform: + configuration: | + module "eip" { + source = "github.com/zzxwill/terraform-alicloud-eip" + name = var.name + bandwidth = var.bandwidth } - output "BUCKET_NAME" { - value = "${alicloud_oss_bucket.bucket-acl.bucket}.${alicloud_oss_bucket.bucket-acl.extranet_endpoint}" - } - - variable "bucket" { - description = "OSS bucket name" - default = "vela-website" + variable "name" { + description = "Name to be used on all resources as prefix. Default to 'TF-Module-EIP'." + default = "TF-Module-EIP" type = string } - variable "acl" { - description = "OSS bucket ACL, supported 'private', 'public-read', 'public-read-write'" - default = "private" - type = string + variable "bandwidth" { + description = "Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second)." + type = number + default = 5 } -``` \ No newline at end of file + + output "EIP_ADDRESS" { + description = "The elastic ip address." + value = module.eip.this_eip_address.0 + } + +``` + +#### 验证 + +你可以通过 `vela show` 命令快速验证 ComponentDefinition。 + +```shell +$ vela show alibaba-eip +# Properties ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ +| NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT | ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ +| name | Name to be used on all resources as prefix. Default to 'TF-Module-EIP'. | string | true | | +| bandwidth | Maximum bandwidth to the elastic public network, measured in Mbps (Mega bit per second). | number | true | | +| writeConnectionSecretToRef | The secret which the cloud resource connection will be written to | [writeConnectionSecretToRef](#writeConnectionSecretToRef) | false | | ++----------------------------+------------------------------------------------------------------------------------------+-----------------------------------------------------------+----------+---------+ + + +## writeConnectionSecretToRef ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +| NAME | DESCRIPTION | TYPE | REQUIRED | DEFAULT | ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +| name | The secret name which the cloud resource connection will be written to | string | true | | +| namespace | The secret namespace which the cloud resource connection will be written to | string | false | | ++-----------+-----------------------------------------------------------------------------+--------+----------+---------+ +``` + +如果表格能正常出来，ComponentDefinition 应该就可以工作了。更进一步，你可以通过文档[部署云资源](../../end-user/components/cloud-services/provider-and-consume-cloud-services)创建一个实际的 EIP 来验证。 +