mirror of https://github.com/kubernetes/kops.git
Create document on asset repositories
This commit is contained in:
John Gardiner Myers
parent
2d86a85b36
commit
a09d10776d
|
|
@ -199,92 +199,7 @@ To achieve this, we can add more parameters to `kops create cluster`.
|
|||
|
||||
### Offline mode
|
||||
|
||||
Here is a naive, uncompleted attempt to provision a cluster in a way minimizing the requirements to the internet because even with some kind of proxies or VPN it's still not that fast and it's always much more expensive than downloading from S3.
|
||||
|
||||
```shell
|
||||
## Setup vars
|
||||
|
||||
KUBERNETES_VERSION=$(curl -fsSL --retry 5 "https://dl.k8s.io/release/stable.txt")
|
||||
KOPS_VERSION=$(curl -fsSL --retry 5 "https://api.github.com/repos/kubernetes/kops/releases/latest" | grep 'tag_name' | cut -d\" -f4)
|
||||
ASSET_BUCKET="some-asset-bucket"
|
||||
ASSET_PREFIX=""
|
||||
|
||||
# Please note that this filename of cni asset may change with kubernetes version
|
||||
# Find this in https://github.com/kubernetes/kops/blob/master/upup/pkg/fi/cloudup/networking.go
|
||||
CNI_FILENAME=cni-plugins-amd64-v0.6.0.tgz
|
||||
|
||||
|
||||
export KOPS_BASE_URL=https://s3.cn-north-1.amazonaws.com.cn/$ASSET_BUCKET/kops/$KOPS_VERSION/
|
||||
export CNI_VERSION_URL=https://s3.cn-north-1.amazonaws.com.cn/$ASSET_BUCKET/kubernetes/network-plugins/$CNI_FILENAME
|
||||
export CNI_ASSET_HASH_STRING=d595d3ded6499a64e8dac02466e2f5f2ce257c9f
|
||||
|
||||
## Download assets
|
||||
|
||||
KUBERNETES_ASSETS=(
|
||||
network-plugins/$CNI_FILENAME
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kube-apiserver.tar
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kube-controller-manager.tar
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kube-proxy.tar
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kube-scheduler.tar
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kubectl
|
||||
release/$KUBERNETES_VERSION/bin/linux/amd64/kubelet
|
||||
)
|
||||
for asset in "${KUBERNETES_ASSETS[@]}"; do
|
||||
dir="kubernetes/$(dirname "$asset")"
|
||||
mkdir -p "$dir"
|
||||
url="https://storage.googleapis.com/kubernetes-release/$asset"
|
||||
wget -P "$dir" "$url"
|
||||
[ "${asset##*.}" != "gz" ] && wget -P "$dir" "$url.sha1"
|
||||
[ "${asset##*.}" == "tar" ] && wget -P "$dir" "${url%.tar}.docker_tag"
|
||||
done
|
||||
|
||||
KOPS_ASSETS=(
|
||||
"images/protokube.tar.gz"
|
||||
"linux/amd64/nodeup"
|
||||
)
|
||||
for asset in "${KOPS_ASSETS[@]}"; do
|
||||
kops_path="kops/$KOPS_VERSION/$asset"
|
||||
dir="$(dirname "$kops_path")"
|
||||
mkdir -p "$dir"
|
||||
url="https://kubeupv2.s3.amazonaws.com/kops/$KOPS_VERSION/$asset"
|
||||
wget -P "$dir" "$url"
|
||||
wget -P "$dir" "$url.sha256"
|
||||
done
|
||||
|
||||
## Upload assets
|
||||
|
||||
## Get default S3 multipart_threshold
|
||||
|
||||
AWS_S3_DEFAULT_MULTIPART_THRESHOLD=$(aws configure get default.s3.multipart_threshold)
|
||||
|
||||
if [ ! -n "$AWS_S3_DEFAULT_MULTIPART_THRESHOLD" ]; then
|
||||
AWS_S3_DEFAULT_MULTIPART_THRESHOLD=8MB
|
||||
fi
|
||||
|
||||
## Set multipart_threshold to 1024MB to prevent Etag not returns MD5 when upload multipart
|
||||
|
||||
aws configure set default.s3.multipart_threshold 1024MB
|
||||
|
||||
aws s3api create-bucket --bucket $ASSET_BUCKET --create-bucket-configuration LocationConstraint=$AWS_REGION
|
||||
for dir in "kubernetes" "kops"; do
|
||||
aws s3 sync --acl public-read "$dir" "s3://$ASSET_BUCKET/$ASSET_PREFIX$dir"
|
||||
done
|
||||
|
||||
aws configure set default.s3.multipart_threshold $AWS_S3_DEFAULT_MULTIPART_THRESHOLD
|
||||
|
||||
```
|
||||
|
||||
When create the cluster, add these parameters to the command line.
|
||||
|
||||
```shell
|
||||
--kubernetes-version https://s3.cn-north-1.amazonaws.com.cn/$ASSET_BUCKET/kubernetes/release/$KUBERNETES_VERSION
|
||||
```
|
||||
|
||||
Now most of the assets required to provision a cluster by `kops` and `kubernetes` will be downloaded from the specified S3 bucket except images like `pause-amd64`, `dns` related, etc. These images not mirrored by the docker hub mirror since they are hosted on `gcr.io`. There will be some [problem][8] if the connection is not good.
|
||||
|
||||
### Assets API
|
||||
|
||||
It hasn't been tested as this approach was only a PR when the author experimenting with provisioning a cluster in AWS China Region. It's the official way to achieve offline mode and should be superior to the previous naive attempt.
|
||||
See [Using local asset repositories](operations/asset-repository.md) for information about copying image and file assets to a local repository.
|
||||
|
||||
|
||||
[1]: http://docs.amazonaws.cn/en_us/aws/latest/userguide/unsupported.html
|
||||
|
|
|
|||
|
|
@ -0,0 +1,62 @@
|
|||
# Using local asset repositories
|
||||
|
||||
You can configure kOps to provision a cluster to download assets (images and files) from local repositories.
|
||||
This is useful when downloading assets from the internet is undersirable, for example:
|
||||
|
||||
* To deploy where the network is offline or internet-restricted.
|
||||
* To avoid rate limits or network transfer costs.
|
||||
* To limit exposure to watering-hole attacks.
|
||||
* To comply with other security requirements, such as the need to scan for vulnerabilities.
|
||||
|
||||
There can be one repository for images and another for files.
|
||||
|
||||
## Configuring
|
||||
|
||||
### Configuring a local image repository
|
||||
|
||||
To configure a local image repository, set either `assets.containerRegistry` or `assets.containerProxy` in the cluster spec.
|
||||
They both do essentially the same thing, but `containerRegistry` avoids using `/` characters in the local image names.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
assets:
|
||||
containerRegistry: example.com/registry
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
assets:
|
||||
containerProxy: example.com/proxy
|
||||
```
|
||||
|
||||
### Configuring a local file repository
|
||||
|
||||
To configure a local file repository, set `assets.fileRepository` in the cluster spec.
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
assets:
|
||||
fileRepository: https://example.com/files
|
||||
```
|
||||
|
||||
## Copying assets into repositories
|
||||
|
||||
{{ kops_feature_table(kops_added_default='1.22') }}
|
||||
|
||||
You can copy assets into their repositories either by running `kops get assets --copy` or through an external process.
|
||||
|
||||
When running `kops get assets --copy`, kOps copies assets into their respective repositories if
|
||||
they do not already exist there.
|
||||
|
||||
For file assets, kOps only supports copying to a repository that is either an S3 or GCS bucket.
|
||||
An S3 bucket must be configured using the [regional naming conventions of S3](https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region).
|
||||
A GCS bucket must be configured with a prefix of `https://storage.googleapis.com/`.
|
||||
|
||||
## Listing assets
|
||||
|
||||
{{ kops_feature_table(kops_added_default='1.22') }}
|
||||
|
||||
You can obtain a list of image and file assets used by a particular cluster by running `kops get assets`. You can get output in table, YAML, or JSON format.
|
||||
You can feed this into a process, external to kOps, for copying the assets to their respective repositories.
|
||||
|
|
@ -8,7 +8,7 @@ This is a document to gather the release notes prior to the release.
|
|||
|
||||
## Instance metadata service version 2
|
||||
|
||||
On AWS, kOps will enable [Instance Metadata Service Version 2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html) by default with a max-hop-limit of 1 on new clusters that use Kubernetes 1.22. This means that any container running in the cluster will be unable to connect to the instance metadata _unless_ the container is running with `hostNetworking: true`. This will increase security by default, but may break some types of workloads. In order to revert to old behavior, add the following to the InstanceGroup:
|
||||
On AWS, kOps will enable [Instance Metadata Service Version 2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html) by default with a max-hop-limit of 1 on new clusters that use Kubernetes 1.22. This means that any container running in the cluster will be unable to connect to the instance metadata _unless_ the container is running with `hostNetworking: true`. This will increase security by default, but may break some types of workloads. In order to revert to old behavior, add the following to the InstanceGroup:
|
||||
|
||||
```
|
||||
spec:
|
||||
|
|
@ -28,6 +28,8 @@ This change only affects dedicated API server nodes and worker nodes. It does no
|
|||
|
||||
* The legacy location for downloads `s3://https://kubeupv2.s3.amazonaws.com/kops/` has been deprecated and will not be used for new releases. The new canonical downloads location is `https://artifacts.k8s.io/binaries/kops/`.
|
||||
|
||||
* The `assets` phase of `kops update cluster` has been removed. It is replaced by the new `kops get assets --copy` command.
|
||||
|
||||
# Required actions
|
||||
|
||||
# Deprecations
|
||||
|
|
@ -58,4 +60,8 @@ This change only affects dedicated API server nodes and worker nodes. It does no
|
|||
|
||||
* It is no longer necessary to set `AWS_SDK_LOAD_CONFIG=1` in the environment when using AWS assumed roles with the `kops` CLI.
|
||||
|
||||
* There is a new command `kops get assets` for listing image and file assets used by a cluster.
|
||||
It also includes a `--copy` flag to copy the assets to local repositories.
|
||||
See the documentation on [Using local asset repositories](../operations/asset-repository.md) for more information.
|
||||
|
||||
# Full change list since 1.21.0 release
|
||||
|
|
|
|||
|
|
@ -76,6 +76,7 @@ nav:
|
|||
- Using Manifests and Customizing: "manifests_and_customizing_via_api.md"
|
||||
- High Availability: "operations/high_availability.md"
|
||||
- Scaling: "operations/scaling.md"
|
||||
- Local asset repositories: "operations/asset-repository.md"
|
||||
- Instancegroup images: "operations/images.md"
|
||||
- Cluster configuration management: "changing_configuration.md"
|
||||
- Cluster Templating: "operations/cluster_template.md"
|
||||
|
|
|
|||
Loading…
Reference in New Issue