diff --git a/_data/toc.yaml b/_data/toc.yaml index a6c3452068..bca183bcbf 100644 --- a/_data/toc.yaml +++ b/_data/toc.yaml @@ -1483,6 +1483,8 @@ manuals: path: /ee/ucp/kubernetes/install-cni-plugin/ - title: Kubernetes network encryption path: /ee/ucp/kubernetes/kubernetes-network-encryption/ + - title: Deploy a CSI plugin + path: /ee/ucp/kubernetes/use-csi/ - sectiontitle: Persistent Storage section: - title: Use NFS Storage diff --git a/ee/ucp/images/csi-host-path-plugin.png b/ee/ucp/images/csi-host-path-plugin.png new file mode 100644 index 0000000000..b23ede1bbb Binary files /dev/null and b/ee/ucp/images/csi-host-path-plugin.png differ diff --git a/ee/ucp/images/csi-plugins.png b/ee/ucp/images/csi-plugins.png new file mode 100644 index 0000000000..6e89812872 Binary files /dev/null and b/ee/ucp/images/csi-plugins.png differ diff --git a/ee/ucp/images/csi-volume-mounted.png b/ee/ucp/images/csi-volume-mounted.png new file mode 100644 index 0000000000..a1df54d4ef Binary files /dev/null and b/ee/ucp/images/csi-volume-mounted.png differ diff --git a/ee/ucp/images/persistent-storage-object.png b/ee/ucp/images/persistent-storage-object.png new file mode 100644 index 0000000000..4ee2519ffa Binary files /dev/null and b/ee/ucp/images/persistent-storage-object.png differ diff --git a/ee/ucp/kubernetes/use-csi.md b/ee/ucp/kubernetes/use-csi.md new file mode 100644 index 0000000000..84103a9c29 --- /dev/null +++ b/ee/ucp/kubernetes/use-csi.md @@ -0,0 +1,84 @@ +--- +title: Using CSI drivers +description: Learn how to deploy a CSI driver with UCP. +keywords: Universal Control Plane, UCP, Docker EE, Kubernetes, storage, volume +--- + +The Container Storage Interface (CSI) is a specification for container orchestrators to manage block and file-based +volumes for storing data. Storage vendors can each create a single CSI driver that works with multiple +container orchestrators. The Kubernetes community maintains sidecar containers that can be used by a containerized +CSI driver to interface with Kubernetes controllers in charge of managing persistent volumes, attaching volumes to +nodes (if applicable), mounting volumes to pods, taking snapshots, and more. These sidecar containers include +a driver registrar, external attacher, external provisioner, and external snapshotter. + +Docker Enterprise 3.0 supports version 1.0+ of the CSI specification. Therefore, UCP 3.2 (as part of Docker Enterprise 3.0) can manage storage backends that ship with an associated CSI driver, as illustrated in the following diagram. + +![Kubernetes and CSI components](ee/ucp/images/csi-plugins.png){: .with-border} + +**Note**: Docker Enterprise does not provide CSI drivers. CSI drivers are provided by enterprise storage vendors. +Kubernetes does not enforce a specific procedure for how Storage Providers (SP) should bundle and distribute CSI drivers. + +Review the [Kubernetes CSI Developer Documentation](https://kubernetes-csi.github.io/docs/) for CSI architecture, +security, and deployment details. + +## Prerequisites + +1. Select a storage provider from the list of [available CSI drivers](https://kubernetes-csi.github.io/docs/drivers.html) +or as documented by your storage vendor. + +> Disclaimer: As of Docker Enterprise 3.0, none of the [available CSI drivers](https://kubernetes-csi.github.io/docs/drivers.html) +are production quality and are considered pre-GA. +2. Install Docker Enterprise 3.0, which includes UCP 3.2. +3. Optionally, set the `--csi-expt-enabled` flag in the UCP install configuration if you want to enable CSI +features that are experimental in Kubernetes 1.14. “VolumeSnapshotDataSource” is an experimental feature in +Kubernetes 1.14 and it controls whether restoring from a CSI volume snapshot is enabled or not. If your storage +plugin of choice supports snapshotting, setting this flag is useful. +4. Install the CSI plugin from your storage provider. For notes regarding installation, refer to your +storage provider’s user manual. +5. Apply RBAC for sidecars and the CSI driver. For details on how to apply RBAC for your specific storage provider, +refer to the storage vendor documentation for specific permissions and roles required for deploying CSI plugins +on the cluster. +6. Perform static or dynamic provisioning of PVs using the CSI plugin as the provisioner. For details on how +to provision volumes for your specific storage provider, refer to the storage provider’s user manual. + +## CSI driver deployment +Refer to documentation from your storage vendor around how to deploy the desired CSI driver. +For easy deployment, storage vendors can package the CSI driver in containers. In the context of +Kubernetes clusters, containerized CSI drivers are typically deployed as `StatefulSets` for +managing the cluster-wide logic and `DaemonSets` for managing node-specific logic. + +You can deploy multiple CSI drivers for different storage backends in the same cluster. + +**Note**: + +- To avoid credential leak to user processes, Kubernetes recommends running CSI Controllers on master nodes and the CSI node plugin on worker nodes. +- UCP allows running privileged pods. This is needed to run CSI drivers. +- The Docker daemon on the hosts must be configured with Shared Mount propagation for CSI to allow the sharing of volumes mounted by one container into other containers in the same pod or even to other pods on the same node. By default, Docker daemon in UCP enables "Bidirectional Mount Propagation". + +For additional information, refer to the [Kubernetes CSI documentation](https://kubernetes-csi.github.io/docs/deploying.html). + +### Role-based access control (RBAC) +Pods containing CSI plugins need the appropriate permissions to access and manipulate Kubernetes objects. +The desired cluster roles and bindings for service accounts associated with CSI driver pods can be configured +through YAML files distributed by the storage vendor. UCP administrators must apply those YAML files to +properly configure RBAC for the service accounts associated with CSI pods. + +## Usage + +### Dynamic provisioning + +Dynamic provisioning of persistent storage depends on the capabilities of the CSI driver and underlying storage backend. The provider of the CSI driver should document the parameters available for configuration. +Refer to [CSI HostPath Driver](https://github.com/kubernetes-csi/csi-driver-host-path) for a generic CSI plugin example. + +### Manage CSI deployment +The UCP user interface (UI) provides information about your CSI deployments, as shown in the following screen capture. In this example, a CSI Host Path Plugin was deployed as a `Pod`: + +![UCP UI with CSI host plugin](ee/ucp/images/csi-host-path-plugin.png) + +In the UCP UI, you can navigate to **Kubernetes** -> **Storage** for information about persistent storage objects such as `StorageClass`, `PersistentVolumeClaim`, and `PersistentVolume`. The following example provides information for objects specifically created using a CSI HostPath plugin: + +![UCP UI with persistent storage object information](ee/ucp/images/persistent-storage-object.png) + +The **Volumes** section on the Pod details page shows that the Pod using this CSI HostPath plugin has a volume mounted into the Pod: + +![UCP UI with CSI volume mount information](ee/ucp/images/csi-volume-mounted.png) diff --git a/ee/ucp/release-notes.md b/ee/ucp/release-notes.md index d8b66cd54f..91c069f307 100644 --- a/ee/ucp/release-notes.md +++ b/ee/ucp/release-notes.md @@ -19,6 +19,149 @@ upgrade your installation to the latest release. * [Version 3.0](#version-30) * [Version 2.2](#version-22) +# Version 3.2.0-beta +(2019-5-16) + +## New features + +### Group Managed Service Accounts (gMSA) +On Windows, you can create or update a service using --credential-spec with the config:// format. +This passes the gMSA credentials file directly to nodes before a container starts. + +### Open Security Controls Assessment Language (OSCAL) +OSCAL API endpoints have been added in Engine and UCP. These endpoints are enabled by default. + +### Container storage interface (CSI) +Version 1.0 of the CSI specification is now supported for container orchestrators to manage storage plugins. +Note: As of May 2019, none of the +(available CSI drivers)[https://kubernetes-csi.github.io/docs/drivers.html] are production quality and are considered pre-GA. + +### Internet Small Computer System Interface (iSCSI) +Using iSCSI, a storage admin can now provision a UCP cluster with persistent storage from which UCP end +users can request storage resources without needing underlying infrastructure knowledge. + +### System for Cross-domain Identity Management (SCIM) +SCIM implementation allows proactive synchronization with UCP and eliminates manual intervention for changing +user status and group membership. + +## Enhancements + +### Backup/restore + +- Backups no longer halt UCP containers. +- Backup contents can now be redirected to a file instead of stdout/err. +- You can now view information for all backups performed, including the date, status, and contents filenames. +Error log information can be accessed for troubleshooting. + +### Upgrade + +- Improved progress information for install and upgrade. +- You can now manually control worker node upgrades. +- User workloads no longer experience downtime during an upgrade. + +## Deprecations +The following features are deprecated in UCP 3.1: + +- Collections + - The ability to create a nested collection of more than 2 layers deep within the root /Swarm/collection is + now deprecated and will not be included in future versions of the product. However, current nested collections + with more than 2 layers are still retained. + - Docker recommends a maximum of two layers when creating collections within UCP under the shared cluster + collection designated as /Swarm/. For example, if a production collection called /Swarm/production is created + under the shared cluster collection /Swarm/, only one level of nesting should be created, for + example, /Swarm/production/app/. See Nested collections for more details. +- UCP `stop` and `restart` + - Additional upgrade functionality has been included which eliminates the need for these commands. + +## Browser support +In order to optimize user experience and security, support for Internet Explorer (IE) version 11 is not provided for Windows 7 with UCP version 3.2. Docker recommends updating to a newer browser version if you plan to use UCP 3.2, or remaining on UCP 3.1.x or older until EOL of IE11 in January 2020. + +## Kubernetes +Updated Kubernetes to version 1.14. + +- Enhancements: + - PodShareProcessNamespace + + - The PodShareProcessNamespace feature, available by default, configures PID namespace sharing within a pod. + See [Share Process Namespace between Containers in a Pod](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/) for more information. + - https://github.com/kubernetes/kubernetes/pull/66507 + - Volume Dynamic Provisioning + + - Combined `VolumeScheduling` and `DynamicProvisioningScheduling`. + - Added allowedTopologies description in kubectl. + - ACTION REQUIRED: The DynamicProvisioningScheduling alpha feature gate has been removed. + The VolumeScheduling beta feature gate is still required for this feature) +https://github.com/kubernetes/kubernetes/pull/67432 + - TokenRequest and TokenRequestProjection + + - Enable these features by starting the API server with the following flags: + * --service-account-issuer + * --service-account-signing-key-file + * --service-account-api-audiences + - https://github.com/kubernetes/kubernetes/pull/67349 + - Removed `--cadvisor-port flag` from kubelet + + - ACTION REQUIRED: The cAdvisor web UI that the kubelet started using `--cadvisor-port` was removed + in 1.12. If cAdvisor is needed, run it via a DaemonSet. + - https://github.com/kubernetes/kubernetes/pull/65707 + - Support for Windows Server Containers + + - Supports Windows Server Containers and Windows Server 2016 as the operating system of + choice for the windows-based kubelet. You can run Windows-based and .NET-based applications inside + containers on Kubernetes. + - https://github.com/kubernetes/enhancements/issues/116 + - Support for Out-of-tree CSI Volume Plugins (stable) with API + + - Allows volume plugins to be developed out-of-tree. + - Not require building volume plugins (or their dependencies) into Kubernetes binaries. + - Not requiring direct machine access to deploy new volume plugins (drivers). + - https://github.com/kubernetes/enhancements/issues/178 + - Server-side Apply leveraged by the UCP GUI for the yaml create page + + - Moved "apply" and declarative object management from kubectl to the apiserver. Added "field ownership". + - https://github.com/kubernetes/enhancements/issues/555 + - The PodPriority admission plugin + + - For `kube-apiserver`, the `Priority` admission plugin is now enabled by default when using `--enable-admission-plugins`. If using `--admission-control` to fully specify the set of admission plugins, the `Priority` admission plugin should be added if using the `PodPriority` feature, which is enabled by default in 1.11. + - The priority admission plugin: + - Allows pod creation to include an explicit priority field if it matches the computed + priority (allows export/import cases to continue to work on the same cluster, between + clusters that match priorityClass values, and between clusters where priority is unused + and all pods get priority:0) + - Preserves existing priority if a pod update does not include a priority value and the old + pod did (allows POST, PUT, PUT, PUT workflows to continue to work, with the admission-set value + on create being preserved by the admission plugin on update) + - https://github.com/kubernetes/kubernetes/pull/65739 + - Volume Topology + + - Made the scheduler aware of a Pod's volume's topology constraints, such as zone or node. + - https://github.com/kubernetes/enhancements/issues/490, Docs pr here: kubernetes/website#10736 + - Admin RBAC role and edit RBAC roles + - The admin RBAC role is aggregated from edit and view. The edit RBAC role is aggregated from a + separate edit and view. + - https://github.com/kubernetes/kubernetes/pull/66684 + - API + - `autoscaling/v2beta2` and `custom_metrics/v1beta2` implement metric selectors for Object and Pods + metrics, as well as allow AverageValue targets on Objects, similar to External metrics. + - https://github.com/kubernetes/kubernetes/pull/64097 +- Version updates + - Client-go libraries bump + - ACTION REQUIRED: the API server and client-go libraries support additional non-alpha-numeric + characters in UserInfo "extra" data keys. Both support extra data containing "/" characters or + other characters disallowed in HTTP headers. + - Old clients sending keys that were %-escaped by the user have their values unescaped by new API servers. + New clients sending keys containing illegal characters (or "%") to old API servers do not have their values unescaped. + - https://github.com/kubernetes/kubernetes/pull/65799 + - audit.k8s.io API group bump + - The audit.k8s.io API group has been bumped to v1. + - Deprecated element metav1.ObjectMeta and Timestamp are removed from audit Events in v1 version. + - Default value of option --audit-webhook-version and --audit-log-version are changed from `audit.k8s.io/v1beta1` + to `audit.k8s.io/v1`. + - https://github.com/kubernetes/kubernetes/pull/65891 +- Known issues + - Backwards-incompatible changes in the Kube API that might affect user workloads will require warnings/documentation in the UCP release notes for Amberjack (list of deprecated features and APIs TBD). + - Does anything need to be noted for Kube 1.12 (deprecations, etc. that is not covered for 1.13?) + # Version 3.1 ## 3.1.7