Merge pull request #1595 from mtanino/raw-block-dyn-support
Spec update of Block volume to support for dynamic provisioning
This commit is contained in:
k8s-ci-robot
GitHub
commit
2c7ff0d3e3
|
|
@ -41,7 +41,7 @@ This document presents a proposal for managing raw block storage in Kubernetes u
|
|||
as bootstrapping
|
||||
* A user wishes to utilize block storage to fully realize the performance of an application tuned to using block devices
|
||||
* A user wishes to read from a block storage device and write to a filesystem (big data analytics processing)
|
||||
Future use cases include dynamically provisioning and intelligent discovery of existing devices, which this proposal sets the
|
||||
Also use cases include dynamically provisioning and intelligent discovery of existing devices, which this proposal sets the
|
||||
foundation for more fully developing these methods.
|
||||
|
||||
|
||||
|
|
@ -68,7 +68,7 @@ This document presents a proposal for managing raw block storage in Kubernetes u
|
|||
|
||||
It is important to note that when a PV is bound, it is either bound as a raw block device or formatted with a filesystem. Therefore,
|
||||
the PVC drives the request and intended usage of the device by specifying the volumeMode as part of the API. This design lends itself
|
||||
to future support of dynamic provisioning by also letting the request initiate from the PVC defining the role for the PV. It also
|
||||
to support of dynamic provisioning by also letting the request initiate from the PVC defining the role for the PV. It also
|
||||
allows flexibility in the implementation and storage plugins to determine their support of this feature. Acceptable values for
|
||||
volumeMode are 'Block' and 'Filesystem'. Where 'Filesystem' is the default value today and not required to be set in the PV/PVC.
|
||||
|
||||
|
|
@ -355,7 +355,7 @@ spec:
|
|||
local:
|
||||
path: /dev/xvdc
|
||||
```
|
||||
## [FUTURE] UC5:
|
||||
## UC5:
|
||||
|
||||
DESCRIPTION: StorageClass with dynamically created volumes
|
||||
|
||||
|
|
@ -374,8 +374,6 @@ provisioner: kubernetes.io/local-block-ssd
|
|||
parameters:
|
||||
```
|
||||
|
||||
***This has implementation details that have yet to be determined. It is included in this proposal for completeness of design ****
|
||||
|
||||
## UC6:
|
||||
|
||||
DESCRIPTION: The developer wishes to request a block device via a Storage Class.
|
||||
|
|
@ -399,6 +397,8 @@ spec:
|
|||
storage: 10Gi
|
||||
```
|
||||
|
||||
**Since the PVC object is passed to the provisioner, it will be responsible for validating and handling whether or not it supports the volumeMode being passed**
|
||||
|
||||
## UC7:
|
||||
|
||||
DESCRIPTION: Admin creates network raw block devices
|
||||
|
|
@ -423,7 +423,6 @@ Spec:
|
|||
gcePersistentDisk:
|
||||
pdName: "gce-disk-1"
|
||||
```
|
||||
**Since the PVC object is passed to the provisioner, it will be responsible for validating and handling whether or not it supports the volumeMode being passed**
|
||||
|
||||
## UC8:
|
||||
|
||||
|
|
@ -731,6 +730,89 @@ The usage of pod device map path is;
|
|||
|
||||
* unspecified defaults to 'file/ext4' today for backwards compatibility and in mount_linux.go
|
||||
|
||||
# Dynamically provisioning
|
||||
|
||||
Using dynamic provisioning, user is able to create block volume via provisioners. Currently,
|
||||
we have two types of provisioners, internal provisioner and external provisioner.
|
||||
During volume creation via dynamic provisioner, user passes persistent volume claim which
|
||||
contains `volumeMode` parameter, then the persistent volume claim object is passed to
|
||||
provisioners. Therefore, in order to create block volume, provisioners need to support
|
||||
`volumeMode` and then create persistent volume with `volumeMode`.
|
||||
|
||||
If a storage and plugin don't have an ability to create raw block type of volume,
|
||||
then `both internal and external provisioner don't need any update` to support `volumeMode`
|
||||
because `volumeMode` in PV and PVC are automatically set to `Filesystem` as a default when
|
||||
these volume object are created.
|
||||
However, there is a case that use specifies `volumeMode` as `Block` even if both plugin and
|
||||
provisioner don't support. As a result, PVC will be created, PV will be provisioned
|
||||
but both of them will stuck Pending status since `volumeMode` between them don't match.
|
||||
For this situation, we will add error propagation into persistent volume controller to make
|
||||
it more clear to the user what's wrong.
|
||||
|
||||
If admin provides external provisioner to provision both filesystem and block volume,
|
||||
admin have to carefully prepare Kubernetes environment for their users because both
|
||||
Kubernetes itself and external provisioner have to support block volume functionality.
|
||||
This means Kubernetes v1.9 or later must be used to provide block volume with external
|
||||
provisioner which supports block volume.
|
||||
|
||||
Regardless of the volumeMode, provisioner can set `FSType` into the plugin's volumeSource
|
||||
but the value will be ignored at the volume plugin side if `volumeMode` is `Block`.
|
||||
|
||||
## Internal provisioner
|
||||
|
||||
If internal plugin has own provisioner, the plugin needs to support `volumeMode` to provision
|
||||
block volume. This is the example implementation of `volumeMode` support for GCE PD plugin.
|
||||
|
||||
```
|
||||
// Obtain volumeMode from PVC Spec VolumeMode
|
||||
var volumeMode v1.PersistentVolumeMode
|
||||
if options.PVC.Spec.VolumeMode != nil {
|
||||
volumeMode = *options.PVC.Spec.VolumeMode
|
||||
}
|
||||
|
||||
// Set volumeMode into PersistentVolumeSpec
|
||||
pv := &v1.PersistentVolume{
|
||||
Spec: v1.PersistentVolumeSpec{
|
||||
VolumeMode: &volumeMode,
|
||||
PersistentVolumeSource: v1.PersistentVolumeSource{
|
||||
GCEPersistentDisk: &v1.GCEPersistentDiskVolumeSource{
|
||||
PDName: options.Parameters["pdName"],
|
||||
FSType: options.Parameters["fsType"],
|
||||
...
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## External provisioner
|
||||
|
||||
We have a "protocol" to allow dynamic provisioning by external software called external provisioner.
|
||||
In order to support block volume via external provisioner, external provisioner needs to support
|
||||
`volumeMode` and then create persistent volume with `volumeMode`. This is the example implementation
|
||||
of `volumeMode` support for external provisioner of Local volume plugin.
|
||||
|
||||
```
|
||||
// Obtain volumeMode from PVC Spec VolumeMode
|
||||
var volumeMode v1.PersistentVolumeMode
|
||||
if options.PVC.Spec.VolumeMode != nil {
|
||||
volumeMode = *options.PVC.Spec.VolumeMode
|
||||
}
|
||||
|
||||
// Set volumeMode into PersistentVolumeSpec
|
||||
pv := &v1.PersistentVolume{
|
||||
Spec: v1.PersistentVolumeSpec{
|
||||
VolumeMode: &volumeMode,
|
||||
PersistentVolumeSource: v1.PersistentVolumeSource{
|
||||
Local: &v1.LocalVolumeSource{
|
||||
Path: options.Parameters["Path"],
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
# Volume binding considerations for dynamically provisioned volumes:
|
||||
The value used for the plugin to indicate is it provisioning block will be plugin dependent and is an opaque parameter. Binding will also be plugin dependent and must handle the parameter being passed and indicate whether or not it supports block.
|
||||
|
|
@ -754,15 +836,9 @@ Feature: Pre-provisioned PVs to precreated devices
|
|||
|
||||
Milestone 7: Initial Plugin changes (FC & Local storage)
|
||||
|
||||
Milestone 8: Disabling of provisioning where volumeMode == Block is not supported
|
||||
|
||||
Phase 2: v1.10
|
||||
Feature: Discovery of block devices
|
||||
|
||||
Milestone 1: Dynamically provisioned PVs to dynamically allocated devices
|
||||
|
||||
Milestone 2: Privileged container concerns
|
||||
|
||||
Milestone 3: Plugin changes with dynamic provisioning support (GCE, AWS & GlusterFS)
|
||||
|
||||
Milestone 4: Flex volume update
|
||||
Milestone 2: Plugin changes with dynamic provisioning support (RBD, iSCSI, GCE, AWS & GlusterFS)
|
||||
|
|
@ -246,6 +246,16 @@ Existing behavior is unchanged for claims that do not specify
|
|||
access mode and PV size. The provisioned volume size MUST NOT be smaller
|
||||
than size requested in the claim, however it MAY be larger.
|
||||
|
||||
* Kubernetes v1.9 or later have functionality to deploy raw block volume
|
||||
instead of filesystem volume as a new feature. To support the feature,
|
||||
we added `volumeMode` parameter which takes values `Filesystem` and
|
||||
`Block` to `pv.Spec` and `pvc.Spec`. In order to deploy block volume
|
||||
via external provisioner, following conditions are REQUIRED.
|
||||
* A storage has ability to create raw block type of volume
|
||||
* Block volume feature has been supported by the volume plugin
|
||||
* External-provisioner MUST set `volumeMode` which matches requirements
|
||||
in `claim.Spec` into `pv.Spec`.
|
||||
|
||||
*So the created PV matches the claim.*
|
||||
|
||||
* `pv.Spec.PersistentVolumeSource` MUST be set to point to the created
|
||||
|
|
@ -259,7 +269,7 @@ Existing behavior is unchanged for claims that do not specify
|
|||
provision a PV for the same claim, however external provisioners can use
|
||||
any mechanism to generate an unique PV name.
|
||||
|
||||
Example of a claim that is to be provisioned by an external provisioner for
|
||||
Example 1) a claim that is to be provisioned by an external provisioner for
|
||||
`foo.org/foo-volume`:
|
||||
|
||||
```yaml
|
||||
|
|
@ -276,13 +286,14 @@ Existing behavior is unchanged for claims that do not specify
|
|||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
volumeMode: Filesystem
|
||||
resources:
|
||||
requests:
|
||||
storage: 4Gi
|
||||
# volumeName: must be empty!
|
||||
```
|
||||
|
||||
Example of the created PV:
|
||||
Example 1) the created PV:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
|
@ -298,6 +309,7 @@ Existing behavior is unchanged for claims that do not specify
|
|||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
volumeMode: Filesystem
|
||||
awsElasticBlockStore:
|
||||
fsType: ext4
|
||||
volumeID: aws://us-east-1d/vol-de401a79
|
||||
|
|
@ -313,6 +325,44 @@ Existing behavior is unchanged for claims that do not specify
|
|||
persistentVolumeReclaimPolicy: Delete
|
||||
```
|
||||
|
||||
Example 2) a claim that provisions `volumeMode: Block` volume:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
...
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
volumeMode: Block
|
||||
resources:
|
||||
requests:
|
||||
storage: 4Gi
|
||||
# volumeName: must be empty!
|
||||
```
|
||||
|
||||
Example 2) the created PV:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
...
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
volumeMode: Block
|
||||
awsElasticBlockStore:
|
||||
volumeID: aws://us-east-1d/vol-de401a79
|
||||
capacity:
|
||||
storage: 4Gi
|
||||
claimRef:
|
||||
...
|
||||
persistentVolumeReclaimPolicy: Delete
|
||||
```
|
||||
|
||||
|
||||
As result, Kubernetes has a PV that represents the storage asset and is bound
|
||||
to the claim. When everything went well, Kubernetes completed binding of the
|
||||
claim to the PV.
|
||||
|
|
|
|||
Loading…
Reference in New Issue