docker
/
docs
mirror of https://github.com/docker/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

adding ucp note per Jira ENGDOCS-58

This commit is contained in:
Dawn W Docker 2019-07-24 13:32:21 -07:00
parent fd260da4d4
commit 1dc3efd6ec
1 changed files with 28 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
ee/ucp/release-notes.md
View File

@ -186,7 +186,34 @@ In order to optimize user experience and security, support for Internet Explorer
### Known issues
- kubelets or Calico-node pods are Down
- Kubelet fails mounting local volumes in "Block" mode on SLES 12 and SLES 15 hosts
The error message from the kubelet looks like this, with mount returning error code 32.
```
Operation for "\"kubernetes.io/local-volume/local-pxjz5\"" failed. No retries permitted until 2019-07-18 20:28:28.745186772 +0000 UTC m=+5936.009498175 (durationBeforeRetry 2m2s). Error: "MountVolume.MountDevice failed for volume \"local-pxjz5\" (UniqueName: \"kubernetes.io/local-volume/local-pxjz5\") pod \"pod-subpath-test-local-preprovisionedpv-l7k9\" (UID: \"364a339d-a98d-11e9-8d2d-0242ac11000b\") : local: failed to mount device /dev/loop0 at /var/lib/kubelet/plugins/kubernetes.io/local-volume/mounts/local-pxjz5 (fstype: ), error exit status 32"
```
Issuing "dmesg" on the system will show something like:
```
[366633.029514] EXT4-fs (loop3): Couldn't mount RDWR because of SUSE-unsupported optional feature METADATA_CSUM. Load module with allow_unsupported=1.
```
Rootcause:
For block volumes, if a specific filesystem is not specified, then "ext4" is used as the default to format the volume. "mke2fs" is the util used for formatting and is part of the hyperkube image. The config file for mke2fs is at /etc/mke2fs.conf. The config file by default has the following line for ext4. Note that the features list includes something called "metadata_csum", which enables storing checksums to ensure filesystem integrity.
```
{{[fs_types]...
ext4 = {features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isizeinode_size = 256}}}
```
"metadata_csum" for ext4 is great. However, SLES12 and SLES15 call this an "experimental feature" and doesnt allow mounting of such blocks. The kubelet's mke2fs util looks up /etc/mke2fs.conf and formats the block volume with the checksum feature. Then the kubelet calls mount to mount the volume. But the kernel refuses to mount such a volume and errors with exit 32.
Resolution:
On SLES12 and SLES15 hosts, use `sed` to remove the `metadata_csum` feature from the ucp-kubelet container:`sed -i 's/metadata_csum,//g' /etc/mke2fs.conf`
This resolution can be automated across your cluster of SLES12 and SLES15 hosts, by creating a docker swarm service as follows. Note that, for this, the hosts should be in "swarm" mode:
Create a global docker service that remove the "metadata_csum" feature from the mke2fs config file (/etc/mke2fs.conf) in ucp-kubelet container. For this, use the UCP client bundle to point to the UCP cluster and run the following swarm commands:
```
docker service create --mode=global --restart-condition none --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock mavenugo/swarm-exec:17.03.0-ce docker exec ucp-kubelet "/bin/bash" "-c" "sed -i 's/metadata_csum,//g' /etc/mke2fs.conf"
```
- Kubelets or Calico-node pods are Down
The symptom of this issue is that kubelets or Calico-node pods are down with one of the following error messages:
- Kubelet is unhealthy