kube-apiserver doesn't expose the healthcheck via a dedicated
endpoint, instead relying on anonyomous-access being enabled. That
has previously forced us to enable the unauthenticated endpoint on
127.0.0.1:8080.
Instead we now run a small sidecar container, which
proxies /healthz and /readyz requests (only) adding appropriate
authentication using a client certificate.
This will also enable better load balancer checks in future, as these
have previously been hampered by the custom CA certificate.
Co-authored-by: John Gardiner Myers <jgmyers@proofpoint.com>
Writing to a hostPath from a non-root container requires file
ownership changes, which is difficult to roll out today. See
discussion in #8454
We were primarily using the logfile for e2e diagnostics, so we're
going to look into collecting the information via other means instead.
We also haven't yet shipped this logfile in a released version (though
we have shipped it in beta releases)
The alpha version of encryption at rest used the following flag:
`--experimental-encryption-provider-config`. As of kubernetes 1.13,
`--encryption-provider-config` should be used instead.
klog can now support logging both to a file and to streams, so we get the output both in docker & logfiles.
A few gotchas:
* The output previously was all on stdout, now it on stderr. That is more correct
* If something writes to stdout or stderr outside of klog, it will no longer end up in the logfile.
* There's some oddities still to be ironed out about the flag syntax https://github.com/kubernetes/klog/issues/60
For the master pods (apiserver, controller manager, scheduler) this is
unlikely to ever matter (the masters aren't expected to run out of
resources and need to evict things) but evictions of kube-proxy from worker
nodes are easy to trigger in clusters with PodPriority enabled. Since these
are static pods the configuration is also somewhat difficult to change.
In 1.12 (kops & kubenetes):
* We default etcd-manager on
* We default to etcd3
* We default to full TLS for etcd (client and peer)
* We stop allowing external access to etcd