When we have multiple writers racing to write /etc/hosts, we could
have file corruption where we see a mix of both files.
We can't use a traditional atomic file write, because we are bind-mounting /etc/hosts.
Instead we write to /etc/hosts, pause, then re-read the contents. If
the contents don't match, we repeat. This will not result in fair
queuing, but will avoid corruption.
* Add a mutex around /etc/hosts updates (for a little extra safety)
* Don't write unchanged files
* Recover from out-of-sequence guard lines
* Add tests
Thanks to granular-ryanbonham for the suggestions & finding the issue!
We don't call klog.InitFlags yet, because that will cause a flag
redefinition error until we get everyone to stop using glog. That
will happen when we update to k8s 1.13.