Add automatic detection of container and system memory limits to control
the Go `GOMEMLIMIT` garbage collector feature. This helps reduced OOMs
by triggering GC when the process approaches system limits.
Signed-off-by: SuperQ <superq@gmail.com>
There are a few documented scenarios where `kube-state-metrics` will
lock up(#995, #1028). I believe a much simpler solution to ensure
`kube-state-metrics` doesn't lock up and require a restart to server
`/metrics` requests is to add default read and write timeouts and to
allow them to be configurable. At Grafana, we've experienced a few
scenarios where `kube-state-metrics` running in larger clusters falls
behind and starts getting scraped multiple times. When this occurs,
`kube-state-metrics` becomes completely unresponsive and requires a
reboot. This is somewhat easily reproduceable(I'll provide a script in
an issue) and causes other critical workloads(KEDA, VPA) to fail in
weird ways.
Adds two flags:
- `server-read-timeout`
- `server-write-timeout`
Updates the metrics http server to set the `ReadTimeout` and
`WriteTimeout` to the configured values.
CatherineF-dev
SuperQ
pokom
Mark
Ovidiu Tirla