podman/libpod
Matthew Heon e19e0de5fa Introduce graph-based pod container removal
Originally, during pod removal, we locked every container in the
pod at once, did a number of validity checks to ensure everything
was safe, and then removed all the containers in the pod.

A deadlock was recently discovered with this approach. In brief,
we cannot lock the entire pod (or much more than a single
container at a time) without causing a deadlock. As such, we
converted to an approach where we just looped over each container
in the pod, removing them individually. Unfortunately, this
removed a lot of the validity checking of the earlier approach,
allowing for a lot of unintended bad things. Infra containers
could be removed while containers in the pod still depended on
them, for example.

There's no easy way to do validity checks while in a simple loop,
so I implemented a version of our graph-traversal logic that
currently handles pod start. This version acts in the reverse
order of startup: startup starts from containers which depend on
nothing and moves outwards, while removal acts on containers which
have nothing depend on them and moves inwards. By doing graph
traversal, we can guarantee that nothing is removed while
something that depends on it still exists - so the infra
container should be the last thing in a pod that is removed, for
example.

In the (unlikely) case that a graph of the pod's containers
cannot be built (most likely impossible without database editing)
the old method of pod removal has been retained to ensure that
even misbehaving pods can be forcibly evicted from the state.

I'm fairly confident that this resolves the problem, but there
are a lot of assumptions around dependency structure built into
the original pod removal code and I am not 100% sure I have
captured all of them.

Fixes #15526

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2022-09-14 13:44:48 -04:00
..
define health check: add on-failure actions 2022-09-09 13:02:05 +02:00
driver bump go module to version 4 2022-01-18 12:47:07 +01:00
events events: Fix spelling of newNullEventer 2022-09-13 08:38:13 +01:00
layers
linkmode go fmt: use go 1.18 conditional-build syntax 2022-03-18 09:11:53 +01:00
lock Merge pull request #15511 from rhatdan/codespell 2022-09-12 16:52:08 +02:00
logs libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
plugin Fix stutters 2022-09-10 07:52:00 -04:00
shutdown libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
boltdb_state.go Fix stutters 2022-09-10 07:52:00 -04:00
boltdb_state_freebsd.go libpod: Re-work the container's network state to help code sharing 2022-09-12 16:11:25 +01:00
boltdb_state_internal.go Fix stutters 2022-09-10 07:52:00 -04:00
boltdb_state_linux.go Fix stutters 2022-09-10 07:52:00 -04:00
boltdb_state_unsupported.go libpod: Add definition of containerPlatformState for FreeBSD 2022-08-23 15:31:00 +01:00
common_test.go bump go module to version 4 2022-01-18 12:47:07 +01:00
container.go Fix stutters 2022-09-10 07:52:00 -04:00
container_api.go Fix stutters 2022-09-10 07:52:00 -04:00
container_commit.go Fix stutters 2022-09-10 07:52:00 -04:00
container_config.go health check: add on-failure actions 2022-09-09 13:02:05 +02:00
container_copy_linux.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
container_copy_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
container_exec.go all: stop using deprecated GenerateNonCryptoID 2022-09-13 16:26:26 -07:00
container_freebsd.go libpod: Re-work the container's network state to help code sharing 2022-09-12 16:11:25 +01:00
container_graph.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
container_graph_test.go bump go module to version 4 2022-01-18 12:47:07 +01:00
container_inspect.go Fix stutters 2022-09-10 07:52:00 -04:00
container_internal.go Merge pull request #15752 from vrothberg/fix-15691 2022-09-13 14:22:34 +02:00
container_internal_common.go libpod: Move platform-specific bind mounts to a per-platform method 2022-09-12 16:11:25 +01:00
container_internal_freebsd.go libpod: Add FreeBSD implementation of container networking 2022-09-12 16:28:35 +01:00
container_internal_linux.go libpod: Move platform-specific bind mounts to a per-platform method 2022-09-12 16:11:25 +01:00
container_internal_linux_test.go test: use `T.TempDir` to create temporary test directory 2022-05-05 21:09:41 +08:00
container_internal_test.go test: fix failing TestPostDeleteHooks 2022-05-05 21:39:46 +08:00
container_internal_unsupported.go libpod: Add FreeBSD implementation of container internals 2022-09-05 10:12:12 +01:00
container_linux.go libpod: Move networkDisable to container_linux.go 2022-09-05 10:12:12 +01:00
container_log.go fix goroutine leaks in events and logs backend 2022-07-20 12:55:34 +02:00
container_log_linux.go fix goroutine leaks in events and logs backend 2022-07-20 12:55:34 +02:00
container_log_unsupported.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
container_path_resolution.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
container_stat_linux.go Fix stutters 2022-09-10 07:52:00 -04:00
container_stat_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
container_top_linux.go Fix stutters 2022-09-10 07:52:00 -04:00
container_top_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
container_unsupported.go libpod: Add definition of containerPlatformState for FreeBSD 2022-08-23 15:31:00 +01:00
container_validate.go health check: add on-failure actions 2022-09-09 13:02:05 +02:00
diff.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
doc.go document that using libpod package directly is not supported 2022-03-23 19:05:29 +01:00
events.go set default EventsLogFilePath on first run 2022-09-12 18:05:18 +02:00
healthcheck.go health check: add on-failure actions 2022-09-09 13:02:05 +02:00
healthcheck_linux.go health checks: restart timers 2022-09-13 10:56:44 +02:00
healthcheck_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
info.go Fix stutters 2022-09-10 07:52:00 -04:00
info_freebsd.go Fix stutters 2022-09-10 07:52:00 -04:00
info_linux.go Fix stutters 2022-09-10 07:52:00 -04:00
info_test.go Robust whitespace split of cpu utilization line from /proc/stat 2022-05-14 00:34:56 +00:00
info_unsupported.go libpod: Enable 'podman info' for FreeBSD 2022-08-24 10:55:52 +01:00
kube.go generate, kube: plug HostUsers 2022-09-08 22:55:52 +02:00
mounts_linux.go go fmt: use go 1.18 conditional-build syntax 2022-03-18 09:11:53 +01:00
networking_common.go libpod: Move ocicniPortsToNetTypesPorts and compareOCICNIPorts to networking_common.go 2022-09-12 16:28:47 +01:00
networking_freebsd.go libpod: Move ocicniPortsToNetTypesPorts and compareOCICNIPorts to networking_common.go 2022-09-12 16:28:47 +01:00
networking_linux.go libpod: Move ocicniPortsToNetTypesPorts and compareOCICNIPorts to networking_common.go 2022-09-12 16:28:47 +01:00
networking_linux_test.go enable unparam linter 2022-04-25 13:23:20 +02:00
networking_machine.go enable gocritic linter 2022-04-26 18:12:22 +02:00
networking_slirp4netns.go Fix stutters 2022-09-10 07:52:00 -04:00
networking_unsupported.go libpod: Add FreeBSD implementation of container networking 2022-09-12 16:28:35 +01:00
oci.go implement podman update 2022-09-01 13:02:01 -04:00
oci_conmon.go
oci_conmon_attach_common.go libpod: Filter out ENOTCONN errors when trying to close unix domain sockets 2022-09-07 07:58:39 +01:00
oci_conmon_attach_freebsd.go libpod: Build oci_conmon_common.go and oci_conmon_attach_common on FreeBSD 2022-08-18 08:07:30 +01:00
oci_conmon_attach_linux.go libpod: Move openUnixSocket to oci_conmon_attach_linux.go 2022-08-18 08:05:42 +01:00
oci_conmon_common.go Merge pull request #15511 from rhatdan/codespell 2022-09-12 16:52:08 +02:00
oci_conmon_exec_common.go Fix stutters 2022-09-10 07:52:00 -04:00
oci_conmon_exec_freebsd.go libpod: Factor out capabilites code from prepareProcessExec 2022-09-07 07:58:37 +01:00
oci_conmon_exec_linux.go libpod: Factor out capabilites code from prepareProcessExec 2022-09-07 07:58:37 +01:00
oci_conmon_freebsd.go libpod: Make sure writeConmonPipeData is called on FreeBSD 2022-08-29 13:04:04 +01:00
oci_conmon_linux.go libpod: Move moveConmonToCgroupAndSignal and GetLimits to oci_conmon_linux.go 2022-08-18 08:05:42 +01:00
oci_conmon_unsupported.go libpod: Build oci_conmon_common.go and oci_conmon_attach_common on FreeBSD 2022-08-18 08:07:30 +01:00
oci_missing.go implement podman update 2022-09-01 13:02:01 -04:00
oci_util.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
options.go health check: add on-failure actions 2022-09-09 13:02:05 +02:00
pod.go resource limits for pods 2022-07-21 14:50:01 -04:00
pod_api.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
pod_internal.go all: stop using deprecated GenerateNonCryptoID 2022-09-13 16:26:26 -07:00
pod_status.go bump go module to version 4 2022-01-18 12:47:07 +01:00
pod_top_linux.go libpod/pod_top_linux.go: s/TODO/NOTE/ 2022-05-25 12:30:19 +02:00
pod_top_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
reset.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
runtime.go set default EventsLogFilePath on first run 2022-09-12 18:05:18 +02:00
runtime_cstorage.go Fix stutters 2022-09-10 07:52:00 -04:00
runtime_ctr.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
runtime_ctr_freebsd.go libpod: Don't mount /dev/shm in containers on FreeBSD 2022-09-05 10:20:50 +01:00
runtime_ctr_linux.go libpod: Don't mount /dev/shm in containers on FreeBSD 2022-09-05 10:20:50 +01:00
runtime_img.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
runtime_migrate.go Fix stutters 2022-09-10 07:52:00 -04:00
runtime_migrate_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
runtime_pod.go libpod/runtime: switch to golang native error wrapping 2022-07-04 15:39:00 +02:00
runtime_pod_linux.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
runtime_pod_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
runtime_renumber.go Fix stutters 2022-09-10 07:52:00 -04:00
runtime_test.go libpod: Ensure that generated container names are random 2022-09-01 10:27:04 +01:00
runtime_volume.go libpod/runtime: switch to golang native error wrapping 2022-07-04 15:39:00 +02:00
runtime_volume_linux.go Introduce graph-based pod container removal 2022-09-14 13:44:48 -04:00
runtime_volume_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
runtime_worker.go work queue: simplify and use a wait group 2022-05-25 10:17:46 +02:00
service.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
state.go libpod: fix wait and exit-code logic 2022-06-23 09:11:57 -04:00
state_test.go bump go module to version 4 2022-01-18 12:47:07 +01:00
stats_common.go libpod: Split out the common code from GetContainerStats 2022-09-14 08:29:26 +01:00
stats_freebsd.go libpod: Add support for 'podman stats' on FreeBSD 2022-09-14 08:29:26 +01:00
stats_linux.go libpod: Split out the common code from GetContainerStats 2022-09-14 08:29:26 +01:00
stats_unsupported.go libpod: Add support for 'podman stats' on FreeBSD 2022-09-14 08:29:26 +01:00
storage.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
util.go Fix stutters 2022-09-10 07:52:00 -04:00
util_linux.go Fix stutters 2022-09-10 07:52:00 -04:00
util_linux_test.go
util_test.go bump go module to version 4 2022-01-18 12:47:07 +01:00
util_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00
volume.go Add support for containers.conf volume timeouts 2022-08-23 15:42:00 -04:00
volume_inspect.go Fix stutters 2022-09-10 07:52:00 -04:00
volume_internal.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
volume_internal_linux.go libpod: switch to golang native error wrapping 2022-07-05 16:06:32 +02:00
volume_internal_unsupported.go libpod: Add stubs for non-linux builds 2022-08-17 11:45:07 +01:00