Commit Graph

28560 Commits

Author SHA1 Message Date
Daniel J Walsh d02a89afb6
Merge branch 'main' into warnings 2022-10-14 13:46:03 -04:00
Daniel J Walsh ebf857fda6
Merge pull request #1381 from mtrmac/shutdown-touch
Remove a manual layerStore.Touch() call from store.Shutdown()
2022-10-14 13:45:44 -04:00
Daniel J Walsh 910667dfca
Merge pull request #1385 from mtrmac/update-names
Simplify the name update call stack
2022-10-14 13:45:26 -04:00
Daniel J Walsh 9a72db8d0e
Merge pull request #1386 from mtrmac/read-layer-lock
Lock layers only for reading
2022-10-14 13:44:15 -04:00
Miloslav Trmač 0b5faf994f Remove a manual layerStore.Touch() call from store.Shutdown()
It's completely unclear why this is necessary; the unmount calls
actually write to the separate mountpoints.json file, with a separate lock.

When this was originally introduced in b046fb5a9a ,
there wasn't a separate mountpoints.json file, and store.go was manually
calling Touch() everywhere; so my best guess is that this is just an
artifact of making those two API changes, and the Touch() is not actually
necessary.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:28:27 +02:00
Miloslav Trmač d498257fd0 Only lock the primary layer store for reading in CreateImage
There seems to be no reason to lock for writing.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:25:08 +02:00
Miloslav Trmač 61c00d1399 Only lock the primary layer store for reading in Layers
... and use readAllLayerStores.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:25:08 +02:00
Miloslav Trmač 703cbe965b Simplify the name update call stack
Instead of going
	3 store methods
	store.updateNames+enum
	3 sub-store methods
	subStore.updateNames+enum
	applyNameOperation+enum,
simplify to
	3 store methods
	store.updateNames+enum
	subStore.updateNames+enum
	applyNameOperation+enum,

Should not change behavior. Looking purely at updateNameOperation,
invalid values would now be detected after doing more work,
but there is no way for an external caller to trigger use of
an invalid value anyway.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:23:08 +02:00
Miloslav Trmač eeadee46d6 Fix races in lockfile_test.go
go { cmd.Run() } means the cmd will be destructed,
and stdin/stdout/stderr closed, concurrently with
other goroutines trying to access stdin/stdout/stderr.

Instead, do it the more traditional way and let the callers
who create those subprocesses explicitly manage their lifetime.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:54 +02:00
Miloslav Trmač fe82b4d4cc Use //nolint
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:54 +02:00
Miloslav Trmač f3316c595f Only pass an io.Reader to invokeUnpack
... which allows us to remove an unnecessary NopCloser.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:54 +02:00
Miloslav Trmač 2d90000b09 Add missing error checks in tests
... and remove one WriteFile that was always failing.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:54 +02:00
Miloslav Trmač d60159bc30 Add more error handling to cmd/containers-storage
Introduce an outputJSON helper to decrease repetition.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:54 +02:00
Miloslav Trmač ace9c41df3 Consolidate error reporting in cmd/containers-storage
Have the action handlers return an error value, and let
main() format that error, if any; this avoids duplicated
error formating code in the action handlers, dropping
89 lines.

This might change the error format in some cases (typically
%v vs. %+v).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 7de868a7c9 Check for errors when creating empty files
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 04a2f46d5f Abort the writer in TarWithOptions on error
It can currently only fail with a broken pipe, or an invalid
pattern; in either case there's no point continuing with the loop.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač c497e66e9a Work around a paralleltest crash
> ERRO [runner] Panic: paralleltest: package "main" (isInitialPkg: true, needAnalyzeSource: true): runtime error: index out of range [0] with length 0: goroutine 5859 [running]:
> ...
> github.com/kunwardeep/paralleltest/pkg/paralleltest.isTestFunction(0x1b7d8c0?)
> 	github.com/kunwardeep/paralleltest@v1.0.6/pkg/paralleltest/paralleltest.go:252 +0x165

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 7f0dae6fea Rename ReadByte to readByte
It is only used internally, and this avoids a warning
about a conflict with io.ByteReader.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 5e410ef763 Misc individual warning fixes
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 767e0cd4ef Move code unused on some architectures
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač ace738a449 Don't compile an unused stub on some Unix platforms
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač a4870b9761 Move code only called on Linux into a Linux-specific file
Only moves unchanged code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 381c1f614a Remove unused stubs
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 26370118ed Exclude internal graphtest functions on platforms that don't call them
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Miloslav Trmač 94f6ec89e6 Use bytes.Equal instead of bytes.Compare
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 17:17:53 +02:00
Daniel J Walsh 8d8d6be5af
Merge pull request #1387 from mtrmac/lock-all-layer-stores
Fix layer locking in CreateContainer
2022-10-14 09:35:23 -04:00
Valentin Rothberg a144fee6f5
Merge pull request #1383 from mtrmac/simple-writers
Consolidate the common parts of simple writers
2022-10-14 09:22:57 +02:00
Miloslav Trmač ad8046ad2a Fix layer locking in CreateContainer
- imageTopLayerForMapping, in the cParentLayer loop, was reading all
  additional layer stores, although only some of them might have been
  locked at that point.
- getAutoUserNS -> getMaxSizeFromImage was assuming that all layer stores
  are locked, but in fact the additional layer stores were all unlocked
  at that point
- getMaxSizeFromImage was calling store.getLayerStore() and the like,
  which could potentially trigger a reload via graphLock and return
  a _different_ store than the one that was locked

So, lock _all_ layer stores in CreateContainer (at least on the path
where images are involved), and pass the locked layer stores down the
call stack.

Also, document a bit more explicitly what's going on.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 02:36:39 +02:00
Miloslav Trmač cae12e3a66 Introduce store.writeToAllStores
... to avoid the repetition of getting, locking,
and reloading, the stores.

This also makes it less error-prone to use a consistent
locking order (but note that other instances of manual
locking still exist).

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 00:01:33 +02:00
Miloslav Trmač f0ada377f3 Introduce store.writeToImageStore
... to avoid the repetition of getting, locking,
and reloading, the store.

Use it at least for the simplest writers.

This could be more elegant with Go generics, returning
(T, error), with T possibly being void = struct{}.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 00:01:32 +02:00
Miloslav Trmač 6f1cdf98a4 Introduce store.writeToContainerStore
... to avoid the repetition of getting, locking,
and reloading, the store.

Use it at least for the simplest writers.

This could be more elegant with Go generic returning
(T, error), with T possibly being void = struct{}.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 00:01:18 +02:00
Miloslav Trmač d4d7d2aa6d Introduce store.writeToLayerStore
... to avoid the repetition of getting, locking,
and reloading, the store.

Use it at least for the simplest writers.

This could be more elegant with Go generics, returning
(T, error), with T possibly being void = struct{}.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-14 00:01:18 +02:00
Daniel J Walsh be70126b99
Merge pull request #1384 from mtrmac/all-readers
Factor out the infrastructure for reading primary + additional stores
2022-10-13 17:44:35 -04:00
Miloslav Trmač 958058a0f1 Add readAllImageStores
... to remove the copy&pasted iteration and copy&pasted locking/
ReloadIfChanged.

That will also make it much easier to add more locking in the future.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-13 21:40:13 +02:00
Miloslav Trmač 6eb9793be0 Add readAllLayerStores
... to remove the copy&pasted iteration and copy&pasted locking/
ReloadIfChanged.

That will also make it much easier to add more locking in the future.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-13 21:40:09 +02:00
Daniel J Walsh 714f4fc6e8
Merge pull request #1380 from mtrmac/misc
Two misc. cleanups
2022-10-13 10:36:30 -04:00
Daniel J Walsh cdd951c718
Merge pull request #1382 from vrothberg/del-ctr
container deletion - EBUSY improvement
2022-10-13 10:36:07 -04:00
Valentin Rothberg d885bfb176 mount.Umount: EBUSY: try at most 50 times
The function is being used in a number of places, notably container
removal and cleanup.  While container removal already loops over EBUSY,
cleanup does not.

To make sure that all callers of Unmount get a fair chance of unmounting
cleanly, also loop there.  I used the same values as containerd: 50
loops with 50ms sleeps.

Context: containers/podman/issues/11594
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-13 12:33:30 +02:00
Valentin Rothberg 92a2cecd24 EnsureRemoveAll: use IsEBUSY
Most likely just a cosmetic change.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-13 12:33:30 +02:00
Valentin Rothberg 0a7dd54b11 DeleteContainer: simplify logic
Return early instead of having a large body in nested branches.
Preserve the logic of returning ErrNotAContainer even on Get().

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-13 12:33:30 +02:00
Daniel J Walsh 12491a6ee2
Merge pull request #1345 from mtrmac/private-apis
Make {container, layer, image} store APIs private
2022-10-13 05:18:19 -04:00
Miloslav Trmač 487febfe07 Remove unused layerStore.LoadLocked
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:29:37 +02:00
Miloslav Trmač b157e14dd0 Remove completely unused *Store.Lookup() methods
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:29:02 +02:00
Miloslav Trmač 821084ee68 Make all the various *Store interfaces, apart from storage.Store, private
There is no public way to obtain implementations of these interfaces; so
replace the public interfaces with private ones, to make it clear that we
_can_ modify them.

For formal API compatibility, preserve the old interface definitions
as copies.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:25:19 +02:00
Miloslav Trmač b997309d26 Make store's accessors of {Container,Layer,Image,ROLayer,ROImage}Store private
It is not directly reachable by any public API, and there are no known
external users.

So, make it obviously private to make it easier to change the returned
interface in the future.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:25:19 +02:00
Miloslav Trmač 55907efbe1 Add store.allImageStores and store.allLayerStores
... to reduce duplication of read implementations of Store.

This can almost certainly be simplified further; this is enough
for the purposes of this PR, where we want to decrease the
number of callers that need to be updated.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:25:19 +02:00
Miloslav Trmač 26c0d64f9a Use ReloadIfChanged everywhere instead of open-coded Modified+Load
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:19:55 +02:00
Miloslav Trmač e2b1c19e34 Remove unused uidMap and gidMap fields from layerStore
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-11 19:19:55 +02:00
Valentin Rothberg 925647d48c
Merge pull request #1376 from mtrmac/recursive-remove
Remove lockfile.Locker.RecursiveLock (alternative to #1344)
2022-10-11 13:48:33 +02:00
Miloslav Trmač 082a8161ed Remove lockfile.Locker.RecursiveLock
It is not valid as currently implemented, and there is no known user.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-07 19:45:14 +02:00