Commit Graph

28626 Commits

Author SHA1 Message Date
Doug Rabson 056232e4e5 pkg/archive: Add support for file flags on FreeBSD
This encodes flag information into the tar stream using
ReadFileFlagsToTarHeader and decodes with WriteFileFlagsFromTarHeader.

To support applying diffs to trees with flags, this adds logic to
reset immutable flags during the UnpackLayer process.  To support
immutable directories, we also need to defer setting flags on
directories until after all modifications to the directory contents.
Fortunately, something similar is already in place for setting
directory modify times.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson 2baf25d67c pkg/archive: Add support for detecting flag changes
Note: the Flags method of system.StatT always returns zero on
non-FreeBSD platforms so this should have no functional change for those
platforms.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson e759a1b2b0 pkg/archive: Add functions for managing file flags
This adds platform-specific entry points to encode and decode file
flags. These are exported so that they can be used in buildah/copier. A
function which can be used to optionally reset the immutable flag on a
file or directory is also added and this will be used when applying
layer changes to trees with immutable files or directories.

These new entry points are implemented for FreeBSD and are noop stubs on
other platforms. The chflags approach could be used on darwin and other
BSD-derived platforms if needed.

On FreeBSD, file flag information is encoded for import/export using the
PAX SCHILY.fflags option. This is compatible with bsdtar (see
https://github.com/libarchive/libarchive). Reading the code for
libarchive, it also has support for encoding/decoding Linux file flags
so this seems like a good reference.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson 9ad4537f01 pkg/system: Add file flags to StatT on FreeBSD
This will also probably work on other BSD derived platforms including
darwin.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson 3ad57b57ad pkg/system: Add file flag names for FreeBSD
Note: these are likely to be identical on other BSDs, including darwin,
but I'm keeping this for just FreeBSD to start with.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson a987773f33 pkg/system: Defer the call to resetFileFlags until after RemoveAll fails
Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Doug Rabson adb0355904 pkg/system: Add Lchflags and use that in resetFileFlags
This also adds an integration test for deleting containers with
immutable files.

Signed-off-by: Doug Rabson <dfr@rabson.org>
2022-10-20 08:58:04 +01:00
Daniel J Walsh 95f15672c2
Merge pull request #1402 from nalind/test-errcheck
pkg/lockfile.TestLockfileMultiprocessModified(): check for wait errors
2022-10-19 15:34:42 -04:00
Nalin Dahyabhai a14e7d309e pkg/lockfile.TestLockfileMultiprocessModified(): check for wait errors
Check for errors from command Wait() methods.  Not sure how this
inconsistency even landed in the first version of this test.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2022-10-19 08:14:26 -04:00
Daniel J Walsh 742493c6ca
Merge pull request #1397 from nalind/test-1391-1396
Add TestLockfileMultiprocessModified()
2022-10-19 07:32:04 -04:00
Daniel J Walsh 2ceb61c634
Merge pull request #1400 from mtrmac/fix-typos
Fix locking bugs
2022-10-19 06:52:53 -04:00
Daniel J Walsh c68c699333
Update pkg/lockfile/lockfile_test.go
Co-authored-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 06:42:03 -04:00
Daniel J Walsh fa02841d87
Merge pull request #1401 from vrothberg/mount-fix
fix break during unmount
2022-10-19 06:41:32 -04:00
Valentin Rothberg f4aa849172 fix break during unmount
Move the break statement out of the switch to actually break out of the
loop as intended.

Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2022-10-19 10:46:14 +02:00
Miloslav Trmač 5c6186fe81 Fix {layerStore,imageStore}.startWriting()
This is a bad bug: we wouldn't reload state from disk
at all in writers.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 06:35:44 +02:00
Miloslav Trmač b35079f911 Fix locking in CreateContainer
Don't immediately unlock the layer store after locking it.

Luckily it only matters for additional stores.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 05:37:40 +02:00
Miloslav Trmač 33c396d1aa
Merge pull request #1398 from nalind/subdirs
Don't create $root/{mounts,tmp}, which we don't use
2022-10-19 01:23:08 +02:00
Nalin Dahyabhai ef7acdd4ad Don't create $root/{mounts,tmp}, which we don't use
Don't create "mounts" or "tmp" subdirectories under the graph root at
startup, since we ended up not using them.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2022-10-18 15:45:54 -04:00
Nalin Dahyabhai 4293c488da Add TestLockfileMultiprocessModified()
Test changes introduced in #1396.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2022-10-18 15:39:37 -04:00
Daniel J Walsh 77f301d916
Merge pull request #1396 from nalind/tweak-1391
Fix lockfiles never detecting changes
2022-10-18 15:17:29 -04:00
Nalin Dahyabhai de366fc7f1 Fix lockfiles never detecting changes
If we ever check the last-writer of a lock file and get back a zero
value, don't only check if subsequent last-writer values match its first
zero bytes, which is always true, as a test to see if the last-writer
value has remained unchanged.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2022-10-18 14:32:27 -04:00
Daniel J Walsh bd3bbad3e4
Merge pull request #1395 from mtrmac/locking-responsibility
Move locking responsibility from store.go to the container/image/layer stores
2022-10-18 06:38:30 -04:00
Giuseppe Scrivano 684d84e256
Merge pull request #1393 from mtrmac/Setenv
Use testing.T.Setenv
2022-10-18 09:40:08 +02:00
Miloslav Trmač d0ad132945 Move layerStore.{ReloadIfChanged,Modified}
.. to be closer to the lock / load set of methods.

Only moves unchanged code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:40:00 +02:00
Miloslav Trmač dbe5fe3059 Remove Lock/Unlock methods from layerStore
They are now only used in the constructors, so use a variant
of startReading/startWriting instead.  This code path is not
performance-critical, so let's share as much code as possible
to ensure consistency in locking.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:33 +02:00
Miloslav Trmač d2defdb9fa Remove Save() from rwLayerStore API
It is done implicitly by all writers already.

Also fix the documentation not to point at an explicit Touch(),
which is not actually necessary.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:27 +02:00
Miloslav Trmač 328ccce601 Remove Load() and ReloadIfChanged() from roLayerStore API
Callers should just use startReading/startWriting.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:27 +02:00
Miloslav Trmač ee9708644f Remove Modified() from the roLayerStore API
It is an internal helper for ReloadIfChanged, with
no external users.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:27 +02:00
Miloslav Trmač 5791a561df Remove unused lockfile forwarders from roLayerStore API
The only callers are internal, have them access r.lockfile directly.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:27 +02:00
Miloslav Trmač 124e526285 Remove a completely unused method from roLayerStore
Exposing the internals of the lock is not necessary, and exposes
too many implementation details.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač c935245fe4 Move the writing lock methods from roLayerStore to rwLayerStore
... for a bit of extra safety.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač f9678cd44b Replace layerStore.{Lock,Unlock} with {startWriting,stopWriting}
This integrates ReloadIfChanged, and makes it clearer that the responsibility
for maintaining locking details is with the layerStore; we can change it
in a single place.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač c4089b6e3d Replace layerStore.{RLock,Unlock} with {startReading,stopReading}
This integrates ReloadIfChanged, and makes it clearer that the responsibility
for maintaining locking details is with the layerStore; we can change it
in a single place.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač d5b48bef6e Copy methods from included interfaces directly into *LayerStore
... so that we can modify/replace them.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 8db2532e49 Move imageStore.ReloadIfChanged
... to be closer to the lock / load set of methods.

Only moves unchanged code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 360bfae52a Remove Lock/Unlock methods from imageStore
They are now only used in the constructors, so use a variant
of startReading/startWriting instead.  This code path is not
performance-critical, so let's share as much code as possible
to ensure consistency in locking.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 8af42de786 Remove Save() from rwImageStore API
It is done implicitly by all writers already.

Also fix the documentation not to point at an explicit Touch(),
which is not actually necessary.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 5f6642c79f Remove a redundant rwImageStore.Save() call
rwImageStore.Create() already calls it.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 097e954764 Remove Load() and ReloadIfChanged() from roImageStore API
Callers should just use startReading/startWriting.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 77ab6e4204 Remove unused lockfile forwarders from roImageStore API
The only callers are internal, have them access r.lockfile directly.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 19659d9a01 Remove a completely unused method from roImageStore
Exposing the internals of the lock is not necessary, and exposes
too many implementation details.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač d312f1dffa Move the writing lock methods from roImageStore to rwImageStore
... for a bit of extra safety. That requires us to be a bit
more explicit in one of the users.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 2f68ba24c1 Replace imageStore.{Lock,Unlock} with {startWriting,stopWriting}
This integrates ReloadIfChanged, and makes it clearer that the responsibility
for maintaining locking details is with the imageStore; we can change it
in a single place.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač a047bc18e4 Replace imageStore.{RLock,Unlock} with {startReading,stopReading}
This integrates ReloadIfChanged, and makes it clearer that the responsibility
for maintaining locking details is with the imageStore; we can change it
in a single place.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač 9844aa39cb Copy methods from included interfaces directly into *ImageStore
... so that we can modify/replace them.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač adae72bf37 Move containerStore.ReloadIfChanged
... to be closer to the lock / load set of methods.

Only moves unchanged code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:19 +02:00
Miloslav Trmač b44c193f3f Remove Lock/Unlock methods from rwContainerStore
They are now only used in the constructor, so use a variant
of startWriting instead.  This code path is not performance-critical,
so let's share as much code as possible to ensure consistency
in locking.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:39:15 +02:00
Miloslav Trmač 4c55fadd03 Remove Save() from rwContainerStore API
It is done implicitly by all writers already.

Also fix the documentation not to point at an explicit Touch(),
which is not actually necessary.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:35:15 +02:00
Miloslav Trmač 58f1120738 Remove Load() and ReloadIfChanged() from rwContainerStore API
Callers should just use startReading/startWriting.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:34:53 +02:00
Miloslav Trmač 409ee3ecbd Remove unused lockfile forwarders from rwContainerStore API
The only callers are internal, have them access r.lockfile directly.

Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-17 21:05:05 +02:00