* Restore succeeds (skipping over the layer) if layer contents are corrupted
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Exporter does not re-use layer from volume cache if layer contents are corrupted
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fixes
Signed-off-by: Natalie Arellano <narellano@vmware.com>
---------
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* add failing test to restorer
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* restorer and exporter working as expected
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* lint
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update phase/restorer.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update phase/cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/image_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/volume_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/volume_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/volume_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/volume_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* Update cache/volume_cache.go
Co-authored-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* update based on feedback
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
* fix log
* temp fix
* this does not work as is. I think we need to modify img utils.
Image utils should fail with a Layer Not found in both ReuseLayer & GetLayer.
For GetLayer, when there is a missing blob, it's return an unexpected EOF error.
For ReuseLayer, when there is a missing blob, it's not returning an error but it should.
* add eof check
* add not exist check
* reuse layer test
* fix test regression
---------
Signed-off-by: Joey Brown <brown.joseph@salesforce.com>
Co-authored-by: Joey Brown <brown.joseph@salesforce.com>
* Consolidate methods that read and write platform spec'd TOML
in platform/files package.
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Calculate lifecycle digest and output version as part of acceptance testing
This can help us diagnose weird failures in CI
Signed-off-by: Natalie Arellano <narellano@vmware.com>
---------
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove deleteOrigImage function from the cache and relative test
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Revert "Remove deleteOrigImage function from the cache and relative test"
This reverts commit 17e646fc39602777a37977dd9416e59aa62f6d04.
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Implemented a new component called cache deleter which takes care of the deletion of the cache images
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Adjusted the name of the struct field
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Move the imade deleter instatiation up to the main
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Add parameter to enable/diable the deletion functionality, for now always set has enabled
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Add feature guard based on the platformAPI version
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Fixing some test titles
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Introduce ImageComparer component
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Add async go-subroutine to the delete call to speed up the process
```go
func Benchmark(b *testing.B) {
mockController := gomock.NewController(b)
fakeImageComparer := cacheMock.NewMockImageComparer(mockController)
testLogger := cmd.DefaultLogger
imageDeleter := NewImageDeleter(fakeImageComparer, testLogger, true)
for i := 0; i < b.N; i++ {
fakeOrigImage := fakes.NewImage("fake-image", "", local.IDIdentifier{ImageID: "fakeImage"})
fakeNewImage := fakes.NewImage("fake-image", "", local.IDIdentifier{ImageID: "fakeNewImage"})
fakeImageComparer.EXPECT().ImagesEq(fakeOrigImage, fakeNewImage).AnyTimes().Return(false, nil)
imageDeleter.DeleteOrigImageIfDifferentFromNewImage(fakeOrigImage, fakeNewImage)
}
}
```
The code above produced this result:
* without the go-subroutine
```
goos: darwin
goarch: amd64
pkg: github.com/buildpacks/lifecycle/cache
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
Benchmark
Benchmark-12 3501 8995839 ns/op
PASS
```
* with the go-subroutine
```
goos: darwin
goarch: amd64
pkg: github.com/buildpacks/lifecycle/cache
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
Benchmark
Benchmark-12 3560 9133704 ns/op
PASS
```
Speed increased by 1.53% ns/op
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
---------
Signed-off-by: Domenico Luciani <dluciani@vmware.com>
* Small fixes from user testing
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fixes from user acceptance
- Removes -stack flag from analyzer
- Fixes debug log message
- Removes deprecated rand.Seed
Signed-off-by: Joe Kimmel <joe-kimmel-vmw@vmware.com>
* Bump linter to a version that will work with go 1.20
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Restorer should ensure ownership of kaniko directory
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* pull the less-than case for flags out of the switch
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* appease the windows linter
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
* Fix weird log output
Signed-off-by: Natalie Arellano <narellano@vmware.com>
---------
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Joe Kimmel <joe-kimmel-vmw@vmware.com>
Signed-off-by: Joe Kimmel <jkimmel@vmware.com>
Co-authored-by: Joe Kimmel <joe-kimmel-vmw@vmware.com>
Co-authored-by: Joe Kimmel <jkimmel@vmware.com>
* Add history when adding buildpack or extension layers
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* REVERTME: point imgutil to feature branch
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Parse extension ID from history so that we can output a useful log message
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Avoid extra wrapping
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Bump imgutil
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix acceptance by overriding imgutil methods for "caching image"
We need to add to the launch cache when adding layers so that the next build will be faster
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add tests for layer factory
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add acceptance test for extender should update history
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Move history methods to imgutil
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix exporter acceptance by fixing imgutil
The digest for a remote image from imgutil
should match the digest for a remote image from ggcr
unless the user specifically requested modifications like overriding history
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add acceptance test; fix layer name when there is only one slice
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Bump imgutil
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix acceptance
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix acceptance
imgutil mutates the image (history) for newer platforms, so the SHA is different
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Point imgutil back to commit on main, also bump containerd
Signed-off-by: Natalie Arellano <narellano@vmware.com>
---------
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Validate the cache image and previous image in the analyze phase
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
* Fix unit tests
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
* Upgrade imgutil
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
* allow-nondistributable-artifacts for windows images
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
* Update docs
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
---------
Signed-off-by: Matthew Robertson <mattrobertson@google.com>
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com>
* Use launch cache when calling SaveAs on caching image
- Failure to use the launch cache will result in slower second builds,
due to the time it takes to pull layers from the daemon
- Improve creator acceptance test
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Try to make test less flakey by forcing first build to be slower
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* ci(lint): auto lint fixes
Signed-off-by: Michael de Senna <desenna@gmail.com>
* fix: replace deprecated funcs
Signed-off-by: Michael de Senna <desenna@gmail.com>
* ci(lint): fix many io/ioutil staticcheck
Signed-off-by: Michael de Senna <desenna@gmail.com>
* fix: import os
Signed-off-by: Michael de Senna <desenna@gmail.com>
Signed-off-by: Michael de Senna <desenna@gmail.com>
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Co-authored-by: Natalie Arellano <narellano@vmware.com>
* If rename fails, fall back to copy and delete on Windows
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Rename internal/io to internal/fsutil
This avoids import collisions with go's io package
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* fsutil.Copy explicitly ignores weird edge cases
Signed-off-by: Natalie Arellano <narellano@vmware.com>
Initialized the cache image to add an empty layer in case is needed
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
# This is the commit message #2:
Fixing compilation error caused by imgutil PR #141
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
* WIP - Initialized the cache image to add an empty layer in case is needed
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
* Pointing to the latest imgutil empty layer branch
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
* Updating imgutil version after PR#142 was merged
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
* Acceptance tests pass with creator commented out
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix creator
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add unit tests for analyze inputs
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add scaffold for analyzer builder unit tests
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add many unit tests, still have some TODOs
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Address most TODOs
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Acceptance tests pass
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* A few more TODOs
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Bring back go 1.16
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add unit test
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix acceptance tests
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix units
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove nolint
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove unneeded things
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Use operations pattern
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Refactor tests to take advantage of operations pattern
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add missing tests
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix lint
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Analyzer factory assigns "nop" services by default
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Try to fix registry handler test
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Move new package from cmd/lifecycle/platform to platform/inputs
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Updates per PR review
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Test the platform instead of the exiter
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Combine cmd/lifecycle/platform and platform
With some small changes to avoid an import cycle, we can make a meaningful platform package
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Clean up constructors
Eventually only the platform package should switch on platform api
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove ForAnalyzer struct
Having the analyzer factory take a list of args will ensure we don't forget
to update the creator when things change.
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Combine cmd/launcher/platform and platform
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove comment
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Remove spec alias
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Move cache metadata back to platform
The lifecycle shouldn't have to depend on the cache package, just the interface
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add comment and remove unused vars
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Add test for buildpack incompatibility error
Signed-off-by: Natalie Arellano <narellano@vmware.com>
* Fix typo
Signed-off-by: Natalie Arellano <narellano@vmware.com>
- Fixes cache images create with wrong format
- imgutil WithPlatform initializes OS-specific images
- Windows sim base layers are generated by imgutil instead of lifecycle
Signed-off-by: Micah Young <ymicah@vmware.com>
- Fixes cache images create with wrong format
- imgutil WithPlatform initializes OS-specific images
- Windows sim base layers are generated by imgutil instead of lifecycle
Signed-off-by: Micah Young <ymicah@vmware.com>
When using image cache, the commit of new cache deletes the old cache. There is a check for if there is a previous cache image, but that check was happenign after the new cache was written to that tag.
https://github.com/buildpacks/lifecycle/issues/453
Signed-off-by: Jesse Brown <jabrown85@gmail.com>
With the reproducibility improvements, it often happens that the old and
new cache images are the same. The original image should not be
deleted in this case.
This is an issue with remote images, where the image is deleted by
digest and both the old and new images share the same digest.
Signed-off-by: Lukas Berger <bergerl@google.com>
* params prefer 'diffID' to 'sha' for specificity
* errors messages prefer 'SHA' to 'diffID' for consistency
* we should consider changing this convention this later
Signed-off-by: Emily Casey <ecasey@pivotal.io>
Jesse Brown