The image name was incorrectly extracted from the given reference,
yielding a trimmed version of the string and breaking the resulting
replacement value.
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
The update procedure is obliged to return a Result struct with all the
objects that were changed, for filling in the commit template. At
present, the result is collated by running each setter on each object
and seeing if the setter is used. This uses the `Set` from kyaml, with
a small amount of glue.
It doesn't quite work, however, because a setter may be used for a
field without changing the value. The result gets an entry for each
policy _mentioned_, whether or not it had a new value. There is no way
to see whether a setter actually changed a field from the outside,
other than by comparing a copy of the object before using the setter
with the object after (which yaml.v3 does not make easy).
A better approach is to get the setter to record whether it changed
anything, since it is there doing the changing. This means
reimplementing kyaml's `Set`. I have stripped it down to the parts
needed for image updates -- so e.g., only field values are examined.
Signed-off-by: Michael Bridgen <michael@weave.works>
With reference to
https://github.com/fluxcd/image-automation-controller/issues/133
this commit adapts the update->result test so that it checks an
additional case: that a field with an update marker that _does_
correspond to a policy, but _doesn't_ get changed, is not included in
the results.
This test fails at present, because the method for determining the
result is to count which setters are referenced, rather than which
fields were changed.
Signed-off-by: Michael Bridgen <michael@weave.works>
This explains the data available to the commit message template in the
API guide. While writing it, I realised it could be made more
convenient, so:
- mask external types by embedding them
- make the most useful parts of an image ref available using a
wrapper struct and interface
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit:
- passes a value including the update result to the commit message
template
- gives the template result a method for enumerating the
objects regardless of file
This means you can access the images updated either by file
(`.Files`), by object (`.Objects()`), or just as a list
(`.Images()`). The additional test case shows how to use these.
Signed-off-by: Michael Bridgen <michael@weave.works>
It's desirable (see #6) to be able to enumerate the updates that were
made by automation, in the commit message and perhaps in an event
announcing success.
Doing this is counter-intuitively difficult. A `kyaml.setters2.Set`
filter will keep a count of the times its used. Previously, one `Set`
was used with the `SetAll` flag set, which would replace any marker
that corresponded to an image, in one traversal. But to keep track of
images individually, you need to have a setter for _each_ image (and
its tag, and its name, since those can be used separately). This means
`3 x policies` traversals of each node! The saving grace, possibly, is
that only files with a marker in them are considered.
Since you might want to dice the results in different ways, the result
returned is a nested map of file->object->image.
Signed-off-by: Michael Bridgen <michael@weave.works>
This contains a number of fixes, in particular for an indentation bug
that seems to have been introduced not long ago, and which would
otherwise cause problems after updating controller-runtime (which
brings a slightly less recent kyaml with it).
This also comes with a nice enhancement: it's no longer necessary to
use the global schema for kio setters, you can just build a schema and
use that. No need to serialise access to the global schema. Yay!
Signed-off-by: Michael Bridgen <michael@weave.works>
In general a copyright notice takes the form:
Copyright year name
.. where the year is the year of first publication, to let people know
from when the copyright applies. It's fairly common in software to
affix additional years in which the software was modified and
released. I have chosen here to use `2020, 2021` for the new and
modified files; it is OK that not _all_ files are updated, since the
important bit is the _first_ year, which they already have.
Signed-off-by: Michael Bridgen <michael@weave.works>
This is intended to address two problems:
- LocalPackage{Reader,Writer} like to reformat the YAML that passes
through them; mostly this is harmless, but occasionally it will end
up fighting format tooling, e.g., prettier.
- It's possible that things like Helm chart templates are lying
around in the git repository to which automation is applied. Those
templates have extensions of ".yaml" but are not usually parseable
as YAML, so would result in errors from the file reader.
This commit changes how updates are run -- firstly, it screens files
by checking for a token (`"$imagepolicy"`) that will be present in
files that might need updating. This cheaply removes some nodes --
likely including Helm chart templates -- from consideration.
Secondly, it now only writes files that were actually updated by an
imagepolicy setter, rather than writing everything that was an
input. This means it's less likely to reformat something that doesn't
need to be touched at all.
Signed-off-by: Michael Bridgen <michael@weave.works>
The initial implementation of image updates required a single image
policy, and updated every image field that used the image, in the git
repo. This mode has limited practical value, and rather than
elaborating on it, it would be better to concentrate on making the
more carefully thought-through "setters" mode.
It will be useful, for kustomizations e.g., to be able to set just the
tag or just the name (repository). This commit adds setters for those
to the schema -- they have the name of the image setter plus a suffix
of `:tag` or `:name`. For example:
newName: ubuntu # {"$imagepolicy": "ns:policy:name"}
newTag: 18.10 # {"$imagepolicy": "ns:policy:tag"}
This adds another means of updating files to the package pkg/update/,
in setters.go (and gives the existing file a better name).
In passing, I changed the test util for comparing before/after
updates, in pkg/files/, to give a little more context when comparing
file contents; and, since the comparison between actual and expected
is not symmetrical, I corrected the order of the args in the tests.
Previously: replace the YNode (yaml.Node) with a new one which is just
the replacement string.
Now: change the value of the YNode, and set it back in place.
This updates the image replacement to switch on the kind of the node,
and change the way the replacements are done for CronJobs.
CronJobs have PodTemplateSpecs embedded deeper in the structure.
This factors out the function that checks directories for equivalence,
and uses it to check that the upstream repo has the expected update
when the controller has pushed its commit.
Just to get points on the board, this gives ImageUpdateAutomation an
`update` field into which you can plug the name of an ImagePolicy
resource (from the image reflector controller). The idea is that the
automation will then replace the image in the policy, anywhere it's
used, with its latest version.
Aurel Canciu
Stefan Prodan
Michael Bridgen
nguyenductoan
Michael Bridgen
Kevin McDermott