It's desirable (see #6) to be able to enumerate the updates that were
made by automation, in the commit message and perhaps in an event
announcing success.
Doing this is counter-intuitively difficult. A `kyaml.setters2.Set`
filter will keep a count of the times its used. Previously, one `Set`
was used with the `SetAll` flag set, which would replace any marker
that corresponded to an image, in one traversal. But to keep track of
images individually, you need to have a setter for _each_ image (and
its tag, and its name, since those can be used separately). This means
`3 x policies` traversals of each node! The saving grace, possibly, is
that only files with a marker in them are considered.
Since you might want to dice the results in different ways, the result
returned is a nested map of file->object->image.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit adds a handful of tests to make sure errors in the
problematic formats are rewritten, and other errors are left alone.
I'm reluctant to test against the actual git providers, since that
would introduce a dependency on them. Thus, these tests won't guard
against the providers changing their messages.
Signed-off-by: Michael Bridgen <michael@weave.works>
libgit2 and go-git both have flaws in the way they treat errors from
the remote. go-git takes only the first line, meaning that it gets a
blank error message from GitLab which like to respond with a
banner. libgit2 returns the whole response, including blank lines and
fences ("=========...").
This commit corrects for both these flaws, by supplying a message if
go-git has taken a blank line, and stripping out blank lines and
fences from libgit2's error. This is unavoidably a brittle approach,
so I have limited it to just the situation that was reported as a
problem: pushing to the upstream git repo.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit rearranges update tests so that those that check that
updates are made can be run against a git server using SSH as well as
HTTP.
The local clone, used to provoke automated updates and to check
results, still uses HTTP. Those operations are not under test.
libgit2 wants to be asked for authentication when using SSH, and will
balk if it's not requested by the server. To avoid that, auth must be
switched on for the git test server.
This also switches auth on for HTTP, so it's necessary to use a git
URL that includes credentials for setting things up with a local
clone. I have also used that URL for the git-over-HTTP tests -- it's
arguable whether it's necessary to test that works, here.
Signed-off-by: Michael Bridgen <michael@weave.works>
The "auth strategy", which depends on the GitImplementation, was
hard-wired to the "gogit" constant, but it should come from the
GitRepository spec. When the implementation is "libgit2" and the git
URL entails SSH, the result would normally include a callback for
checking the host key against known_hosts; but since it was
hard-wired, it was missing that callback.
This explains at least some instances of the error `user cancelled
hostkey check` from #106. The error, or a close relative, might also
arise if the callback rejects the host key because the host as it
appears in the known_hosts doesn't match that host as passed to the
callback -- see
https://github.com/fluxcd/source-controller/issues/287.
Signed-off-by: Michael Bridgen <michael@weave.works>
- set fsGroup to allow AWS IAM Role bindings
- fix the group assignment in Alpine
- bump Alpine to 3.13
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Using the helper from `pkg/runtime/pprof`, which follows the suggestion
from controller-runtime to use `AddMetricsExtraHandler`.
Signed-off-by: Hidde Beydals <hello@hidde.co>
The `.spec.update` field has a default, which means you can leave it
out of new objects, and it will still be a valid spec. However,
existing objects will not be valid, because they will have a value for
`.spec.update` (so it won't get the default) which doesn't have a
value for `.strategy` (which is required, and an enum).
So: default the strategy field as well, so that existing objects are
still valid. This doesn't change the meaning of any existing objects,
since the outcome is the default, which is the only possible value
anyway.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit updates the generated and written API refs to account for
the new update strategy types.
Signed-off-by: Michael Bridgen <michael@weave.works>
It's convenient to be able to leave out the update strategy, since
there is only one possible value at present; and if there were
alternatives, the present choice would still be a reasonable
default. However, with the format as it is, this doesn't work with
OpenAPIv3 schema, so you have to supply a value, even though there are
no parameters:
```yaml
spec:
update:
setters: {}
```
A more self-explanatory format which _does_ work with defaulting is to
name the strategy rather than relying on the presence of a field:
```yaml
spec:
update:
strategy: Setters
```
The whole `update` field can be elided and left to default. This
doesn't preclude having other strategies later, even those with
parameters, e.g.,
```yaml
spec:
update:
strategy: Foo
fooParam: 5
```
This commit changes the API types and code that uses them, and the CRD
manifest, and adds a test that checks the defaulting actually works.
Signed-off-by: Michael Bridgen <michael@weave.works>
The build now incorporates libgit2; however, most Linux distributions,
notably Ubuntu and Debian, have a version of libgit2 that's over a
year old. This copies the approach used in source-controller, which is
to use a container image that can be preloaded with an up to date
release of libgit2.
Signed-off-by: Michael Bridgen <michael@weave.works>
This follows up the stage-setting in prior commits, by respecting the
GitImplementation field given in the GitRepository object. NB it only
matters for cloning and pushing, so gogit is used in the "middle" to
record the commit in the local checkout.
Signed-off-by: Michael Bridgen <michael@weave.works>
This separates the commit and push steps, since the commit step just
uses gogit, while the push step will dispatch based on the git
implementation.
Signed-off-by: Michael Bridgen <michael@weave.works>
This copies the approach used in source-controller: add the
libgit2-dev package (i.e., headers) to the build container, then add
the shared lib to the _image_ image.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit changes the clone and push code to use libgit2. In the
case of clone, this means simply passing the const representing the
libgit2 implementation to the source-controller/pkg/git function. In
the case of push, this means adding a small helper to do the required
invocation.
NB:
- there's no need to use libgit2 for operations other than clone and
push; in particular, commits can have a single, go-git
implementation.
- libgit2's push is quite sensitive to the refspec it's given;
`<branch>:refs/heads/<branch>` didn't work, and supplying no
refspecs makes it time out.
- libgit2 push will only work with a repository on disk that was
cloned by libgit2 -- it's initialising the repo differently to
go-git. This is surprising (a git repo is a git repo, isn't it?),
but fine -- any given automation will use _either_ go-git or
libgit2 for both of clone and push.
Signed-off-by: Michael Bridgen <michael@weave.works>
Michael Bridgen
Hidde Beydals
Hidde Beydals
Stefan Prodan