This commit adds a handful of tests to make sure errors in the
problematic formats are rewritten, and other errors are left alone.
I'm reluctant to test against the actual git providers, since that
would introduce a dependency on them. Thus, these tests won't guard
against the providers changing their messages.
Signed-off-by: Michael Bridgen <michael@weave.works>
libgit2 and go-git both have flaws in the way they treat errors from
the remote. go-git takes only the first line, meaning that it gets a
blank error message from GitLab which like to respond with a
banner. libgit2 returns the whole response, including blank lines and
fences ("=========...").
This commit corrects for both these flaws, by supplying a message if
go-git has taken a blank line, and stripping out blank lines and
fences from libgit2's error. This is unavoidably a brittle approach,
so I have limited it to just the situation that was reported as a
problem: pushing to the upstream git repo.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit rearranges update tests so that those that check that
updates are made can be run against a git server using SSH as well as
HTTP.
The local clone, used to provoke automated updates and to check
results, still uses HTTP. Those operations are not under test.
libgit2 wants to be asked for authentication when using SSH, and will
balk if it's not requested by the server. To avoid that, auth must be
switched on for the git test server.
This also switches auth on for HTTP, so it's necessary to use a git
URL that includes credentials for setting things up with a local
clone. I have also used that URL for the git-over-HTTP tests -- it's
arguable whether it's necessary to test that works, here.
Signed-off-by: Michael Bridgen <michael@weave.works>
The "auth strategy", which depends on the GitImplementation, was
hard-wired to the "gogit" constant, but it should come from the
GitRepository spec. When the implementation is "libgit2" and the git
URL entails SSH, the result would normally include a callback for
checking the host key against known_hosts; but since it was
hard-wired, it was missing that callback.
This explains at least some instances of the error `user cancelled
hostkey check` from #106. The error, or a close relative, might also
arise if the callback rejects the host key because the host as it
appears in the known_hosts doesn't match that host as passed to the
callback -- see
https://github.com/fluxcd/source-controller/issues/287.
Signed-off-by: Michael Bridgen <michael@weave.works>
- set fsGroup to allow AWS IAM Role bindings
- fix the group assignment in Alpine
- bump Alpine to 3.13
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Using the helper from `pkg/runtime/pprof`, which follows the suggestion
from controller-runtime to use `AddMetricsExtraHandler`.
Signed-off-by: Hidde Beydals <hello@hidde.co>
The `.spec.update` field has a default, which means you can leave it
out of new objects, and it will still be a valid spec. However,
existing objects will not be valid, because they will have a value for
`.spec.update` (so it won't get the default) which doesn't have a
value for `.strategy` (which is required, and an enum).
So: default the strategy field as well, so that existing objects are
still valid. This doesn't change the meaning of any existing objects,
since the outcome is the default, which is the only possible value
anyway.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit updates the generated and written API refs to account for
the new update strategy types.
Signed-off-by: Michael Bridgen <michael@weave.works>
It's convenient to be able to leave out the update strategy, since
there is only one possible value at present; and if there were
alternatives, the present choice would still be a reasonable
default. However, with the format as it is, this doesn't work with
OpenAPIv3 schema, so you have to supply a value, even though there are
no parameters:
```yaml
spec:
update:
setters: {}
```
A more self-explanatory format which _does_ work with defaulting is to
name the strategy rather than relying on the presence of a field:
```yaml
spec:
update:
strategy: Setters
```
The whole `update` field can be elided and left to default. This
doesn't preclude having other strategies later, even those with
parameters, e.g.,
```yaml
spec:
update:
strategy: Foo
fooParam: 5
```
This commit changes the API types and code that uses them, and the CRD
manifest, and adds a test that checks the defaulting actually works.
Signed-off-by: Michael Bridgen <michael@weave.works>
The build now incorporates libgit2; however, most Linux distributions,
notably Ubuntu and Debian, have a version of libgit2 that's over a
year old. This copies the approach used in source-controller, which is
to use a container image that can be preloaded with an up to date
release of libgit2.
Signed-off-by: Michael Bridgen <michael@weave.works>
This follows up the stage-setting in prior commits, by respecting the
GitImplementation field given in the GitRepository object. NB it only
matters for cloning and pushing, so gogit is used in the "middle" to
record the commit in the local checkout.
Signed-off-by: Michael Bridgen <michael@weave.works>
This separates the commit and push steps, since the commit step just
uses gogit, while the push step will dispatch based on the git
implementation.
Signed-off-by: Michael Bridgen <michael@weave.works>
This copies the approach used in source-controller: add the
libgit2-dev package (i.e., headers) to the build container, then add
the shared lib to the _image_ image.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit changes the clone and push code to use libgit2. In the
case of clone, this means simply passing the const representing the
libgit2 implementation to the source-controller/pkg/git function. In
the case of push, this means adding a small helper to do the required
invocation.
NB:
- there's no need to use libgit2 for operations other than clone and
push; in particular, commits can have a single, go-git
implementation.
- libgit2's push is quite sensitive to the refspec it's given;
`<branch>:refs/heads/<branch>` didn't work, and supplying no
refspecs makes it time out.
- libgit2 push will only work with a repository on disk that was
cloned by libgit2 -- it's initialising the repo differently to
go-git. This is surprising (a git repo is a git repo, isn't it?),
but fine -- any given automation will use _either_ go-git or
libgit2 for both of clone and push.
Signed-off-by: Michael Bridgen <michael@weave.works>
This updates the source controller module to v0.5.2, and adjusts the
use of the git package therein. The main change there is that it now
accounts for two different git implementations -- go-git and libgit2
-- but I have not exposed that difference, just made it work as it did
before.
Signed-off-by: Michael Bridgen <michael@weave.works>
This commit upgrades the `controller-runtime` dependency to `v0.7.0`,
including all changes required to make all wiring work again.
- Upgrade `runtime` to v0.6.2 to include `controller-runtime` changes.
- Logger has been removed from the reconciler, and is now retrieved
from the `context.Context` passed to the `Reconcile` method and
downwards functions.
- Logger configuration flags are now bound to the flag set using
`BindFlags` from `runtime/logger`, ensuring the same contract across
GitOps Toolkit controllers, and the `--log-json` flag has been
deprecated in favour of the `--log-encoding=json` default.
- The `ChangePredicate` from `runtime` has changed to a
`ReconcileRequestedPredicate`, and is now chained with the
`GenerationChangedPredicate` from `controller-runtime` using
`predicate.Or`.
- Signatures that made use of `runtime.Object` have changed to
`client.Object`, removing the requirement to e.g. call
`runtime.Object#Object`.
- The `leader-election-role` was changed, as leader election now works
via the `coordination/v1` API.
Other notable changes:
- Upgrade of `image-reflector-controller` API package to include
controller-runtime changes.
- Upgrade of `source-controller` API package to v0.6.1.
Signed-off-by: Hidde Beydals <hello@hidde.co>
This contains a number of fixes, in particular for an indentation bug
that seems to have been introduced not long ago, and which would
otherwise cause problems after updating controller-runtime (which
brings a slightly less recent kyaml with it).
This also comes with a nice enhancement: it's no longer necessary to
use the global schema for kio setters, you can just build a schema and
use that. No need to serialise access to the global schema. Yay!
Signed-off-by: Michael Bridgen <michael@weave.works>
Michael Bridgen
Stefan Prodan
Hidde Beydals
Hidde Beydals