Adds a finalizer to the Cluster resource so we can delete the namespace related to it on deletion.
Refers to: https://github.com/rancher/fleet/issues/3985
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
While Fleet config files are typically named `fleet.yaml`, the recent
introduction of user-driven bundle scanning enables a config file to be
named arbitrarily, in which case it must still be excluded from the
corresponding bundle. Fleet now takes care of this without any action
needed from the user.
Integration tests also demonstrate that `.fleetignore` files can be
leveraged to exclude config files living in the same directory as
the config file referenced explicitly through user-driven bundle
scanning. Fleet would otherwise not exclude those files from the bundle.
While we had recently added support for globbing for path-based secrets
to classic bundle scanning, that had been omitted for user-driven bundle
scanning.
This commit remedies this, ensuring that globbing support works
regardless of the chosen bundle scanning mode.
A bundle must only be reconciled upon changes impacting its spec, or a
related cluster or bundle deployment. Changes to the bundle's status
must not lead to new reconciles, which may cause endless loops.
When a chart version cannot be retrieved from a Helm repository,
including the response body in the error message brings little context
and can lead to excessively long error status messages.
Therefore, Fleet now skips the response body in such cases, relying
solely on the status code.
* Replace deprecated Requeue with RequeueAfter in controllers
Replace all usages of reconcile.Result{Requeue: true} with appropriate
RequeueAfter durations to align with controller-runtime best practices.
* Improve stability of helmops controller tests
Increase retry attempts and add delays to reduce race conditions in
HelmOp updates.
* Improve stability of gitrepo e2e tests
Following the changes for Replace deprecated Requeue with RequeueAfter
in controllers.
The Helm deployer now honours Helm's `atomic` flag for installs, not just
updates.
It also supports overwriting releases in `pending-install` state, which can
unblock deployments.
A new integration test case simulates an existing release with
`pending-install` status, by creating the corresponding Helm secret, as
Fleet uses Helm's secret storage back-end, then validates that Fleet can
install a release with the same name on top of it.
---------
Co-authored-by: Mario Manno <mario.manno@suse.com>
Co-authored-by: Corentin Néau <tan.neau@suse.com>
Values files may be referenced by file name alone, or with a directory
prefix. Both variants now lead to values files being excluded from
bundles resources, regardless of where a `fleet.yaml` file referencing
them lives (inside or outside of a chart directory).
* Trims the v prefix in versions before checking if it's a valid semver
As decribed in the related issue, some helm repositories identify versions using the "v" prefix (like v1.5.6).
Although that's not recommended by Helm, those repositories are still valid.
This PR trims the "v" prefix (if exists) before checking if the version stored in the Bundle spec is semver compliant.
Refers to: https://github.com/rancher/fleet/issues/3953
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
A bundle's name can be computed from a GitRepo's name and the path
leading to the bundle. However, overriding that name is also possible,
by populating the `name` field in a `fleet.yaml` file.
A new integration test case covers and demonstrates this.
* Support globbing in helmSecretNameForPaths
Helm access secrets referenced in a GitRepo's `helmSecretNameForPaths`
field now support globbing, such as `my-path/*`, or `foo?bar`.
When more than one path listed in the `helmSecretNameForPaths` secret
matches a given GitRepo path, then the first match listed in the
`helmSecretNameForPaths` secret will be used.
* Ensure deterministic order of iteration over auth map
When more than one patterns match a given bundle path, the first
matching pattern, in lexical order, will be used.
This resets the HelmOps conditions in the following scenarios:
* When getting a version from the helm repository if there was a previous error
* When getting a new version from the helm repository
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
* Check Helm for static version
When a HelmOp resource references a static chart version, the HelmOps
controller checks Helm for existence of that version.
This prevents errors about the version not existing in the Helm index
from going undetected until a bundle deployment is created and an agent
attempts to deploy it.
Consequently, this also ensures that errors about the version not
existing are produced even if no target cluster is found for the HelmOp.
* Remove extra wrapping from error message
When a Helm chart version cannot be found, the error is already wrapped
with a message stating `could not get a chart version`. No need to wrap
it with a similar message.
* Reduce risk of conflicts in HelmOps status updates
The HelmOps status reconciler now uses a `Patch` operation, updating
only changed status fields as opposed to replacing a full HelmOps status
subresource.
* Fix `GotNewChartVersion` visibility issue
The condition on emitting the event was wrong, as a new, not yet found
version should be determined based on the status of the HelmOp, which is
later updated by the same polling job.
* Add HelmOps to cluster status
With HelmOps no longer an experimental feature, HelmOp resources are now
reflected in cluster statuses, similarly to GitRepos, with desired ready
and actual ready counts.
* Add desired and ready HelmOps to cluster metrics
HelmOp resources are now reflected next to GitRepos in cluster metrics.
* Validate HelmOps fields in reconciler
Only a subset of all possible combinations of repo, chart and version
fields make sense. The HelmOps reconciler now ensures that unsupported
combinations lead to error messages being output in the HelmOp's status,
without any bundle being created.
* Enable HelmOps OCI chart rendering
When installing a Helm chart through an OCI reference, HelmOps are
expected to store that reference in the `repo` field, unlike GitOps Helm
options which use the `chart` field to that effect.
Since we do not want to break compatibility with existing gitOps setups,
chart URL resolution and contents rendering is now a bit more flexible
than it used to be, supporting both variants.
By the same token, reporting of failures to install charts is improved
in the HelmOps status, which previously skipped status updates when the
`chart` field was empty.
* Add end-to-end test for HelmOps with tarball
A new end-to-end test case validates that HelmOps supports installing a
Helm chart referenced through a tarball.
Kubernetes resources required by Fleet to manage HelmOps resources are
now created if, and only if, Helm value `helmops.enabled` is `true`.
That value is `true` by default, enabling HelmOps, as was already the case
for GitOps. Both features can now be toggled in a consistent manner.
* Enables OCI storage by default
Changes the env variable from `EXPERIMENTAL_OCI_STORAGE` to `OCI_STORAGE`
The feature is enabled by default and in order to disable it users should pass `OCI_STORAGE=false` in `extraEnv`
Refers to: https://github.com/rancher/fleet/issues/3818
---------
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
The HelmOps reconciler would previously only update a HelmOp's
`Accepted` condition in case of an error, meaning that that condition
would stay false even after an update, for instance setting the chart
version to a valid constraint resolving to one or more existing
versions.
Such updates now reset the `Accepted` condition on a HelmOps, which
reflects its actual state.
* Schedule polling jobs for HelmOps
The HelmOps controller is now able to schedule a polling job for a
HelmOp resource meeting both of the following conditions:
- a version constraint resolving to a non-static version
- a non-zero polling interval
A job will poll the Helm registry pointed to by a HelmOp resource, and
update the bundle created from that HelmOp resource accordingly.
Polling failures are tested through integration tests, while the happy
case is validated in end-to-end tests.
In the meantime, templating of test Kubernetes resources now
relies on `text/template` instead of `html/template`, to prevent version
constraints such as `< 1.0` from being escaped as `< 1.0`, which
Fleet would then interpret as invalid semver constraints.
* Generate event from Helm polling job
The Helm polling job now generates events when it successfully obtains a
new version for a Helm chart, and when it fails for various reasons as
well.
* Update polling time
When a polling job is able to successfully retrieve a new Helm chart
version and apply it to the bundle, the job updates the parent HelmOp's
status with a timestamp indicating when Helm was last polled.
The polling job also updates the latest polling timestamp in the HelmOp
status when operations which run after polling the Helm registry, such as
getting a bundle, fail. This should make troubleshooting easier.
* Skip HelmOp version update from reconciler if polling applies
Ensuring that a HelmOp's version fields are only updated from a single
place limits risks of race conditions. The polling job will be
responsible for updating those fields if polling applies to the HelmOp.
Status updates run by the HelmOps reconciler, on the other hand, are now
done using a `Patch` operation, more lightweight than a full `Update`,
and skipped if the patch is empty.
* Skip deployment of HelmOps bundle with version constraint
When a bundle is of HelmOps type and still bears a version constraint,
for instance because the initial HelmOps polling job has not yet updated
the bundle with its chart version resolved from Helm, the bundle
reconciler will not create bundle deployments. Instead, it will return
an error and feed it into the bundle status' `Ready` condition, for
better visibility in the Rancher UI.
This prevents bundle deployments from being created with a version which
does not resolve to a single Helm version, which may lead to
inconsistencies in deployments of that bundle among target clusters.
* Adds a label to internal/cloned secrets.
It also adds an event when the oci artifact is purged and could not be deleted for whatever reason.
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
* Skip setup for separate check on same test case
Creating a new HelmOp is not needed to check for finalizers, especially
as that check was clearly meant as additional validation on an existing
test case.
* Support semver constraints in HelmOp chart versions
The underlying implementation for fetching a chart version actually
already supported semantic versioning. That support simply needed to be
exposed and tested.
* Detect bundle naming collision in HelmOps reconciler
When building a HelmOps bundle, the reconciler now aborts with an error
if a non-HelmOps bundle with the same name exists in the same namespace.
* Detect bundle naming collision in `fleet apply`
When creating or updating a bundle, `fleet apply` checks the spec of any
existing bundle with the same name in the same namespace. If one bundle
is a HelmOps bundle while the other is not, `fleet apply` will abort the
update and output an error message.
* Simplify collision detection in `fleet apply`
Relying on the fact that `fleet apply` does not support creating HelmOps
bundles at this point, there is no need to generalise collision
detection to any mismatch in HelmOps state between a possibly existing
bundle and the one to be created or updated.
Instead, checking whether a HelmOps bundle already exists is enough to
establish a collision.
* Use controller-runtime client in fleet apply
This change is made to eliminate dependencies on Wrangler in the Fleet CLI and also to support future proposed changes related to reading secrets for OCI registries.
It also facilitates the creation of new integration tests, as the client interface is simplified and a single mock is used.
* Delete cluster instantiation and cache start in the cli
* Avoid adding new verbs that are not strictly required
* Separate gitOps-specific bundle deployment options
Those options are not relevant for HelmOps bundles. This commit
therefore separates them into a distinct struct to make this more
explicit, while limiting the risk of breaking the API.
* Move GitOps-specific Helm options to embedded struct
This clarifies that values files are not relevant to HelmOps bundles,
since no repository is used in such cases to store those files.
While users create and reference access secrets, of types which cannot
be enforced without adding hurdles to workload creation, Fleet now
applies specific secret types when cloning access secrets downstream,
namely:
* `fleet.cattle.io/bundle-helmops-access/v1alpha1` for Helm access when
deploying HelmOps bundles
* `fleet.cattle.io/bundle-oci-storage/v1alpha1` for OCI storage access
when storing bundle resources in an OCI registry instead of etcd.
This should help users identify secrets created in downstream clusters,
and uses of such secrets.
This renames all structs, CRDs and files related to HelmOps to feature
`helmop` instead of `helmapp`, with similar changes being made for and
differently-cased variants.
* fleet cli json output
Uses a JSONFormatter in fleet cli.
It also filters out the output received from fleet cli so the error set in the conditions is only the one sent by the fleet cli code and not from any used libraries.
Refers to: https://github.com/rancher/fleet/issues/3234
---------
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
Progress is not working. This also wraps more errors and skips .git
folders when scanning cloned git repos.
Co-authored-by: Mario Manno <mario.manno@suse.com>
* Small refactor to use typed functions
* Use a delay for enqueing GitRepo status reconciliations from bundle updates
* Allow tests to wait for delayed GitRepo/Helmapp status update
* Adds driven bundle scan
This adds a new feature to let the user define the bundles to be considered by Fleet.
It adds a new field `bundles` that is a list of path and config files.
The path is the base path for the resources contained in the Bundle and the config file is optional
and it's the Bundle's fleet.yaml file (that may have any custom name).
This is an example showing the new fields:
```yaml
kind: GitRepo
apiVersion: fleet.cattle.io/v1alpha1
metadata:
name: driven
namespace: fleet-local
spec:
repo: https://github.com/0xavi0/fleet-test-data
branch: driven-scan-example
bundles:
- path: driven/helm
- path: driven/simple
- path: driven/kustomize
options: dev.yaml
- path: driven/kustomize
options: test.yaml
```
Given the root driven folder with the contents:
```
driven
|___helm
| |__ fleet.yaml
|
|___simple
| |__ configmap.yaml
| |__ service.yaml
|
|___kustomize
|__ base
| |__ kustomization.yaml
| |__ secret.yaml
|
|__ overlays
| |__ dev
| | |__ kustomization.yaml
| | |__ secret.yaml
| |__ prod
| | |__ kustomization.yaml
| | |__ secret.yaml
| |__ test
| |__ kustomization.yaml
| |__ secret.yaml
|__ dev.yaml
|__ prod.yaml
|__ test.yaml
```
The GitRepo above would describe the following bundles:
**Helm based bundle (path: `driven/helm`, no config file)**
This one has a `fleet.yaml` file in the root folder and we're not specifying any config file.
In this case Fleet will read the `fleet.yaml` file in the root folder and will load the helm chart resources.
**Raw yaml folder based bundle (path: `driven/simple`, no config file)**
In this case the folder contains 2 yamls and no `fleet.yaml` in its root. We are not specifying any config file,
so the folder is considered as a bundle made of raw yaml files.
It will contain:
- configmap.yaml
- service.yaml
**Kustomize bundle (path: `driven/kustomize`, config file: `dev.yaml`)**
In this case we are specifying a config file (with the format of a `fleet.yaml` file).
Fleet will load all the resources found in the `drive/kustomize` folder and will load the config found
in the `dev.yaml` file.
The `dev.yaml` has the following content:
```yaml
namespace: kustomize-dev
kustomize:
dir: "overlays/dev"
```
Which means it points to the `overlays/dev` kustomization.
This Bundle will load all the resources, but will only apply the `overlays/dev` kustomization
**Kustomize bundle (path: `driven/kustomize`, config file: `test.yaml`)**
This Bundle will be equivalent to the `dev` one described above, but pointint to the `overlays/test` kustomization.
Note that the `prod` kustomization is not used in this case, which shows that not all the options defined in the kustomization must be deployed.
Possible improvements:
As we can be loading the same folder multiple times it would be better to just load the resources once and cache them for the next repeated folder.
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
Each Bundle and BundleDeployment can have a secret to store options in. The secrets have the same name and namespace as the resource. For now they only store the helm values.
This adds a new chart value to toggle strict host key checks.
When that value is set to `false`, Fleet will enforce strict host key
checks, meaning that it will fail to establish any SSH connections to
hosts for which no matching `known_hosts` entry can be found.
`known_hosts` entries are sourced in priority from secrets
referenced in `GitRepo`s, e.g. `helmSecretName` for accessing Helm
charts or `clientSecretName` for cloning git repositories.
If no such secret exists, or no `known_hosts` entries are available
in that secret, then Fleet uses its own `known-hosts` config map,
newly created at installation time with static entries for the most
widely used git providers.
Host key fingerprints added to the config map are sourced, respectively:
* from here [1] for Github
* from here [2] for Gitlab
* from here [3] for Bitbucket, which also provides a `curl` command to
fetch them in `known_hosts`-friendly format: `curl
https://bitbucket.org/site/ssh`
Additional Azure DevOps fingerprints could not be found at this point.
The absence of the config map, should no secret be available, is
considered a symptom of an incomplete Fleet deployment, and reported as
such.
[1]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints
[2]: https://docs.gitlab.com/ee/user/gitlab_com/index.html#ssh-known_hosts-entries
[3]: https://support.atlassian.com/bitbucket-cloud/docs/configure-ssh-and-two-step-verification/
Delete Bundle deployments that exist in the Bundle, that are also deployed, but that are no longer targeted.
When changing either the target customizations or targets in the `GitRepo` definition, Fleet does not delete
bundle deployments that are no longer targeted using the new target rules.
Refers to: https://github.com/rancher/fleet/issues/2995
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
* Remove unused function
This function became obsolete when using a delaying enqueue handler for
drift detection; see fleet#3401 [1].
[1]: https://github.com/rancher/fleet/pull/3401
* Modernise output setting in deployer integration tests
This makes use of `SetOut` and `SetErr` instead of `SetOutput`, which is
deprecated.
* Remove unused parameter from test helper methods
That parameter is no longer necessary now that our test infrastructure
uses `default` as its namespace.
This checks bundle deployment options for resources to ignore,
supporting a new `ignore` operation in a bundle deployment's
`Spec.Diff.Options.ComparePatches`.
Fleet will delete any such resources from the set of resources
considered missing in a bundle deployment's status.
The test already checks for meaningful status fields and `ResourceVersion` is part of the kubernetes engine.
The test is flaky because the value of the original `ResourceVersion` could be already updated when it reads it.
Signed-off-by: Xavi Garcia <xavi.garcia@suse.com>
Tim Hardeck