Commit Graph

296 Commits

Author SHA1 Message Date
Stefan Prodan d1a686235c
Add test for build panic
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-04-27 12:58:56 +03:00
Steven E. Harris e6beca11b1
Confirm format detection of file sources via tests
Signed-off-by: Steven E. Harris <seh@panix.com>
2022-04-26 11:07:17 -04:00
Steven E. Harris ee0e4c1674
Remove invalid "EnvSources" entry from test case
Signed-off-by: Steven E. Harris <seh@panix.com>
2022-04-26 11:07:17 -04:00
Steven E. Harris 7162f5e6fa
Use Secret generator keys for SOPS format hint
Rather than inspecting the source file name supplied to kustomize's
Secret generator to determine the format of the SOPS-encrypted file
content, instead inspect the Secret key (when supplied separately from
the source file name) as a more reliable heuristic.

Doing so allows kustomization authors to name their SOPS-encrypted
output files with a ".json" extension accurately reflecting the format
in which SOPS writes its encrypted output, even if the encrypted
content itself is not in JSON format.

Signed-off-by: Steven E. Harris <seh@panix.com>
2022-04-26 11:07:17 -04:00
Hidde Beydals cda51febc7
Merge pull request #631 from cwyl02/ychen/exp-backoff-retry 2022-04-22 09:45:45 +02:00
Sanskar Jaiswal d499ed8063 Ensure generated temp dir is absolute on all OSes
Signed-off-by: Sanskar Jaiswal <sanskar.jaiswal@weave.works>
2022-04-22 09:01:46 +02:00
York Chen 827f814f2f
feat: flags to configure exponential backoff retry
Signed-off-by: York Chen <ychen@d2iq.com>
2022-04-21 10:09:39 -04:00
Hidde Beydals 880587f20d generator: ensure remote build dirs can be reached
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-20 17:12:33 +02:00
Hidde Beydals f4528fb25d controllers: use own Kustomize FS implementation
For details, see: https://github.com/fluxcd/pkg/pull/262

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-15 15:48:23 +02:00
Hidde Beydals 105ebd9f47 controllers: improve decryptor and add tests
- Refactored recursion while iterating over Kustomization files.
  References of files that have been visited are cached, and not
  visited again. In addition, symlinks are confirmed to not traverse
  outside the working directory.
- Optimized various bits around (un)marshalling (encrypted) data, and
  YAML -> JSON -> YAML roundtrips are prevented where not required.
- Added support for decrypting INI Kustomize EnvSource references using
  the dedicated SOPS store for the format.
- Introduced support for decrypting Kustomize FileSources:
  https://pkg.go.dev/sigs.k8s.io/kustomize@v1.0.2/pkg/types#DataSources

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-14 09:17:43 +02:00
Hidde Beydals 62dafa1baf decryptor: switch Printf to Errorf
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-05 13:07:01 +02:00
Hidde Beydals bf4cefe011 decryptor: mention Secret name in import errors
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-05 12:44:35 +02:00
Hidde Beydals ffdda3f3da sops/azkv: add Token utility type
This adds a new Token type which can be used to configure an
azcore.TokenCredential on a master key. Due to this introduction, the
server only has to go through the AADConfig once.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-05 12:44:35 +02:00
Hidde Beydals dfbeae9487 sops/age: add ParsedIdentities utility type
This adds a new ParsedIdentities type which can be used to parse a set
of age identities just once, instead of parsing them for every Decrypt
request.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-05 12:44:35 +02:00
Hidde Beydals ed49f8427f sops/pgp: add GnuPGHome utility type
This adds a new GnuPGHome type which can be used to create a new
contained GnuPG home directory. The type is self-validating, ensuring
it adheres to e.g. permission rules set out by GnuPG, and allows for
importing keys from armored bytes and files.

Because of this introduction, the decryptor service no longer has to
write data from a Secret to a temporary file, but is instead able to
directly import them into the keyring from the Secret entry's bytes.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-04 23:57:46 +02:00
Hidde Beydals e5c2730342 sops/keyservice: allow config of server via opts
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-04 23:57:46 +02:00
Hidde Beydals 31bfd5a057 sops/keyservice: remove prompt capabilities
The original keyservice implementation was forked from SOPS, keeping
the prompt capabilities intact. However, since the implementations have
started to grow, it is better to remove this capability and resort to
a hardcoded `false` default for the fallback server. As the controller
is never expected to have an interactive TTY/shell.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-04-04 23:57:46 +02:00
Paulo Gomes d29032e914
Add flags to control kubeconfig support
Two new flags were added to allow users to enable the
use of user.Exec and InsecureTLS in the kubeconfigs
provided remote apply reconciliations.

Breaking change: both functionalities are no longer
enabled by default.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-04-01 10:13:55 +01:00
Hidde Beydals 069a1065fc sops/keyservice: properly fallback to default
This solves a regression bug introduced in `v0.22.0`, which caused the
keyservice to not properly fall back to the default for Azure Key Vault
decryption requests.

A couple of nitpicks that I ran into while inspecting the code have
been adressed as well.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-25 18:03:17 +01:00
Hidde Beydals db5777f06c Ensure annotations are prefixed with Group FQDN
This to facilitate improvements on the notification-controller side,
where annotations prefixed with the FQDN of the Group of the Involved
Object will be transformed into "fields".

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-23 22:30:31 +01:00
Hidde Beydals 855d57c491 Use new standardized meta and runtime pkgs
This includes an update of the source-controller to v0.22.0, to pull in
the v1beta2 API which makes use of the same packages.

Co-authored-by: Sunny <darkowlzz@protonmail.com>
Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-18 12:04:39 +01:00
Hidde Beydals 9f768615a9 Document SOPS Azure Key Vault in spec
This includes a refactor of the other entries, to start moving guides
to the website while containing minimal technical (instructions)
in-spec.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-14 10:29:02 +01:00
Hidde Beydals 369193ee59 sops/azkv: update to SDK including azidentity
This updates to the `github.com/Azure/azure-sdk-for-go` SDK, which is
the (apparent) successor of the previous SDK, and allows for easier
configuration of credentials through the `azidentity` package.

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-14 10:28:46 +01:00
Hidde Beydals a72e687607 sops/azkv: refactor initial Azure Keyvault impl
- Ensure key source follows upstream SOPS contracts as closely as
  possible (e.g. `MasterKey` interface).
- Prevent unnecesary FS operations by allowing token creation and
  and authorizer configuration to be factored from file bytes.
- Ensure a limited number of configuration option is taken into
  account, excluding e.g. file path references.
- Ensure server maintains backwards compatibility with previously
  supported "global" Azure configuration, _without_ relying on file
  assumptions and/or inspections (but rather, server configurations).

Signed-off-by: Hidde Beydals <hello@hidde.co>
2022-03-14 10:28:46 +01:00
David Quagebeur 343f937e59 sops/azkv: support for credentials via SecretRef
Signed-off-by: David Quagebeur <david.quagebeur@worldline.com>
2022-03-14 10:27:55 +01:00
Stefan Prodan d56695a93b
Allow shared ownership of in-cluster objects applied with kubectl
Introduce an annotation that configures kustomize-controller to co-manage objects applied with kubectl.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-03-04 12:37:30 +02:00
Stefan Prodan f2f9395c4d
Merge pull request #573 from harshi1122/Typo
Fixed typo : If applied, this commit will ensure no grammatical errors
2022-02-25 15:23:31 +02:00
Stefan Prodan 55ebe211ed
Convert `stringData` to `data` before applying Secrets
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-02-22 17:54:33 +02:00
harshi1122 f123852771 'Done'
Signed-off-by: harshi1122 <harshitha1155@gmail.com>
2022-02-18 14:55:40 +05:30
Steven E. Harris eba4168672
Tolerate absence of resources in post-build subst.
In a Kustomization's post-build substitution sources, introduce a new
"Optional" field to allow referencing a Kubernetes ConfigMap or Secret
that may not exist at time of reconciliation. Treat substitution when
the referenced object is missing as if the object had been present but
empty, lacking any variable bindings.

Retain the longstanding behavior of interpreting references to
Kubernetes objects being mandatory by default, such that
reconciliation fails if such a referenced object does not exist. Only
when the "Optional" field is set to true will reconciliation tolerate
finding the referenced object to be missing.

Signed-off-by: Steven E. Harris <seh@panix.com>
2022-02-15 09:27:41 -05:00
Steven E. Harris e665bccf89
Trap failure to create Vault instance in tests
When the Docker service isn't running, the test suite can't create a
Vault instance. Trap this failure earlier in the test program to
preclude panicking after tests relying on Vault have failed.

Signed-off-by: Steven E. Harris <seh@panix.com>
2022-02-13 16:13:55 -05:00
Steven E. Harris e437cb83c2
Use strings.ReplaceAll function when applicable
Signed-off-by: Steven E. Harris <seh@panix.com>
2022-02-13 16:13:54 -05:00
Stefan Prodan 65aaa1d69a
Ensure object are finalized under impersonation
If the service account used for impersonation has been deleted, skip pruning, log the error and continue with finalization to allow tenants removals from clusters.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-31 13:20:12 +02:00
Stefan Prodan f353ba44a7
Introduce a dedicated manager for status updates to avoid conflicts
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-31 11:29:54 +02:00
Stefan Prodan 38541078fa
Revoke kubectl managed fields ownership
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-31 11:25:11 +02:00
Stefan Prodan 4d7cba91b0
Allow setting a default service account for impersonation
Introduce the flag `--default-service-account` for allowing cluster admins to enforce impersonation for resources reconciliation.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-27 18:25:02 +02:00
Stefan Prodan 518c8a021b
Allow disabling cross-namespace references
Introduce the flag `--no-cross-namespace-refs` (defaults to false) for allowing cluster admins to disable cross-namespace references to sources.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-26 22:50:10 +02:00
Stefan Prodan 4ee01a2db0
Fix preflight validation
Validate that the resources built with kustomize conform to the Kubernetes API conventions before passing them to the server-side apply engine.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-21 13:05:40 +02:00
Soule BA c579e71430
add native support for sops decryption/encryption with Vault
If implemented, the kustomize controller will be able to retrieve a
secret containing a VAULT TOKEN and use it to decrypt the sops encrypted
master key. It will then use it to decrypt the data key and finally use the data
key to decrypt the final data.

Signed-off-by: Soule BA <bah.soule@gmail.com>
2022-01-19 21:59:10 +01:00
Paulo Gomes 50c043eb4f
Refactor fuzzing
Structure the fuzz implementation to be closer to what go native will support.
Add Makefile target to enable smoketesting fuzzers.
Add smoketest as CI workflow.

Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2022-01-14 15:38:24 +00:00
Stefan Prodan b18584a652
Merge pull request #535 from kingdonb/patch-finalizers
Use patch instead of update when adding finalizers
2022-01-14 08:24:26 +02:00
Kingdon Barrett 441b48aeef Use patch instead of update when adding finalizers
Signed-off-by: Kingdon Barrett <kingdon@weave.works>
2022-01-13 19:44:10 -05:00
Stefan Prodan 02c6708362
Update fluxcd/pkg/ssa to exclude the status field from apply
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2022-01-13 18:50:56 +02:00
Florian Fl Bauer 8435a5ba41 If applied, this commit will solve race condition when using two Kustomizations with the same SourceRef
Signed-off-by: Florian Fl Bauer <florian.fl.bauer@deutschebahn.com>
2022-01-07 14:17:40 +01:00
Stefan Prodan 00257e0cc9
Merge pull request #478 from fluxcd/go-v1.17
Update Go to v1.17 and controller-runtime to v0.11
2022-01-05 18:08:38 +02:00
Paulo Gomes facda8b422
Check EventRecorder is not nil
Signed-off-by: Paulo Gomes <paulo.gomes@weave.works>
2021-12-21 21:10:43 +00:00
Aurel Canciu ec9fdb1550
Update flux pkg components
Signed-off-by: Aurel Canciu <aurelcanciu@gmail.com>
2021-12-20 14:50:41 +01:00
Somtochi Onyekwere f6403df16f Add test for varsub in secret
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
2021-11-25 12:34:20 +01:00
Stefan Prodan 1badc828b4
Replace deprecated dependencies
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-23 14:21:21 +02:00
Stefan Prodan bedb53e0fa
Verify artifacts integrity
After downloading an artifact, compute its checksum and verify that it matches the original checksum advertised by source-controller.

Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
2021-11-12 14:03:58 +02:00