Commit Graph

710 Commits

Author SHA1 Message Date
Muvaffak Onus 4ed7edad48
Add kubeconfig key to be used in connection secrets of kubernetes clusters
Signed-off-by: Muvaffak Onus <onus.muvaffak@gmail.com>
2019-11-14 20:20:18 +03:00
Nic Cope 8686ab0b71
Merge pull request #77 from hasheddan/gomodfailci
Check for dirty go modules in CI
2019-11-12 12:43:56 -08:00
hasheddan 37470966c8 modules: check that go mod tidy does not produce diff in CI
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2019-11-11 08:10:14 -06:00
hasheddan 3389ce4034 modules: don't vendor when running make reviewable
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
2019-11-11 08:09:28 -06:00
Nic Cope 723f72b3ea
Merge pull request #75 from negz/optimistic
Don't resolve references when being deleted
2019-11-05 05:14:31 -08:00
Nic Cope 29347c9f0a Don't resolve references when being deleted
Per the comment, there's too high a chance we'll get stuck and not process the
delete because we reference resources that are also being deleted. I considered:

* Processing references at delete time but not blocking on accessor errors. I
  felt this introduced too much complexity for little gain compared to just not
  trying resolution at all.
* Making AttributeReferencers no-ops when the field they would set already had a
  value. I think this is an avenue we should investigate (issue forthcoming) but
  it is awkward to implement with the current AttributeReferencer interface.
* Enforcing ordered deletes, such that a referenced resource cannot be deleted
  (at least not by Crossplane) until its referencers have all been deleted. This
  is the most bulletproof, but also the most complicated solution and would
  require further design investigation to pursue.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-05 00:12:57 -08:00
Nic Cope 444e96ae02
Merge pull request #73 from negz/subdefault
Have claim reconciler default to using the status subresource
2019-11-03 19:06:59 -08:00
Nic Cope 8d85a75dac Have claim reconciler default to using the status subresource
All resource claims use the status subresource, but at the time the claim
reconciler was introduced most managed resources did not. This is no longer
true - all managed resources use the status subresource with the exception of a
few stragglers.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-03 18:53:16 -08:00
Nic Cope 11ec9aceb4
Merge pull request #70 from negz/finality
Add managed resource finalizer immediately before creating
2019-11-03 17:53:02 -08:00
Nic Cope 8eb89d374c
Merge pull request #72 from negz/rmcm
Don't update claim statuses after they've been deleted
2019-11-03 17:47:01 -08:00
Nic Cope 8fc49848d6 Set binding phase after update when unbinding
Previously we set the binding phase to unbound, then called Update(), which
reset the binding status to the API server value before we called
Status().Update() to persist it.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-02 02:16:32 -07:00
Nic Cope 6e49f843c2 Don't update claim statuses after they've been deleted
Unless someone else added a finalizer (and didn't yet remove it) the claim will
cease to exist as soon as the finalizer is removed, so there's nothing to update.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-02 01:42:27 -07:00
Nic Cope 105d2dd705 Fix grammar in managed reconciler commentary
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 16:49:39 -07:00
Nic Cope 3ec572329d Add claim finalizer before binding, not creation
The claim reconciler uses the finalizer to unbind the managed resource. If we
never bound to the resource there's nothing to do. Keep in mind we currently
rely on garbage collection to ensure dynamically provisioned managed resources
are deleted when the claim is deleted.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 16:49:37 -07:00
Nic Cope a3a5f918ac Add managed resource finalizer immediately before creating
This commit moves where we set the finalizer for managed resources to right
before creating them, not at the beginning of the reconcile. This means we'll be
less likely to encounter issues where we can't delete a managed resource because
we could never create it in the first place, but we added a finalizer.

* By the time we get here we know our Observe call worked. If (for example) our
  cloud provider credentials were completely wrong, we'd never proceed far
  enough to add the finalizer.
* If Observe works but Create fails (for example because we had RO cloud
  provider credentials) we would already have added the finalizer, but...
* When the managed resource was deleted we'd be able to Observe that the
  external resource does not exist (because we were never able to Create it) and
  thus would not call Delete on the external resource and go straight to
  unpublishing credentials and removing the finalizer.

This commit also renames and refactors a bunch of our interfaces to use less
obtuse names. Previously sometimes a "finalize" method unbound a managed
resource, while at other times it removed the finalizer. Similarly, finalizers
were added in "initialize". We now have a 'Binder' interface with bind and
unbind methods, and two 'Finalizer' interfaces (one for Claim, and one for
Managed) that add and remove finalizers, as you would expect.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 15:48:09 -07:00
Nic Cope e834b8ab98 GoDoc tweaks
Clarify the purpose of a few types, and make their documentation a little more
similar to the documentation of existing similar patterns.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 15:48:09 -07:00
Nic Cope 8f84243a5a
Merge pull request #68 from negz/deleteharder
Don't trigger a requeue after a successful deletion
2019-11-01 15:47:08 -07:00
Nic Cope b01c234701 Don't attempt to update resource status after finalizer removal
This status update is only useful when:

* Some other controller also added a finalizer to this managed resource.
* The other controller has not yet processed the delete and removed its finalizer.

Given that this is a rare (or non-existent) edge case, and given that attempting
to update the status of a non-existent managed resource results in spurious
errors in the logs, I think it's safe to remove this logic.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 15:14:07 -07:00
Nic Cope 78e91b4b29 Don't trigger a requeue after a successful deletion
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-11-01 15:00:10 -07:00
Nic Cope 1b6638be27
Merge pull request #71 from negz/scheduleharder
Requeue when no resource claims match labels or default annotations
2019-11-01 09:59:14 -07:00
Nic Cope a63f5b1c93 Requeue when no resource claims match labels or default annotations
Previously if no controller could schedule or default new claims they would
never try again.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 20:00:18 -07:00
Nic Cope 3ad494e53c
Merge pull request #66 from negz/nooprefs
Resolve references on every reconcile
2019-10-31 17:06:25 -07:00
Nic Cope 9b74c6d769 Make resource.CanReference a type of runtime.Object
We no longer need CanReference types to satisfy the metav1.Object interface. It
was used only to determine the namespace of the referencing object before all
such objects became cluster scoped.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 16:11:10 -07:00
Nic Cope bc04c33519 Resolve references on every reconcile
Reference resolution is now a no-op if nothing changes, so we run it on every
reconcile. We also run it after delete has been handled, so unresolved
references will only block creates and updates.

This commit means we'll make more get calls to the cache (or API) in order to
resolve our references each reconcile, and also risk potentially changing the
values of 'immutable' fields automatically if and when our references resources
change. I believe we should address this by having referencers be no-ops when
the field value they would set is already set.

I attempted to move reference resolution to right before we call create or
update (i.e. after observe and delete), but it turns out certain resources
(specifically GCP Connections) could need references to be resolved in order to
observe the external resource.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 16:11:10 -07:00
Nic Cope 5468d81383 Use real conditioned and binding statuses in mock objects
The mock binding status was identical to the real one, while the mock
conditioned status set only the most recent condition, leading to a few
slightly broken managed resource reconciler tests.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 13:27:40 -07:00
Nic Cope d5dadd4e74 Requeue after a short wait when references are not ready
the established convention is for the managed resource reconciler to requeue
after a short wait (typically 30 seconds) when it knows it is waiting for an
operation.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 13:27:30 -07:00
Nic Cope 5b01de5a4d Make reference resolution a no-op if assignments did not change anything
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 13:27:30 -07:00
Nic Cope 604cfc074e Fix grammar in reference resolution condition message
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 13:27:30 -07:00
Nic Cope 242629474a Remove attributereferencer tag check, make referencer finder pluggable
This commit refactors ResolveReferencers to allow the code that finds types
within a struct that satisfy AttributeReferencer to be swapped out. It also
updates the default AttributeReferencerFinder to avoid checking struct tags.

Previously errors were returned when:

1. A struct field tagged as a referencer did not satisfy AttributeReferencer
2. A struct field not tagged as a referencer satisfied AttributeReferencer

If either of these scenarios occurred, ResolveReferences would panic with the
returned error the first time it encountered an incorrectly written API type.
My feeling is that both of these conditions are testing for programmer errors
that would be better caught at build time than at runtime.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 13:27:30 -07:00
Nic Cope 702f6cb7c3
Merge pull request #67 from negz/rectests
Clean up managed reconciler tests
2019-10-31 13:14:02 -07:00
Nic Cope 4fbddd0d00 Clean up managed resource reconciler tests
No functional updates to the code here, just tidying up the tests a bit to
ensure they all mock out their various moving parts via options.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-31 12:35:28 -07:00
Nic Cope 6db5ced56a Trivial comment updates
Just wrapping a comment and updating the linter override to reflect that we've
moved beyond "slightly" over our cyclomatic complexity goal. :(

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-28 16:01:38 -07:00
Jared Watts 6916b9475f
Merge pull request #60 from jbw976/ci-builds
build: Jenkinsfile updates for this package only crossplane-runtime repo
2019-10-26 09:53:13 -07:00
Jared Watts 2d3075505d
build: Makefile and Jenkinsfile updates for this package only crossplane-runtime repo
Signed-off-by: Jared Watts <jbw976@gmail.com>
2019-10-26 09:07:27 -07:00
Nic Cope 78072ef19d
Merge pull request #55 from negz/nonsense
Fix static provisioning edge cases
2019-10-24 21:30:10 -07:00
Nic Cope 551efff860 Add tests for external name annotation back-propagation
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 21:07:40 -07:00
Nic Cope 799c22bda6 Set managed resource claim ref at binding time
We already set this reference at dynamic provisioning time, but we need it set
for (bound) statically provisioned managed resources too, so we set it
(potentially again) at binding time.

We must still set the reference at dynamic provisioning time in order to ensure
reconciles are queued for dynamically provisioned managed resources when they
become available.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 21:07:40 -07:00
Nic Cope 94a6b45721 Don't assume the class reference is set by dynamic provisioning time
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 02:30:11 -07:00
Nic Cope 6a97932337 Demonstrate panic when a resource references a claim without references
Despite past negz's optimistic comment to the contrary, we can't assume the
class reference is set by the time we get to the dynamic provisioning stage.
We are queued for managed resources that reference a claim, so we could reach
the dynamic provisioning stage without a class reference being set if a managed
resource referenced a claim that had no class reference or resource reference.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 02:29:46 -07:00
Nic Cope bd7fed5ad4 Requeue when a referenced managed resource does not yet exist
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 02:29:45 -07:00
Nic Cope 2baf308c91 Demonstrate panic when a claim references a non-existent resource
If a resource claim specifies a managed resource reference to a non-existent
managed resource we'll panic because we proceed and try to operate on our
managed resource's nil class reference.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-24 02:29:38 -07:00
Nic Cope 0f37bea549
Merge pull request #48 from negz/notsoclassy
Introduce label selector based class scheduling
2019-10-23 14:56:52 -07:00
Nic Cope a81fb19350 Increase claim scheduling and defaulting max jitter to 1.5 seconds
200ms seems low enough that GCP consistently beats Azure when scheduling
RedisCluster claims in my experiments.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope eb0bb8b34f Work around strange deepcopy bug
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope 701387a302 Add a HasResourceClass predicate
For resource claim controllers, now that we're unconcerned with indirect
resource classes.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope d7b4db0562 Add claim scheduling and defaulting controllers
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope a2e24876e0 Use custom types for secret references
This allows us to require that name (and namespace where appropriate) are set at
the CRD level. In the case of cluster scoped resources that reference secrets
this is less surprising than defaulting to the `default` namespace when the
namespace is omitted.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope 9f5188e0cb Assume resources, classes, and providers are cluster scoped
Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:13:21 -07:00
Nic Cope a310ee6a4a Remove support for portable resource classes
This commit renames "non portable resource class" back to "resource class", and
requires that resource claims reference a (non portable) resource class in any
namespace.

Signed-off-by: Nic Cope <negz@rk0n.org>
2019-10-22 17:12:36 -07:00
Nic Cope 1647c08184
Merge pull request #54 from muvaf/port-key
Add port key to connection secret key constants
2019-10-22 12:31:31 -07:00