Commit Graph

132 Commits

Author SHA1 Message Date
Dave Protasowski accfe36491
Satisfy linter (#3132)
* drop deprecated linter

* fix uint64=>int64 overflow

* fix unnecessary fmt.Sprintf

* ignore false positive fatcontext

* fix integer overflows

* update golangci-lint config - properly was moved

* fix formatting calls

* use new integer range syntax

* adjust nolint comments

* stop using deprecated k8s types
2025-01-10 15:06:18 +00:00
Dave Protasowski ee1db869c7
Update linter config and address lint warnings/failures (#3068)
* fix whitespace linter

* fix usestdlibvars

* fix staticheck

* ignore spancheck warning

* fix linter perfsprint

* fix nolintlint feedback

* fix nilerr lint checks

* fix misspell

* fix mirror lint

* fix intrange linter

* fix gofumpt linter

* fix gosec linter - ignore warning since default min tls version is 1.3

* fix gocritic linter

* fix whitespace

* fix fatcontext linter

* fix errorlint

* fix errname linter

* fix copyloopvar (go1.22) linter

* fix bodyclose linter

* update linter config

* add script to format code

* fix unit test
2024-06-25 14:49:36 +00:00
Dave Protasowski 8535fcc248
gofumpt the repo (#3067)
* gofumpt the repo

* don't prefix numbers with 0 - otherwise they're octal
2024-06-25 07:27:07 +00:00
Prashant Rewar fef5da99bd
Replace all usages of deprecated wait.PollImmediate with wait.PollUntilContextTimeout (#3004)
Signed-off-by: Prashant Rewar <108176843+prashantrewar@users.noreply.github.com>
2024-04-02 14:44:03 +00:00
Reto Lehmann 1860700f6f
update k8s min version to 1.26.x and go deps to 1.27.x (#2832)
* update k8s min version to 1.26.0

* bump k8s go deps to latest v0.26 version

* Fix version_test

* bump k8s go deps to latest v0.27 version
2023-10-02 12:57:59 +00:00
Kenjiro Nakayama c11003ae6d
Use go sync/atomic instead of go.uber.org/atomic (#2777)
* wip

* tiny fix

* Fix controller/controller_test.go

* fix metrics.go

* Fix profiling/server.go

* Fix reconciler/testing

* update ./test/spoof/spoof_test.go

* Fix ./test/zipkin/util.go

* update go.uber.org/atomic
2023-08-02 16:26:48 +00:00
Pierangelo Di Pilato 2fdd6bf8be
Allow reconcilers to listen to leader promotion events (#2688)
* Generator: allow reconcilers to listen to leader promotion events

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

* Run hack/update-codegen.sh

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

---------

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
2023-02-22 15:02:28 +00:00
Dave Protasowski 8532ae0645
update OWNERS files to use teams with active members (#2672)
* update OWNERS files to use teams with active members

* add eventing-wg leads as approvers to /apis
2023-02-06 16:58:45 +00:00
Dave Protasowski 2daa86aa8f
Electors can provide an initial set of buckets (#2473)
The controller package now promotes those buckets
prior to starting reconciliation and leader election

The unopposed elector now seeds the univeral bucket

This helps avoid races in unit tests were the univeral
bucket isn't promoted in time when reconciling an resource
2022-03-25 11:21:49 -07:00
Dave Protasowski 80c511aa34
Wait for reconciler/controllers to return prior to exiting the process (#2437) 2022-02-22 13:12:04 -08:00
Julian Friedman d569db39a8
Add context to reconcile error logs (#2281) 2021-09-09 03:21:58 -07:00
Matt Moore d60f1a4998
Avoid double-resyncs without leader election. (#2252)
* Avoid double-resyncs without leader election.

tl;dr Without leader election enabled, we've been suffering from double resyncs, this fixes that.

Essentially when the informers are started, prior to starting the controllers, they fill the controller's workqueue with ~the world.  Once we start the controllers, the leader election code calls `Promote` on a `Bucket` so that it starts processing them as their leader, and then is requeues every key that falls into that `Bucket`.

When leader election is disabled, we use the `UniversalBucket`, which always owns everything.  This means that the first pass through the workqueue was already hitting every key, so re-enqueuing every key results in superfluous processing of those keys.

This change makes the `LeaderAwareFuncs` elide the call to `PromoteFunc` when the `enq` function passed is `nil`, and makes the `unopposedElector` have a `nil` `enq` field when we explicitly pass it the `UniversalBucket` because leader election is disabled.

* Add more unit test coverage
2021-09-02 09:46:06 -07:00
Matt Moore e957ee54fa
Drop several deprecated APIs. (#2228) 2021-08-30 13:25:54 -07:00
Matt Moore 6ef763ddee
Remove long deprecated NewImplWithStats (#2223) 2021-08-21 14:43:34 -07:00
Matt Moore 9c7fd8e14f
Introduce `NewContext`, deprecate `NewImplFull`. (#2222)
* Introduce `NewContext`, deprecate `NewImplFull`.

Our generated `NewImpl` methods have long taken `context.Context`, but despite many iterations the forms we expose from our `controller` package never have.  This change contains several elements:
1. Expose a new `NewContext` method that takes `context.Context` in addition to the current `NewImplFull` signature.
2. Call `NewContext` instead of the deprecated `NewImpl` from our generated controller code.
3. Call `NewContext` from all our webhook reconcilers.

* Add a Tracker to controller.Impl to cut down on downstream boilerplate.
2021-08-21 14:00:34 -07:00
Julian Friedman 50410e0b83
Use consistent case for "Deprecated" comments (#2216)
* Use consistent case for "Deprecated" comments

Not the most important thing ever, but the canonical string to use for
Deprecated warnings is case sensitive, and also it's nice to be
consistent.

* Add nolint comment
2021-08-17 09:15:16 -07:00
Matt Moore 3826bb2436
Add a new mechanism for requeuing a key. (#2201)
This is modelled after some of the semantics available in controller-runtime's `Result` type, which allows reconcilers to indicate a desire to reprocess a key either immediately or after a delay.

I worked around our lack of this when I reworked Tekton's timeout handling logic by exposing a "snooze" function to the Reconciler that wrapped `EnqueueAfter`, but this feels like a cleaner API for folks to use in general, and is consistent with our `NewPermanentError` and `NewSkipKey` functions in terms of influencing the queuing behaviors in our reconcilers via wrapped errors.

You can see some discussion of this here: https://knative.slack.com/archives/CA4DNJ9A4/p1627524921161100
2021-07-30 08:31:33 -07:00
Ben Moss 7d1b0f19ef
Add ability to filter objects on injection controller promotion (#2180)
Currently we enqueue every object with no way to filter, which causes
problems for eventing's source controller which reconciles duck CRDs.
2021-07-13 10:55:50 -07:00
zhaojizhuang 4cdacd0473
add concurrency for each controller (#2160) 2021-06-25 12:41:44 -07:00
Markus Thömmes 980a33719a
Fix revive related linting issues (#2131) 2021-05-26 01:10:28 -07:00
Evan Anderson 728bc4ad4e
Update OWNERS_ALIASES to match autogen in community (#2078) 2021-04-08 07:42:51 -07:00
Markus Thömmes 04fdbd775b
Add WaitForCacheSyncQuick and use it in tests (#2045) 2021-03-05 07:07:20 -08:00
Victor Agababov 8fbab7ebb7
Redo the comment a bit further (#2042)
I think `thus` is quite wrong here (I commented on the pr), but
also took it a bit further.
2021-03-03 11:22:15 -08:00
Markus Thömmes 08fc6268bf
Fix comment for skipKeyError (#2041)
* Fix comment for skipKeyError

* Review
2021-03-03 03:19:15 -08:00
Lionel Villard 07b5ddfaea
add demoteFunc controller option (#2033)
* add demoteFunc controller option

* use tab instead of space for indent

* run codegen
2021-02-25 14:10:47 -08:00
Matt Moore 8a9bf766d3
Add symmetric filter helped based on OwnerRefable. (#2032)
Most resources stamped out by knative controllers have OwnerReferences synthesized via `kmeta.NewOwnerRef`, which requires the parent resource to implement `kmeta.OwnerRefable` for accessing the `GroupVersionKind`.

However, where we setup informer watches on those same children resources we have essentially relied on direct synthesis of the `Group[Version]Kind` where we could instead provide an empty instance of the controller resource and leverage `GetGroupVersionKind` to provide the GVK used for filtration.

So where before folks would write:

```golang
FilterFunc: controller.FilterControllerGK(v1alpha1.WithKind("MyType"))
```

They may now write:

```golang
FilterFunc: controller.FilterController(&v1alpha1.MyType{})
```

The latter is preferable in part because it is more strongly typed, but also shorter.
2021-02-19 08:47:03 -08:00
Stavros Kontopoulos 84c98f3c3e
remove reflector metrics (#2020) 2021-02-15 08:55:23 -08:00
Stavros Kontopoulos 448ae657fb
Metric unit fixes (#2018)
* metric unit fixes

* additional fixes
2021-02-12 12:38:35 -08:00
Markus Thömmes a02dcff9ee
Bump a few assorted dependencies to their latest versions (#2013)
* Bump a few assorted dependencies to their latest versions

* Use new uuid helper

* Some more slight adjustments
2021-02-08 09:52:52 -08:00
Dave Protasowski 5bb97df49b
fix duration logging (#1992) 2021-01-15 12:20:20 -08:00
Dave Protasowski a74906c7fb
Use structured logging to augment our logger vs. naming (#1991)
* Use structured logging to augment our logger vs. naming

* remove unused line
2021-01-15 12:08:21 -08:00
Matt Moore f0ea5e6b9c
Use special error type to designate skips. (#1988)
This change introduces a new `controller.NewSkipKey` method to designate certain reconciliations as "skipped".

The primary motivation for this is to squelch useless logging on non-leader replicas, which currently report success with trivial latency.

I have plumbed this through existing reconcilers and the code-gen so most things downstream should get this for free.  In places where a key is observed, I do not mark the reconcile as skipped as the reconciler did some processing for which the awareness of side-effects and reported latency may be interesting.
2021-01-14 14:30:20 -08:00
Markus Thömmes 261c9b4624
Remove unnecessary intermediate loggers (#1969)
* Remove unnecessary intermediate loggers

* Make linter happy

* Collapse logging variable into if contexts
2020-12-16 08:27:58 -08:00
rusde 3154117dcf
Add tags to knative provided reconcile metrics (#1916)
* Add  tags to knative provided reconcile metrics

* Remove resource tag to reduce metric cardinality

* Removing unknown tags

* Update controller/testing/fake_stats_reporter.go

Co-authored-by: Victor Agababov <vagababov@gmail.com>

Co-authored-by: Victor Agababov <vagababov@gmail.com>
2020-11-24 12:42:35 -08:00
Markus Thömmes d9c4e5c439
Fix a few more occurrences of divisive language (#1902) 2020-11-12 06:41:59 -08:00
Markus Thömmes 565516e224
Add errorlint and fix all existing issues (#1855) 2020-10-29 01:14:35 -07:00
Markus Thömmes 385c8b9c0e
Fix nolint warnings and adhere to best practices (#1823) 2020-10-20 09:33:59 -07:00
Markus Thömmes 3d42810561
Make sure all controllers finish before ending test (#1818) 2020-10-19 10:02:58 -07:00
Josh Soref b39d5da935
Spelling (#1797)
* spelling: adopted

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: aliased

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: apierrs

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: assignment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: available

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: coexistence

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: commit

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversions

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: creates

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: custom

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: determine

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: different

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: distribution

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: duplicate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: editing

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: endpoint

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: generate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: implementation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: identified

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: ignore

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: indicates

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: interface

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: interleaved

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: labels

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: label

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: mimic

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: namespaced

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: necessary

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: organization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: populatable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: prometheus

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: refer

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reference

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: response

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: something

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: specable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: spoofing

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: synchronized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: this

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: trailing

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsupported

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validation

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* chore: reviewdog go header boilerplate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-10-18 14:22:57 -07:00
Victor Agababov 7bad843466
Enable golint and exclude some other generated or additional dirs (#1783)
* Enable golint and exclude some other generated or additional dirs

Also remove `test` ignore, since it's covered by path ignore rule.

* meh

* fixes

* more

* progressing

* further

* like a boss
2020-10-07 14:58:20 -07:00
Victor Agababov a371418524
v2 (#1754) 2020-09-29 13:18:29 -07:00
Markus Thömmes 5fbbde31b3
Align linters with serving (enables stylecheck and asciicheck) (#1738) 2020-09-23 07:37:40 -07:00
Julian Friedman 6e0430fd94
Fix flakes in EnqueueAfter tests (#1710)
* Fix flakes in EnqueueAfter tests

* only call q.Len() once
2020-09-16 10:15:41 -07:00
Zbynek Roubalik 2d4efecc6b
bump to k8s 1.18 (#1428)
* bump to k8s 1.18.8

Signed-off-by: Zbynek Roubalik <zroubali@redhat.com>

* plumbing ctx through

Signed-off-by: Zbynek Roubalik <zroubali@redhat.com>

* add more ctx plumbing

Signed-off-by: Zbynek Roubalik <zroubali@redhat.com>

* ctx WithCancel()

Signed-off-by: Zbynek Roubalik <zroubali@redhat.com>
2020-09-11 07:54:00 -07:00
Victor Agababov 9c75061487
Make controller tests reentrant (#1671)
The stats tests were completely wonky checking some random things
and despite attempt to make them reentrant they weren't so.

So fix that \o/!
             |
             /
2020-08-31 14:06:15 -07:00
Victor Agababov 55cef32af0
Remove code duplication by reusing funcs (#1669)
and consts
2020-08-31 12:11:15 -07:00
Victor Agababov dba58d1d78
Modernize the controller tests (#1661)
- remove mutexes where we can get by with an atomic
- make things private that are not public
- etc
2020-08-28 21:54:07 -07:00
Victor Agababov 34838c4559
Remove all ClearAll calls from pkg UTs (#1654)
We need this in order to deprecate the function.
Serving is already free of those.
2020-08-27 13:13:07 -07:00
Antoine Cotten c56f5e203b
Reduce delays in controller tests (#1603)
* TestEnqueue: remove unnecessary calls to Sleep

The rate limiter applies only when multiple items are put onto the
workqueue, which is not the case in those tests.

Execution: ~7.6s -> ~2.1s

* TestEnqueueAfter: remove assumptions on execution times

Instead of sleeping for a conservative amount of time, keep watching the
state of the workqueue in a goroutine, and notify the test logic as soon
as the item is observed.

Execution: ~1s -> ~0.05s

* TestEnqueueKeyAfter: remove assumptions on execution times

Instead of sleeping for a conservative amount of time, keep watching the
state of the workqueue in a goroutine, and notify the test logic as soon
as the item is observed.

Execution: ~1s -> ~0.05s

* TestStartAndShutdownWithErroringWork: remove sleep

Instead of sleeping for a conservative amount of time, keep watching the
state number of requeues in a goroutine, and notify the test logic as
soon as the expected threshold is reached.

Logs, for an idea of timings
----------------------------
Started workers
Processing from queue bar (depth: 0)
Reconcile error {"error": "I always error"}
Requeuing key bar due to non-permanent error (depth: 0)
Reconcile failed. Time taken: 104µs     {"knative.dev/key": "bar"}
Processing from queue bar (depth: 0)
Reconcile error {"error": "I always error"}
Requeuing key bar due to non-permanent error (depth: 0)
Reconcile failed. Time taken: 48.2µs    {"knative.dev/key": "bar"}

Execution: ~1s -> ~0.01s

* TestStart*/TestRun*: reduce sleep time

There is no need to sleep for that long. If an error was returned, it
would activate the second select case immediately.

Execution: ~1s -> ~0.05s

* TestImplGlobalResync: reduce sleep time

We know the fast lane is empty in this test, so we can safely assume
immediate enqueuing of all items on the slow lane.

Logs, for an idea of timings
----------------------------
Started workers
Processing from queue foo/bar (depth: 0)
Reconcile succeeded. Time taken: 11.5µs {"knative.dev/key": "foo/bar"}
Processing from queue bar/foo (depth: 1)
Processing from queue fizz/buzz (depth: 0)
Reconcile succeeded. Time taken: 9.7µs  {"knative.dev/key": "fizz/buzz"}
Reconcile succeeded. Time taken: 115µs  {"knative.dev/key": "bar/foo"}
Shutting down workers

Execution: ~4s -> ~0.05s

* review: Replace for/select with PollUntil

* review: Remove redundant duration multiplier

* review: Replace defer with t.Cleanup
2020-08-26 10:09:06 -07:00
Victor Agababov d5c09d2aef
Optimize and clean the controller code (#1639)
- fix outdated comments
- reorder the code to reduce the  number of defer calls
- various other nits
"
2020-08-21 22:20:46 -07:00