Commit Graph

89 Commits

Author SHA1 Message Date
Maciej Borsz 02f98184b3 Add APF's priorityLevel to httplog.go
Kubernetes-commit: 6db63869b0fd0551382dec1b30b649fda4289444
2021-08-13 13:00:07 +02:00
Mike Spreitzer 7d5430cfba Fix extra latency and add tests for that and width
Added missing dispatching after delayed release of seats.

Updated logging for all six situations of execution completion and
seat release.

Added behavioral tests for non-zero extra latency and non-unit width.

Also added two tests for baseline functionality.

Also improved some comments and other logging in `queueset.go`.

Kubernetes-commit: d2a27a58f0af20c6185fa1c21890d666e9d3746b
2021-08-12 16:48:02 -04:00
Abu Kashem da50ca4c6e apf: free seats in use after additional latency
Kubernetes-commit: d68186452d9150b113489e6a722caf82f898857f
2021-06-27 13:04:20 -04:00
Mike Spreitzer 8c2108bc80 Refactor goroutine counting
Add comment outlining TestContextCancel.

Stop calling `t.Errorf` from wrong goroutine.

Package up queueNoteFn expectation checking.

Add counting of goroutine in req1 exec fn.

Remove unnecessary assignment to `_`.

Make TestContextCancel wait on fake clock, to insulate timing check
from scheduler noise.

Factor goroutine counting out of queueset.go, into queueset_test.go,
where it matters.

Refactor promise: Use a simple channel-based implementation for normal
code, a mutex-based one for testing code.

Took all the panics out of queueset.go

Shrink the timeouts in promise tests to 1 second.

Kubernetes-commit: 1db36ae3b30e30d70972998a22987a7db470479b
2021-07-29 00:35:25 -04:00
Mike Spreitzer 904cd74454 Some cleanup of the package for event clocks
Rename from `clock` to `eventclock`.

Simplify by removing the prohibition on an EventFunc suspending and
resuming activity.

Remove "EventClock" from names to avoid stuttering.

Start to consolidate test code under fairqueuing/testing/.

Kubernetes-commit: 80ca6a4ae6ff571c32962a7155efd55edefff9e6
2021-08-06 02:06:43 -04:00
Abu Kashem f013b63777 apf: use EventClock rather than a PassiveClock for queueset
Kubernetes-commit: c2a3b793d3ec62e781dd20704370d09f7e1be706
2021-07-21 17:06:48 -04:00
Abu Kashem bd2ef9810a apf: estimate list width
Kubernetes-commit: 296c18ec323328bf1c6a621a9ca3a094d78bb9d0
2021-06-22 19:38:00 -04:00
Manu Gupta b09300b3f0 fix unsafe json construction for digestConfigObjects.
fix unsafe json creation by creating intermediate objects
while creating patch bytes.

Kubernetes-commit: dfde50b18564cc9a8e99fdcba2f136d770ec6943
2021-08-01 18:43:40 -07:00
Mike Spreitzer 0c550377cf Introduce event clocks based on k8s.io/utils/clock
So we can move off of the apimachinery clock package.

Switch queueset to new clocks.

Removed event clocks based on apimachinery clocks,
because this PR introduces ones based on k8s.io/utils/clock .

Removed interface that is implemented by only one interesting type.

Simplify RealEventClock::EventAfterTime.

Kubernetes-commit: dcb298c9552de44e27ed52f5e2b58a0dd7cd8d54
2021-07-21 16:56:11 -04:00
Abu Kashem feb4eefe1c apiserver: add callback to get notified of object count
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
wojtekt 719fda2a8b Simplify APF promise to what is really used in the code
Kubernetes-commit: 9f735e71bbb7d0dde67a718891641d8afd20a8bc
2021-07-23 13:30:34 +02:00
wojtekt d6e0cf5d71 Remove unused promise code from APF
Kubernetes-commit: a1cf44eab44f21c3929ff2d79501c56e6fbcddc3
2021-07-21 12:14:30 +02:00
wojtekt 9209ed5928 Get rid of unused flowSchemaRVs in P&F
Kubernetes-commit: c79a0a08828ef86e07941e96dafefb7d8bc51990
2021-07-13 15:16:17 +02:00
wojtekt b4c306e1e8 Rename width to workEstimate in P&F code
Kubernetes-commit: 73211256e8f15cf84ee69d6fe8258c3a912e0f94
2021-07-13 15:10:58 +02:00
wojtekt 242c8c8a1c Optimize APF support for watch initialization to fix the pod startup time regression.
Kubernetes-commit: ef435b85b47618d0d463ac93e9758d75c2998681
2021-07-13 10:22:30 +02:00
Abu Kashem b206f9ea02 apf: fix virtual clock
Kubernetes-commit: a570008cbd6c2142a2f270570fe49bf982825c19
2021-07-06 19:29:16 -04:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
wojtekt 9606daa5ef Refinements to pick queue logic in P&F
Kubernetes-commit: 7f1c4977d7ad9fdfd44549ee8ffe289127be4d81
2021-07-07 08:58:49 +02:00
wojtekt fbc127e994 Add watch tracker to APF for request cost estimation
Kubernetes-commit: cea1dcfeed2fc4e8ab89cd43e5a0e402251c8df5
2021-06-15 10:49:42 +02:00
wojtekt a35bb85f09 Update the logic to pick the best queue in P&F
Kubernetes-commit: 0ecc7ba311ab33b16c5d907ebb1120e3e51a947d
2021-06-17 09:22:00 +02:00
Abu Kashem cf5c77fde9 apf: add additional latency into width
Kubernetes-commit: 24e19229101d242d924ce98a562be3864dde9eae
2021-06-27 12:45:24 -04:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Abu Kashem e9708ba84d apf: calculation of dR/dt should use seats in use
Kubernetes-commit: 6203d4fc6af36b1fd60177e8a88b8d4d45acb029
2021-06-21 05:38:07 -04:00
Mike Spreitzer 4dcb848926 Track generalization for width
Kubernetes-commit: 3930629e130f1c98821f8033941870ab582db284
2021-06-18 01:37:22 -04:00
Mike Spreitzer d28ccb4224 Add APF metrics about R(t)
Kubernetes-commit: 676f0450ed37eeec92b67246719cc46e7567e512
2021-06-14 16:48:27 -04:00
youhonglian 56e6dac0bb update github.com/pkg/errors to go native errors pkg in staging
Kubernetes-commit: 4b1402187474a1ebd0f71c824f45448fea160fc6
2021-06-22 11:54:56 +08:00
Abu Kashem e1aec4ecae apf: take seats into account when dispatching request
Kubernetes-commit: ff716cef508f948b50e1026e980e6df5ee475538
2021-06-14 12:19:06 -04:00
Abu Kashem abdd901971 apf: always include seats in virtual time
Kubernetes-commit: 2f7456076e0cb29a95d86cd0f54c34a04b4722ab
2021-06-14 09:52:00 -04:00
Abu Kashem 345d1c6ff9 apf: add a gauge for the number of seats currently in use
Kubernetes-commit: c710f99ef730a791a6911e63cc3b9d26cced6bd3
2021-06-10 17:34:50 -04:00
Abu Kashem 3c7f54740f apf: add plumbing to estimate width" of a request
- add plumbing that allows us to estimated "width" of a request
- the default implementation returns 1 as the "width" of all
  incoming requests, this is in keeping with the current behavior.

Kubernetes-commit: 9b72eb1929a64b9d5a5234090a631ba312fb4d41
2021-05-11 07:03:05 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Abu Kashem ec22c8bdd8 apf: add "width" for request
all requests have a width of 1 to maintain current behavior.

Kubernetes-commit: b50507d98bd12503592ea62d2be2aadef49bdf70
2021-05-11 07:03:05 -04:00
Abu Kashem 4e12e0cd34 apf: use a list instead of slice for queueset
Kubernetes-commit: 69f9bc181f155ded7c5d5cc0ca9f026a6b42f431
2021-03-29 11:31:12 -04:00
Abu Kashem 7c07064678 apf: fix test flake
Kubernetes-commit: fa0952ee778aaf7df3e593305575f5d121b55cf1
2021-03-29 16:09:49 -04:00
David Eads 4c81a6672b Enable, rate limit, and test APF controller fights
Using real time.

Kubernetes-commit: 80ff06fe846c5e565e0cbd4b70f5f1e8a4ea7295
2020-12-02 16:50:05 -05:00
yoyinzyc 1a8abfc56f add context to metrics in util/flowcontrol.
Kubernetes-commit: 57d0bc301a017c41d890baee0a3a287f448c664d
2020-12-16 17:08:43 -08:00
Adhityaa Chandrasekar ec9a97b26e flowcontrol: make TestConfigConsumer less time-consuming
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: c1f3e90969d38f5b8410b67f0640d5df0d098e6b
2021-02-02 11:20:18 +00:00
Mike Spreitzer b2c3c4f86c Tweak up TestSampler in response to review
Kubernetes-commit: 736a87a4ee516a3feca0a79a4c48806ad4916aa3
2021-01-24 23:38:48 -05:00
Mike Spreitzer 39f5df86ff Added comment explaining the new unit test
Kubernetes-commit: 42f8d4e28fc05f3532f5cf7d2294049e3314896a
2021-01-18 00:57:09 -05:00
Mike Spreitzer 60021e2fed Explain warnings in new test
Add info message that explains what warnings are deliberate and expected.

Kubernetes-commit: 194c22fed54ba89d985085412795a9850bedeaf3
2021-01-06 12:56:16 -05:00
Mike Spreitzer 71c9d22768 Add unit test for sample-and-watermark histograms
Kubernetes-commit: 5fe48be506fe2b946d4db9cd7976ad0c4d4e7e1a
2021-01-05 23:31:06 -05:00
pacoxu 2343af38bb fix all keps links 404 for kep folder migration
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: a10bdfed09e96f3c2703454caafa2c884914e064
2021-01-14 14:03:21 +08:00
Abu Kashem 84ce907827 fix apf controller unit test
- don't expose the internal states of the apf controller to the caller
- return a boolean, instead of the priority level states

Kubernetes-commit: f20c6cb2d9060920cae9ff5cade1739c7e0b7f7a
2021-01-13 11:06:17 -05:00
Mike Spreitzer 16288a4d61 move all variables in sampleAndWaterMarkHistograms::innerSet
to tiptoe around https://github.com/golang/go/issues/43570 for #97685

Kubernetes-commit: 611184aa59d0cd40466bc3bc4b40a3712a038171
2021-01-08 13:32:38 -05:00
Mike Spreitzer 41788f53e2 Relax apf_controller's reaction to certain errors
When the error is due to the object having been deleted, the
controller does not need to do anything before the coming
notification.

Kubernetes-commit: ef1e2039b5fc7f955ec4f9c636a64aa403cba2ab
2020-12-17 10:46:14 -05:00
Abu Kashem eea0d66fcd clean up executing request on panic
Kubernetes-commit: 13cedca0eb5337b13e5176983ea5e784ec38df22
2020-12-10 12:57:21 -05:00
Abu Kashem 59a718ed18 fix the deadlock in priority and fairness config controller
Kubernetes-commit: 7114319b3e66e331250c116ac926f1a57352072f
2020-12-15 12:19:56 -05:00
Mike Spreitzer a011399e98 Define TestableConfig in k/apiserver/pkg/util/flowcontrol
Collect the parameters of newTestableController into a named type.

Also tolerate the surprising situation in which a request's user
groups include neither `system:authenticated` nor
`system:unauthenticated` --- because this is observed to happen in
some tests.

Also a few other minor fixups.

Kubernetes-commit: 10df6d459b5bca7b42471f9409182417fbc3f253
2020-12-16 18:58:45 -05:00
Mike Spreitzer 2ceb0fcbe8 Create OWNERS for most of the API Priority and Fairness impl
Kubernetes-commit: 23de8061fbb63928c62aa6020b443a62491a0a1d
2020-12-11 17:54:59 -05:00
corehello b6f30b469b Fix typo for ServiceAccount
Kubernetes-commit: c20b2c8cd79f9e7dafa86fb4245dee9dd2d795c7
2020-12-02 01:38:46 +08:00