6adfddf535
Clarify APF metric wrt all three stages of execution
...
Kubernetes-commit: 88f8e8448bf873cf41035cb858422a10a1d03018
2021-11-30 11:45:53 -05:00
6bd59a523a
apf: add a metric to count seat samples
...
Kubernetes-commit: bb15bdf15c1cc4d5a4380f3f6ed46d4adc9662a1
2021-11-23 11:36:09 -05:00
1d83e4074a
apf: ensure exempt request notes the classification
...
Kubernetes-commit: 8b2dd74c277d6a56a14e99830d39b23c5788c62e
2021-12-05 11:29:15 -05:00
56a3a30ae1
Check in OWNERS modified by update-yamlfmt.sh
...
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
40993f6319
apf: add new label for request_execution_seconds metric
...
Kubernetes-commit: 54439e934371a3018f49e629cdc68f0944e08af0
2021-10-06 11:55:12 -04:00
d9e2f76835
Correct Generator calls for executing seat count
...
Kubernetes-commit: 95964c5b351657820c4a8db0da0b772cb222fe99
2021-11-29 14:50:11 -05:00
6dbeb046f3
prune enum when OpenAPIEnums is disabled.
...
Kubernetes-commit: 9c05de2f31582b47da05d9dd3f9bbf89588bea82
2021-11-09 13:00:00 -08:00
8c46c2c1d9
Fix sample_and_watermark_test.go for bad luck, repeated test
...
Kubernetes-commit: 06e17165b16a4706ecf86b089de2bc1c0faea264
2021-11-10 15:30:50 -05:00
4098be7694
Factored TimedObserver into less surprising pieces
...
Kubernetes-commit: ab64e852023965fd8873abcd50ff09cf79814d11
2021-11-15 14:59:30 -05:00
56b220f8cd
Add metrics about watch counts seen by APF
...
Kubernetes-commit: 154bf6aab33c2486a9066f66ab3a056c1095cb9a
2021-10-25 03:31:47 -04:00
6a2631848c
Add sample-and-watermark for seats occupied during all of execution
...
Kubernetes-commit: 945f960cfb8fc018b093c1a08e5d4cdd362b1fc6
2021-10-25 01:13:52 -04:00
64f47d35c4
P&F fix watch tracker bug
...
Kubernetes-commit: b1d0fde370ae409ff028c557c6a60ff69491e499
2021-11-03 15:02:51 +01:00
44154d94e6
Enable support for indexes in WatchTracker
...
Kubernetes-commit: 21ec77dc19047215094a538b717e24035ca000ab
2021-10-28 14:57:47 +02:00
9ad8b586fc
P&F: Update WatchTracker interface to pass more information
...
Kubernetes-commit: 12746f4bc15458d585ffd4c6e9d6066810e27361
2021-10-28 12:41:41 +02:00
bdc01eb8e6
P&F: update mutating request estimation
...
Kubernetes-commit: 4700cf622bfcf5c2b159ddc4e37d7f51d1c41e68
2021-10-27 10:25:38 +02:00
55b43e446f
P&F: move seat-seconds to a better location
...
Kubernetes-commit: e262db7a4daf5218520e49b423789ea55a94af75
2021-10-27 10:30:25 +02:00
9c98ce48a5
P&F: clean up mutating work estimator tests
...
Kubernetes-commit: 943bc38c0e0076935701d32067d5a2d8d512be8a
2021-10-27 10:05:13 +02:00
3f529d0551
apf: call metrics.AddReject for decisionCancel
...
Kubernetes-commit: f6dcf17a10dfd2cb5ce0ea7923723547c92c2e55
2021-10-18 13:08:56 -04:00
3af709f483
aggregator: pass apiServiceRegistrationControllerInitiated signal directly to apiserviceRegistration controller
...
Kubernetes-commit: 5116a508a7bf84844f4987ab2db14af88bfd296f
2021-10-20 11:06:27 +02:00
67be998d0f
rename to muxAndDiscoveryComplete
...
Kubernetes-commit: 9e2bdfee02a6851fbb13ffe28611e9d2b6242785
2021-10-19 12:24:00 +02:00
1fc498576a
apiserver: indroduces NotFoundHanlder
...
The new handler is meant to be executed at the end of the delegation chain.
It simply checks if the request have been made before the server has installed all known HTTP paths.
In that case it returns a 503 response otherwise it returns a 404.
We don't want to add additional checks to the readyz path as it might prevent fixing bricked clusters.
This specific handler is meant to "protect" requests that arrive before the paths and handlers are fully initialized.
Kubernetes-commit: 53867975e72c7a2d2dd94aac6bd2869411f92094
2021-10-15 17:37:16 +02:00
5283383fb5
Clarify metrics help wrt APF execution phases
...
Kubernetes-commit: d7a3bf0d260a0c291941cda68492f10e5010ac91
2021-10-24 22:32:13 -04:00
7324607749
Free APF seats for watches handled by an aggregated apiserver.
...
Kubernetes-commit: 1873915be6be40df20fe35a8e0c7e8e0a620111a
2021-10-06 10:16:46 -04:00
5ab6f3fe6b
Remove presumptions about what decision has been made
...
Kubernetes-commit: e70999becd98aa00fd00bc622ffdca0476ec7340
2021-10-18 02:10:21 -06:00
b40e786ba3
apf: return nil for a request that has been removed from queue
...
Kubernetes-commit: cd06ba502cf85603242008cc9f61234790ede6de
2021-10-18 12:46:54 -04:00
c5a0365136
Fix nits noticed in recent code review
...
Kubernetes-commit: 1844a052776bce33322ce20c11b2902403655ef8
2021-10-18 23:51:48 -05:00
fb57181f8d
apf: include seat information in request dump
...
Kubernetes-commit: 8e33a3b2cc97813724b64be36fe977290db3053a
2021-10-14 09:27:54 -04:00
20ef863037
apf: include seats in use in queue dump
...
Kubernetes-commit: edf657a33a70ff942d9a0695f602bed13e5e9628
2021-10-13 10:39:37 -04:00
88992ba87a
apf: change QueueDump to use SeatSeconds
...
Kubernetes-commit: 3ef5752edf5b3f7076041368b1c1b5396ee534d7
2021-10-13 10:22:41 -04:00
e4ae91b4a2
apf: include queue sum stats in debug
...
Kubernetes-commit: 87c7401eb853678da804ff503e266daeef5ad126
2021-10-13 10:14:30 -04:00
f578e7708d
Disable P&F for watch requests
...
Kubernetes-commit: 763e6d1dbb6371c66b62e4ea4c1fcbee1c113bda
2021-10-13 07:43:05 +02:00
d69d77c659
Update queueset_test.go for FinalSeats
...
Track the introduction of FinalSeats.
Give up on calculating expected results for tests with added latency,
because I did not find an easy and obvious way to do it.
Kubernetes-commit: 0fc595e03360ba7fc4c3e251d4b41f39172aca72
2021-10-08 22:27:39 -07:00
c3ef02ad27
Adjust final seats if they don't fit the limit
...
Kubernetes-commit: c5a77d8a761b0651b53864f9e396d6f23efd01d2
2021-10-08 11:14:11 +02:00
c18ab3e1b1
Estimate width of the request based on watchers count in P&F
...
Kubernetes-commit: 223f9be59778b6ec2e44fd57df523f00e246bd95
2021-07-07 10:48:29 +02:00
56c6fd034b
Unconfused logging wrt additional latency
...
Fixed confusion in queueSet logging wrt seats and additional latency.
Kubernetes-commit: 42f698daad73d9c010027bb60fff9728168c71da
2021-10-11 13:21:46 -07:00
f7bfb170d7
Keep the progress meter R from overflowing
...
Also add test for that situation.
Kubernetes-commit: a797fbd96de8c67aaed58aef54fbe9f0eb94a2c2
2021-10-01 22:04:05 -07:00
487fea8e61
Update log messages in finishRequestLocked
...
Make them clearer and consistent.
Kubernetes-commit: 3906e187a685cebddef85226950fc36f65e8ddb4
2021-10-09 23:48:41 -07:00
1b1389676f
Relax TestDifferentWidths
...
Make the margin a little wider because flakiness was reported.
Kubernetes-commit: 10326282f9d1abcd4a45b737628286d40971efea
2021-10-07 16:09:53 -07:00
a5192405d9
Calculate the work in each request just once
...
Kubernetes-commit: f2c46c8f9d0b360cf913e22c222d9954b4ff9a76
2021-10-07 17:20:56 -07:00
9560ec6e92
introduce final seats for work estimate
...
Kubernetes-commit: 3d6cc118fee15313419bf7aa0082a2a608ec62f6
2021-09-24 15:18:27 -04:00
dc449969cc
Use SeatSeconds
...
Kubernetes-commit: 4b5e1398199282f471d0f332eefeb5c2415bdb01
2021-10-01 15:33:37 -07:00
0cb46ec2f7
Draft datatype for seat-seconds
...
Kubernetes-commit: d866f9445831687ab3254d754b13a4acf271882f
2021-10-01 13:29:35 -07:00
863c48fbc2
apf: rename WorkEstimate.Seats to InitialSeats
...
Kubernetes-commit: 5d67896adedbce27f01b59eb5f2054919a047f2b
2021-09-24 09:41:38 -04:00
72ff8a6261
Improve queueset sharding and dispatching
...
New anti-windup technique: use the request arrival time as the floor
on the virtual dispatch time. Prevent bound violations where they
might arise rather than fixing up just one queue at dispatch time,
so that the fixed up dispatch times figure into the dispatching choice.
Two tweaks to the shuffle sharding. Take seats of executing requests
into account as well as seats of waiting requests. Do not always
consider the generated hand in the same order.
Rename the queueset methods that do shuffle sharding and finding the
queue to dispatch from, because the old names were confusingly
similar.
Tighten up some request margins.
Name the test cases in TestNoRestraint and TestWindup.
Kubernetes-commit: 4b9cba85874158b25b5c994773a4ec04343820c2
2021-09-20 15:45:24 -04:00
8d3036922c
More test tweaks
...
Canonicalize listing of test cases.
Make TestNoRestraint try both cases: competition and none.
Kubernetes-commit: 0ee1a7b4ff9012b050bd447055ad5e1e8c57c30e
2021-09-20 15:45:24 -04:00
c505aa64af
Update TestNoRestraint and TestWindup
...
Make TestNoRestraint verify that fairness is NOT achieved
when there is real competition.
Make TestWindup run two cases, to show that 0.1 is too narrow
a margin and 0.26 is wide enough.
Kubernetes-commit: c4945fdf0c14ba2032a5c8edf192678d9fe00374
2021-09-17 01:40:16 -04:00
e417abf592
Migrate apiserver/pkg/util/flowcontrol to use k8s.io/utils/clock
...
.. instead of apimachinery/pkt/util/clock
Kubernetes-commit: 9f45c0f8c07cd0adfe38c887aa618d33b8a4ee1c
2021-09-17 15:14:42 -04:00
f17b9e7b4b
Track recent reorg in testing clockery
...
Kubernetes-commit: 20ccd735f6a55fdd9ecbe8cddb889310f0bc033b
2021-09-13 16:15:28 -04:00
db8aff032b
apf: update apf logic to use v1beta2
...
Kubernetes-commit: 28f2b42a4116a9223113e8b152e02a4f1e602ff4
2021-08-16 17:53:57 -04:00
7964b85d9c
apf: update e2e test to use v1beta2
...
Kubernetes-commit: d99497ef6d5581b787e7aebfca943697ed8fe164
2021-08-16 17:12:42 -04:00
de042674ed
Widen margins of TestDifferentWidths and TestTooWide
...
These behavioral unit tests of queueset were failing because the
evaluation criteria were too strict.
Kubernetes-commit: 59d319ec06bb33289a87036418b4a61ed3bb215f
2021-09-09 17:07:58 -04:00
de227d1d37
Change execution duration guess from 1 minute to 3 milliseconds
...
So that the width estimate has some effect but not a grossly excessive
one.
Added the fifo::Peek method to simplify the fifo client code.
Also renamed the queueSet::estimatedServiceTime field to
estimatedServiceSeconds to make the units clear.
Kubernetes-commit: a0c161f2f6908ee424ea888ff40f75ff071bd20a
2021-09-07 00:46:50 -04:00
1efcc1c03a
Refine locking in API Priority and Fairness config controller
...
Instead of a plain `Mutex`, use an `RWMutex` so that the common
operations can proceed in parallel.
Kubernetes-commit: 58927c1abede11ce7a8a74104328cf823df1b39e
2021-09-08 00:34:52 -04:00
d5beafeb68
Avoid expensive go-cmp/cmp comparison
...
The cmp comparison is relatively expensive (#104821 ). If we're not
going to log it, we shouldn't make the comparison.
Kubernetes-commit: f9f556dc7061df1dfc8c1628db983eeb97149317
2021-09-07 13:36:50 -04:00
d6c30c1553
Adjust LIST work estimator to match current code
...
Kubernetes-commit: 6f160ca11b0ce9e9fda9571e37eec2de1b97bcc5
2021-08-25 23:22:13 -04:00
cc873bafd2
Increase debug logging in waitGroupCounter::Add
...
Now tries a little harder to get to meaningful stack frames.
This revision seems to strike a balance that is useful in the queueset tests.
Kubernetes-commit: b56dd725032cb7a14aa27e4c50c1c9d7c6d23eb1
2021-08-06 14:32:19 -04:00
02f98184b3
Add APF's priorityLevel to httplog.go
...
Kubernetes-commit: 6db63869b0fd0551382dec1b30b649fda4289444
2021-08-13 13:00:07 +02:00
7d5430cfba
Fix extra latency and add tests for that and width
...
Added missing dispatching after delayed release of seats.
Updated logging for all six situations of execution completion and
seat release.
Added behavioral tests for non-zero extra latency and non-unit width.
Also added two tests for baseline functionality.
Also improved some comments and other logging in `queueset.go`.
Kubernetes-commit: d2a27a58f0af20c6185fa1c21890d666e9d3746b
2021-08-12 16:48:02 -04:00
bbc4211158
Update unit tests to handle go1.17 certificate parsing error messages
...
Kubernetes-commit: 0e925f266f78d72ed05a5a2fe48faa5e2a87d107
2021-08-17 11:24:03 -04:00
0741f109f6
Add a new webhook metric tracking request totals.
...
Also add a 1.0s bucket boundary to the webhook latency metric.
Kubernetes-commit: 8ed1628a6e75f4029853502dbac44fdb0edac5fc
2021-06-22 22:32:47 +00:00
da50ca4c6e
apf: free seats in use after additional latency
...
Kubernetes-commit: d68186452d9150b113489e6a722caf82f898857f
2021-06-27 13:04:20 -04:00
8c2108bc80
Refactor goroutine counting
...
Add comment outlining TestContextCancel.
Stop calling `t.Errorf` from wrong goroutine.
Package up queueNoteFn expectation checking.
Add counting of goroutine in req1 exec fn.
Remove unnecessary assignment to `_`.
Make TestContextCancel wait on fake clock, to insulate timing check
from scheduler noise.
Factor goroutine counting out of queueset.go, into queueset_test.go,
where it matters.
Refactor promise: Use a simple channel-based implementation for normal
code, a mutex-based one for testing code.
Took all the panics out of queueset.go
Shrink the timeouts in promise tests to 1 second.
Kubernetes-commit: 1db36ae3b30e30d70972998a22987a7db470479b
2021-07-29 00:35:25 -04:00
904cd74454
Some cleanup of the package for event clocks
...
Rename from `clock` to `eventclock`.
Simplify by removing the prohibition on an EventFunc suspending and
resuming activity.
Remove "EventClock" from names to avoid stuttering.
Start to consolidate test code under fairqueuing/testing/.
Kubernetes-commit: 80ca6a4ae6ff571c32962a7155efd55edefff9e6
2021-08-06 02:06:43 -04:00
f013b63777
apf: use EventClock rather than a PassiveClock for queueset
...
Kubernetes-commit: c2a3b793d3ec62e781dd20704370d09f7e1be706
2021-07-21 17:06:48 -04:00
bd2ef9810a
apf: estimate list width
...
Kubernetes-commit: 296c18ec323328bf1c6a621a9ca3a094d78bb9d0
2021-06-22 19:38:00 -04:00
b09300b3f0
fix unsafe json construction for digestConfigObjects.
...
fix unsafe json creation by creating intermediate objects
while creating patch bytes.
Kubernetes-commit: dfde50b18564cc9a8e99fdcba2f136d770ec6943
2021-08-01 18:43:40 -07:00
0c550377cf
Introduce event clocks based on k8s.io/utils/clock
...
So we can move off of the apimachinery clock package.
Switch queueset to new clocks.
Removed event clocks based on apimachinery clocks,
because this PR introduces ones based on k8s.io/utils/clock .
Removed interface that is implemented by only one interesting type.
Simplify RealEventClock::EventAfterTime.
Kubernetes-commit: dcb298c9552de44e27ed52f5e2b58a0dd7cd8d54
2021-07-21 16:56:11 -04:00
feb4eefe1c
apiserver: add callback to get notified of object count
...
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
719fda2a8b
Simplify APF promise to what is really used in the code
...
Kubernetes-commit: 9f735e71bbb7d0dde67a718891641d8afd20a8bc
2021-07-23 13:30:34 +02:00
d6e0cf5d71
Remove unused promise code from APF
...
Kubernetes-commit: a1cf44eab44f21c3929ff2d79501c56e6fbcddc3
2021-07-21 12:14:30 +02:00
9209ed5928
Get rid of unused flowSchemaRVs in P&F
...
Kubernetes-commit: c79a0a08828ef86e07941e96dafefb7d8bc51990
2021-07-13 15:16:17 +02:00
b4c306e1e8
Rename width to workEstimate in P&F code
...
Kubernetes-commit: 73211256e8f15cf84ee69d6fe8258c3a912e0f94
2021-07-13 15:10:58 +02:00
242c8c8a1c
Optimize APF support for watch initialization to fix the pod startup time regression.
...
Kubernetes-commit: ef435b85b47618d0d463ac93e9758d75c2998681
2021-07-13 10:22:30 +02:00
785c5626f9
add tracing to webhook requests
...
Kubernetes-commit: 9dd59017c4c218135b12949a2bd3af07e892199a
2021-07-08 20:13:33 -07:00
b206f9ea02
apf: fix virtual clock
...
Kubernetes-commit: a570008cbd6c2142a2f270570fe49bf982825c19
2021-07-06 19:29:16 -04:00
7afffd8ef3
Revert "apiserver: add callback to get notified of object count"
...
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
9606daa5ef
Refinements to pick queue logic in P&F
...
Kubernetes-commit: 7f1c4977d7ad9fdfd44549ee8ffe289127be4d81
2021-07-07 08:58:49 +02:00
fe1610f3fe
switch from golang-lru to the one in k8s.io/utils
...
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Kubernetes-commit: 79d0c6cdc10293c9bfe644ce31dc186a936579b0
2021-07-07 13:45:07 -04:00
fbc127e994
Add watch tracker to APF for request cost estimation
...
Kubernetes-commit: cea1dcfeed2fc4e8ab89cd43e5a0e402251c8df5
2021-06-15 10:49:42 +02:00
a35bb85f09
Update the logic to pick the best queue in P&F
...
Kubernetes-commit: 0ecc7ba311ab33b16c5d907ebb1120e3e51a947d
2021-06-17 09:22:00 +02:00
cf5c77fde9
apf: add additional latency into width
...
Kubernetes-commit: 24e19229101d242d924ce98a562be3864dde9eae
2021-06-27 12:45:24 -04:00
d929410e13
apiserver: add callback to get notified of object count
...
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
e9708ba84d
apf: calculation of dR/dt should use seats in use
...
Kubernetes-commit: 6203d4fc6af36b1fd60177e8a88b8d4d45acb029
2021-06-21 05:38:07 -04:00
4dcb848926
Track generalization for width
...
Kubernetes-commit: 3930629e130f1c98821f8033941870ab582db284
2021-06-18 01:37:22 -04:00
d28ccb4224
Add APF metrics about R(t)
...
Kubernetes-commit: 676f0450ed37eeec92b67246719cc46e7567e512
2021-06-14 16:48:27 -04:00
56e6dac0bb
update github.com/pkg/errors to go native errors pkg in staging
...
Kubernetes-commit: 4b1402187474a1ebd0f71c824f45448fea160fc6
2021-06-22 11:54:56 +08:00
e1aec4ecae
apf: take seats into account when dispatching request
...
Kubernetes-commit: ff716cef508f948b50e1026e980e6df5ee475538
2021-06-14 12:19:06 -04:00
abdd901971
apf: always include seats in virtual time
...
Kubernetes-commit: 2f7456076e0cb29a95d86cd0f54c34a04b4722ab
2021-06-14 09:52:00 -04:00
345d1c6ff9
apf: add a gauge for the number of seats currently in use
...
Kubernetes-commit: c710f99ef730a791a6911e63cc3b9d26cced6bd3
2021-06-10 17:34:50 -04:00
3c7f54740f
apf: add plumbing to estimate width" of a request
...
- add plumbing that allows us to estimated "width" of a request
- the default implementation returns 1 as the "width" of all
incoming requests, this is in keeping with the current behavior.
Kubernetes-commit: 9b72eb1929a64b9d5a5234090a631ba312fb4d41
2021-05-11 07:03:05 -04:00
5290a24345
Implement support for watch initialization in P&F
...
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
ec22c8bdd8
apf: add "width" for request
...
all requests have a width of 1 to maintain current behavior.
Kubernetes-commit: b50507d98bd12503592ea62d2be2aadef49bdf70
2021-05-11 07:03:05 -04:00
5289a15a1a
client-go transport: assert that final CA data is valid
...
Signed-off-by: Monis Khan <mok@vmware.com>
Kubernetes-commit: 440ea3ef49e0ac77353ceeaebc2aad6c995d5b35
2021-05-03 10:11:54 -04:00
7eb7056b38
webhooks,aggregation: add metrics to count certs missing SAN
...
Adds counters to generic webhook code and to the kube aggregator
so that it is possible to effectively measure the impact of
Golang 1.15's deprecation of x509 cert CN hostname
verification.
Kubernetes-commit: 9d6a19efff05a40081337a2fcb3dce6331b04022
2020-10-06 13:02:52 +02:00
4e12e0cd34
apf: use a list instead of slice for queueset
...
Kubernetes-commit: 69f9bc181f155ded7c5d5cc0ca9f026a6b42f431
2021-03-29 11:31:12 -04:00
af9424d2c9
remove go-openapi/spec
...
Signed-off-by: Gautier Delorme <gautier.delorme@gmail.com>
Kubernetes-commit: 34b0fcef5fc47e3fcddf7f6ca1b3e6176b2a5323
2021-04-20 17:48:33 +02:00
d7e2a0bbee
Update kube-openapi and gnostic dependencies
...
Pulling in https://github.com/kubernetes/kube-openapi/pull/220
Kubernetes-commit: a849c8998c1ed71f25387a560f8359596aec8bd3
2021-03-23 08:33:15 -07:00
7c07064678
apf: fix test flake
...
Kubernetes-commit: fa0952ee778aaf7df3e593305575f5d121b55cf1
2021-03-29 16:09:49 -04:00
4c81a6672b
Enable, rate limit, and test APF controller fights
...
Using real time.
Kubernetes-commit: 80ff06fe846c5e565e0cbd4b70f5f1e8a4ea7295
2020-12-02 16:50:05 -05:00
1a8abfc56f
add context to metrics in util/flowcontrol.
...
Kubernetes-commit: 57d0bc301a017c41d890baee0a3a287f448c664d
2020-12-16 17:08:43 -08:00
Mike Spreitzer