Commit Graph

241 Commits

Author SHA1 Message Date
Mike Spreitzer 2ceb0fcbe8 Create OWNERS for most of the API Priority and Fairness impl
Kubernetes-commit: 23de8061fbb63928c62aa6020b443a62491a0a1d
2020-12-11 17:54:59 -05:00
corehello b6f30b469b Fix typo for ServiceAccount
Kubernetes-commit: c20b2c8cd79f9e7dafa86fb4245dee9dd2d795c7
2020-12-02 01:38:46 +08:00
Ken Sipe 0da065589a fix S1023 redundant return statement
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 1d795002919675b8cf3bf462b6543fef843b5b54
2020-06-25 17:29:42 -05:00
Adhityaa Chandrasekar 43cf7154be APF matching: fallback to catch-all if nothing matches
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 794a3ec0997caffa116c2e6f78feb53ebfca6052
2020-11-13 03:58:07 +00:00
yue9944882 a4a3fc9b87 APF: graduate API and types to beta
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 849be447f563fc93a27a0827fb1185b885b57114
2020-11-04 16:33:14 +08:00
Mike Spreitzer 70bbff23ac Fixed quantization and made monotic time reversal not panic
Changed quantization to use monotonic clock readings.

Stopping panicing when monotonic clock readings go backwards because
https://github.com/kubernetes/kubernetes/issues/96459 reported it
happening.

Re-organized sampleAndWaterMarkHistograms::innerSet to `defer` the
Unlock call, because that is the preferred pattern.

Kubernetes-commit: 0809220fd04d26892f0fcfd3bf54cc4baeda22da
2020-11-12 15:17:07 -05:00
Adhityaa Chandrasekar 8b21b5725d APF metrics: set StabilityLevel to ALPHA
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: b16f36b251ddbfef5f12fed58640de53512631f0
2020-11-05 15:35:39 +00:00
Adhityaa Chandrasekar ebe254b2e6 APF: use snake_case in metric labels
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: f9d57a8d5db3e58f79a1b1958d80c049c63d6cde
2020-11-04 22:19:52 +00:00
yue9944882 5474822749 fixes max-min fairness
Kubernetes-commit: fd889ec8ae37437a9e75386542291bd0e2cc605e
2020-10-29 18:57:38 +08:00
Mike Spreitzer 86c572cf3f Remove phantoms from dump_requests output
No longer writes a single info-free line for each exempt priority level.

Kubernetes-commit: 6aeca5fb345256e104b9b2224a0f8797c1921214
2020-07-24 00:35:26 -04:00
Ken Sipe 4ad3e91efa changes in imports was unintentional
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 09cd56b561d0c31421dfc6374b68e0b18c6cc746
2020-06-28 15:26:40 -05:00
Ken Sipe 32533315c9 fix S1000 simplify ch switch cases
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 268c2f81c7ab94cbab68a8d6c00725144b81fa09
2020-06-26 10:45:30 -05:00
Mike Spreitzer b1ede52e21 Make sampleAndWaterMarkHistograms not fall very far behind
Kubernetes-commit: 9e89b92a92c02cdd2c70c0f52a30936e9c3309c7
2020-08-20 16:43:11 -04:00
yue9944882 543449d042 removes empty columns and fixes request details
Kubernetes-commit: 40e3feb7f1b7f54a512785af7e28810c0874ac63
2020-07-23 14:25:11 +08:00
Mike Spreitzer e28ab56bd4 Introduce more metrics on concurrency
Introduce min, average, and standard deviation for the number of
executing mutating and readOnly requests.

Introduce min, max, average, and standard deviation for the number
waiting and number waiting per priority level.

Later:

Revised to use a series of windows

Use three individuals instead of array of powers

Later:

Add coarse queue count metrics, removed windowed avg and stddev

Add metrics for number of queued mutating and readOnly requests,
to complement metrics for number executing.

Later:

Removed windowed average and standard deviation because consumers can
derive such from integrals of consumer's chosen window.

Also replaced "requestKind" Prometheus label with "request_kind".

Later:

Revised to focus on sampling

Make the clock intrinsic to a TimedObserver

... so that the clock can be read while holding the observer's lock;
otherwise, forward progress is not guaranteed (and violations were
observed in testing).

Bug fixes and histogram buckets revision

SetX1 to 1 when queue length limit is zero, beause dividing by zero is nasty.

Remove obsolete argument in gen_test.go.

Add a bucket boundary at 0 for sample-and-water-mark histograms, to
distinguish zeroes from non-zeros.

This includes adding Integrator test.

Simplified test code.

More pervasively used "ctlr" instead of "ctl" as abbreviation for
"controller".

Kubernetes-commit: 57ecea22296797a93b0157169db0ff2e477f58d0
2020-05-17 01:02:25 -04:00
Mike Spreitzer f0320d394e Add test for windup problem in APF
The current design for Fair Queueing for Server Requests has a
problem: if the min-max fair result stays different from an even
division for a long time and no queue involved in the imbalance goes
empty then the imbalance keeps accruing in queue virtual state times.

This commit adds a test that demonstrates the problem.

It also has some other tweaks to make other tests less flaky.

Factor the big scenario-testing func into pieces, with supporting
structs.

Kubernetes-commit: f3fdd5cf9f51f2a03ebfddef82bc2014e04b62a0
2020-06-04 00:15:08 -04:00
yue9944882 7f1f8c2578 introduce apf debug endpoint
Kubernetes-commit: 567becd5eedd9a1f3802f2b3b0b6b8efc445d2a1
2020-05-22 13:02:10 +08:00
Mike Spreitzer 705ee42713 Clean up fairqueuing/interface.go
Fixed outdated comment on QueueSet::StartRequest.

Removed unused interface EmptyHandler.

Kubernetes-commit: 28671fc4cf34e9e34eee545312c848ce40ab148b
2020-05-17 22:36:30 -04:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
Mike Spreitzer 9df60c9fe6 Renaming: "Change" -> "Add" for consistency with underlying method
Kubernetes-commit: c7b098ac6c276d65a79db6cfeb04f5f0f86eb315
2020-03-05 15:17:33 -05:00
Mike Spreitzer 8235385f97 Fix queued request accounting, extended queueset test
Kubernetes-commit: 8a1b60320986eca05cb281bcce45332e0969268e
2020-03-05 15:13:46 -05:00
Mike Spreitzer 6ae3e470a2 Make some metrics finer-grained, add dispatch counts, note immediate reject
Also add testing of metrics for queuesets.

Kubernetes-commit: f535a9c9ed4b6a0def47c354acad0ac2a8f961b0
2020-03-01 20:22:58 -05:00
Mike Spreitzer 219ab512ff Generalized NonResourcePolicyRule.NonResourceURLs impl
... to match the comment on that field.

Also generalized the test case generator to exercise the new
generality.

Kubernetes-commit: 2e97d3c8732147c3ba2f11d668f50b44e6374348
2020-03-04 00:00:39 -05:00
Mike Spreitzer f9c1780311 Hopefully plainer test strings
Kubernetes-commit: 4a4852ca9a1ffc439e2c476d7057a8be9f081055
2020-03-04 21:47:19 -05:00
Mike Spreitzer f44645645b Added non-randomized tests of matching FlowSchema rules
Kubernetes-commit: 21f78f3348736f242541f7b66e191aa1dce78c47
2020-02-27 23:36:19 -05:00
yue9944882 f452a698b0 register metrics from comp-base
Kubernetes-commit: 11656478be93d4a9e54129ec35cd2b9558e901ac
2020-02-27 17:04:17 +08:00
yue9944882 f93a7a8312 homogenize metrics naming
Kubernetes-commit: a1523a049ff9fc47d7dc2c4354b16b69d2eb4be2
2020-02-19 16:34:49 +08:00
Ted Yu f2eace2821 Do not dereference qcAPI which maybe nil
Kubernetes-commit: 583f804652ffff898437c80ce357e4f0c379bfb9
2020-02-14 13:30:40 -08:00
Mike Spreitzer 1ef67fe134 Version the API Priority and Fairness FieldManager values
Kubernetes-commit: b142868eaf46b24fce0e6b01e06087c924351857
2020-02-14 13:50:11 -05:00
Mike Spreitzer 1c79014daa Added API Priority and Fairness filter and config consumer
Kubernetes-commit: 73614ddd4e42728a36c7ac6b7b20f27c8032cafb
2020-01-25 19:47:12 -05:00
Mike Spreitzer fffb0fade5 brushed up according to review
Added LockingWriteMultipleOnly and LockingWriteOnceOnly interfaces,
so that further extensions are possible (in this package or others).

Moved common SetLocked behavior into promisoid.

Made comments say things that were implied.

Kubernetes-commit: cbdd3a279e6161d73f2c4e8a2b916ae74b258621
2020-01-24 14:33:15 -05:00
Mike Spreitzer 27a36b04f5 Tweak new names
Kubernetes-commit: a65f525aed8d9f28f677028f77e2ab41313ae037
2020-01-21 20:35:45 -05:00
Mike Spreitzer 3a091e906d Simplified logic around context cancel, removing bugs
Previously, a `decisionCancel` could overwrite a `decisionReject` or
`decisionExecute`, causing confusion.  Now a request gets exactly one
decision and there is no confusion.

Also added write-once to the promise package and refactored.

Kubernetes-commit: 1c092bf635954bde9c9c363672fa156b9430206b
2020-01-09 14:03:13 -05:00
Mike Spreitzer 4fead639b7 Refactored QueueSet configuration into two phases
So that errors can be detected before resolving concurrency shares
into concurrency counts.

Kubernetes-commit: 1e170637c3ce6c4ccd378275d9e52192f4be12b7
2020-01-18 01:46:11 -05:00
Mike Spreitzer d0f4b93c91 Brushed up fairqueuing package
This commit responds to the comments on PR #85192 that were not yet
addressed at the time it merged, apart from the one fixed in PR

Generalized fairqueuing to allow for zero queues, to support a
priority level that limits concurrency but does no queuing.

Kubernetes-commit: b123a43e7117e977606bacd31d77f4a30d2ed212
2019-11-13 23:34:10 -05:00
Ted Yu 82980a5db1 Correct the checking of robinIndex
Kubernetes-commit: 2bdd379a1eb9d0344df41fff0339123849682e7c
2019-11-13 20:13:29 -08:00
Mike Spreitzer cd34d8d0ce Fixed lint and staticcheck oversights
Kubernetes-commit: e10acc78dee5d90d93fc7bc0e76e97bc7bc0b3a3
2019-11-13 09:20:00 -05:00
Mike Spreitzer 63ad2ccdad Brushing up queueset
(1) Replaced random-looking assortment of counter increments and
decrements with something hopefully more principalled-looking.  Most
importantly, introduced the MutablePromise abstraction to neatly wrap
up the complicated business of unioning multiple sources of
unblocking.

(2) Improved debug logging.

(3) Somewhat more interesting test cases, and a bug fix wrt round
robin index.

Kubernetes-commit: 1c31b2bdc65377f502c2306dbdf32a802eb1afb7
2019-11-13 01:52:05 -05:00
Aaron Prindle e231e56df2 review changes - *Locked updates
Kubernetes-commit: 6619df1798859d49bbb52b1c029533035384824e
2019-11-12 09:24:56 -08:00
Aaron Prindle 572fbfc84d review changes
Kubernetes-commit: 396e2d4aa33bb7289cd8e7466e4465f56a73b7d0
2019-11-12 08:51:49 -08:00
Aaron Prindle a222f282e1 fairqueuing implementation with unit tests
Kubernetes-commit: 24065cf5be6bed995da7b7abb37ee78ff95230f0
2019-10-29 21:54:16 -07:00