Commit Graph

201 Commits

Author SHA1 Message Date
Mike Spreitzer b2c3c4f86c Tweak up TestSampler in response to review
Kubernetes-commit: 736a87a4ee516a3feca0a79a4c48806ad4916aa3
2021-01-24 23:38:48 -05:00
Mike Spreitzer 39f5df86ff Added comment explaining the new unit test
Kubernetes-commit: 42f8d4e28fc05f3532f5cf7d2294049e3314896a
2021-01-18 00:57:09 -05:00
Mike Spreitzer 60021e2fed Explain warnings in new test
Add info message that explains what warnings are deliberate and expected.

Kubernetes-commit: 194c22fed54ba89d985085412795a9850bedeaf3
2021-01-06 12:56:16 -05:00
Mike Spreitzer 71c9d22768 Add unit test for sample-and-watermark histograms
Kubernetes-commit: 5fe48be506fe2b946d4db9cd7976ad0c4d4e7e1a
2021-01-05 23:31:06 -05:00
pacoxu 2343af38bb fix all keps links 404 for kep folder migration
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: a10bdfed09e96f3c2703454caafa2c884914e064
2021-01-14 14:03:21 +08:00
Mike Spreitzer 5d4804815d Less demanding test cases in TestUniformDistribution
Also a bug fix in the evaluation.

Kubernetes-commit: 1f1e8e21d81c4ff75078adc163bf2872d1b4650e
2021-01-25 01:44:08 -05:00
Abu Kashem 84ce907827 fix apf controller unit test
- don't expose the internal states of the apf controller to the caller
- return a boolean, instead of the priority level states

Kubernetes-commit: f20c6cb2d9060920cae9ff5cade1739c7e0b7f7a
2021-01-13 11:06:17 -05:00
Mike Spreitzer 16288a4d61 move all variables in sampleAndWaterMarkHistograms::innerSet
to tiptoe around https://github.com/golang/go/issues/43570 for #97685

Kubernetes-commit: 611184aa59d0cd40466bc3bc4b40a3712a038171
2021-01-08 13:32:38 -05:00
Abu Kashem 3ba02b7f93 handle webhook authenticator and authorizer error
webhook.WithExponentialBackoff returns an error, and the priority is:
- A: if the last invocation of the webhook function returned an error
  that error should be returned, otherwise
- B: the error associated with the context if it has been canceled or
  it has expired, or the ErrWaitTimeout returned by the wait package
  once all retries have been exhausted.

caller should check the error returned by webhook.WithExponentialBackoff
to handle both A and B. Currently, we only handle A.

Kubernetes-commit: ae2b353fbf519b29d168c534f88c373fd67a1c31
2021-01-07 16:14:18 -05:00
Mike Spreitzer 41788f53e2 Relax apf_controller's reaction to certain errors
When the error is due to the object having been deleted, the
controller does not need to do anything before the coming
notification.

Kubernetes-commit: ef1e2039b5fc7f955ec4f9c636a64aa403cba2ab
2020-12-17 10:46:14 -05:00
Abu Kashem eea0d66fcd clean up executing request on panic
Kubernetes-commit: 13cedca0eb5337b13e5176983ea5e784ec38df22
2020-12-10 12:57:21 -05:00
Abu Kashem 59a718ed18 fix the deadlock in priority and fairness config controller
Kubernetes-commit: 7114319b3e66e331250c116ac926f1a57352072f
2020-12-15 12:19:56 -05:00
Mike Spreitzer a011399e98 Define TestableConfig in k/apiserver/pkg/util/flowcontrol
Collect the parameters of newTestableController into a named type.

Also tolerate the surprising situation in which a request's user
groups include neither `system:authenticated` nor
`system:unauthenticated` --- because this is observed to happen in
some tests.

Also a few other minor fixups.

Kubernetes-commit: 10df6d459b5bca7b42471f9409182417fbc3f253
2020-12-16 18:58:45 -05:00
Mike Spreitzer 2ceb0fcbe8 Create OWNERS for most of the API Priority and Fairness impl
Kubernetes-commit: 23de8061fbb63928c62aa6020b443a62491a0a1d
2020-12-11 17:54:59 -05:00
corehello b6f30b469b Fix typo for ServiceAccount
Kubernetes-commit: c20b2c8cd79f9e7dafa86fb4245dee9dd2d795c7
2020-12-02 01:38:46 +08:00
Ken Sipe 0da065589a fix S1023 redundant return statement
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 1d795002919675b8cf3bf462b6543fef843b5b54
2020-06-25 17:29:42 -05:00
Ken Sipe b60943f03a fix S1019 use of make
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: a4d05d51d95d30ebe90ec8f652c9a794caa44d53
2020-06-25 17:26:42 -05:00
Adhityaa Chandrasekar 43cf7154be APF matching: fallback to catch-all if nothing matches
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 794a3ec0997caffa116c2e6f78feb53ebfca6052
2020-11-13 03:58:07 +00:00
yue9944882 a4a3fc9b87 APF: graduate API and types to beta
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 849be447f563fc93a27a0827fb1185b885b57114
2020-11-04 16:33:14 +08:00
Mike Spreitzer 70bbff23ac Fixed quantization and made monotic time reversal not panic
Changed quantization to use monotonic clock readings.

Stopping panicing when monotonic clock readings go backwards because
https://github.com/kubernetes/kubernetes/issues/96459 reported it
happening.

Re-organized sampleAndWaterMarkHistograms::innerSet to `defer` the
Unlock call, because that is the preferred pattern.

Kubernetes-commit: 0809220fd04d26892f0fcfd3bf54cc4baeda22da
2020-11-12 15:17:07 -05:00
Adhityaa Chandrasekar 8b21b5725d APF metrics: set StabilityLevel to ALPHA
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: b16f36b251ddbfef5f12fed58640de53512631f0
2020-11-05 15:35:39 +00:00
Adhityaa Chandrasekar ebe254b2e6 APF: use snake_case in metric labels
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: f9d57a8d5db3e58f79a1b1958d80c049c63d6cde
2020-11-04 22:19:52 +00:00
yue9944882 5474822749 fixes max-min fairness
Kubernetes-commit: fd889ec8ae37437a9e75386542291bd0e2cc605e
2020-10-29 18:57:38 +08:00
Abu Kashem 5254108841 make backoff parameters configurable for webhook
Currently webhook retry backoff parameters are hard coded, we want
to have the ability to configure the backoff parameters for webhook
retry logic.

Kubernetes-commit: 53a1307f68ccf6c9ffd252eeea2b333e818c1103
2020-10-30 11:25:32 -04:00
Abu Kashem 67f1bb6022 Webhook: handle error when calling wait.ExponentialBackoff
- Inside WithExponentialBackoff function, handle error returned
by wait.ExponentialBackoff.
- Ensure that the wait time is bound to the given context.

Kubernetes-commit: f8e35de156f212b6989b465e608dd99b525bd5dc
2020-10-26 11:37:11 -04:00
Andrew Sy Kim 6746ccadda apiserver: support egress selection name 'controlplane' and deprecate 'master'
Signed-off-by: Andrew Sy Kim <kim.andrewsy@gmail.com>

Kubernetes-commit: a0aebf96ec2eef6517e2611335f0e6c9375dd807
2020-10-26 10:24:16 -04:00
Mike Spreitzer 86c572cf3f Remove phantoms from dump_requests output
No longer writes a single info-free line for each exempt priority level.

Kubernetes-commit: 6aeca5fb345256e104b9b2224a0f8797c1921214
2020-07-24 00:35:26 -04:00
Ken Sipe 4ad3e91efa changes in imports was unintentional
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 09cd56b561d0c31421dfc6374b68e0b18c6cc746
2020-06-28 15:26:40 -05:00
Ken Sipe 32533315c9 fix S1000 simplify ch switch cases
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 268c2f81c7ab94cbab68a8d6c00725144b81fa09
2020-06-26 10:45:30 -05:00
Mike Spreitzer b1ede52e21 Make sampleAndWaterMarkHistograms not fall very far behind
Kubernetes-commit: 9e89b92a92c02cdd2c70c0f52a30936e9c3309c7
2020-08-20 16:43:11 -04:00
yue9944882 543449d042 removes empty columns and fixes request details
Kubernetes-commit: 40e3feb7f1b7f54a512785af7e28810c0874ac63
2020-07-23 14:25:11 +08:00
Mike Spreitzer e28ab56bd4 Introduce more metrics on concurrency
Introduce min, average, and standard deviation for the number of
executing mutating and readOnly requests.

Introduce min, max, average, and standard deviation for the number
waiting and number waiting per priority level.

Later:

Revised to use a series of windows

Use three individuals instead of array of powers

Later:

Add coarse queue count metrics, removed windowed avg and stddev

Add metrics for number of queued mutating and readOnly requests,
to complement metrics for number executing.

Later:

Removed windowed average and standard deviation because consumers can
derive such from integrals of consumer's chosen window.

Also replaced "requestKind" Prometheus label with "request_kind".

Later:

Revised to focus on sampling

Make the clock intrinsic to a TimedObserver

... so that the clock can be read while holding the observer's lock;
otherwise, forward progress is not guaranteed (and violations were
observed in testing).

Bug fixes and histogram buckets revision

SetX1 to 1 when queue length limit is zero, beause dividing by zero is nasty.

Remove obsolete argument in gen_test.go.

Add a bucket boundary at 0 for sample-and-water-mark histograms, to
distinguish zeroes from non-zeros.

This includes adding Integrator test.

Simplified test code.

More pervasively used "ctlr" instead of "ctl" as abbreviation for
"controller".

Kubernetes-commit: 57ecea22296797a93b0157169db0ff2e477f58d0
2020-05-17 01:02:25 -04:00
Mike Spreitzer f0320d394e Add test for windup problem in APF
The current design for Fair Queueing for Server Requests has a
problem: if the min-max fair result stays different from an even
division for a long time and no queue involved in the imbalance goes
empty then the imbalance keeps accruing in queue virtual state times.

This commit adds a test that demonstrates the problem.

It also has some other tweaks to make other tests less flaky.

Factor the big scenario-testing func into pieces, with supporting
structs.

Kubernetes-commit: f3fdd5cf9f51f2a03ebfddef82bc2014e04b62a0
2020-06-04 00:15:08 -04:00
yue9944882 7f1f8c2578 introduce apf debug endpoint
Kubernetes-commit: 567becd5eedd9a1f3802f2b3b0b6b8efc445d2a1
2020-05-22 13:02:10 +08:00
RainbowMango efc2982011 Fix a wrong usage of recover in apiserver.
Kubernetes-commit: d1a883293930336b5c561448900e40d9c4fe4676
2020-05-29 17:03:43 +08:00
Mike Spreitzer 705ee42713 Clean up fairqueuing/interface.go
Fixed outdated comment on QueueSet::StartRequest.

Removed unused interface EmptyHandler.

Kubernetes-commit: 28671fc4cf34e9e34eee545312c848ce40ab148b
2020-05-17 22:36:30 -04:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
Gaurav Singh 43ce4f9092 Remove double import of k8s.io/apimachinery/pkg/util/net
Kubernetes-commit: 29ceb8d5f2fc48d1fe4974110b7f60579668c00e
2020-04-08 16:25:23 -04:00
Jordan Liggitt 08f23b5217 github.com/googleapis/gnostic v0.4.1
Kubernetes-commit: 93c7b24562d80959f45c308e7412456a410b9b25
2020-03-31 17:18:56 -04:00
Davanum Srinivas c15d16953f Move k8s.io/apiserver/pkg/util/term to k8s.io/component-base/term
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 1d057da2f73118893b5cc27c15d59ff03beb271e
2020-03-16 16:27:44 -04:00
Mike Spreitzer 9df60c9fe6 Renaming: "Change" -> "Add" for consistency with underlying method
Kubernetes-commit: c7b098ac6c276d65a79db6cfeb04f5f0f86eb315
2020-03-05 15:17:33 -05:00
Mike Spreitzer 8235385f97 Fix queued request accounting, extended queueset test
Kubernetes-commit: 8a1b60320986eca05cb281bcce45332e0969268e
2020-03-05 15:13:46 -05:00
Mike Spreitzer 6ae3e470a2 Make some metrics finer-grained, add dispatch counts, note immediate reject
Also add testing of metrics for queuesets.

Kubernetes-commit: f535a9c9ed4b6a0def47c354acad0ac2a8f961b0
2020-03-01 20:22:58 -05:00
Mike Spreitzer 219ab512ff Generalized NonResourcePolicyRule.NonResourceURLs impl
... to match the comment on that field.

Also generalized the test case generator to exercise the new
generality.

Kubernetes-commit: 2e97d3c8732147c3ba2f11d668f50b44e6374348
2020-03-04 00:00:39 -05:00
Mike Spreitzer f9c1780311 Hopefully plainer test strings
Kubernetes-commit: 4a4852ca9a1ffc439e2c476d7057a8be9f081055
2020-03-04 21:47:19 -05:00
Mike Spreitzer f44645645b Added non-randomized tests of matching FlowSchema rules
Kubernetes-commit: 21f78f3348736f242541f7b66e191aa1dce78c47
2020-02-27 23:36:19 -05:00
yue9944882 f452a698b0 register metrics from comp-base
Kubernetes-commit: 11656478be93d4a9e54129ec35cd2b9558e901ac
2020-02-27 17:04:17 +08:00
Jefftree 13613a1c15 Address comment and remove if condition
Kubernetes-commit: 61fa4e6c098559b65fe28c1bf55cb817697e38e5
2020-02-27 17:18:57 -08:00
Jefftree f1c9537c7b pass Dialer instead of egressselector to webhooks
Kubernetes-commit: 1b38199ea8b220be0b645af8a4cbdef4c87ce7fc
2019-12-05 17:28:59 -08:00
Jefftree aa55f94611 authentication webhook via network proxy
Kubernetes-commit: d318e52ffe0ba156a96cb5507026de6827d543ca
2019-12-03 15:20:49 -08:00