Commit Graph

40 Commits

Author SHA1 Message Date
Jordan Liggitt 20ceb8964c Revert "Revert "Merge pull request 107797 from tkashem/revert-107456""
This reverts commit b0b460921b81b260473d5c393d85beeb5a03e834.

Kubernetes-commit: 363a8beaab2d753413c5599a9ceb41c526bf7270
2022-03-25 14:34:41 -04:00
Patrick Ohly ba3b8e9322 enhance and fix log calls
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:

- Logging a message only above a certain verbosity threshold without
  recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
  this matters when using a logging backend which records the verbosity
  level.

- Passing a format string with parameters to a logging function that
  doesn't do string formatting.

All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.

In some cases it reports false positives, but those can be suppressed with
source code comments.

Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
2022-02-16 12:17:47 +01:00
Abu Kashem 11ce69f338 Revert "Merge pull request #107797 from tkashem/revert-107456"
This reverts commit 83ca74541216405323ddfb67f5f80ad5717da826, reversing
changes made to 1c216c6ec86e700170620fe4c75fa3a2a2817530.

Kubernetes-commit: b0b460921b81b260473d5c393d85beeb5a03e834
2022-02-27 13:40:28 -05:00
Abu Kashem d089edccf5 Revert "Merge pull request #107456 from tkashem/apf-ssa"
This reverts commit 6faa4f001008a5a29476f5722f66430c35f48229, reversing
changes made to 33a2c50bce334467640e016f68cf19e9382ba1a7.

Kubernetes-commit: 8fb33338635565f2f755a4557b94c26039c175d9
2022-01-26 13:54:26 -05:00
Abu Kashem ac4e99eacb apf: change controller to use SSA for patches
Kubernetes-commit: d9f4d6507b6a43342a3fe39756dc72b73be1fb6d
2022-01-10 14:15:42 -05:00
Patrick Ohly ec795ae204 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.

Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
2021-12-11 12:10:21 +01:00
Abu Kashem 1d83e4074a apf: ensure exempt request notes the classification
Kubernetes-commit: 8b2dd74c277d6a56a14e99830d39b23c5788c62e
2021-12-05 11:29:15 -05:00
Mike Spreitzer d9e2f76835 Correct Generator calls for executing seat count
Kubernetes-commit: 95964c5b351657820c4a8db0da0b772cb222fe99
2021-11-29 14:50:11 -05:00
Mike Spreitzer 4098be7694 Factored TimedObserver into less surprising pieces
Kubernetes-commit: ab64e852023965fd8873abcd50ff09cf79814d11
2021-11-15 14:59:30 -05:00
Mike Spreitzer 56b220f8cd Add metrics about watch counts seen by APF
Kubernetes-commit: 154bf6aab33c2486a9066f66ab3a056c1095cb9a
2021-10-25 03:31:47 -04:00
Mike Spreitzer 6a2631848c Add sample-and-watermark for seats occupied during all of execution
Kubernetes-commit: 945f960cfb8fc018b093c1a08e5d4cdd362b1fc6
2021-10-25 01:13:52 -04:00
Mike Spreitzer e417abf592 Migrate apiserver/pkg/util/flowcontrol to use k8s.io/utils/clock
.. instead of apimachinery/pkt/util/clock

Kubernetes-commit: 9f45c0f8c07cd0adfe38c887aa618d33b8a4ee1c
2021-09-17 15:14:42 -04:00
Abu Kashem db8aff032b apf: update apf logic to use v1beta2
Kubernetes-commit: 28f2b42a4116a9223113e8b152e02a4f1e602ff4
2021-08-16 17:53:57 -04:00
Mike Spreitzer 1efcc1c03a Refine locking in API Priority and Fairness config controller
Instead of a plain `Mutex`, use an `RWMutex` so that the common
operations can proceed in parallel.

Kubernetes-commit: 58927c1abede11ce7a8a74104328cf823df1b39e
2021-09-08 00:34:52 -04:00
Justin SB d5beafeb68 Avoid expensive go-cmp/cmp comparison
The cmp comparison is relatively expensive (#104821).  If we're not
going to log it, we shouldn't make the comparison.

Kubernetes-commit: f9f556dc7061df1dfc8c1628db983eeb97149317
2021-09-07 13:36:50 -04:00
Maciej Borsz 02f98184b3 Add APF's priorityLevel to httplog.go
Kubernetes-commit: 6db63869b0fd0551382dec1b30b649fda4289444
2021-08-13 13:00:07 +02:00
Manu Gupta b09300b3f0 fix unsafe json construction for digestConfigObjects.
fix unsafe json creation by creating intermediate objects
while creating patch bytes.

Kubernetes-commit: dfde50b18564cc9a8e99fdcba2f136d770ec6943
2021-08-01 18:43:40 -07:00
wojtekt d6e0cf5d71 Remove unused promise code from APF
Kubernetes-commit: a1cf44eab44f21c3929ff2d79501c56e6fbcddc3
2021-07-21 12:14:30 +02:00
wojtekt 9209ed5928 Get rid of unused flowSchemaRVs in P&F
Kubernetes-commit: c79a0a08828ef86e07941e96dafefb7d8bc51990
2021-07-13 15:16:17 +02:00
wojtekt b4c306e1e8 Rename width to workEstimate in P&F code
Kubernetes-commit: 73211256e8f15cf84ee69d6fe8258c3a912e0f94
2021-07-13 15:10:58 +02:00
wojtekt fbc127e994 Add watch tracker to APF for request cost estimation
Kubernetes-commit: cea1dcfeed2fc4e8ab89cd43e5a0e402251c8df5
2021-06-15 10:49:42 +02:00
Abu Kashem cf5c77fde9 apf: add additional latency into width
Kubernetes-commit: 24e19229101d242d924ce98a562be3864dde9eae
2021-06-27 12:45:24 -04:00
youhonglian 56e6dac0bb update github.com/pkg/errors to go native errors pkg in staging
Kubernetes-commit: 4b1402187474a1ebd0f71c824f45448fea160fc6
2021-06-22 11:54:56 +08:00
Abu Kashem 3c7f54740f apf: add plumbing to estimate width" of a request
- add plumbing that allows us to estimated "width" of a request
- the default implementation returns 1 as the "width" of all
  incoming requests, this is in keeping with the current behavior.

Kubernetes-commit: 9b72eb1929a64b9d5a5234090a631ba312fb4d41
2021-05-11 07:03:05 -04:00
David Eads 4c81a6672b Enable, rate limit, and test APF controller fights
Using real time.

Kubernetes-commit: 80ff06fe846c5e565e0cbd4b70f5f1e8a4ea7295
2020-12-02 16:50:05 -05:00
Abu Kashem 84ce907827 fix apf controller unit test
- don't expose the internal states of the apf controller to the caller
- return a boolean, instead of the priority level states

Kubernetes-commit: f20c6cb2d9060920cae9ff5cade1739c7e0b7f7a
2021-01-13 11:06:17 -05:00
Mike Spreitzer 41788f53e2 Relax apf_controller's reaction to certain errors
When the error is due to the object having been deleted, the
controller does not need to do anything before the coming
notification.

Kubernetes-commit: ef1e2039b5fc7f955ec4f9c636a64aa403cba2ab
2020-12-17 10:46:14 -05:00
Abu Kashem 59a718ed18 fix the deadlock in priority and fairness config controller
Kubernetes-commit: 7114319b3e66e331250c116ac926f1a57352072f
2020-12-15 12:19:56 -05:00
Mike Spreitzer a011399e98 Define TestableConfig in k/apiserver/pkg/util/flowcontrol
Collect the parameters of newTestableController into a named type.

Also tolerate the surprising situation in which a request's user
groups include neither `system:authenticated` nor
`system:unauthenticated` --- because this is observed to happen in
some tests.

Also a few other minor fixups.

Kubernetes-commit: 10df6d459b5bca7b42471f9409182417fbc3f253
2020-12-16 18:58:45 -05:00
Ken Sipe 0da065589a fix S1023 redundant return statement
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 1d795002919675b8cf3bf462b6543fef843b5b54
2020-06-25 17:29:42 -05:00
Adhityaa Chandrasekar 43cf7154be APF matching: fallback to catch-all if nothing matches
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 794a3ec0997caffa116c2e6f78feb53ebfca6052
2020-11-13 03:58:07 +00:00
yue9944882 a4a3fc9b87 APF: graduate API and types to beta
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 849be447f563fc93a27a0827fb1185b885b57114
2020-11-04 16:33:14 +08:00
Mike Spreitzer b1ede52e21 Make sampleAndWaterMarkHistograms not fall very far behind
Kubernetes-commit: 9e89b92a92c02cdd2c70c0f52a30936e9c3309c7
2020-08-20 16:43:11 -04:00
Mike Spreitzer e28ab56bd4 Introduce more metrics on concurrency
Introduce min, average, and standard deviation for the number of
executing mutating and readOnly requests.

Introduce min, max, average, and standard deviation for the number
waiting and number waiting per priority level.

Later:

Revised to use a series of windows

Use three individuals instead of array of powers

Later:

Add coarse queue count metrics, removed windowed avg and stddev

Add metrics for number of queued mutating and readOnly requests,
to complement metrics for number executing.

Later:

Removed windowed average and standard deviation because consumers can
derive such from integrals of consumer's chosen window.

Also replaced "requestKind" Prometheus label with "request_kind".

Later:

Revised to focus on sampling

Make the clock intrinsic to a TimedObserver

... so that the clock can be read while holding the observer's lock;
otherwise, forward progress is not guaranteed (and violations were
observed in testing).

Bug fixes and histogram buckets revision

SetX1 to 1 when queue length limit is zero, beause dividing by zero is nasty.

Remove obsolete argument in gen_test.go.

Add a bucket boundary at 0 for sample-and-water-mark histograms, to
distinguish zeroes from non-zeros.

This includes adding Integrator test.

Simplified test code.

More pervasively used "ctlr" instead of "ctl" as abbreviation for
"controller".

Kubernetes-commit: 57ecea22296797a93b0157169db0ff2e477f58d0
2020-05-17 01:02:25 -04:00
yue9944882 7f1f8c2578 introduce apf debug endpoint
Kubernetes-commit: 567becd5eedd9a1f3802f2b3b0b6b8efc445d2a1
2020-05-22 13:02:10 +08:00
Davanum Srinivas 5879417a28 switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 442a69c3bdf6fe8e525b05887e57d89db1e2f3a5
2020-04-17 15:25:06 -04:00
Mike Spreitzer 6ae3e470a2 Make some metrics finer-grained, add dispatch counts, note immediate reject
Also add testing of metrics for queuesets.

Kubernetes-commit: f535a9c9ed4b6a0def47c354acad0ac2a8f961b0
2020-03-01 20:22:58 -05:00
Ted Yu f2eace2821 Do not dereference qcAPI which maybe nil
Kubernetes-commit: 583f804652ffff898437c80ce357e4f0c379bfb9
2020-02-14 13:30:40 -08:00
Mike Spreitzer 1ef67fe134 Version the API Priority and Fairness FieldManager values
Kubernetes-commit: b142868eaf46b24fce0e6b01e06087c924351857
2020-02-14 13:50:11 -05:00
Mike Spreitzer 1c79014daa Added API Priority and Fairness filter and config consumer
Kubernetes-commit: 73614ddd4e42728a36c7ac6b7b20f27c8032cafb
2020-01-25 19:47:12 -05:00