Commit Graph

420 Commits

Author SHA1 Message Date
Jordan Liggitt 5f8feb467c Tolerate additional error messages in TLS unit tests
Kubernetes-commit: cff4eeef9f1880b42b8c3d3b8f3a27a89540dbe0
2022-03-29 08:57:31 -04:00
Stanislav Laznicka 034ebc2d9b x509metrics: rename 'missing_san*.go' files to match their new purpose better
Kubernetes-commit: 5479306b1326bcbf29ae2ca0694a36555ed1ce35
2022-03-25 17:03:13 +01:00
Stanislav Laznicka cd3019f777 webhooks, aggregation: add metrics to count certs with SHA1 signatures
Extends the certificate attribute deprecation RoundTrippers wrapper with
a checker that counts certificates with sha-1 signatures in server responses.

Non-root non-self-signed SHA-1 certificate signatures were deprecated in
Golang 1.18.

Kubernetes-commit: 499ee65a9b0cd45d41716b513fae0d537f7f9c88
2022-03-24 01:08:02 -04:00
Stanislav Laznicka a0f7d7ad7c webhooks: regenerate certificates for additional sha-1 signatures
Kubernetes-commit: b2c6de170b0b880ccca0ae783bae09559f8dff04
2022-03-25 16:52:40 +01:00
Stanislav Laznicka d98bb60d70 webhooks: gencerts.sh: add intermediate and server certs with sha-1 signatures
Kubernetes-commit: fee08cc580e5147d11f3f12a37452f8a86cee3b8
2022-03-25 16:52:07 +01:00
Jordan Liggitt 20ceb8964c Revert "Revert "Merge pull request 107797 from tkashem/revert-107456""
This reverts commit b0b460921b81b260473d5c393d85beeb5a03e834.

Kubernetes-commit: 363a8beaab2d753413c5599a9ceb41c526bf7270
2022-03-25 14:34:41 -04:00
Jiahui Feng d3152849fc add test case for enabled OpenAPIEnums
Kubernetes-commit: 1f493f6f91d0ead75eabd14874900fe6e9ae15f0
2022-03-22 17:58:16 -07:00
Patrick Ohly ba3b8e9322 enhance and fix log calls
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:

- Logging a message only above a certain verbosity threshold without
  recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
  this matters when using a logging backend which records the verbosity
  level.

- Passing a format string with parameters to a logging function that
  doesn't do string formatting.

All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.

In some cases it reports false positives, but those can be suppressed with
source code comments.

Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
2022-02-16 12:17:47 +01:00
Abu Kashem 11ce69f338 Revert "Merge pull request #107797 from tkashem/revert-107456"
This reverts commit 83ca74541216405323ddfb67f5f80ad5717da826, reversing
changes made to 1c216c6ec86e700170620fe4c75fa3a2a2817530.

Kubernetes-commit: b0b460921b81b260473d5c393d85beeb5a03e834
2022-02-27 13:40:28 -05:00
Monis Khan 2d35015235 webhook: use rest.Config instead of kubeconfig file as input
This change updates the generic webhook logic to use a rest.Config
as its input instead of a kubeconfig file.  This exposes all of the
rest.Config knobs to the caller instead of the more limited set
available through the kubeconfig format.  This is useful when this
code is being used as a library outside of core Kubernetes. For
example, a downstream consumer may want to override the webhook's
internals such as its TLS configuration.

Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: fef7d0ef1e1fbff65e8d445256036704bb9dbcbd
2021-07-23 11:15:47 -04:00
Jefftree 9b81dbc7e0 googleapis/gnostic -> google/gnostic
Kubernetes-commit: 8a1d5947ad34ba275192341baa4e5fef8e6c7f24
2022-03-15 20:36:21 -07:00
Abu Kashem d089edccf5 Revert "Merge pull request #107456 from tkashem/apf-ssa"
This reverts commit 6faa4f001008a5a29476f5722f66430c35f48229, reversing
changes made to 33a2c50bce334467640e016f68cf19e9382ba1a7.

Kubernetes-commit: 8fb33338635565f2f755a4557b94c26039c175d9
2022-01-26 13:54:26 -05:00
Joe Julian 7cac225c86 remove unwanted values returned from dry-run
Remove the uid and the resourceVersion from dry-run results per kep 576

https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/576-dry-run/README.md#generated-values

Kubernetes-commit: 60c1d58d02c7374645c00281dda3fd656264e1c5
2021-12-16 12:44:02 -08:00
Abu Kashem ac4e99eacb apf: change controller to use SSA for patches
Kubernetes-commit: d9f4d6507b6a43342a3fe39756dc72b73be1fb6d
2022-01-10 14:15:42 -05:00
Abu Kashem 44e5395e0e apf: add metric to track dispatch with no accommodation
Kubernetes-commit: 30c0485e0cba3ec6b19e092e7e78059b3fd4f18c
2021-11-23 10:55:31 -05:00
Patrick Ohly ec795ae204 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.

Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
2021-12-11 12:10:21 +01:00
erwinvaneyk 23f238960a Improve the error message when trying to proxy unsupported service type
Kubernetes-commit: 342ef6234d4c7acf3ffca0f01831de316fb7ac11
2021-11-17 12:24:49 +01:00
Wojciech Tyczyński bfcf29ce7a P&F: enable mutating work estimator
Kubernetes-commit: 8f5ece0548b59b1f8706f79af814d9f7700acb5c
2021-11-02 12:54:29 +01:00
Mike Spreitzer 6adfddf535 Clarify APF metric wrt all three stages of execution
Kubernetes-commit: 88f8e8448bf873cf41035cb858422a10a1d03018
2021-11-30 11:45:53 -05:00
Abu Kashem 6bd59a523a apf: add a metric to count seat samples
Kubernetes-commit: bb15bdf15c1cc4d5a4380f3f6ed46d4adc9662a1
2021-11-23 11:36:09 -05:00
Abu Kashem 1d83e4074a apf: ensure exempt request notes the classification
Kubernetes-commit: 8b2dd74c277d6a56a14e99830d39b23c5788c62e
2021-12-05 11:29:15 -05:00
Davanum Srinivas 56a3a30ae1 Check in OWNERS modified by update-yamlfmt.sh
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 9405e9b55ebcd461f161859a698b949ea3bde31d
2021-12-09 21:31:26 -05:00
Abu Kashem 40993f6319 apf: add new label for request_execution_seconds metric
Kubernetes-commit: 54439e934371a3018f49e629cdc68f0944e08af0
2021-10-06 11:55:12 -04:00
Mike Spreitzer d9e2f76835 Correct Generator calls for executing seat count
Kubernetes-commit: 95964c5b351657820c4a8db0da0b772cb222fe99
2021-11-29 14:50:11 -05:00
Jiahui Feng 6dbeb046f3 prune enum when OpenAPIEnums is disabled.
Kubernetes-commit: 9c05de2f31582b47da05d9dd3f9bbf89588bea82
2021-11-09 13:00:00 -08:00
Mike Spreitzer 8c46c2c1d9 Fix sample_and_watermark_test.go for bad luck, repeated test
Kubernetes-commit: 06e17165b16a4706ecf86b089de2bc1c0faea264
2021-11-10 15:30:50 -05:00
Mike Spreitzer 4098be7694 Factored TimedObserver into less surprising pieces
Kubernetes-commit: ab64e852023965fd8873abcd50ff09cf79814d11
2021-11-15 14:59:30 -05:00
Mike Spreitzer 56b220f8cd Add metrics about watch counts seen by APF
Kubernetes-commit: 154bf6aab33c2486a9066f66ab3a056c1095cb9a
2021-10-25 03:31:47 -04:00
Mike Spreitzer 6a2631848c Add sample-and-watermark for seats occupied during all of execution
Kubernetes-commit: 945f960cfb8fc018b093c1a08e5d4cdd362b1fc6
2021-10-25 01:13:52 -04:00
Wojciech Tyczyński 64f47d35c4 P&F fix watch tracker bug
Kubernetes-commit: b1d0fde370ae409ff028c557c6a60ff69491e499
2021-11-03 15:02:51 +01:00
Wojciech Tyczyński 44154d94e6 Enable support for indexes in WatchTracker
Kubernetes-commit: 21ec77dc19047215094a538b717e24035ca000ab
2021-10-28 14:57:47 +02:00
Wojciech Tyczyński 9ad8b586fc P&F: Update WatchTracker interface to pass more information
Kubernetes-commit: 12746f4bc15458d585ffd4c6e9d6066810e27361
2021-10-28 12:41:41 +02:00
Wojciech Tyczyński bdc01eb8e6 P&F: update mutating request estimation
Kubernetes-commit: 4700cf622bfcf5c2b159ddc4e37d7f51d1c41e68
2021-10-27 10:25:38 +02:00
Wojciech Tyczyński 55b43e446f P&F: move seat-seconds to a better location
Kubernetes-commit: e262db7a4daf5218520e49b423789ea55a94af75
2021-10-27 10:30:25 +02:00
Wojciech Tyczyński 9c98ce48a5 P&F: clean up mutating work estimator tests
Kubernetes-commit: 943bc38c0e0076935701d32067d5a2d8d512be8a
2021-10-27 10:05:13 +02:00
Abu Kashem 3f529d0551 apf: call metrics.AddReject for decisionCancel
Kubernetes-commit: f6dcf17a10dfd2cb5ce0ea7923723547c92c2e55
2021-10-18 13:08:56 -04:00
Lukasz Szaszkiewicz 3af709f483 aggregator: pass apiServiceRegistrationControllerInitiated signal directly to apiserviceRegistration controller
Kubernetes-commit: 5116a508a7bf84844f4987ab2db14af88bfd296f
2021-10-20 11:06:27 +02:00
Lukasz Szaszkiewicz 67be998d0f rename to muxAndDiscoveryComplete
Kubernetes-commit: 9e2bdfee02a6851fbb13ffe28611e9d2b6242785
2021-10-19 12:24:00 +02:00
Lukasz Szaszkiewicz 1fc498576a apiserver: indroduces NotFoundHanlder
The new handler is meant to be executed at the end of the delegation chain.
It simply checks if the request have been made before the server has installed all known HTTP paths.
In that case it returns a 503 response otherwise it returns a 404.

We don't want to add additional checks to the readyz path as it might prevent fixing bricked clusters.
This specific handler is meant to "protect" requests that arrive before the paths and handlers are fully initialized.

Kubernetes-commit: 53867975e72c7a2d2dd94aac6bd2869411f92094
2021-10-15 17:37:16 +02:00
Mike Spreitzer 5283383fb5 Clarify metrics help wrt APF execution phases
Kubernetes-commit: d7a3bf0d260a0c291941cda68492f10e5010ac91
2021-10-24 22:32:13 -04:00
Ben Luddy 7324607749 Free APF seats for watches handled by an aggregated apiserver.
Kubernetes-commit: 1873915be6be40df20fe35a8e0c7e8e0a620111a
2021-10-06 10:16:46 -04:00
Mike Spreitzer 5ab6f3fe6b Remove presumptions about what decision has been made
Kubernetes-commit: e70999becd98aa00fd00bc622ffdca0476ec7340
2021-10-18 02:10:21 -06:00
Abu Kashem b40e786ba3 apf: return nil for a request that has been removed from queue
Kubernetes-commit: cd06ba502cf85603242008cc9f61234790ede6de
2021-10-18 12:46:54 -04:00
Mike Spreitzer c5a0365136 Fix nits noticed in recent code review
Kubernetes-commit: 1844a052776bce33322ce20c11b2902403655ef8
2021-10-18 23:51:48 -05:00
Abu Kashem fb57181f8d apf: include seat information in request dump
Kubernetes-commit: 8e33a3b2cc97813724b64be36fe977290db3053a
2021-10-14 09:27:54 -04:00
Abu Kashem 20ef863037 apf: include seats in use in queue dump
Kubernetes-commit: edf657a33a70ff942d9a0695f602bed13e5e9628
2021-10-13 10:39:37 -04:00
Abu Kashem 88992ba87a apf: change QueueDump to use SeatSeconds
Kubernetes-commit: 3ef5752edf5b3f7076041368b1c1b5396ee534d7
2021-10-13 10:22:41 -04:00
Abu Kashem e4ae91b4a2 apf: include queue sum stats in debug
Kubernetes-commit: 87c7401eb853678da804ff503e266daeef5ad126
2021-10-13 10:14:30 -04:00
wojtekt f578e7708d Disable P&F for watch requests
Kubernetes-commit: 763e6d1dbb6371c66b62e4ea4c1fcbee1c113bda
2021-10-13 07:43:05 +02:00
Mike Spreitzer d69d77c659 Update queueset_test.go for FinalSeats
Track the introduction of FinalSeats.

Give up on calculating expected results for tests with added latency,
because I did not find an easy and obvious way to do it.

Kubernetes-commit: 0fc595e03360ba7fc4c3e251d4b41f39172aca72
2021-10-08 22:27:39 -07:00
wojtekt c3ef02ad27 Adjust final seats if they don't fit the limit
Kubernetes-commit: c5a77d8a761b0651b53864f9e396d6f23efd01d2
2021-10-08 11:14:11 +02:00
wojtekt c18ab3e1b1 Estimate width of the request based on watchers count in P&F
Kubernetes-commit: 223f9be59778b6ec2e44fd57df523f00e246bd95
2021-07-07 10:48:29 +02:00
Mike Spreitzer 56c6fd034b Unconfused logging wrt additional latency
Fixed confusion in queueSet logging wrt seats and additional latency.

Kubernetes-commit: 42f698daad73d9c010027bb60fff9728168c71da
2021-10-11 13:21:46 -07:00
Mike Spreitzer f7bfb170d7 Keep the progress meter R from overflowing
Also add test for that situation.

Kubernetes-commit: a797fbd96de8c67aaed58aef54fbe9f0eb94a2c2
2021-10-01 22:04:05 -07:00
Mike Spreitzer 487fea8e61 Update log messages in finishRequestLocked
Make them clearer and consistent.

Kubernetes-commit: 3906e187a685cebddef85226950fc36f65e8ddb4
2021-10-09 23:48:41 -07:00
Mike Spreitzer 1b1389676f Relax TestDifferentWidths
Make the margin a little wider because flakiness was reported.

Kubernetes-commit: 10326282f9d1abcd4a45b737628286d40971efea
2021-10-07 16:09:53 -07:00
Mike Spreitzer a5192405d9 Calculate the work in each request just once
Kubernetes-commit: f2c46c8f9d0b360cf913e22c222d9954b4ff9a76
2021-10-07 17:20:56 -07:00
Abu Kashem 9560ec6e92 introduce final seats for work estimate
Kubernetes-commit: 3d6cc118fee15313419bf7aa0082a2a608ec62f6
2021-09-24 15:18:27 -04:00
Mike Spreitzer dc449969cc Use SeatSeconds
Kubernetes-commit: 4b5e1398199282f471d0f332eefeb5c2415bdb01
2021-10-01 15:33:37 -07:00
Mike Spreitzer 0cb46ec2f7 Draft datatype for seat-seconds
Kubernetes-commit: d866f9445831687ab3254d754b13a4acf271882f
2021-10-01 13:29:35 -07:00
Abu Kashem 863c48fbc2 apf: rename WorkEstimate.Seats to InitialSeats
Kubernetes-commit: 5d67896adedbce27f01b59eb5f2054919a047f2b
2021-09-24 09:41:38 -04:00
Mike Spreitzer 72ff8a6261 Improve queueset sharding and dispatching
New anti-windup technique: use the request arrival time as the floor
on the virtual dispatch time.  Prevent bound violations where they
might arise rather than fixing up just one queue at dispatch time,
so that the fixed up dispatch times figure into the dispatching choice.

Two tweaks to the shuffle sharding.  Take seats of executing requests
into account as well as seats of waiting requests.  Do not always
consider the generated hand in the same order.

Rename the queueset methods that do shuffle sharding and finding the
queue to dispatch from, because the old names were confusingly
similar.

Tighten up some request margins.

Name the test cases in TestNoRestraint and TestWindup.

Kubernetes-commit: 4b9cba85874158b25b5c994773a4ec04343820c2
2021-09-20 15:45:24 -04:00
Mike Spreitzer 8d3036922c More test tweaks
Canonicalize listing of test cases.

Make TestNoRestraint try both cases: competition and none.

Kubernetes-commit: 0ee1a7b4ff9012b050bd447055ad5e1e8c57c30e
2021-09-20 15:45:24 -04:00
Mike Spreitzer c505aa64af Update TestNoRestraint and TestWindup
Make TestNoRestraint verify that fairness is NOT achieved
when there is real competition.

Make TestWindup run two cases, to show that 0.1 is too narrow
a margin and 0.26 is wide enough.

Kubernetes-commit: c4945fdf0c14ba2032a5c8edf192678d9fe00374
2021-09-17 01:40:16 -04:00
Mike Spreitzer e417abf592 Migrate apiserver/pkg/util/flowcontrol to use k8s.io/utils/clock
.. instead of apimachinery/pkt/util/clock

Kubernetes-commit: 9f45c0f8c07cd0adfe38c887aa618d33b8a4ee1c
2021-09-17 15:14:42 -04:00
Mike Spreitzer f17b9e7b4b Track recent reorg in testing clockery
Kubernetes-commit: 20ccd735f6a55fdd9ecbe8cddb889310f0bc033b
2021-09-13 16:15:28 -04:00
Abu Kashem db8aff032b apf: update apf logic to use v1beta2
Kubernetes-commit: 28f2b42a4116a9223113e8b152e02a4f1e602ff4
2021-08-16 17:53:57 -04:00
Abu Kashem 7964b85d9c apf: update e2e test to use v1beta2
Kubernetes-commit: d99497ef6d5581b787e7aebfca943697ed8fe164
2021-08-16 17:12:42 -04:00
Mike Spreitzer de042674ed Widen margins of TestDifferentWidths and TestTooWide
These behavioral unit tests of queueset were failing because the
evaluation criteria were too strict.

Kubernetes-commit: 59d319ec06bb33289a87036418b4a61ed3bb215f
2021-09-09 17:07:58 -04:00
Mike Spreitzer de227d1d37 Change execution duration guess from 1 minute to 3 milliseconds
So that the width estimate has some effect but not a grossly excessive
one.

Added the fifo::Peek method to simplify the fifo client code.

Also renamed the queueSet::estimatedServiceTime field to
estimatedServiceSeconds to make the units clear.

Kubernetes-commit: a0c161f2f6908ee424ea888ff40f75ff071bd20a
2021-09-07 00:46:50 -04:00
Mike Spreitzer 1efcc1c03a Refine locking in API Priority and Fairness config controller
Instead of a plain `Mutex`, use an `RWMutex` so that the common
operations can proceed in parallel.

Kubernetes-commit: 58927c1abede11ce7a8a74104328cf823df1b39e
2021-09-08 00:34:52 -04:00
Justin SB d5beafeb68 Avoid expensive go-cmp/cmp comparison
The cmp comparison is relatively expensive (#104821).  If we're not
going to log it, we shouldn't make the comparison.

Kubernetes-commit: f9f556dc7061df1dfc8c1628db983eeb97149317
2021-09-07 13:36:50 -04:00
Mike Spreitzer d6c30c1553 Adjust LIST work estimator to match current code
Kubernetes-commit: 6f160ca11b0ce9e9fda9571e37eec2de1b97bcc5
2021-08-25 23:22:13 -04:00
Mike Spreitzer cc873bafd2 Increase debug logging in waitGroupCounter::Add
Now tries a little harder to get to meaningful stack frames.
This revision seems to strike a balance that is useful in the queueset tests.

Kubernetes-commit: b56dd725032cb7a14aa27e4c50c1c9d7c6d23eb1
2021-08-06 14:32:19 -04:00
Maciej Borsz 02f98184b3 Add APF's priorityLevel to httplog.go
Kubernetes-commit: 6db63869b0fd0551382dec1b30b649fda4289444
2021-08-13 13:00:07 +02:00
Mike Spreitzer 7d5430cfba Fix extra latency and add tests for that and width
Added missing dispatching after delayed release of seats.

Updated logging for all six situations of execution completion and
seat release.

Added behavioral tests for non-zero extra latency and non-unit width.

Also added two tests for baseline functionality.

Also improved some comments and other logging in `queueset.go`.

Kubernetes-commit: d2a27a58f0af20c6185fa1c21890d666e9d3746b
2021-08-12 16:48:02 -04:00
Jordan Liggitt bbc4211158 Update unit tests to handle go1.17 certificate parsing error messages
Kubernetes-commit: 0e925f266f78d72ed05a5a2fe48faa5e2a87d107
2021-08-17 11:24:03 -04:00
Ryan Moriarty 0741f109f6 Add a new webhook metric tracking request totals.
Also add a 1.0s bucket boundary to the webhook latency metric.

Kubernetes-commit: 8ed1628a6e75f4029853502dbac44fdb0edac5fc
2021-06-22 22:32:47 +00:00
Abu Kashem da50ca4c6e apf: free seats in use after additional latency
Kubernetes-commit: d68186452d9150b113489e6a722caf82f898857f
2021-06-27 13:04:20 -04:00
Mike Spreitzer 8c2108bc80 Refactor goroutine counting
Add comment outlining TestContextCancel.

Stop calling `t.Errorf` from wrong goroutine.

Package up queueNoteFn expectation checking.

Add counting of goroutine in req1 exec fn.

Remove unnecessary assignment to `_`.

Make TestContextCancel wait on fake clock, to insulate timing check
from scheduler noise.

Factor goroutine counting out of queueset.go, into queueset_test.go,
where it matters.

Refactor promise: Use a simple channel-based implementation for normal
code, a mutex-based one for testing code.

Took all the panics out of queueset.go

Shrink the timeouts in promise tests to 1 second.

Kubernetes-commit: 1db36ae3b30e30d70972998a22987a7db470479b
2021-07-29 00:35:25 -04:00
Mike Spreitzer 904cd74454 Some cleanup of the package for event clocks
Rename from `clock` to `eventclock`.

Simplify by removing the prohibition on an EventFunc suspending and
resuming activity.

Remove "EventClock" from names to avoid stuttering.

Start to consolidate test code under fairqueuing/testing/.

Kubernetes-commit: 80ca6a4ae6ff571c32962a7155efd55edefff9e6
2021-08-06 02:06:43 -04:00
Abu Kashem f013b63777 apf: use EventClock rather than a PassiveClock for queueset
Kubernetes-commit: c2a3b793d3ec62e781dd20704370d09f7e1be706
2021-07-21 17:06:48 -04:00
Abu Kashem bd2ef9810a apf: estimate list width
Kubernetes-commit: 296c18ec323328bf1c6a621a9ca3a094d78bb9d0
2021-06-22 19:38:00 -04:00
Manu Gupta b09300b3f0 fix unsafe json construction for digestConfigObjects.
fix unsafe json creation by creating intermediate objects
while creating patch bytes.

Kubernetes-commit: dfde50b18564cc9a8e99fdcba2f136d770ec6943
2021-08-01 18:43:40 -07:00
Mike Spreitzer 0c550377cf Introduce event clocks based on k8s.io/utils/clock
So we can move off of the apimachinery clock package.

Switch queueset to new clocks.

Removed event clocks based on apimachinery clocks,
because this PR introduces ones based on k8s.io/utils/clock .

Removed interface that is implemented by only one interesting type.

Simplify RealEventClock::EventAfterTime.

Kubernetes-commit: dcb298c9552de44e27ed52f5e2b58a0dd7cd8d54
2021-07-21 16:56:11 -04:00
Abu Kashem feb4eefe1c apiserver: add callback to get notified of object count
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
wojtekt 719fda2a8b Simplify APF promise to what is really used in the code
Kubernetes-commit: 9f735e71bbb7d0dde67a718891641d8afd20a8bc
2021-07-23 13:30:34 +02:00
wojtekt d6e0cf5d71 Remove unused promise code from APF
Kubernetes-commit: a1cf44eab44f21c3929ff2d79501c56e6fbcddc3
2021-07-21 12:14:30 +02:00
wojtekt 9209ed5928 Get rid of unused flowSchemaRVs in P&F
Kubernetes-commit: c79a0a08828ef86e07941e96dafefb7d8bc51990
2021-07-13 15:16:17 +02:00
wojtekt b4c306e1e8 Rename width to workEstimate in P&F code
Kubernetes-commit: 73211256e8f15cf84ee69d6fe8258c3a912e0f94
2021-07-13 15:10:58 +02:00
wojtekt 242c8c8a1c Optimize APF support for watch initialization to fix the pod startup time regression.
Kubernetes-commit: ef435b85b47618d0d463ac93e9758d75c2998681
2021-07-13 10:22:30 +02:00
David Ashpole 785c5626f9 add tracing to webhook requests
Kubernetes-commit: 9dd59017c4c218135b12949a2bd3af07e892199a
2021-07-08 20:13:33 -07:00
Abu Kashem b206f9ea02 apf: fix virtual clock
Kubernetes-commit: a570008cbd6c2142a2f270570fe49bf982825c19
2021-07-06 19:29:16 -04:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
wojtekt 9606daa5ef Refinements to pick queue logic in P&F
Kubernetes-commit: 7f1c4977d7ad9fdfd44549ee8ffe289127be4d81
2021-07-07 08:58:49 +02:00
Davanum Srinivas fe1610f3fe switch from golang-lru to the one in k8s.io/utils
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 79d0c6cdc10293c9bfe644ce31dc186a936579b0
2021-07-07 13:45:07 -04:00
wojtekt fbc127e994 Add watch tracker to APF for request cost estimation
Kubernetes-commit: cea1dcfeed2fc4e8ab89cd43e5a0e402251c8df5
2021-06-15 10:49:42 +02:00
wojtekt a35bb85f09 Update the logic to pick the best queue in P&F
Kubernetes-commit: 0ecc7ba311ab33b16c5d907ebb1120e3e51a947d
2021-06-17 09:22:00 +02:00
Abu Kashem cf5c77fde9 apf: add additional latency into width
Kubernetes-commit: 24e19229101d242d924ce98a562be3864dde9eae
2021-06-27 12:45:24 -04:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Abu Kashem e9708ba84d apf: calculation of dR/dt should use seats in use
Kubernetes-commit: 6203d4fc6af36b1fd60177e8a88b8d4d45acb029
2021-06-21 05:38:07 -04:00
Mike Spreitzer 4dcb848926 Track generalization for width
Kubernetes-commit: 3930629e130f1c98821f8033941870ab582db284
2021-06-18 01:37:22 -04:00
Mike Spreitzer d28ccb4224 Add APF metrics about R(t)
Kubernetes-commit: 676f0450ed37eeec92b67246719cc46e7567e512
2021-06-14 16:48:27 -04:00
youhonglian 56e6dac0bb update github.com/pkg/errors to go native errors pkg in staging
Kubernetes-commit: 4b1402187474a1ebd0f71c824f45448fea160fc6
2021-06-22 11:54:56 +08:00
Abu Kashem e1aec4ecae apf: take seats into account when dispatching request
Kubernetes-commit: ff716cef508f948b50e1026e980e6df5ee475538
2021-06-14 12:19:06 -04:00
Abu Kashem abdd901971 apf: always include seats in virtual time
Kubernetes-commit: 2f7456076e0cb29a95d86cd0f54c34a04b4722ab
2021-06-14 09:52:00 -04:00
Abu Kashem 345d1c6ff9 apf: add a gauge for the number of seats currently in use
Kubernetes-commit: c710f99ef730a791a6911e63cc3b9d26cced6bd3
2021-06-10 17:34:50 -04:00
Abu Kashem 3c7f54740f apf: add plumbing to estimate width" of a request
- add plumbing that allows us to estimated "width" of a request
- the default implementation returns 1 as the "width" of all
  incoming requests, this is in keeping with the current behavior.

Kubernetes-commit: 9b72eb1929a64b9d5a5234090a631ba312fb4d41
2021-05-11 07:03:05 -04:00
wojtekt 5290a24345 Implement support for watch initialization in P&F
Kubernetes-commit: 0cc217647ca8be0820973b970124a072c27b6575
2021-05-07 12:49:06 +02:00
Abu Kashem ec22c8bdd8 apf: add "width" for request
all requests have a width of 1 to maintain current behavior.

Kubernetes-commit: b50507d98bd12503592ea62d2be2aadef49bdf70
2021-05-11 07:03:05 -04:00
Monis Khan 5289a15a1a client-go transport: assert that final CA data is valid
Signed-off-by: Monis Khan <mok@vmware.com>

Kubernetes-commit: 440ea3ef49e0ac77353ceeaebc2aad6c995d5b35
2021-05-03 10:11:54 -04:00
Stanislav Laznicka 7eb7056b38 webhooks,aggregation: add metrics to count certs missing SAN
Adds counters to generic webhook code and to the kube aggregator
so that it is possible to effectively measure the impact of
Golang 1.15's deprecation of x509 cert CN hostname
verification.

Kubernetes-commit: 9d6a19efff05a40081337a2fcb3dce6331b04022
2020-10-06 13:02:52 +02:00
Abu Kashem 4e12e0cd34 apf: use a list instead of slice for queueset
Kubernetes-commit: 69f9bc181f155ded7c5d5cc0ca9f026a6b42f431
2021-03-29 11:31:12 -04:00
Gautier Delorme af9424d2c9 remove go-openapi/spec
Signed-off-by: Gautier Delorme <gautier.delorme@gmail.com>

Kubernetes-commit: 34b0fcef5fc47e3fcddf7f6ca1b3e6176b2a5323
2021-04-20 17:48:33 +02:00
John Howard d7e2a0bbee Update kube-openapi and gnostic dependencies
Pulling in https://github.com/kubernetes/kube-openapi/pull/220

Kubernetes-commit: a849c8998c1ed71f25387a560f8359596aec8bd3
2021-03-23 08:33:15 -07:00
Abu Kashem 7c07064678 apf: fix test flake
Kubernetes-commit: fa0952ee778aaf7df3e593305575f5d121b55cf1
2021-03-29 16:09:49 -04:00
David Eads 4c81a6672b Enable, rate limit, and test APF controller fights
Using real time.

Kubernetes-commit: 80ff06fe846c5e565e0cbd4b70f5f1e8a4ea7295
2020-12-02 16:50:05 -05:00
yoyinzyc 1a8abfc56f add context to metrics in util/flowcontrol.
Kubernetes-commit: 57d0bc301a017c41d890baee0a3a287f448c664d
2020-12-16 17:08:43 -08:00
Adhityaa Chandrasekar ec9a97b26e flowcontrol: make TestConfigConsumer less time-consuming
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: c1f3e90969d38f5b8410b67f0640d5df0d098e6b
2021-02-02 11:20:18 +00:00
Mike Spreitzer b2c3c4f86c Tweak up TestSampler in response to review
Kubernetes-commit: 736a87a4ee516a3feca0a79a4c48806ad4916aa3
2021-01-24 23:38:48 -05:00
Mike Spreitzer 39f5df86ff Added comment explaining the new unit test
Kubernetes-commit: 42f8d4e28fc05f3532f5cf7d2294049e3314896a
2021-01-18 00:57:09 -05:00
Mike Spreitzer 60021e2fed Explain warnings in new test
Add info message that explains what warnings are deliberate and expected.

Kubernetes-commit: 194c22fed54ba89d985085412795a9850bedeaf3
2021-01-06 12:56:16 -05:00
Mike Spreitzer 71c9d22768 Add unit test for sample-and-watermark histograms
Kubernetes-commit: 5fe48be506fe2b946d4db9cd7976ad0c4d4e7e1a
2021-01-05 23:31:06 -05:00
pacoxu 2343af38bb fix all keps links 404 for kep folder migration
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: a10bdfed09e96f3c2703454caafa2c884914e064
2021-01-14 14:03:21 +08:00
Mike Spreitzer 5d4804815d Less demanding test cases in TestUniformDistribution
Also a bug fix in the evaluation.

Kubernetes-commit: 1f1e8e21d81c4ff75078adc163bf2872d1b4650e
2021-01-25 01:44:08 -05:00
Abu Kashem 84ce907827 fix apf controller unit test
- don't expose the internal states of the apf controller to the caller
- return a boolean, instead of the priority level states

Kubernetes-commit: f20c6cb2d9060920cae9ff5cade1739c7e0b7f7a
2021-01-13 11:06:17 -05:00
Mike Spreitzer 16288a4d61 move all variables in sampleAndWaterMarkHistograms::innerSet
to tiptoe around https://github.com/golang/go/issues/43570 for #97685

Kubernetes-commit: 611184aa59d0cd40466bc3bc4b40a3712a038171
2021-01-08 13:32:38 -05:00
Abu Kashem 3ba02b7f93 handle webhook authenticator and authorizer error
webhook.WithExponentialBackoff returns an error, and the priority is:
- A: if the last invocation of the webhook function returned an error
  that error should be returned, otherwise
- B: the error associated with the context if it has been canceled or
  it has expired, or the ErrWaitTimeout returned by the wait package
  once all retries have been exhausted.

caller should check the error returned by webhook.WithExponentialBackoff
to handle both A and B. Currently, we only handle A.

Kubernetes-commit: ae2b353fbf519b29d168c534f88c373fd67a1c31
2021-01-07 16:14:18 -05:00
Mike Spreitzer 41788f53e2 Relax apf_controller's reaction to certain errors
When the error is due to the object having been deleted, the
controller does not need to do anything before the coming
notification.

Kubernetes-commit: ef1e2039b5fc7f955ec4f9c636a64aa403cba2ab
2020-12-17 10:46:14 -05:00
Abu Kashem eea0d66fcd clean up executing request on panic
Kubernetes-commit: 13cedca0eb5337b13e5176983ea5e784ec38df22
2020-12-10 12:57:21 -05:00
Abu Kashem 59a718ed18 fix the deadlock in priority and fairness config controller
Kubernetes-commit: 7114319b3e66e331250c116ac926f1a57352072f
2020-12-15 12:19:56 -05:00
Mike Spreitzer a011399e98 Define TestableConfig in k/apiserver/pkg/util/flowcontrol
Collect the parameters of newTestableController into a named type.

Also tolerate the surprising situation in which a request's user
groups include neither `system:authenticated` nor
`system:unauthenticated` --- because this is observed to happen in
some tests.

Also a few other minor fixups.

Kubernetes-commit: 10df6d459b5bca7b42471f9409182417fbc3f253
2020-12-16 18:58:45 -05:00
Mike Spreitzer 2ceb0fcbe8 Create OWNERS for most of the API Priority and Fairness impl
Kubernetes-commit: 23de8061fbb63928c62aa6020b443a62491a0a1d
2020-12-11 17:54:59 -05:00
corehello b6f30b469b Fix typo for ServiceAccount
Kubernetes-commit: c20b2c8cd79f9e7dafa86fb4245dee9dd2d795c7
2020-12-02 01:38:46 +08:00
Ken Sipe 0da065589a fix S1023 redundant return statement
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 1d795002919675b8cf3bf462b6543fef843b5b54
2020-06-25 17:29:42 -05:00
Ken Sipe b60943f03a fix S1019 use of make
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: a4d05d51d95d30ebe90ec8f652c9a794caa44d53
2020-06-25 17:26:42 -05:00
Adhityaa Chandrasekar 43cf7154be APF matching: fallback to catch-all if nothing matches
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 794a3ec0997caffa116c2e6f78feb53ebfca6052
2020-11-13 03:58:07 +00:00
yue9944882 a4a3fc9b87 APF: graduate API and types to beta
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: 849be447f563fc93a27a0827fb1185b885b57114
2020-11-04 16:33:14 +08:00
Mike Spreitzer 70bbff23ac Fixed quantization and made monotic time reversal not panic
Changed quantization to use monotonic clock readings.

Stopping panicing when monotonic clock readings go backwards because
https://github.com/kubernetes/kubernetes/issues/96459 reported it
happening.

Re-organized sampleAndWaterMarkHistograms::innerSet to `defer` the
Unlock call, because that is the preferred pattern.

Kubernetes-commit: 0809220fd04d26892f0fcfd3bf54cc4baeda22da
2020-11-12 15:17:07 -05:00
Adhityaa Chandrasekar 8b21b5725d APF metrics: set StabilityLevel to ALPHA
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: b16f36b251ddbfef5f12fed58640de53512631f0
2020-11-05 15:35:39 +00:00
Adhityaa Chandrasekar ebe254b2e6 APF: use snake_case in metric labels
Signed-off-by: Adhityaa Chandrasekar <adtac@google.com>

Kubernetes-commit: f9d57a8d5db3e58f79a1b1958d80c049c63d6cde
2020-11-04 22:19:52 +00:00
yue9944882 5474822749 fixes max-min fairness
Kubernetes-commit: fd889ec8ae37437a9e75386542291bd0e2cc605e
2020-10-29 18:57:38 +08:00
Abu Kashem 5254108841 make backoff parameters configurable for webhook
Currently webhook retry backoff parameters are hard coded, we want
to have the ability to configure the backoff parameters for webhook
retry logic.

Kubernetes-commit: 53a1307f68ccf6c9ffd252eeea2b333e818c1103
2020-10-30 11:25:32 -04:00
Abu Kashem 67f1bb6022 Webhook: handle error when calling wait.ExponentialBackoff
- Inside WithExponentialBackoff function, handle error returned
by wait.ExponentialBackoff.
- Ensure that the wait time is bound to the given context.

Kubernetes-commit: f8e35de156f212b6989b465e608dd99b525bd5dc
2020-10-26 11:37:11 -04:00
Andrew Sy Kim 6746ccadda apiserver: support egress selection name 'controlplane' and deprecate 'master'
Signed-off-by: Andrew Sy Kim <kim.andrewsy@gmail.com>

Kubernetes-commit: a0aebf96ec2eef6517e2611335f0e6c9375dd807
2020-10-26 10:24:16 -04:00
Mike Spreitzer 86c572cf3f Remove phantoms from dump_requests output
No longer writes a single info-free line for each exempt priority level.

Kubernetes-commit: 6aeca5fb345256e104b9b2224a0f8797c1921214
2020-07-24 00:35:26 -04:00
Ken Sipe 4ad3e91efa changes in imports was unintentional
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 09cd56b561d0c31421dfc6374b68e0b18c6cc746
2020-06-28 15:26:40 -05:00
Ken Sipe 32533315c9 fix S1000 simplify ch switch cases
Signed-off-by: Ken Sipe <kensipe@gmail.com>

Kubernetes-commit: 268c2f81c7ab94cbab68a8d6c00725144b81fa09
2020-06-26 10:45:30 -05:00
Mike Spreitzer b1ede52e21 Make sampleAndWaterMarkHistograms not fall very far behind
Kubernetes-commit: 9e89b92a92c02cdd2c70c0f52a30936e9c3309c7
2020-08-20 16:43:11 -04:00
yue9944882 543449d042 removes empty columns and fixes request details
Kubernetes-commit: 40e3feb7f1b7f54a512785af7e28810c0874ac63
2020-07-23 14:25:11 +08:00