Commit Graph

363 Commits

Author SHA1 Message Date
Jordan Liggitt b0b89e7e65 Prune trailing newlines when enums are disabled
Kubernetes-commit: d0967bbe5e78dbcdd5bc189d5fc2becafc4be0f2
2023-01-06 00:58:23 -05:00
Wojciech Tyczyński 299c158ba3 Reuse generic List test for watchcache
Kubernetes-commit: ef8c4fbca8e5bed1e7edc162b95c412a7f1a758e
2022-11-04 22:01:37 +01:00
Jordan Liggitt c6b4e07ee4 Fix indentation/spacing in comments to render correctly in godoc
Kubernetes-commit: 78cb3862f11225135afdf76f3424e2d7b33104c7
2022-12-17 17:31:05 -05:00
Dario Tranchitella 0477735c3a fix(docs): wrong comment block for ResolveEndpoint function
Kubernetes-commit: 01c7559856f6c81d0deb7d042c06bee15be65527
2022-09-06 14:38:30 +02:00
Mike Spreitzer 770f2e1fa4 apiserver: finish implementation of borrowing in APF
Also make some design changes exposed in testing and review.

Do not remove the ambiguous old metric
`apiserver_flowcontrol_request_concurrency_limit` because reviewers
though it is too early.  This creates a problem, that metric can not
keep both of its old meanings.  I chose the configured concurrency
limit.

Testing has revealed a design flaw, which concerns the initialization
of the seat demand state tracking.  The current design in the KEP is
as follows.

> Adjustment is also done on configuration change … For a newly
> introduced priority level, we set HighSeatDemand, AvgSeatDemand, and
> SmoothSeatDemand to NominalCL-LendableSD/2 and StDevSeatDemand to
> zero.

But this does not work out well at server startup.  As part of its
construction, the APF controller does a configuration change with zero
objects read, to initialize its request-handling state.  As always,
the two mandatory priority levels are implicitly added whenever they
are not read.  So this initial reconfig has one non-exempt priority
level, the mandatory one called catch-all --- and it gets its
SmoothSeatDemand initialized to the whole server concurrency limit.
From there it decays slowly, as per the regular design.  So for a
fairly long time, it appears to have a high demand and competes
strongly with the other priority levels.  Its Target is higher than
all the others, once they start to show up.  It properly gets a low
NominalCL once other levels show up, which actually makes it compete
harder for borrowing: it has an exceptionally high Target and a rather
low NominalCL.

I have considered the following fix.  The idea is that the designed
initialization is not appropriate before all the default objects are
read.  So the fix is to have a mode bit in the controller.  In the
initial state, those seat demand tracking variables are set to zero.
Once the config-producing controller detects that all the default
objects are pre-existing, it flips the mode bit.  In the later mode,
the seat demand tracking variables are initialized as originally
designed.

However, that still gives preferential treatment to the default
PriorityLevelConfiguration objects, over any that may be added later.

So I have made a universal and simpler fix: always initialize those
seat demand tracking variables to zero.  Even if a lot of load shows
up quickly, remember that adjustments are frequent (every 10 sec) and
the very next one will fully respond to that load.

Also: revise logging logic, to log at numerically lower V level when
there is a change.

Also: bug fix in float64close.

Also, separate imports in some file

Co-authored-by: Han Kang <hankang@google.com>

Kubernetes-commit: feb42277884bc7cfbd6f0bb1d875cc63b1b6caac
2022-10-31 16:13:25 -07:00
Mike Spreitzer 2c528b2e56 apiserver: add solver for APF concurrency rebalancing
Kubernetes-commit: b2f69fb88772be64504e5c8e36aba4a0ffb4beef
2022-08-30 22:34:16 -04:00
Tim Hockin b304fee293 Clarify a few validation messages
Kubernetes-commit: 6d4241fe5dc7341e3207b8ab96513ab1ddeae8a2
2022-08-12 12:33:41 -07:00
Mike Spreitzer 084f1abd96 apiserver: define metrics for API Priority and Fairness borrowing
Kubernetes-commit: ba5ec78916ae5fe9e400a298da6879515029a12f
2022-10-31 15:09:39 -07:00
Mike Spreitzer 17134f6412 apiserver: remove redundant field from seatDemandStats
Kubernetes-commit: 4ad1c0f9cb8aa54efa127975aaa9d5bd3080e8d0
2022-11-03 10:41:50 -07:00
Marcel Zięba 6887520e10 Fix APF width estimate for creating service account's token
Kubernetes-commit: 2f7b4ca6851aa3d479c9af3c14a168b4974f2fee
2022-10-18 12:27:50 +00:00
Mike Spreitzer 413be63b46 Add instrumentation for seat borrowing
Kubernetes-commit: 9b684579e230f105bcaa743f06bc07c39af703df
2022-10-20 15:21:09 -04:00
Tim Allclair bd7c7f52c2 Consolidate AuditContext
Kubernetes-commit: f1d684b7b60b39b7dc1eb4156307c593f0ba74e1
2022-07-12 11:53:57 -07:00
Mike Spreitzer b6fd67c2a7 Introduce constants for the two parameters of seat borrowing
Kubernetes-commit: 8d826ee9f8e19f499f5c0a982bb256ae268a9010
2022-10-20 00:47:01 -04:00
Abu Kashem 538eaa3c92 apiserver: use SSA for apf configuration client
Kubernetes-commit: ecc640f1b40f14a894269e4b2ae6c80158626e93
2022-09-19 14:03:16 -04:00
Abu Kashem 4ecff81419 rename assuredConcurrencyShares for flowcontrol v1beta3
Kubernetes-commit: 66fc0d703794f309c9715028d3b63f64c281a5fd
2022-09-21 15:40:33 -04:00
Abu Kashem 98ffe5507d apiserver: update apf logic to use v1beta3
Kubernetes-commit: 0a99e6ebb1e241bf421f6df44b15a5a16063a9f2
2022-09-10 07:26:31 -04:00
Abu Kashem e888dae968 apiserver: update apf tests to use v1beta3
Kubernetes-commit: 6edc2516863994905b57f4386450f3e8f818bbc9
2022-09-10 07:05:20 -04:00
Monis Khan 25ccbfa75c Use https links for k8s KEPs, issues, PRs, etc
Signed-off-by: Monis Khan <mok@microsoft.com>

Kubernetes-commit: b738be9b46a899571303c8c887e32bf4d5b71a0a
2022-09-23 16:13:22 -04:00
Marcel Zięba 395682d7b7 Fix list estimator for lists that are executed as gets
Kubernetes-commit: 09f1ce9ec821c14013c775ba106e4888cb29c2c3
2022-09-19 07:42:45 +00:00
Mike Spreitzer 3419387b18 Call queueSet::boundNextDispatchLocked enough
Fix the one path where boundNextDispatchLocked was not being called
after modifying a queue.

Also check for negative work in a request.

These are motivated by
https://github.com/kubernetes/kubernetes/issues/112169 but I do not
have a way to reproduce it and so can not check that these changes
actually remove that symptom.  But these changes are good anyway.

Kubernetes-commit: 6ee93e2cee695203a6ce4935da1b9a807b624260
2022-09-01 22:54:53 -04:00
leilei.wan.cn 9f91290828 not reset when exempt pl panic
Kubernetes-commit: aef47c3c3b2e5a66b38f073ac05825f831eeb3fd
2022-08-18 19:13:47 +08:00
wanlei bb7342b40b fix(apf): not reset apf when panic
Kubernetes-commit: 22b0be9842640eabac961132a42239a9f22cdf71
2022-08-15 10:38:50 +08:00
David Eads 6a7c6a0940 Revert "Add an option to conditionally disable compression based on client ip."
This reverts commit 023583a15586328569ccab505db2f57f398e04b3.

Kubernetes-commit: 2f3ffbed2cffaaba63304318bc1d09b0144600ff
2022-08-17 15:08:39 -04:00
Sally O'Malley 4f9e133507 kubelet tracing
Signed-off-by: Sally O'Malley <somalley@redhat.com>
Co-authored-by: David Ashpole <dashpole@google.com>

Kubernetes-commit: 47e7d8034ff3be8e198dde6a671d05a11c30e333
2021-10-10 09:17:27 -04:00
Maciej Borsz 40280f9889 Add an option to conditionally disable compression based on client ip.
Kubernetes-commit: 023583a15586328569ccab505db2f57f398e04b3
2022-07-29 08:44:14 +00:00
jupblb 16f776a534 Switch initial/final seats type to uint64
Kubernetes-commit: 3c46482eb09d7343e0f98a930a9aaa158237e278
2022-07-28 10:48:40 +02:00
Davanum Srinivas 7e94033a61 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Davanum Srinivas a13e0b29d5 fix patch_test for gofmt issue
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: ea7dff551f648d8a46087f59e40767216d00f60e
2022-07-26 10:12:54 -04:00
Mike Spreitzer eb15930b31 Fix APF metric denominator problems
Co-authored-by: JUN YANG <yang.jun22@zte.com.cn>

Kubernetes-commit: fdd921cad0cd9308ec62c1b86c9c1cc5d12e5d21
2022-05-22 23:39:49 -04:00
Wojciech Tyczyński 8f7c120935 Eliminate MaintainObservations function in P&F
Kubernetes-commit: badf436ac4451590e5e84e537f2234e3632ea3b4
2021-11-25 12:44:50 +01:00
jupblb 738a050cda Introduce config for API Priority and Fairness
Linked all the default values with a single config structure.

Kubernetes-commit: 1c594e7e01a899807431c806cd11c1d27c885c9c
2022-07-20 11:33:45 +02:00
Mike Spreitzer 959fbf9f84 Use timing ratio histograms instead of sample-and-watermark histograms
Kubernetes-commit: 0c0b7ca49f9ade72b990bf3a6f568485586af8b4
2022-05-18 02:56:48 -04:00
Mike Spreitzer 32237d0a84 Add timing ratio histograms
Following is the result of benchmarking the two kinds of histograms on
my laptop.

% go test  -benchmem -run=^$ -bench Histogram .
goos: darwin
goarch: amd64
pkg: k8s.io/apiserver/pkg/util/flowcontrol/metrics
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkSampleAndWaterMarkHistogramsVecEltSafeEarly-16    	  980143	      1230 ns/op	       0 B/op	       0 allocs/op
BenchmarkSampleAndWaterMarkHistogramsVecEltSafeLate-16     	  932380	      1216 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimingRatioHistogram-16                           	12665247	        94.13 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimingRatioHistogramVecElementSimple-16           	11015806	       100.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimingRatioHistogramVecElementSafeEarly-16        	 7142589	       172.0 ns/op	       0 B/op	       0 allocs/op
BenchmarkTimingRatioHistogramVecElementSafeLate-16         	11487517	        96.67 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	k8s.io/apiserver/pkg/util/flowcontrol/metrics	7.718s

Kubernetes-commit: 8039f9d600c08450a81b60494b392a6aada0c14b
2022-05-18 02:30:06 -04:00
Mike Spreitzer 0796534fe5 Remove the PairVec types
Kubernetes-commit: 1f1cfba2a3fb35a8542bbf64a46746214355674c
2022-06-11 00:57:19 -04:00
Mike Spreitzer 0f5737dda8 Remove unhelpful pairing of members of read_vs_write_request_count_samples
Members are not used in (waiting,executing) pairs, so stopped
using the wrapper that adds such pairing.

Kubernetes-commit: cd33c7cf2260b351dd345497223a944e80bc7b61
2022-05-22 22:39:06 -04:00
Mike Spreitzer 10da00a048 Rename observer.go to interface.go
Kubernetes-commit: a4c18fa7fc3e370a8cfbbafc66bbdf4d1d834713
2022-05-23 16:11:05 -04:00
Mike Spreitzer cae328fb1c Give apf metrics abstractions more familiar names
The logic is similar to Prometheus gauges and vectors,
adopt that terminology.

Kubernetes-commit: 7d64a93a1407f91b5e13bf540a0fa834a41622eb
2022-05-17 23:27:47 -04:00
Mike Spreitzer 8628966894 Fix more initial numerators
Kubernetes-commit: ba690c2257af76bd971d0dfb6bef13ff9099e549
2022-05-18 00:22:30 -04:00
Wojciech Tyczyński f5d65d90e9 Avoid leaking StorageObjectCountTracker goroutine
Kubernetes-commit: 564b376812836fb1e77452d478ab16eee5101447
2022-05-15 09:57:26 +02:00
Maciej Borsz d1014d8c99 For each call, log apf_execution_time
Kubernetes-commit: 488a19184e847ed88e1cc81995fabfe1f88f5347
2022-03-29 14:55:52 +00:00
Jordan Liggitt 268935d657 Revert "remove unwanted values returned from dry-run"
This reverts commit 60c1d58d02c7374645c00281dda3fd656264e1c5.

Kubernetes-commit: adb76219198235bb069bb67e924455dceaaba7c4
2022-04-06 12:34:42 -04:00
Wojciech Tyczyński b6f111beb9 Fix the overestimated cost of deletaged API requests in P&F
Kubernetes-commit: 14a4fd385353b914265acb233fa8d2a426af7f90
2022-03-31 09:47:41 +02:00
Jordan Liggitt 689a39c38a Add audit annotations and log prefixes to deprecated cert warnings
Kubernetes-commit: f1dd9a611415120af5158f3feb0a0ef6a5c6d7d1
2022-03-29 10:41:40 -04:00
Jordan Liggitt 5f8feb467c Tolerate additional error messages in TLS unit tests
Kubernetes-commit: cff4eeef9f1880b42b8c3d3b8f3a27a89540dbe0
2022-03-29 08:57:31 -04:00
Stanislav Laznicka 034ebc2d9b x509metrics: rename 'missing_san*.go' files to match their new purpose better
Kubernetes-commit: 5479306b1326bcbf29ae2ca0694a36555ed1ce35
2022-03-25 17:03:13 +01:00
Stanislav Laznicka cd3019f777 webhooks, aggregation: add metrics to count certs with SHA1 signatures
Extends the certificate attribute deprecation RoundTrippers wrapper with
a checker that counts certificates with sha-1 signatures in server responses.

Non-root non-self-signed SHA-1 certificate signatures were deprecated in
Golang 1.18.

Kubernetes-commit: 499ee65a9b0cd45d41716b513fae0d537f7f9c88
2022-03-24 01:08:02 -04:00
Stanislav Laznicka a0f7d7ad7c webhooks: regenerate certificates for additional sha-1 signatures
Kubernetes-commit: b2c6de170b0b880ccca0ae783bae09559f8dff04
2022-03-25 16:52:40 +01:00
Stanislav Laznicka d98bb60d70 webhooks: gencerts.sh: add intermediate and server certs with sha-1 signatures
Kubernetes-commit: fee08cc580e5147d11f3f12a37452f8a86cee3b8
2022-03-25 16:52:07 +01:00
Jordan Liggitt 20ceb8964c Revert "Revert "Merge pull request 107797 from tkashem/revert-107456""
This reverts commit b0b460921b81b260473d5c393d85beeb5a03e834.

Kubernetes-commit: 363a8beaab2d753413c5599a9ceb41c526bf7270
2022-03-25 14:34:41 -04:00
Jiahui Feng d3152849fc add test case for enabled OpenAPIEnums
Kubernetes-commit: 1f493f6f91d0ead75eabd14874900fe6e9ae15f0
2022-03-22 17:58:16 -07:00