Commit Graph

116 Commits

Author SHA1 Message Date
Patrick Ohly b84662911c remove import doc comments
The "// import <path>" comment has been superseded by Go modules.
We don't have to remove them, but doing so has some advantages:

- They are used inconsistently, which is confusing.
- We can then also remove the (currently broken) hack/update-vanity-imports.sh.
- Last but not least, it would be a first step towards avoiding the k8s.io domain.

This commit was generated with
   sed -i -e 's;^package \(.*\) // import.*;package \1;' $(git grep -l '^package.*// import' | grep -v 'vendor/')

Everything was included, except for
   package labels // import k8s.io/kubernetes/pkg/util/labels
because that package is marked as "read-only".

Kubernetes-commit: 8a908e0c0bd96a3455edf7e3b5f5af90564e65b0
2024-12-02 14:43:58 +01:00
José Carlos Chávez f099bff723 chore: adds consistent vanity import to files and provides tooling for verifying and updating them. (#120642)
* chore: drops update vanity imports from script.

* chore: changes copyright year to 2024.

* chore: makes lint happy.

Kubernetes-commit: 6d6398ef9266abce3518a4c9a3d4e4d8feeffdc1
2024-02-08 14:10:27 +00:00
SataQiu 7751f0aa90 remove import hack about k8s.io/utils/clock/testing
Kubernetes-commit: 81d040d538101b89bd8edd51bb78a58ea5bf793c
2023-11-16 12:30:14 +08:00
weilaaa febd537a31 use build-in max and min func to instead of k8s.io/utils/integer funcs
Kubernetes-commit: eb8f3f194fed16484162aebdaab69168e02f8cb4
2023-12-15 15:09:11 +08:00
Abu Kashem 04acc93a12 apf: request ejected from queue should use reason 'time-out'
Kubernetes-commit: 6297067ff1afaa2d63d83183a23e8744865d7c2b
2023-08-29 16:30:02 -04:00
Abu Kashem d64c9b18da apf: remove RequestWaitLimit from queueset config
Kubernetes-commit: 11ef9514dad6f46a4315198978fee14132c4bbca
2023-08-29 12:11:08 -04:00
Abu Kashem 290096a4d0 apf: remove timeoutOldRequestsAndRejectOrEnqueueLocked function
Kubernetes-commit: da8a472206623d0727ba486489d34780c4b6c1d9
2023-08-28 17:26:11 -04:00
Abu Kashem 27772523df apf: refactor promise to use a context
Kubernetes-commit: 0039f24d74d0f57c8ba868ae361821d37fd908d6
2023-08-21 15:19:31 -04:00
Andrew Sy Kim 066c7cb8cc apiserver: add flow control metric current_inqueue_seats
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

Kubernetes-commit: fb9646fd60d4b8e79223b729c1cb54fc6818fdd1
2023-07-24 19:40:05 +00:00
Mike Spreitzer b8bc556baa Add tracking and reporting of executing requests
Signed-off-by: Mike Spreitzer <mspreitz@us.ibm.com>

Kubernetes-commit: a8a2fb317c8bc9c64ced023988802b2517d34f81
2023-06-30 22:55:35 -04:00
Andrew Sy Kim 73f18d34af promote the following APF metrics to beta:
apiserver_flowcontrol_request_wait_duration_seconds
apiserver_flowcontrol_request_concurrency_in_use
apiserver_flowcontrol_request_concurrency_limit
apiserver_flowcontrol_rejected_requests_total
apiserver_flowcontrol_dispatched_requests_total
apiserver_flowcontrol_current_inqueue_requests
apiserver_flowcontrol_current_executing_requests

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

Kubernetes-commit: 0bb419b1498a664d1dda3b487e9f15fd220ea363
2023-07-05 18:19:36 +00:00
Mike Spreitzer d03cf9927b Introduce apiserver_flowcontrol_current_executing_seats metric
This is a duplicate of
`apiserver_flowcontrol_request_concurrency_in_use` but with a better
name.  Hopefully we can later remove the copy with the inferior name.

Signed-off-by: Mike Spreitzer <mspreitz@us.ibm.com>

Kubernetes-commit: 65e818d4ecfb7bf2a165897fb1caf29bf42f4f83
2023-06-29 01:43:17 -04:00
Mike Spreitzer 078694d35d Make QueueSet support exempt behavior; use it
Signed-off-by: Mike Spreitzer <mspreitz@us.ibm.com>

Kubernetes-commit: f269acd12b225f6a2dbbfae64a475f73f448b918
2023-06-28 22:55:30 -04:00
RuquanZhao bc5f595633 fix undefined convertion
Signed-off-by: Ruquan Zhao ruquan.zhao@arm.com

Kubernetes-commit: 65f3454c1d926a1f119710684794bb54350ef4b1
2023-04-20 17:16:46 +08:00
Li Bo 193a91e8ec refine code
Kubernetes-commit: 2d98d2412a153154396c7089fbe355a56e8a2329
2023-02-14 15:10:54 +08:00
Li Bo d334d385d8 enhance priority-level dumping by adding total requests of dispatched,timed out and rejected
Kubernetes-commit: c0bb425d8f3de4bd3468db9ba83962b777c6331b
2022-09-06 20:06:04 +08:00
Andrew Sy Kim f86340dad2 increase expected fairness margin in TestDifferentWidths
Signed-off-by: Andrew Sy Kim <andrewsy@google.com>

Kubernetes-commit: 736720128824264b4246f247b9ec0d09f5383cf0
2022-10-21 11:39:11 -04:00
Jordan Liggitt c6b4e07ee4 Fix indentation/spacing in comments to render correctly in godoc
Kubernetes-commit: 78cb3862f11225135afdf76f3424e2d7b33104c7
2022-12-17 17:31:05 -05:00
Mike Spreitzer 770f2e1fa4 apiserver: finish implementation of borrowing in APF
Also make some design changes exposed in testing and review.

Do not remove the ambiguous old metric
`apiserver_flowcontrol_request_concurrency_limit` because reviewers
though it is too early.  This creates a problem, that metric can not
keep both of its old meanings.  I chose the configured concurrency
limit.

Testing has revealed a design flaw, which concerns the initialization
of the seat demand state tracking.  The current design in the KEP is
as follows.

> Adjustment is also done on configuration change … For a newly
> introduced priority level, we set HighSeatDemand, AvgSeatDemand, and
> SmoothSeatDemand to NominalCL-LendableSD/2 and StDevSeatDemand to
> zero.

But this does not work out well at server startup.  As part of its
construction, the APF controller does a configuration change with zero
objects read, to initialize its request-handling state.  As always,
the two mandatory priority levels are implicitly added whenever they
are not read.  So this initial reconfig has one non-exempt priority
level, the mandatory one called catch-all --- and it gets its
SmoothSeatDemand initialized to the whole server concurrency limit.
From there it decays slowly, as per the regular design.  So for a
fairly long time, it appears to have a high demand and competes
strongly with the other priority levels.  Its Target is higher than
all the others, once they start to show up.  It properly gets a low
NominalCL once other levels show up, which actually makes it compete
harder for borrowing: it has an exceptionally high Target and a rather
low NominalCL.

I have considered the following fix.  The idea is that the designed
initialization is not appropriate before all the default objects are
read.  So the fix is to have a mode bit in the controller.  In the
initial state, those seat demand tracking variables are set to zero.
Once the config-producing controller detects that all the default
objects are pre-existing, it flips the mode bit.  In the later mode,
the seat demand tracking variables are initialized as originally
designed.

However, that still gives preferential treatment to the default
PriorityLevelConfiguration objects, over any that may be added later.

So I have made a universal and simpler fix: always initialize those
seat demand tracking variables to zero.  Even if a lot of load shows
up quickly, remember that adjustments are frequent (every 10 sec) and
the very next one will fully respond to that load.

Also: revise logging logic, to log at numerically lower V level when
there is a change.

Also: bug fix in float64close.

Also, separate imports in some file

Co-authored-by: Han Kang <hankang@google.com>

Kubernetes-commit: feb42277884bc7cfbd6f0bb1d875cc63b1b6caac
2022-10-31 16:13:25 -07:00
Mike Spreitzer 084f1abd96 apiserver: define metrics for API Priority and Fairness borrowing
Kubernetes-commit: ba5ec78916ae5fe9e400a298da6879515029a12f
2022-10-31 15:09:39 -07:00
Mike Spreitzer 413be63b46 Add instrumentation for seat borrowing
Kubernetes-commit: 9b684579e230f105bcaa743f06bc07c39af703df
2022-10-20 15:21:09 -04:00
Mike Spreitzer 3419387b18 Call queueSet::boundNextDispatchLocked enough
Fix the one path where boundNextDispatchLocked was not being called
after modifying a queue.

Also check for negative work in a request.

These are motivated by
https://github.com/kubernetes/kubernetes/issues/112169 but I do not
have a way to reproduce it and so can not check that these changes
actually remove that symptom.  But these changes are good anyway.

Kubernetes-commit: 6ee93e2cee695203a6ce4935da1b9a807b624260
2022-09-01 22:54:53 -04:00
jupblb 16f776a534 Switch initial/final seats type to uint64
Kubernetes-commit: 3c46482eb09d7343e0f98a930a9aaa158237e278
2022-07-28 10:48:40 +02:00
Davanum Srinivas 7e94033a61 Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: a9593d634c6a053848413e600dadbf974627515f
2022-07-19 20:54:13 -04:00
Mike Spreitzer eb15930b31 Fix APF metric denominator problems
Co-authored-by: JUN YANG <yang.jun22@zte.com.cn>

Kubernetes-commit: fdd921cad0cd9308ec62c1b86c9c1cc5d12e5d21
2022-05-22 23:39:49 -04:00
Wojciech Tyczyński 8f7c120935 Eliminate MaintainObservations function in P&F
Kubernetes-commit: badf436ac4451590e5e84e537f2234e3632ea3b4
2021-11-25 12:44:50 +01:00
Mike Spreitzer 0796534fe5 Remove the PairVec types
Kubernetes-commit: 1f1cfba2a3fb35a8542bbf64a46746214355674c
2022-06-11 00:57:19 -04:00
Mike Spreitzer cae328fb1c Give apf metrics abstractions more familiar names
The logic is similar to Prometheus gauges and vectors,
adopt that terminology.

Kubernetes-commit: 7d64a93a1407f91b5e13bf540a0fa834a41622eb
2022-05-17 23:27:47 -04:00
Mike Spreitzer 8628966894 Fix more initial numerators
Kubernetes-commit: ba690c2257af76bd971d0dfb6bef13ff9099e549
2022-05-18 00:22:30 -04:00
Patrick Ohly ba3b8e9322 enhance and fix log calls
Some of these changes are cosmetic (repeatedly calling klog.V instead of
reusing the result), others address real issues:

- Logging a message only above a certain verbosity threshold without
  recording that verbosity level (if klog.V().Enabled() { klog.Info... }):
  this matters when using a logging backend which records the verbosity
  level.

- Passing a format string with parameters to a logging function that
  doesn't do string formatting.

All of these locations where found by the enhanced logcheck tool from
https://github.com/kubernetes/klog/pull/297.

In some cases it reports false positives, but those can be suppressed with
source code comments.

Kubernetes-commit: edffc700a43e610f641907290a5152ca593bad79
2022-02-16 12:17:47 +01:00
Abu Kashem 44e5395e0e apf: add metric to track dispatch with no accommodation
Kubernetes-commit: 30c0485e0cba3ec6b19e092e7e78059b3fd4f18c
2021-11-23 10:55:31 -05:00
Patrick Ohly ec795ae204 avoid klog Info calls without verbosity
In the following code pattern, the log message will get logged with v=0 in JSON
output although conceptually it has a higher verbosity:

   if klog.V(5).Enabled() {
       klog.Info("hello world")
   }

Having the actual verbosity in the JSON output is relevant, for example for
filtering out only the important info messages. The solution is to use
klog.V(5).Info or something similar.

Whether the outer if is necessary at all depends on how complex the parameters
are. The return value of klog.V can be captured in a variable and be used
multiple times to avoid the overhead for that function call and to avoid
repeating the verbosity level.

Kubernetes-commit: 9eaa2dc554e0c3d4485d4c916dfdbc2f517db2e0
2021-12-11 12:10:21 +01:00
Mike Spreitzer 6adfddf535 Clarify APF metric wrt all three stages of execution
Kubernetes-commit: 88f8e8448bf873cf41035cb858422a10a1d03018
2021-11-30 11:45:53 -05:00
Mike Spreitzer 4098be7694 Factored TimedObserver into less surprising pieces
Kubernetes-commit: ab64e852023965fd8873abcd50ff09cf79814d11
2021-11-15 14:59:30 -05:00
Mike Spreitzer 6a2631848c Add sample-and-watermark for seats occupied during all of execution
Kubernetes-commit: 945f960cfb8fc018b093c1a08e5d4cdd362b1fc6
2021-10-25 01:13:52 -04:00
Wojciech Tyczyński 55b43e446f P&F: move seat-seconds to a better location
Kubernetes-commit: e262db7a4daf5218520e49b423789ea55a94af75
2021-10-27 10:30:25 +02:00
Abu Kashem 3f529d0551 apf: call metrics.AddReject for decisionCancel
Kubernetes-commit: f6dcf17a10dfd2cb5ce0ea7923723547c92c2e55
2021-10-18 13:08:56 -04:00
Mike Spreitzer 5283383fb5 Clarify metrics help wrt APF execution phases
Kubernetes-commit: d7a3bf0d260a0c291941cda68492f10e5010ac91
2021-10-24 22:32:13 -04:00
Mike Spreitzer 5ab6f3fe6b Remove presumptions about what decision has been made
Kubernetes-commit: e70999becd98aa00fd00bc622ffdca0476ec7340
2021-10-18 02:10:21 -06:00
Abu Kashem b40e786ba3 apf: return nil for a request that has been removed from queue
Kubernetes-commit: cd06ba502cf85603242008cc9f61234790ede6de
2021-10-18 12:46:54 -04:00
Mike Spreitzer c5a0365136 Fix nits noticed in recent code review
Kubernetes-commit: 1844a052776bce33322ce20c11b2902403655ef8
2021-10-18 23:51:48 -05:00
Abu Kashem fb57181f8d apf: include seat information in request dump
Kubernetes-commit: 8e33a3b2cc97813724b64be36fe977290db3053a
2021-10-14 09:27:54 -04:00
Abu Kashem 88992ba87a apf: change QueueDump to use SeatSeconds
Kubernetes-commit: 3ef5752edf5b3f7076041368b1c1b5396ee534d7
2021-10-13 10:22:41 -04:00
Abu Kashem e4ae91b4a2 apf: include queue sum stats in debug
Kubernetes-commit: 87c7401eb853678da804ff503e266daeef5ad126
2021-10-13 10:14:30 -04:00
Mike Spreitzer d69d77c659 Update queueset_test.go for FinalSeats
Track the introduction of FinalSeats.

Give up on calculating expected results for tests with added latency,
because I did not find an easy and obvious way to do it.

Kubernetes-commit: 0fc595e03360ba7fc4c3e251d4b41f39172aca72
2021-10-08 22:27:39 -07:00
wojtekt c3ef02ad27 Adjust final seats if they don't fit the limit
Kubernetes-commit: c5a77d8a761b0651b53864f9e396d6f23efd01d2
2021-10-08 11:14:11 +02:00
Mike Spreitzer 56c6fd034b Unconfused logging wrt additional latency
Fixed confusion in queueSet logging wrt seats and additional latency.

Kubernetes-commit: 42f698daad73d9c010027bb60fff9728168c71da
2021-10-11 13:21:46 -07:00
Mike Spreitzer f7bfb170d7 Keep the progress meter R from overflowing
Also add test for that situation.

Kubernetes-commit: a797fbd96de8c67aaed58aef54fbe9f0eb94a2c2
2021-10-01 22:04:05 -07:00
Mike Spreitzer 487fea8e61 Update log messages in finishRequestLocked
Make them clearer and consistent.

Kubernetes-commit: 3906e187a685cebddef85226950fc36f65e8ddb4
2021-10-09 23:48:41 -07:00
Mike Spreitzer 1b1389676f Relax TestDifferentWidths
Make the margin a little wider because flakiness was reported.

Kubernetes-commit: 10326282f9d1abcd4a45b737628286d40971efea
2021-10-07 16:09:53 -07:00