Commit Graph

2363 Commits

Author SHA1 Message Date
Mike Spreitzer d69d77c659 Update queueset_test.go for FinalSeats
Track the introduction of FinalSeats.

Give up on calculating expected results for tests with added latency,
because I did not find an easy and obvious way to do it.

Kubernetes-commit: 0fc595e03360ba7fc4c3e251d4b41f39172aca72
2021-10-08 22:27:39 -07:00
wojtekt c3ef02ad27 Adjust final seats if they don't fit the limit
Kubernetes-commit: c5a77d8a761b0651b53864f9e396d6f23efd01d2
2021-10-08 11:14:11 +02:00
wojtekt c18ab3e1b1 Estimate width of the request based on watchers count in P&F
Kubernetes-commit: 223f9be59778b6ec2e44fd57df523f00e246bd95
2021-07-07 10:48:29 +02:00
Mike Spreitzer 56c6fd034b Unconfused logging wrt additional latency
Fixed confusion in queueSet logging wrt seats and additional latency.

Kubernetes-commit: 42f698daad73d9c010027bb60fff9728168c71da
2021-10-11 13:21:46 -07:00
Dr. Stefan Schimanski c0ff3feb85 apiserver/watchcache: doc watch event fan-out to blocked watchers better
Kubernetes-commit: 94b2bb1d3dcfe7973329cc56024f449adde4086c
2021-10-08 17:05:27 +02:00
Mike Spreitzer f7bfb170d7 Keep the progress meter R from overflowing
Also add test for that situation.

Kubernetes-commit: a797fbd96de8c67aaed58aef54fbe9f0eb94a2c2
2021-10-01 22:04:05 -07:00
Mike Spreitzer 487fea8e61 Update log messages in finishRequestLocked
Make them clearer and consistent.

Kubernetes-commit: 3906e187a685cebddef85226950fc36f65e8ddb4
2021-10-09 23:48:41 -07:00
Antonio Ojea 9bb5c950e8 apiserver aggregator upgrade unit test
Co-authored-by: Chao Xu <xuchao@google.com>

Kubernetes-commit: 3627462ef3e741f2b8e6de7ed3f51be3122cbc6d
2021-10-05 00:19:43 +02:00
Dr. Stefan Schimanski 2f0da0d2ea apiserver/storage: free etcd kv early in list decoding loop
Kubernetes-commit: dbf98e42664c8a3a104fe06e0e362497fca2902d
2021-10-08 14:05:26 +02:00
Mike Spreitzer 1b1389676f Relax TestDifferentWidths
Make the margin a little wider because flakiness was reported.

Kubernetes-commit: 10326282f9d1abcd4a45b737628286d40971efea
2021-10-07 16:09:53 -07:00
Mike Spreitzer a5192405d9 Calculate the work in each request just once
Kubernetes-commit: f2c46c8f9d0b360cf913e22c222d9954b4ff9a76
2021-10-07 17:20:56 -07:00
Abu Kashem 9560ec6e92 introduce final seats for work estimate
Kubernetes-commit: 3d6cc118fee15313419bf7aa0082a2a608ec62f6
2021-09-24 15:18:27 -04:00
Abu Kashem 733c3f75e9 apf: print watch init latency in httplog
Kubernetes-commit: 9b21e11cf9f4cda7c985a588f60925f0766fac02
2021-10-01 10:18:19 -04:00
Abu Kashem a1d74a4e54 remove httplog filter from handler chain when klog.v < 3
Kubernetes-commit: b05d805fd3337d3e3ea6a195883a4eb9c66a7b8a
2021-08-24 14:29:49 -04:00
Mike Spreitzer dc449969cc Use SeatSeconds
Kubernetes-commit: 4b5e1398199282f471d0f332eefeb5c2415bdb01
2021-10-01 15:33:37 -07:00
Mike Spreitzer 0cb46ec2f7 Draft datatype for seat-seconds
Kubernetes-commit: d866f9445831687ab3254d754b13a4acf271882f
2021-10-01 13:29:35 -07:00
Abu Kashem 6013da2b94 apiserver: wrap ResponseWriter using abstraction
Kubernetes-commit: 0d50c969c587c8a6c16e0962118305ac652c5a6b
2021-09-20 11:21:56 -04:00
Abu Kashem 02a51f0c60 apiserver: introduce abstraction for wrapping ResponseWriter
Kubernetes-commit: 96e8fd69e392cc2acf12930a76edc66b801b3c93
2021-09-20 11:21:02 -04:00
Abu Kashem 0e3e7334bb apiserver: refactor PolicyRuleEvaluator to return a struct
Kubernetes-commit: a748fdc6775c63b52a1a963e2332ac774890d2a9
2021-09-20 17:44:11 -04:00
Abu Kashem d53acfe201 apiserver: store (event, evaluated policy) pair in request context
Kubernetes-commit: 8be823b0b0270e1b979b3d4c6e683e1daa0f2e01
2021-09-20 17:43:16 -04:00
astraw99 2e5bed6053 fix typo `registry`
Kubernetes-commit: 6d16238fe53bf1135d781c73d3df4283ecef7fed
2021-08-08 18:37:16 +08:00
Abu Kashem c491eeb785 make lifecycle signal thread safe
Kubernetes-commit: d3e2e9ede295fd743c991e15789a1bc2e8d123f7
2021-09-30 15:24:38 -04:00
CKchen0726 5c4bb8bb6a remove invalid nil check
Kubernetes-commit: 3f1675312042cd855a39200c4bff8020cc0922de
2021-09-13 21:04:31 +08:00
cici37 a7d4ad2f8f Add feature gate CustomResourceValidationExpressions
Kubernetes-commit: 117ef89d2bb4f4bf907280c1c67a12b35e2152df
2021-09-17 12:43:27 -07:00
wojtekt a7cc30c9b3 Optimize watchcache by not starting a gorotuine for all Get/List requests setting RV=0
Kubernetes-commit: 37f93fc63d988c2f1ea11f467d2f73a43d1fa5b9
2021-09-24 14:14:19 +02:00
Abu Kashem 863c48fbc2 apf: rename WorkEstimate.Seats to InitialSeats
Kubernetes-commit: 5d67896adedbce27f01b59eb5f2054919a047f2b
2021-09-24 09:41:38 -04:00
Anish Ramasekar 1e3c9bfcdb fix typo in kms encryption config logs
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>

Kubernetes-commit: 63295a126e316fb7b8630dbc57e98041e747cbed
2021-09-16 18:18:59 +00:00
Mike Spreitzer 72ff8a6261 Improve queueset sharding and dispatching
New anti-windup technique: use the request arrival time as the floor
on the virtual dispatch time.  Prevent bound violations where they
might arise rather than fixing up just one queue at dispatch time,
so that the fixed up dispatch times figure into the dispatching choice.

Two tweaks to the shuffle sharding.  Take seats of executing requests
into account as well as seats of waiting requests.  Do not always
consider the generated hand in the same order.

Rename the queueset methods that do shuffle sharding and finding the
queue to dispatch from, because the old names were confusingly
similar.

Tighten up some request margins.

Name the test cases in TestNoRestraint and TestWindup.

Kubernetes-commit: 4b9cba85874158b25b5c994773a4ec04343820c2
2021-09-20 15:45:24 -04:00
Mike Spreitzer 889106eff7 Add metrics about handling LIST requests
Add metrics that illuminate the costs of handling LIST requests.

Kubernetes-commit: bf424292db185546c474b27b963fd27bdbafc72f
2021-09-01 18:18:23 -04:00
Mike Spreitzer 8d3036922c More test tweaks
Canonicalize listing of test cases.

Make TestNoRestraint try both cases: competition and none.

Kubernetes-commit: 0ee1a7b4ff9012b050bd447055ad5e1e8c57c30e
2021-09-20 15:45:24 -04:00
Mike Spreitzer c505aa64af Update TestNoRestraint and TestWindup
Make TestNoRestraint verify that fairness is NOT achieved
when there is real competition.

Make TestWindup run two cases, to show that 0.1 is too narrow
a margin and 0.26 is wide enough.

Kubernetes-commit: c4945fdf0c14ba2032a5c8edf192678d9fe00374
2021-09-17 01:40:16 -04:00
Mike Spreitzer e417abf592 Migrate apiserver/pkg/util/flowcontrol to use k8s.io/utils/clock
.. instead of apimachinery/pkt/util/clock

Kubernetes-commit: 9f45c0f8c07cd0adfe38c887aa618d33b8a4ee1c
2021-09-17 15:14:42 -04:00
wojtekt b898581360 Migrate to k8s.io/utils/clock in apiserver
Kubernetes-commit: 859a98c0358610e2c127cd2fba1be601ca975188
2021-09-14 20:36:07 +02:00
Lukasz Szaszkiewicz 8fa48ed99b etcd-client starts retrying transient errors from the etcd cluster
This PR enables unaryClientInterceptor in conjunction with Prometheus interceptor.
Previously it was simply overwritten by the Prometheus interceptor.
As a result etcd client didn't attempt to retry certain errors.

The unaryClientInterceptor is important because it knows how to retry all sorts of errors from the etcd cluster. It will make the API server more resilient to failures -  end users won't see certain errors.
The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72

Kubernetes-commit: 83171562b0954b2e19eb69943f01a44779cc7a8f
2021-09-15 16:40:44 +02:00
Paco Xu d1458891b4 remove deprecated validEgressSelectorNames 'master' (#102242)
* remove deprecated validEgressSelectorNames 'master'

Signed-off-by: pacoxu <paco.xu@daocloud.io>

* update gce configure: replace deprecated egress name 'master' with 'controlplane'

Signed-off-by: pacoxu <paco.xu@daocloud.io>

* add dup error for EgressSelection & fix converting alpha/beta to v1 name

Kubernetes-commit: a48a2efbd45ad77901dd09f2665d8cc1e1d8dbf6
2021-09-16 22:09:46 +08:00
Mike Spreitzer fa81877012 Rename httplog entry from "apf_d" to "apf_fd"
Because it is for the "Flow Distinguisher".

Kubernetes-commit: be717abb8317ed309a9d2139856a8e87fad76a16
2021-09-15 17:10:11 -04:00
Jiahui Feng 578621b17f add feature gate OpenAPIEnums.
KEP-2887 OpenAPI Enum Types

Kubernetes-commit: 2ddcf13e916c68030ccedd9d32d155258817aa2e
2021-09-15 13:58:44 -07:00
Mike Spreitzer f17b9e7b4b Track recent reorg in testing clockery
Kubernetes-commit: 20ccd735f6a55fdd9ecbe8cddb889310f0bc033b
2021-09-13 16:15:28 -04:00
Mike Spreitzer 31ec50ec43 Plumb the schema.GroupResource into etcd3 Store struct
Kubernetes-commit: 6f8019aae95db85552381f7e7066a52fef61be7a
2021-09-13 16:54:34 -04:00
Abu Kashem db8aff032b apf: update apf logic to use v1beta2
Kubernetes-commit: 28f2b42a4116a9223113e8b152e02a4f1e602ff4
2021-08-16 17:53:57 -04:00
Abu Kashem 7964b85d9c apf: update e2e test to use v1beta2
Kubernetes-commit: d99497ef6d5581b787e7aebfca943697ed8fe164
2021-08-16 17:12:42 -04:00
Mike Spreitzer de042674ed Widen margins of TestDifferentWidths and TestTooWide
These behavioral unit tests of queueset were failing because the
evaluation criteria were too strict.

Kubernetes-commit: 59d319ec06bb33289a87036418b4a61ed3bb215f
2021-09-09 17:07:58 -04:00
tiloso ab3cca3647 Fix staticcheck in apiserver and client-go pkgs
Kubernetes-commit: 830a137d2ea70663cd94403595313b95ac40ffe8
2021-06-19 22:03:46 +02:00
Mike Spreitzer de227d1d37 Change execution duration guess from 1 minute to 3 milliseconds
So that the width estimate has some effect but not a grossly excessive
one.

Added the fifo::Peek method to simplify the fifo client code.

Also renamed the queueSet::estimatedServiceTime field to
estimatedServiceSeconds to make the units clear.

Kubernetes-commit: a0c161f2f6908ee424ea888ff40f75ff071bd20a
2021-09-07 00:46:50 -04:00
Mike Spreitzer 1efcc1c03a Refine locking in API Priority and Fairness config controller
Instead of a plain `Mutex`, use an `RWMutex` so that the common
operations can proceed in parallel.

Kubernetes-commit: 58927c1abede11ce7a8a74104328cf823df1b39e
2021-09-08 00:34:52 -04:00
Justin SB d5beafeb68 Avoid expensive go-cmp/cmp comparison
The cmp comparison is relatively expensive (#104821).  If we're not
going to log it, we shouldn't make the comparison.

Kubernetes-commit: f9f556dc7061df1dfc8c1628db983eeb97149317
2021-09-07 13:36:50 -04:00
Vince Prignano 2b1166c385 Object creation with generateName should return a proper error
Signed-off-by: Vince Prignano <vincepri@vmware.com>

Kubernetes-commit: 8a9d61278f6c2177309f58bf2655f2269e8f6afd
2021-08-31 18:04:59 -07:00
Dr. Stefan Schimanski 5210be757e server-side-apply: print object name in SHOULD NOT HAPPEN message
Kubernetes-commit: dc6f471e2cceb7aebb62fe2e4bd7c9ce53df6ff0
2021-09-06 08:35:47 +02:00
Mike Spreitzer 06debee006 Remove race condition from TestApfExecuteWatchRequestsWithInitializationSignal
Kubernetes-commit: b78baaad5ff4bcaf12e5de9e7e35f8fadba1c25b
2021-08-31 17:40:03 -04:00
Mike Spreitzer d6c30c1553 Adjust LIST work estimator to match current code
Kubernetes-commit: 6f160ca11b0ce9e9fda9571e37eec2de1b97bcc5
2021-08-25 23:22:13 -04:00
Mike Spreitzer b225af44fe Introduce storagebackend.ConfigForResource
This is a Config specialized for a GroupResource.
It will support generating new resource-specific metrics.

Kubernetes-commit: 85bcd243aa3c8769a5904a1aea44ce704f5e7174
2021-08-29 01:06:12 -04:00
Abu Kashem 033ff70436 Revert "Merge pull request #104281 from tkashem/not-ready-429"
This reverts commit fc5863b8b276e0789f717859e8cce58d7d060181, reversing
changes made to 027fe2554fd18343b8be39eddc8ff6570a6c390f.

Kubernetes-commit: f9f08725907b7db2104ee5fe9f82ab0752726533
2021-08-31 10:10:46 -04:00
Abu Kashem a687b3b7a9 Revert "Merge pull request #104630 from tkashem/remove-option"
This reverts commit edb0a72cff0e43bab72a02cada8486d562ee1cd5, reversing
changes made to 80feff6f407be9f0898c449ba3f9d4d013f05ec9.

Kubernetes-commit: 8844d3092a46a40915b4df6e3b9944d5081f8268
2021-08-31 10:06:12 -04:00
Mike Spreitzer cc873bafd2 Increase debug logging in waitGroupCounter::Add
Now tries a little harder to get to meaningful stack frames.
This revision seems to strike a balance that is useful in the queueset tests.

Kubernetes-commit: b56dd725032cb7a14aa27e4c50c1c9d7c6d23eb1
2021-08-06 14:32:19 -04:00
Abu Kashem ee2c906b05 apiserver: remove server option startup-send-retry-after-until-ready
Kubernetes-commit: 7adc79a4ea33cc44aa8d694c4949e01219e016de
2021-08-27 10:29:37 -04:00
Stephen Augustus 771ffe6475 generated: Run hack/update-gofmt.sh
Signed-off-by: Stephen Augustus <foo@auggie.dev>

Kubernetes-commit: 481cf6fbe753b9eb2a47ced179211206b0a99540
2021-08-12 17:13:11 -04:00
Nicolas Mitchell 1cd62ac37f add backticks to comments containing `<` or `>`
Kubernetes-commit: 340555476a4926e10c9ffac8b283cd77ca088c5d
2021-08-21 10:59:50 -07:00
Maciej Borsz 02f98184b3 Add APF's priorityLevel to httplog.go
Kubernetes-commit: 6db63869b0fd0551382dec1b30b649fda4289444
2021-08-13 13:00:07 +02:00
Abu Kashem cafef859a7 apiserver: add key/value pair to httplog
Kubernetes-commit: bdedd2a4c16487e075db45e778c9e47ddd57448f
2021-08-19 17:07:24 -04:00
Tim Hockin cf0112f87e REST: Document mutable inputs on Create()
If one doesn't DeepCopy() on the way into Create, we can end up writing
into the original object.  This is by design, and should not be a
problem EXCEPT for tests.  If a test compares the input to this function
with the result, but the input was mutated in-situ, it may hide errors,
resulting in tests that pass, but shouldn't.

Kubernetes-commit: 6dfae64d9bebb2c40680bbd6e8270f69839ab013
2021-08-13 11:54:00 -07:00
Tim Hockin 578388b70b Fix registry tests to look at result objects
Kubernetes-commit: 42c7e62180446910f41d3a064016c0d7cdebb407
2021-08-13 11:54:21 -07:00
Mike Spreitzer 7d5430cfba Fix extra latency and add tests for that and width
Added missing dispatching after delayed release of seats.

Updated logging for all six situations of execution completion and
seat release.

Added behavioral tests for non-zero extra latency and non-unit width.

Also added two tests for baseline functionality.

Also improved some comments and other logging in `queueset.go`.

Kubernetes-commit: d2a27a58f0af20c6185fa1c21890d666e9d3746b
2021-08-12 16:48:02 -04:00
Antonio Ojea 38c6ad936b run hack/update-netparse-cve.sh
Kubernetes-commit: 0cd75e8fec62a2531637e80bb950ac9983cac1b0
2021-08-20 01:16:14 +02:00
Abu Kashem 450b7e8f12 rename audit Checker interface
Kubernetes-commit: 27f150351475adaef416bd893403e7066b70d33a
2021-03-24 13:07:21 -04:00
jyz0309 8a6934a20b add new metric
Signed-off-by: jyz0309 <45495947@qq.com>

add comment

Signed-off-by: jyz0309 <45495947@qq.com>

add promlint

Signed-off-by: jyz0309 <45495947@qq.com>

address comment

Signed-off-by: jyz0309 <45495947@qq.com>

go fmt code

Signed-off-by: jyz0309 <45495947@qq.com>

Kubernetes-commit: 97ba475941f64f7ab41a92c51adeb4401146eed1
2021-07-20 19:28:33 +08:00
Jordan Liggitt bbc4211158 Update unit tests to handle go1.17 certificate parsing error messages
Kubernetes-commit: 0e925f266f78d72ed05a5a2fe48faa5e2a87d107
2021-08-17 11:24:03 -04:00
Antoine Pelisse 56458d94fe Add name and namespace to structuredmerge errors
Kubernetes-commit: 5b3c226c34af328d6bd46866e806589189d44f4d
2021-08-12 15:53:54 -07:00
Abu Kashem f3ae70d0cf send retry-after until ready
Kubernetes-commit: 6e3923d0a4f4720d2d9f628eb9c073d2d3ee291a
2021-08-10 12:03:21 -04:00
Abu Kashem 030819c510 apiserver: refactor WithRetryAfter server filter
Kubernetes-commit: 83889ae5940036d89b9822a1e38f0f939308e408
2021-08-09 18:25:29 -04:00
Ryan Moriarty 0741f109f6 Add a new webhook metric tracking request totals.
Also add a 1.0s bucket boundary to the webhook latency metric.

Kubernetes-commit: 8ed1628a6e75f4029853502dbac44fdb0edac5fc
2021-06-22 22:32:47 +00:00
Abu Kashem da50ca4c6e apf: free seats in use after additional latency
Kubernetes-commit: d68186452d9150b113489e6a722caf82f898857f
2021-06-27 13:04:20 -04:00
Abu Kashem ffb869e08f apiserver: add a new mode for graceful termination
add a new mode for graceful termination with the new server run option
'shutdown-send-retry-after'
- shutdown-send-retry-after=true: we initiate shutdown of the
  HTTP Server when all in-flight request(s) have been drained. during
  this window all incoming requests are rejected with status code
  429 and the following response headers:
    - 'Retry-After: N' - client should retry after N seconds
    - 'Connection: close' - tear down the TCP connection
- shutdown-send-retry-after=false: we initiate shutdown of the
  HTTP Server as soon as shutdown-delay-duration has elapsed. This
  is in keeping with the current behavior.

Kubernetes-commit: 3182b69e970bd1fd036ff839fdf811f14e790244
2021-07-14 10:39:29 -04:00
Abu Kashem 5d12abfebe apiserver: rename test variables
Kubernetes-commit: e8381733068f63f4a3e30bae52cea2c7abdde365
2021-07-14 10:29:53 -04:00
Mike Spreitzer 8c2108bc80 Refactor goroutine counting
Add comment outlining TestContextCancel.

Stop calling `t.Errorf` from wrong goroutine.

Package up queueNoteFn expectation checking.

Add counting of goroutine in req1 exec fn.

Remove unnecessary assignment to `_`.

Make TestContextCancel wait on fake clock, to insulate timing check
from scheduler noise.

Factor goroutine counting out of queueset.go, into queueset_test.go,
where it matters.

Refactor promise: Use a simple channel-based implementation for normal
code, a mutex-based one for testing code.

Took all the panics out of queueset.go

Shrink the timeouts in promise tests to 1 second.

Kubernetes-commit: 1db36ae3b30e30d70972998a22987a7db470479b
2021-07-29 00:35:25 -04:00
Mike Spreitzer 904cd74454 Some cleanup of the package for event clocks
Rename from `clock` to `eventclock`.

Simplify by removing the prohibition on an EventFunc suspending and
resuming activity.

Remove "EventClock" from names to avoid stuttering.

Start to consolidate test code under fairqueuing/testing/.

Kubernetes-commit: 80ca6a4ae6ff571c32962a7155efd55edefff9e6
2021-08-06 02:06:43 -04:00
Abu Kashem f013b63777 apf: use EventClock rather than a PassiveClock for queueset
Kubernetes-commit: c2a3b793d3ec62e781dd20704370d09f7e1be706
2021-07-21 17:06:48 -04:00
Abu Kashem bd2ef9810a apf: estimate list width
Kubernetes-commit: 296c18ec323328bf1c6a621a9ca3a094d78bb9d0
2021-06-22 19:38:00 -04:00
Jordan Liggitt 29227fd4ee Avoid spurious calls to update/delete validation
Kubernetes-commit: 561ce035b5b7d577ddb5bb30657bc1f8b399e109
2021-08-05 17:21:40 -04:00
wojtekt c6719af51e Fix metrics reporting for the deprecated watch path
Kubernetes-commit: d9d41b70f64da65525d769ab279e3930d59435fe
2021-08-05 16:17:13 +02:00
Quan Tian 0737519ac6 Improve dynamic cert file change detection
DynamicFileCAContent and DynamicCertKeyPairContent used periodical job
to check whether the file content has changed, leading to 1 minute of
delay in worst case. This patch improves it by leveraging fsnotify
watcher. The content change will be reflected immediately.

Kubernetes-commit: 3cfe3d048ff37c1c6994d131ed8557f3c8bddc8a
2021-08-03 21:28:01 +08:00
Manu Gupta b09300b3f0 fix unsafe json construction for digestConfigObjects.
fix unsafe json creation by creating intermediate objects
while creating patch bytes.

Kubernetes-commit: dfde50b18564cc9a8e99fdcba2f136d770ec6943
2021-08-01 18:43:40 -07:00
j2gg0s 1f934a9364 apiserver: avoid repeated loading context for requestInfo in Namer.Name
Kubernetes-commit: 9cee586ee98bf7b5377ea04a8980f7f3fdbe5ec2
2021-07-28 19:04:48 +08:00
Jordan Liggitt 1e6293c86f Set idle and readheader timeouts
Kubernetes-commit: db48793269ef9eca63fa12abb1a25d017c0e09bb
2021-07-27 11:58:45 -04:00
Mike Spreitzer 0c550377cf Introduce event clocks based on k8s.io/utils/clock
So we can move off of the apimachinery clock package.

Switch queueset to new clocks.

Removed event clocks based on apimachinery clocks,
because this PR introduces ones based on k8s.io/utils/clock .

Removed interface that is implemented by only one interesting type.

Simplify RealEventClock::EventAfterTime.

Kubernetes-commit: dcb298c9552de44e27ed52f5e2b58a0dd7cd8d54
2021-07-21 16:56:11 -04:00
Abu Kashem feb4eefe1c apiserver: add callback to get notified of object count
Kubernetes-commit: 2c60feffbee690af4632d068158e640abe10f678
2021-07-14 16:44:34 -04:00
Alexi Kessler 80b28d7c2c Update doc description for --audit-log-maxbackup
Per https://pkg.go.dev/gopkg.in/natefinch/lumberjack.v1 a value of 0 will retain all logs. Not understanding this led to an outage for my team.

Kubernetes-commit: 94977dce8d13ec1e8b4bd8b449f555af685c3ab6
2021-07-22 09:42:30 -04:00
wojtekt 719fda2a8b Simplify APF promise to what is really used in the code
Kubernetes-commit: 9f735e71bbb7d0dde67a718891641d8afd20a8bc
2021-07-23 13:30:34 +02:00
wojtekt d6e0cf5d71 Remove unused promise code from APF
Kubernetes-commit: a1cf44eab44f21c3929ff2d79501c56e6fbcddc3
2021-07-21 12:14:30 +02:00
Antonio Ojea 42a99cd0a2 decouple timeBudget from real clock
instead of using a goroutine refreshing the budget, obtain
the value from the last time the budget was accessed.

Kubernetes-commit: dd2c38306000eeb1720afc8346165a6caab09259
2021-07-20 13:54:59 +02:00
Yan Huang 8febdc018a Deprecate apiserver_longrunning_guage and apiserver_registered_watchers
Deprecate apiserver_longrunning_guage and apiserver_registered_watchers

Update

Fix name

update

Add back empty space

Update the deprecated version to 1.23

Kubernetes-commit: e2b6816953c9c52b98c129a3e9c2dac4ee925ef0
2021-07-19 23:18:35 -07:00
wojtekt 9209ed5928 Get rid of unused flowSchemaRVs in P&F
Kubernetes-commit: c79a0a08828ef86e07941e96dafefb7d8bc51990
2021-07-13 15:16:17 +02:00
wojtekt b4c306e1e8 Rename width to workEstimate in P&F code
Kubernetes-commit: 73211256e8f15cf84ee69d6fe8258c3a912e0f94
2021-07-13 15:10:58 +02:00
Kevin Delgado 4a4149c8b0 remove apiserver impor from client-go
Kubernetes-commit: af11c4ac9be74115dfd35ce6f18cfa0440fc3458
2021-07-27 20:46:40 +00:00
Kevin Delgado a8a3bb4200 rename cache, add to integration test
Kubernetes-commit: 604db6eb2ada446e78152685943253455b5ec888
2021-07-19 20:04:27 +00:00
Kevin Delgado 613f696b0b Manually set GVK in extract, add commentary to extractor
Kubernetes-commit: dda31bbf2e7a94624e3679e3db56c95e66509ce0
2021-07-08 01:56:58 +00:00
Kevin Delgado 09e56ead9d Initial UnstructuredExtract without caching
Kubernetes-commit: 6e481c5db19cda1ef1b7506c22c0046e440c7fe2
2021-07-03 00:05:11 +00:00
Sergiusz Urbaniak 2402d951d2 Revert "Add a namespace label to admission metrics and expand histogram range to 0-10s"
Kubernetes-commit: 1a87ae19a62d0c61afa6b381a54c6798effa49eb
2021-07-30 14:34:45 +02:00
Andrew Rynhard bfbd0aaa7d Do not try to create an audit log file named "-"
That PR fixes --audit-log-path=- support.
It now logs to stdout as in 1.21.

Kubernetes-commit: 7728428f017350d5fb9a91e6e5dc3ccf86348478
2021-07-23 14:26:28 +00:00
wojtekt 280558d490 Add additional APF test for handling other panic types
Kubernetes-commit: faed88bb7200cc4693b7a3f9cecff1a99ea1fc95
2021-07-14 16:36:38 +02:00
wojtekt 242c8c8a1c Optimize APF support for watch initialization to fix the pod startup time regression.
Kubernetes-commit: ef435b85b47618d0d463ac93e9758d75c2998681
2021-07-13 10:22:30 +02:00
David Ashpole 0562c83329 add apiserver tracing integration test, and fix endpoint validation
Kubernetes-commit: e5a1f86e0ac21de6bacbb9fed242853ff20c3aaf
2021-07-15 16:21:48 -07:00
David Ashpole 785c5626f9 add tracing to webhook requests
Kubernetes-commit: 9dd59017c4c218135b12949a2bd3af07e892199a
2021-07-08 20:13:33 -07:00
Paco Xu d0159b4bbf Revert "use PermitWithoutStream=true for etcd: send pings even without active stream"
Kubernetes-commit: 1280a365e4a2407efb9aba9e701eeb1a5a69a2f0
2021-07-12 11:47:55 +08:00
Abu Kashem b206f9ea02 apf: fix virtual clock
Kubernetes-commit: a570008cbd6c2142a2f270570fe49bf982825c19
2021-07-06 19:29:16 -04:00
David Eads 526d12c09b prevent mutation of deletion options during delete collection
Kubernetes-commit: 649b87aaf85dbb6e8190bf7d16c5dc903b5ecedc
2021-03-10 15:41:03 -05:00
Ryan Phillips 7afffd8ef3 Revert "apiserver: add callback to get notified of object count"
Kubernetes-commit: d95b14e1abfb5ec87248e5dd826b89d0c738af42
2021-07-08 13:56:39 -05:00
David Ashpole 4d188ce1c3 Add distributed tracing to the etcd client
Kubernetes-commit: 71f810bb71ed7b87e02c91ebb879363f32dcfaac
2021-06-25 11:54:53 -07:00
Harry Zhang 0321f0c67e fix CleanScope so we can resolve correct verb for apiserver_request_terminations_total
Kubernetes-commit: a3f57886a267c28d5a0193ca7193d4de9e9212d9
2021-07-07 19:07:49 -07:00
wojtekt 9606daa5ef Refinements to pick queue logic in P&F
Kubernetes-commit: 7f1c4977d7ad9fdfd44549ee8ffe289127be4d81
2021-07-07 08:58:49 +02:00
Davanum Srinivas fe1610f3fe switch from golang-lru to the one in k8s.io/utils
Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 79d0c6cdc10293c9bfe644ce31dc186a936579b0
2021-07-07 13:45:07 -04:00
pacoxu 6640c869f9 use PermitWithoutStream=true for etcd: send pings even without active streams
Signed-off-by: pacoxu <paco.xu@daocloud.io>

Kubernetes-commit: 849dbe034b3e4398922c383e09900b464a60db69
2021-04-29 16:27:08 +08:00
wojtekt cd8b524e3f Avoid code duplication in watchcache
Kubernetes-commit: 2df05df6982df93e4ced37d1236f0366069d5838
2021-07-07 09:40:58 +02:00
wojtekt fbc127e994 Add watch tracker to APF for request cost estimation
Kubernetes-commit: cea1dcfeed2fc4e8ab89cd43e5a0e402251c8df5
2021-06-15 10:49:42 +02:00
astraw99 532c5c3d69 fix delete nil pointer panic
Kubernetes-commit: af19d7f415d70a1de3d67a00921865ba2868b425
2021-06-26 18:06:20 +08:00
Margo Crawford 6c62752c02 This introduces an Impersonate-Uid header to server side code.
UserInfo contains a uid field alongside groups, username and extra.
This change makes it possible to pass a UID through as an impersonation header like you
can with Impersonate-Group, Impersonate-User and Impersonate-Extra.

This PR contains:

* Changes to impersonation.go to parse the Impersonate-Uid header and authorize uid impersonation
* Unit tests for allowed and disallowed impersonation cases
* An integration test that creates a CertificateSigningRequest using impersonation,
  and ensures that the API server populates the correct impersonated spec.uid upon creation.

Kubernetes-commit: 74f5ed6b17287100b339a2b3a43fd4c6fb200978
2021-03-04 15:19:52 -08:00
wojtekt a35bb85f09 Update the logic to pick the best queue in P&F
Kubernetes-commit: 0ecc7ba311ab33b16c5d907ebb1120e3e51a947d
2021-06-17 09:22:00 +02:00
Abu Kashem cf5c77fde9 apf: add additional latency into width
Kubernetes-commit: 24e19229101d242d924ce98a562be3864dde9eae
2021-06-27 12:45:24 -04:00
David Ashpole fe620be9c7 change tracing service from kube-apiserver to apiserver
Kubernetes-commit: 8972efc65fe7f4d2ed840ba1a2bd8ff31e829a7a
2021-07-02 07:04:26 -07:00
Abu Kashem d929410e13 apiserver: add callback to get notified of object count
Kubernetes-commit: 1002b0d163dd948334f10b0e25d0e91d253791e8
2021-06-18 10:44:07 -04:00
Lukasz Szaszkiewicz aefd8ed86f adds HasBeenReady signal that fires when the readyz endpoint succeeds
Kubernetes-commit: 58b91ffca9efe3afb20d80914cdc33c6b0acdef2
2021-07-02 15:18:19 +02:00
Lukasz Szaszkiewicz 279d11fb1e readyz signals when the handler succeeds for the first time.
Co-authored-by: Dr. Stefan Schimanski <stefan.schimanski@gmail.com>

Kubernetes-commit: ca108d109d55e927c292e3e558fc4f761a3a4e7c
2021-06-30 11:04:44 +02:00
Abu Kashem e9708ba84d apf: calculation of dR/dt should use seats in use
Kubernetes-commit: 6203d4fc6af36b1fd60177e8a88b8d4d45acb029
2021-06-21 05:38:07 -04:00
Lukasz Szaszkiewicz d8c0235e05 remove logging from the Signal method
Kubernetes-commit: 6c88a62cb4c849e3844dcc3870073a1b5e05d301
2021-07-02 12:50:20 +02:00
Lukasz Szaszkiewicz 3f407f0668 rename terminationSignals to lifecycleSignals
Kubernetes-commit: dae08bc3a735e50845af7cf639bdbb8971a2115a
2021-07-02 12:28:07 +02:00
David Ashpole 71612b014d move tracing instantiation further up, and check for nil
Kubernetes-commit: b0ffaa93f5c607325203f978c1ae685d6b64d053
2021-07-01 10:42:11 -07:00
David Ashpole 5053bddd06 add tracing to the apiserver's client-go requests
Kubernetes-commit: 79d400c4416f903af604fb2cf07d2c34bb4b98f7
2021-06-26 09:16:32 -07:00
Lukasz Szaszkiewicz 9ff2637133 adds metrics for authorization webhook
Kubernetes-commit: 4a2aef00d6dd2543b011aa7e5af28df598a0cd72
2021-03-17 16:30:40 +01:00
Lukasz Szaszkiewicz 69433684ce adds metrics for authentication webhook
Kubernetes-commit: 322c18c147da08dd2eea25ba3c2b9630a228cf5f
2021-02-23 08:39:25 +01:00
Mike Spreitzer 4dcb848926 Track generalization for width
Kubernetes-commit: 3930629e130f1c98821f8033941870ab582db284
2021-06-18 01:37:22 -04:00
Mike Spreitzer d28ccb4224 Add APF metrics about R(t)
Kubernetes-commit: 676f0450ed37eeec92b67246719cc46e7567e512
2021-06-14 16:48:27 -04:00
Shiming Zhang b76bf120f0 Allow delete with grace period 1s even if metadata period is negative
Kubernetes-commit: a8d4cfac933485b9b4e5e802e267723ce9d2b90f
2021-06-21 15:12:17 +08:00
JD Nurme 288c8d4acf Updated to use konnectivity client v0.0.21, and implemented placeholder context
Kubernetes-commit: 21fd224ebdb17fbaafbb73a36a18aafcab6e4efa
2021-06-23 19:29:14 +00:00
Abu Kashem 4f3f9c7b9e add unit test to verify graceful termination behavior
Kubernetes-commit: 913c449a42f13da758fca2740474bb33d3b65da8
2021-06-24 16:15:12 -04:00
Abu Kashem c1c949723d apiserver: NonBlockingRun should return a listener stopped channel
NonBlockingRun should also return a channel that gets closed when the
underlying http Server has stopped listening (during the graceful
shutdown period)

Kubernetes-commit: a84c1b71005930e8253c1348515020132c5c175b
2021-06-24 16:04:54 -04:00
Abu Kashem 5c1642946b apiserver: refactor graceful termination logic
- refactor graceful termination logic so we can write unit tests
  to assert on the expected behavior.

Kubernetes-commit: d85619030e3a5fec5960ad00136e8d9bd030b5f8
2021-06-24 15:51:38 -04:00
David Ashpole 5927da0040 Add distributed tracing to the apiserver using OpenTelemetry
Kubernetes-commit: 79550ed40c67a70534c1cb697e1fb7e7dbf96335
2021-06-25 05:20:16 -07:00
njuptlzf 4e8f1a1c5a Fix staticcheck failures
Kubernetes-commit: 93365919b501f4b7dcd0cf3ce84c34a584efc17f
2021-04-02 12:23:30 +08:00
Jordan Liggitt 4bb06f3c4d Simplify running embedded etcd server in tests
Kubernetes-commit: 01760927b82cf28bc6ba5b6364fda3c9f8c5c0b1
2021-06-15 12:07:46 -04:00
Mike Spreitzer feae5f3e79 Add config checking for inflight limits
When API Priority and Fairness is enabled, the inflight limits must
add up to something positive.
This rejects the configuration that prompted
https://github.com/kubernetes/kubernetes/issues/102885

Update help for max inflight flags

Kubernetes-commit: 0762f492c5b850471723a305cfa7390e44851145
2021-06-21 16:53:07 -04:00
youhonglian 56e6dac0bb update github.com/pkg/errors to go native errors pkg in staging
Kubernetes-commit: 4b1402187474a1ebd0f71c824f45448fea160fc6
2021-06-22 11:54:56 +08:00
lala123912 5c11e2a763 staging/src/k8s.io/apiserver/pkg/registry:migrate logs to structured logs
Kubernetes-commit: 8e7b5ff480b04342f78b6d321814f05e31abc4bb
2021-01-22 10:25:47 +08:00
Antonio Ojea 5a7fb85083 deflake TestClientReceivedGOAWAY test
The test was flaking because the test was creating more connections
than expected.

Disabling connection pooling removes the flakes, and no more connections
are created that the necessary.

Kubernetes-commit: 4d11c3cd8cb18c1e246a7a6b8e9a791177c49d31
2021-06-18 00:38:02 +02:00
tiloso c94ba7a71a Fix staticcheck in cluster & k8s.io/{apimachinery,apiserver}
Kubernetes-commit: c5d70e9db1e8f36e5eb6dae6ca72ad62c1c837d9
2021-06-05 22:15:15 +02:00
Daniel Smith a08d7d5ddf Make logged verbs match metric-reported verbs
Kubernetes-commit: 54fde5d119f8bb55aec18cb5aef01b6ea58dfea0
2021-06-16 17:08:54 -07:00
Abu Kashem e1aec4ecae apf: take seats into account when dispatching request
Kubernetes-commit: ff716cef508f948b50e1026e980e6df5ee475538
2021-06-14 12:19:06 -04:00
Davanum Srinivas c2350dc583 Switch to github.com/coreos/go-systemd/v22 and drop older package
- We use the new v22 module released on May 10
- We drop the unmaintained `github.com/coreos/pkg`

Signed-off-by: Davanum Srinivas <davanum@gmail.com>

Kubernetes-commit: 7fcdbbef06d0bc8c4416db1d2cbba9f30d30e8c4
2021-06-16 11:14:02 -04:00
Jordan Liggitt aafb67c9b2 Quiet grpc info logs in apiserver
Kubernetes-commit: 503f5e1b52812f1bc5bb66d9b02a8f32cc5dccd9
2021-06-15 18:50:54 -04:00
Abu Kashem abdd901971 apf: always include seats in virtual time
Kubernetes-commit: 2f7456076e0cb29a95d86cd0f54c34a04b4722ab
2021-06-14 09:52:00 -04:00
Jordan Liggitt 2f8b9e4dbc Fix etcd egress dialer addr parsing
Kubernetes-commit: a26c392de176494f2c425f712bc49fc399e9ce6d
2021-06-14 18:02:59 -04:00
Jordan Liggitt aaf5f32453 Quiet embedded etcd logs
Kubernetes-commit: f283deee6b65be2b435ec192b6444f969f9d2a42
2021-06-04 00:43:11 -04:00