Commit Graph

34 Commits

Author SHA1 Message Date
Chao Xu 034ed25bbe Add RemainingItemCount to ListMeta
Kubernetes-commit: 58c18309a84f9e0fe05b92c202616e4ecf8062f7
2019-04-01 14:44:19 -07:00
Joe Betz f90442e3fb Paginate List call performed by Reflector's ListAndWatch call
Kubernetes-commit: 84723c2d3ef5ff5d30aadd1ad72068bf2254358c
2019-03-14 13:26:19 -07:00
wojtekt 1fd6bb259f Expose etcd client latency metrics
Kubernetes-commit: d5c9ad80499a9148a40b8a6c33c165cf12578649
2019-04-01 12:46:06 +02:00
Andrew Kim 2b43930b32 replace k8s.io/apiserver/pkg/util/trace with k8s.io/utils/trace
Kubernetes-commit: 93b086f6eafc1e94214fc1d946a1917482751cf3
2019-01-24 11:50:30 -05:00
Davanum Srinivas 2710b17b80 Move from glog to klog
- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
  * github.com/kubernetes/repo-infra
  * k8s.io/gengo/
  * k8s.io/kube-openapi/
  * github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods

Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135

Kubernetes-commit: 954996e231074dc7429f7be1256a579bedd8344c
2018-11-09 13:49:10 -05:00
Jordan Liggitt bd604a62aa Remove deprecated --etcd-quorum-read flag
Kubernetes-commit: cff79c542130831f4a212099974570244a0c9586
2018-10-08 11:04:28 -04:00
Mike Danese f8e0930b86 storage: propagate TransformFromStorage errors from List
Like we do everywhere else we use TranformFromStorage. The current
behavior is causing all service account tokens to be regenerated,
invalidating old service account tokens and unrecoverably breaking apps
that are using InClusterConfig or exported service account tokens.

If we are going to break stuff, let's just break the Lists so that
misconfiguration of encryption config or checkpoint corruption are
obvious.

Kubernetes-commit: e7bda4431da05b55b4e8f66ed308d4ed90efd2df
2018-10-03 13:30:29 -07:00
Chao Xu cc84cfddc9 support continueToken for inconsistent list
Kubernetes-commit: 0a7286c6b21a858f7397a0835776cb5900d98e87
2018-08-10 10:27:40 -07:00
Antoine Pelisse 30e2071cad storage: Move precondition check as a method of preconditions
Kubernetes-commit: b5258a53809bde48cac07848d787a0fc7db16b2a
2018-07-31 13:53:27 -07:00
Cong Ding 5081ebb434 cleanup: remove deadcode
Kubernetes-commit: 3bacb04a5f9805bb83e016e341a49d0f13a43598
2018-06-22 16:39:13 -07:00
fisherxu dd4ba939ef collapse the resource version parse
Kubernetes-commit: 68ec856b4506d4440b327141051bdf31fc92be42
2018-06-23 17:29:06 +08:00
Cong Ding f230b000db etcd: reuse leases for keys in a time window
Reuse leases for keys in a time window, to reduce the overhead to etcd
caused by using massive number of leases

Fixes #47532

Kubernetes-commit: 163529bc202054d991f0ce2e21738cc18ffd6022
2018-05-30 17:27:00 -07:00
wojtekt 5d15c5549a Fix incorrectly set resource version in List
Kubernetes-commit: a3578c864ed7db5a73daa88839eed11d86af209b
2018-05-22 15:32:25 +02:00
hzxuzhonghu 6ab99203b7 Replace "golang.org/x/net/context" with "context"
Kubernetes-commit: 70e45eccf27726f0e63dd1024924ccc7e2cd35a0
2018-02-28 12:20:22 +08:00
Marek Grabowski e36f8069aa Add a metric exposing number of objects per type
Kubernetes-commit: f6e9ebffa2df10f7792fbea0a0fbe5ab8e388a26
2018-02-12 15:58:57 +00:00
Daniel Smith 4b163fbe32 resource version parsing should all be in one place
Kubernetes-commit: 023895d597be6539a1a16fa54d60e47a17d85dc1
2018-01-10 10:43:59 -08:00
Jordan Liggitt 8e603ae547 Recheck if transformed data is stale when doing live lookup during update
Kubernetes-commit: 070089c6bfccc18ba6039eee45b982633087c87e
2018-01-17 01:21:52 -05:00
Di Xu 7a46c8324d remove FilterFunc and use SelectionPredicate everywhere
Kubernetes-commit: 3ae7bdd211e3df1350446b6def142b4d31c75e52
2017-11-22 16:57:06 +08:00
Cao Shufeng 837b7e4418 partial fix crd patch failing
partial fixes https://github.com/kubernetes/kubernetes/issues/53379

Kubernetes-commit: 1e800350a2e05899f7435385bebe7fc101142d95
2017-11-03 16:06:14 +08:00
Clayton Coleman 5f039fac0c Avoid intermediate List allocations as items added to the list
Pick a reasonable middle ground between allocating larger chunks of
memory (2048 * ~500b for pod slices) and having many small allocations
as the list is resized by preallocating capacity based on the expected
list size. At worst, we'll allocate a 1M slice for pods and only add
a single pod to it (if the selector is very specific).

Kubernetes-commit: ce0dc76901bd1ce36ca20c5cf96b89088d0e95a2
2017-10-09 22:16:13 -04:00
Clayton Coleman f24913f06e Promote continuation token schema to v1
Change the filtering logic to require a leading path and clean that
afterwards.

Kubernetes-commit: ac8808b792f624fa04aa7c589bd5aca1b9afde39
2017-09-24 18:09:54 -04:00
Clayton Coleman 2f9b480e69 Fill partial pages on the server rather than forcing client to
The etcd3 storage now attempts to fill partial pages to prevent clients
having to make more round trips (latency from server to etcd is lower
than client to server). The server makes repeated requests to etcd of
the current page size, then uses the filter function to eliminate any
matches. After this change the apiserver will always return full pages,
but we leave the language in place that clients must tolerate it.

Reduces tail latency of large filtered lists, such as viewing pods
assigned to a node.

Kubernetes-commit: da7124e5e5c0385dd5bcfc72ef035effc7708913
2017-09-24 18:06:57 -04:00
Kubernetes Publisher 0f62a50c16 etcd3 store: retry w/live object on conflict
In GuaranteedUpdate, if it was called with a suggestion (e.g. via the
watch cache), and the suggested object is stale, perform a live lookup
and then retry the update.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>

Kubernetes-commit: bf33df16b52508974ddedacd814010cfe0fb79f0
2017-09-22 11:42:06 +00:00
Clayton Coleman 460257fd61 Server side implementation of paging for etcd3
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.

Adds GetContinue() to ListMeta.

Kubernetes-commit: 8952a0cb722b77459cf2701632a30f5b264f5aba
2017-09-03 14:04:12 +00:00
Clayton Coleman a9bfd91dd9 Do not persist SelfLink into etcd storage
This behavior regressed in an earlier release. Clearing the self link
ensures that a new version is always written and reduces the size of the
stored object by a small amount. Add tests to verify that Create and
Update result in no SelfLink stored in etcd.

Kubernetes-commit: 461c3701f0915acbf49c339f5321fa86879a963e
2017-07-16 04:08:42 +00:00
Clayton Coleman 2f829d739b GuaranteedUpdate must write if stored data is not canonical
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data.

This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.

Kubernetes-commit: b851614adfe2b39941d518485480ff527fa4f0c1
2017-07-04 08:39:44 +00:00
Saksham Sharma 0b1c13686c Add configuration options for encryption providers
Add location transformer, config for transformers

Location transformer helps choose the most specific transformer for
read/write operations depending on the path of resource being accessed.

Configuration allows use of --experimental-encryption-provider-config
to set up encryption providers. Only AEAD is supported at the moment.

Add new files to BUILD, AEAD => k8s-aes-gcm

Use group resources to select encryption provider

Update tests for configuration parsing

Remove location transformer

Allow specifying providers per resource group in configuration

Add IdentityTransformer configuration option

Fix minor issues with initial AEAD implementation

Unified parsing of all configurations

Parse configuration using a union struct

Run configuration parsing in APIserver, refactor parsing

More gdoc, fix minor bugs

Add test coverage for combined transformers

Use table driven tests for encryptionconfig

Kubernetes-commit: 9760d00d08ef0619e30a7b1b90fd290cab960069
2017-06-13 20:47:30 +00:00
Clayton Coleman f27996225a Update etcd3 storage to leverage storage/value interfaces
Adds context argument which must be set for AES GCM authenticated data
to be passed.

Kubernetes-commit: a73990a33f95713f026ee7ae9ae6741255aaf8e4
2017-05-17 17:27:53 +00:00
deads2k 5e858945af remove objectmetafor
Kubernetes-commit: 04460c8750b9a47672aa708251de0e703cfb7266
2017-04-11 20:35:21 +00:00
Jordan Liggitt a22f78080a Preserve custom etcd prefix compatibility for etcd3
Kubernetes-commit: 6853e4d71ea128ff955fad32972ad9edcb376dfb
2017-03-18 19:56:09 +00:00
Clayton Coleman 74dc1360f1 Allow ValueTransformer to indicate resource is stale
Allows a transformer (such as an encrypter) to force an update if a new
key is in use, thus allowing simple writes to the REST layer to
trivially migrate keys.
2017-02-13 07:36:41 -05:00
Clayton Coleman 3d8615c851 Allow values to be wrapped prior to serialization in etcd3 2017-02-13 07:36:41 -05:00
Dr. Stefan Schimanski 4b46da7a0d Mechanical import fixup: pkg/storage 2017-02-13 07:36:41 -05:00
Dr. Stefan Schimanski 4a2fe15be7 Mechanical move: pkg/storage -> k8s.io/apiserver 2017-02-13 07:36:41 -05:00