Commit Graph

53 Commits

Author SHA1 Message Date
Jordan Liggitt bd604a62aa Remove deprecated --etcd-quorum-read flag
Kubernetes-commit: cff79c542130831f4a212099974570244a0c9586
2018-10-08 11:04:28 -04:00
Christoph Blecker 92e87e143a Update gofmt for go1.11
Kubernetes-commit: 97b2992dc191a357e2167eff5035ce26237a4799
2018-10-05 12:59:38 -07:00
Mike Danese f8e0930b86 storage: propagate TransformFromStorage errors from List
Like we do everywhere else we use TranformFromStorage. The current
behavior is causing all service account tokens to be regenerated,
invalidating old service account tokens and unrecoverably breaking apps
that are using InClusterConfig or exported service account tokens.

If we are going to break stuff, let's just break the Lists so that
misconfiguration of encryption config or checkpoint corruption are
obvious.

Kubernetes-commit: e7bda4431da05b55b4e8f66ed308d4ed90efd2df
2018-10-03 13:30:29 -07:00
Chao Xu cc84cfddc9 support continueToken for inconsistent list
Kubernetes-commit: 0a7286c6b21a858f7397a0835776cb5900d98e87
2018-08-10 10:27:40 -07:00
Antoine Pelisse 30e2071cad storage: Move precondition check as a method of preconditions
Kubernetes-commit: b5258a53809bde48cac07848d787a0fc7db16b2a
2018-07-31 13:53:27 -07:00
David Eads cb5cac48ee make package name match all the import aliases
Kubernetes-commit: d3bd0eb1d5cefc25e4476d8dc086ebd90439ef4e
2018-08-01 10:01:32 -04:00
Mikhail Mazurskiy 0f7bbcadfb Add missing error handling in schema-related code
Kubernetes-commit: bfe313d5f351dfae086a85a97e7103183173e5b5
2018-06-03 14:59:58 +10:00
Cong Ding 5081ebb434 cleanup: remove deadcode
Kubernetes-commit: 3bacb04a5f9805bb83e016e341a49d0f13a43598
2018-06-22 16:39:13 -07:00
fisherxu dd4ba939ef collapse the resource version parse
Kubernetes-commit: 68ec856b4506d4440b327141051bdf31fc92be42
2018-06-23 17:29:06 +08:00
Cong Ding f230b000db etcd: reuse leases for keys in a time window
Reuse leases for keys in a time window, to reduce the overhead to etcd
caused by using massive number of leases

Fixes #47532

Kubernetes-commit: 163529bc202054d991f0ce2e21738cc18ffd6022
2018-05-30 17:27:00 -07:00
Jordan Liggitt 1f9c7bdd99 Quiet verbose apiserver logs
Kubernetes-commit: 862f8567bb4b1d4e9f63330cf3a516af61ae7cc5
2018-06-11 22:32:46 -04:00
wojtekt 5d15c5549a Fix incorrectly set resource version in List
Kubernetes-commit: a3578c864ed7db5a73daa88839eed11d86af209b
2018-05-22 15:32:25 +02:00
hzxuzhonghu 2515aae7de remove unused code
Kubernetes-commit: fffa40552cc9f6d372607d1ed40a44d97cecb314
2018-03-20 18:50:19 +08:00
hzxuzhonghu dff0e6ac00 check etcd servers by a random order
Kubernetes-commit: ea627f987507de38dfc46ab2d9c924690eba5ee9
2018-03-15 10:04:40 +08:00
hzxuzhonghu 6ab99203b7 Replace "golang.org/x/net/context" with "context"
Kubernetes-commit: 70e45eccf27726f0e63dd1024924ccc7e2cd35a0
2018-02-28 12:20:22 +08:00
Kubernetes Publisher 627fa76a8b sync: initially remove files BUILD */BUILD BUILD.bazel */BUILD.bazel 2018-03-15 09:38:17 +00:00
Marek Grabowski 25b054a72f generated
Kubernetes-commit: fb7101ef7c9892e0a5d3a718038b93e84b9314b5
2018-02-14 16:15:12 +00:00
Marek Grabowski e36f8069aa Add a metric exposing number of objects per type
Kubernetes-commit: f6e9ebffa2df10f7792fbea0a0fbe5ab8e388a26
2018-02-12 15:58:57 +00:00
Jeff Grafton 1ab12b2dc8 Autogenerated: hack/update-bazel.sh
Kubernetes-commit: ef56a8d6bb3800ab7803713eafc4191e8202ad6e
2018-02-16 13:43:01 -08:00
halfcrazy 6f8c3a80da fix typo in package apiserver
Kubernetes-commit: 0da91a8577ddfdeaff985cbb6c0da69d5a2ffc81
2018-02-01 03:04:33 +08:00
Daniel Smith 4b163fbe32 resource version parsing should all be in one place
Kubernetes-commit: 023895d597be6539a1a16fa54d60e47a17d85dc1
2018-01-10 10:43:59 -08:00
YuxiJin-tobeyjin 57076f11ae Should use Fatalf while need to format the output
Kubernetes-commit: 10751e54e6f01ed771a8fb75abd90346ee4ab501
2017-10-25 16:20:32 +08:00
Jordan Liggitt 8e603ae547 Recheck if transformed data is stale when doing live lookup during update
Kubernetes-commit: 070089c6bfccc18ba6039eee45b982633087c87e
2018-01-17 01:21:52 -05:00
Jeff Grafton c8a97ee31a Autogenerate BUILD files
Kubernetes-commit: efee0704c60a2ee3049268a41535aaee7f661f6c
2017-12-23 13:06:26 -08:00
Di Xu 7a46c8324d remove FilterFunc and use SelectionPredicate everywhere
Kubernetes-commit: 3ae7bdd211e3df1350446b6def142b4d31c75e52
2017-11-22 16:57:06 +08:00
Cao Shufeng 837b7e4418 partial fix crd patch failing
partial fixes https://github.com/kubernetes/kubernetes/issues/53379

Kubernetes-commit: 1e800350a2e05899f7435385bebe7fc101142d95
2017-11-03 16:06:14 +08:00
Jeff Grafton f4dbe23125 update BUILD files
Kubernetes-commit: aee5f457dbfd70c2d15c33e392dce6a3ca710116
2017-10-12 13:52:10 -07:00
Clayton Coleman 5f039fac0c Avoid intermediate List allocations as items added to the list
Pick a reasonable middle ground between allocating larger chunks of
memory (2048 * ~500b for pod slices) and having many small allocations
as the list is resized by preallocating capacity based on the expected
list size. At worst, we'll allocate a 1M slice for pods and only add
a single pod to it (if the selector is very specific).

Kubernetes-commit: ce0dc76901bd1ce36ca20c5cf96b89088d0e95a2
2017-10-09 22:16:13 -04:00
Clayton Coleman f24913f06e Promote continuation token schema to v1
Change the filtering logic to require a leading path and clean that
afterwards.

Kubernetes-commit: ac8808b792f624fa04aa7c589bd5aca1b9afde39
2017-09-24 18:09:54 -04:00
Clayton Coleman 2f9b480e69 Fill partial pages on the server rather than forcing client to
The etcd3 storage now attempts to fill partial pages to prevent clients
having to make more round trips (latency from server to etcd is lower
than client to server). The server makes repeated requests to etcd of
the current page size, then uses the filter function to eliminate any
matches. After this change the apiserver will always return full pages,
but we leave the language in place that clients must tolerate it.

Reduces tail latency of large filtered lists, such as viewing pods
assigned to a node.

Kubernetes-commit: da7124e5e5c0385dd5bcfc72ef035effc7708913
2017-09-24 18:06:57 -04:00
Hitoshi Mitake a7bf68f0d6 storage, etcd3: add an option for configuring interval of compaction requests from apiserver
This commit adds an option for controlling request of compaction to
etcd3 from apiserver. There is a situation that apiserver cannot fully
own its etcd cluster (e.g. sharing it with canal). In such a case,
apiserver should have limited access in terms of etcd's auth
functionality so it don't have a priviledge to issue compaction
requests. It means that the compaction requests should be issued by
other component and apiserver's compaction requests are needless.

For such use cases, this commit adds a new flag
`storagebackend.Config.CompactionInterval`. If the flag is non 0,
apiserver issues the compaction requests like current behaviour (the
default is 5 minutes). If it is 0, apiserver doesn't issue the
requests. It can be configured with a newly added option of apiserver
`--etcd-compaction-interval`.

Kubernetes-commit: 87d4d3e92be6b93517f189082b0451cee6957ee5
2017-09-01 14:06:25 +09:00
Kubernetes Publisher 0f62a50c16 etcd3 store: retry w/live object on conflict
In GuaranteedUpdate, if it was called with a suggestion (e.g. via the
watch cache), and the suggested object is stale, perform a live lookup
and then retry the update.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>

Kubernetes-commit: bf33df16b52508974ddedacd814010cfe0fb79f0
2017-09-22 11:42:06 +00:00
Clayton Coleman 460257fd61 Server side implementation of paging for etcd3
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.

Adds GetContinue() to ListMeta.

Kubernetes-commit: 8952a0cb722b77459cf2701632a30f5b264f5aba
2017-09-03 14:04:12 +00:00
Jeff Grafton 6c539a43c6 Use buildozer to delete licenses() rules except under third_party/
Kubernetes-commit: a7f49c906df816123e7d4ccbd4cebab411519465
2017-08-29 13:15:24 +00:00
Jeff Grafton 6caa2933ae Use buildozer to remove deprecated automanaged tags
Kubernetes-commit: 33276f06be5e872bf53ca62a095fcf0a6b6c11a8
2017-08-29 13:15:24 +00:00
Jeff Grafton 44942b068a Run hack/update-bazel.sh to generate BUILD files
Kubernetes-commit: 3579017b865ddbc5449d6bba87346f086e4b93ff
2017-08-29 13:13:51 +00:00
bjhaid 7dfcb9c56f This adds an etcd health check endpoint to kube-apiserver
addressing https://github.com/kubernetes/kubernetes/issues/48215.

Kubernetes-commit: 47d748c5dc989ea46142569bf42636c622fe128a
2017-08-29 13:13:05 +00:00
Clayton Coleman a9bfd91dd9 Do not persist SelfLink into etcd storage
This behavior regressed in an earlier release. Clearing the self link
ensures that a new version is always written and reduces the size of the
stored object by a small amount. Add tests to verify that Create and
Update result in no SelfLink stored in etcd.

Kubernetes-commit: 461c3701f0915acbf49c339f5321fa86879a963e
2017-07-16 04:08:42 +00:00
Clayton Coleman 2f829d739b GuaranteedUpdate must write if stored data is not canonical
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data.

This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.

Kubernetes-commit: b851614adfe2b39941d518485480ff527fa4f0c1
2017-07-04 08:39:44 +00:00
NickrenREN 080796b69a Lower etcd compacted loglevel
Kubernetes-commit: 151b6a04e1355c1b47191f46283a3bfe98dfc393
2017-06-28 00:14:31 +00:00
Clayton Coleman 5fa08b8c5e Allow initialization of resources
Add support for creating resources that are not immediately visible to
naive clients, but must first be initialized by one or more privileged
cluster agents. These controllers can mark the object as initialized,
allowing others to see them.

Permission to override initialization defaults or modify an initializing
object is limited per resource to a virtual subresource "RESOURCE/initialize"
via RBAC.

Initialization is currently alpha.

Kubernetes-commit: 331eea67d8000e5c4b37e2234a90903c15881c2f
2017-06-13 20:47:30 +00:00
Saksham Sharma 0b1c13686c Add configuration options for encryption providers
Add location transformer, config for transformers

Location transformer helps choose the most specific transformer for
read/write operations depending on the path of resource being accessed.

Configuration allows use of --experimental-encryption-provider-config
to set up encryption providers. Only AEAD is supported at the moment.

Add new files to BUILD, AEAD => k8s-aes-gcm

Use group resources to select encryption provider

Update tests for configuration parsing

Remove location transformer

Allow specifying providers per resource group in configuration

Add IdentityTransformer configuration option

Fix minor issues with initial AEAD implementation

Unified parsing of all configurations

Parse configuration using a union struct

Run configuration parsing in APIserver, refactor parsing

More gdoc, fix minor bugs

Add test coverage for combined transformers

Use table driven tests for encryptionconfig

Kubernetes-commit: 9760d00d08ef0619e30a7b1b90fd290cab960069
2017-06-13 20:47:30 +00:00
Monis Khan 6794013a5b Panic server on watch errors in test environment
This change makes it so that errors during watch decoding panic the
server if it is in a test environment.  This allows us to catch coder
errors related to storing incompatible types at the same location in
etcd.

Signed-off-by: Monis Khan <mkhan@redhat.com>

Kubernetes-commit: a13f026fd012859f04467e6007e2cafe4a788927
2017-06-13 20:47:29 +00:00
Clayton Coleman 0bd7c5925e generated: bazel
Kubernetes-commit: 7827899b1dd22074eb230e241f63e69499046fb6
2017-05-17 17:27:53 +00:00
Clayton Coleman f27996225a Update etcd3 storage to leverage storage/value interfaces
Adds context argument which must be set for AES GCM authenticated data
to be passed.

Kubernetes-commit: a73990a33f95713f026ee7ae9ae6741255aaf8e4
2017-05-17 17:27:53 +00:00
Mike Danese 2aab760a2a autogenerated
Kubernetes-commit: a05c3c0efdc5822049e34b1a5a1ee259c5fb1906
2017-04-15 20:35:23 +00:00
deads2k 5e858945af remove objectmetafor
Kubernetes-commit: 04460c8750b9a47672aa708251de0e703cfb7266
2017-04-11 20:35:21 +00:00
Cao Shufeng 1e6400df1f delete etcd socket file for unit tests
This change clean up the environment for etcd3 unit test.
Without this change, "make test" will leave some socket files in
workspace. And these socket files make hack/verify-generated-protobuf.sh
fails.

Kubernetes-commit: 4e9dcf3da536a9b956eb092fe9dbc5b2081cfb9d
2017-03-31 20:37:15 +00:00
Jordan Liggitt a22f78080a Preserve custom etcd prefix compatibility for etcd3
Kubernetes-commit: 6853e4d71ea128ff955fad32972ad9edcb376dfb
2017-03-18 19:56:09 +00:00
Clayton Coleman 74dc1360f1 Allow ValueTransformer to indicate resource is stale
Allows a transformer (such as an encrypter) to force an update if a new
key is in use, thus allowing simple writes to the REST layer to
trivially migrate keys.
2017-02-13 07:36:41 -05:00