The DestroyFunc functions returned by generic.NewRawStorage is never
called when we do a StartTestServer() in the test suite. For a quick
hack for now, added TrackStorageCleanup/RegisterStorageCleanup and
CleanupStorage. Note that unless TrackStorageCleanup is called (which
is called only from the test suite) the other two methods are
no-ops essentially. So no change in behavior at runtime. This vastly
brings down the number of goroutines that are left behind when this
test is executed and should reduce if not eliminate the flakiness
of TestCRD
Kubernetes-commit: 00bcbd1311af711f70c771d790137b93ce48309a
Pick a reasonable middle ground between allocating larger chunks of
memory (2048 * ~500b for pod slices) and having many small allocations
as the list is resized by preallocating capacity based on the expected
list size. At worst, we'll allocate a 1M slice for pods and only add
a single pod to it (if the selector is very specific).
Kubernetes-commit: ce0dc76901bd1ce36ca20c5cf96b89088d0e95a2
The etcd3 storage now attempts to fill partial pages to prevent clients
having to make more round trips (latency from server to etcd is lower
than client to server). The server makes repeated requests to etcd of
the current page size, then uses the filter function to eliminate any
matches. After this change the apiserver will always return full pages,
but we leave the language in place that clients must tolerate it.
Reduces tail latency of large filtered lists, such as viewing pods
assigned to a node.
Kubernetes-commit: da7124e5e5c0385dd5bcfc72ef035effc7708913
This commit adds an option for controlling request of compaction to
etcd3 from apiserver. There is a situation that apiserver cannot fully
own its etcd cluster (e.g. sharing it with canal). In such a case,
apiserver should have limited access in terms of etcd's auth
functionality so it don't have a priviledge to issue compaction
requests. It means that the compaction requests should be issued by
other component and apiserver's compaction requests are needless.
For such use cases, this commit adds a new flag
`storagebackend.Config.CompactionInterval`. If the flag is non 0,
apiserver issues the compaction requests like current behaviour (the
default is 5 minutes). If it is 0, apiserver doesn't issue the
requests. It can be configured with a newly added option of apiserver
`--etcd-compaction-interval`.
Kubernetes-commit: 87d4d3e92be6b93517f189082b0451cee6957ee5
In GuaranteedUpdate, if it was called with a suggestion (e.g. via the
watch cache), and the suggested object is stale, perform a live lookup
and then retry the update.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
Kubernetes-commit: bf33df16b52508974ddedacd814010cfe0fb79f0
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.
Adds GetContinue() to ListMeta.
Kubernetes-commit: 8952a0cb722b77459cf2701632a30f5b264f5aba
Adding myself as a reviewer for the following areas:
- API
- auth
- registry
- storage (etcd)
Signed-off-by: Monis Khan <mkhan@redhat.com>
Kubernetes-commit: dd06794bc20ef1e0889af576c7a4f7a2f607e49d
This behavior regressed in an earlier release. Clearing the self link
ensures that a new version is always written and reduces the size of the
stored object by a small amount. Add tests to verify that Create and
Update result in no SelfLink stored in etcd.
Kubernetes-commit: 461c3701f0915acbf49c339f5321fa86879a963e
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data.
This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.
Kubernetes-commit: b851614adfe2b39941d518485480ff527fa4f0c1
Add support for creating resources that are not immediately visible to
naive clients, but must first be initialized by one or more privileged
cluster agents. These controllers can mark the object as initialized,
allowing others to see them.
Permission to override initialization defaults or modify an initializing
object is limited per resource to a virtual subresource "RESOURCE/initialize"
via RBAC.
Initialization is currently alpha.
Kubernetes-commit: 331eea67d8000e5c4b37e2234a90903c15881c2f
Add location transformer, config for transformers
Location transformer helps choose the most specific transformer for
read/write operations depending on the path of resource being accessed.
Configuration allows use of --experimental-encryption-provider-config
to set up encryption providers. Only AEAD is supported at the moment.
Add new files to BUILD, AEAD => k8s-aes-gcm
Use group resources to select encryption provider
Update tests for configuration parsing
Remove location transformer
Allow specifying providers per resource group in configuration
Add IdentityTransformer configuration option
Fix minor issues with initial AEAD implementation
Unified parsing of all configurations
Parse configuration using a union struct
Run configuration parsing in APIserver, refactor parsing
More gdoc, fix minor bugs
Add test coverage for combined transformers
Use table driven tests for encryptionconfig
Kubernetes-commit: 9760d00d08ef0619e30a7b1b90fd290cab960069
This change makes it so that errors during watch decoding panic the
server if it is in a test environment. This allows us to catch coder
errors related to storing incompatible types at the same location in
etcd.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Kubernetes-commit: a13f026fd012859f04467e6007e2cafe4a788927
The underlying storage has always returned the old object on watch
delete events when filtering. The cache watcher does not, which means a
downsteam caller gets different behavior.
This fixes the cache watcher to be consistent with our long term
behavior for watch. It may result in a behavior change (the filter
becomes more precise) but this was a regression in behavior.
Kubernetes-commit: e9e69356e4907fa4d0f45ea7e7768357ba71aba9
Tweak the ValueTransformer interface slightly to support additional
context information (to allow authenticated data to be generated by the
store and passed to the transformer). Add a prefix transformer that
looks for known matching prefixes and uses them. Add an AES GCM
transformer that performs AEAD on the values coming in and out of the
store.
Kubernetes-commit: f418468c87d3071c5d9ed14ce850996c77251080
change import of client-go/api/helper to kubernetes/api/helper
remove unnecessary use of client-go/api.registry
change use of client-go/pkg/util to kubernetes/pkg/util
remove dependency on client-go/pkg/apis/extensions
remove unnecessary invocation of k8s.io/client-go/extension/intsall
change use of k8s.io/client-go/pkg/apis/authentication to v1
Kubernetes-commit: c354076aa41e3cf417b291d5f0eff2b70395ac30
This change clean up the environment for etcd3 unit test.
Without this change, "make test" will leave some socket files in
workspace. And these socket files make hack/verify-generated-protobuf.sh
fails.
Kubernetes-commit: 4e9dcf3da536a9b956eb092fe9dbc5b2081cfb9d
Allows a transformer (such as an encrypter) to force an update if a new
key is in use, thus allowing simple writes to the REST layer to
trivially migrate keys.
This adds a new value transformer to the etcd2 store that can transform
the value from etcd on read and write. This will allow the store to
implement encryption at rest or otherwise transform the value prior to
persistence.
YuxiJin-tobeyjin
Jordan Liggitt
Ryan Phillips
Joe Betz
hzxuzhonghu
Christoph Blecker
Jeff Grafton
Wojciech Tyczynski
Di Xu
Saksham Sharma
Dr. Stefan Schimanski
Cao Shufeng
Davanum Srinivas
supereagle
Clayton Coleman
Hitoshi Mitake
Kubernetes Publisher
Tim Hockin
Slava Semushin
Monis Khan
deads2k
bjhaid
Tim Allclair
Chao Xu
NickrenREN
David Ashpole
Saksham Sharma
Mike Danese
Derek Carr
fate-grand-order