community/contributors/devel/strategic-merge-patch.md

Strategic Merge Patch

Background

Kubernetes supports a customized version of JSON merge patch called strategic merge patch. This patch format is used by kubectl apply, kubectl edit and kubectl patch, and contains specialized directives to control how specific fields are merged.

In the standard JSON merge patch, JSON objects are always merged but lists are always replaced. Often that isn't what we want. Let's say we start with the following Pod:

spec:
  containers:
    - name: nginx
      image: nginx-1.0

and we POST that to the server (as JSON). Then let's say we want to add a container to this Pod.

PATCH /api/v1/namespaces/default/pods/pod-name
spec:
  containers:
    - name: log-tailer
      image: log-tailer-1.0

If we were to use standard Merge Patch, the entire container list would be replaced with the single log-tailer container. However, our intent is for the container lists to merge together based on the name field.

To solve this problem, Strategic Merge Patch uses the go struct tag of the API objects to determine what lists should be merged and which ones should not. Currently the metadata is available as struct tags on the API objects themselves, but will become available to clients as Swagger annotations in the future. In the above example, the patchStrategy metadata for the containers field would be merge and the patchMergeKey would be name.

Basic Patch Format

Strategic Merge Patch supports special operations through directives.

There are multiple directives:

• replace
• merge
• delete
• delete from primitive list

replace, merge and delete are mutual exclusive.

replace Directive

Purpose

replace directive indicates that the element that contains it should be replaced instead of being merged.

Syntax

replace directive is used in both patch with directive marker and go struct tags.

Example usage in the patch:

$patch: replace

Example

replace directive can be used on both map and list.

Map

To indicate that a map should not be merged and instead should be taken literally:

$patch: replace  # recursive and applies to all fields of the map it's in
containers:
- name: nginx
  image: nginx-1.0

List of Maps

To override the container list to be strictly replaced, regardless of the default:

containers:
  - name: nginx
    image: nginx-1.0
  - $patch: replace   # any further $patch operations nested in this list will be ignored

delete Directive

Purpose

delete directive indicates that the element that contains it should be deleted.

Syntax

delete directive is used only in the patch with directive marker. It can be used on both map and list of maps.

$patch: delete

Example

List of Maps

To delete an element of a list that should be merged:

containers:
  - name: nginx
    image: nginx-1.0
  - $patch: delete
    name: log-tailer  # merge key and value goes here

Note: Delete operation will delete all entries in the list that match the merge key.

Maps

One way to delete a map is using delete directive. Applying this patch will delete the rollingUpdate map.

rollingUpdate:
  $patch: delete

An equivalent way to delete this map is

rollingUpdate: null

merge Directive

Purpose

merge directive indicates that the element that contains it should be merged instead of being replaced.

Syntax

merge directive is used only in the go struct tags.

deleteFromPrimitiveList Directive

Purpose

We have two patch strategies for lists of primitives: replace and merge. Replace is the default patch strategy for list, which will replace the whole list on update and it will preserve the order; while merge strategy works as an unordered set. We call a primitive list with merge strategy an unordered set. The patch strategy is defined in the go struct tag of the API objects.

deleteFromPrimitiveList directive indicates that the elements in this list should be deleted from the original primitive list.

Syntax

It is used only as the prefix of the key in the patch.

$deleteFromPrimitiveList/<keyOfPrimitiveList>: [a primitive list]

Example

List of Primitives (Unordered Set)

finalizers uses merge as patch strategy.

Finalizers []string `json:"finalizers,omitempty" patchStrategy:"merge" protobuf:"bytes,14,rep,name=finalizers"`

Suppose we have defined a finalizers and we call it the original finalizers:

finalizers:
  - a
  - b
  - c

To delete items "b" and "c" from the original finalizers, the patch will be:

# The directive includes the prefix $deleteFromPrimitiveList and
# followed by a '/' and the name of the list.
# The values in this list will be deleted after applying the patch.
$deleteFromPrimitiveList/finalizers:
  - b
  - c

After applying the patch on the original finalizers, it will become:

finalizers:
  - a

Note: When merging two set, the primitives are first deduplicated and then merged. In an erroneous case, the set may be created with duplicates. Deleting an item that has duplicates will delete all matching items.

Changing patch format

As issues and limitations have been discovered with the strategic merge patch implementation, it has been necessary to change the patch format to support additional semantics - such as merging lists of primitives and defining order when merging lists.

Requirments for any changes to the patch format

Note: Changes to the strategic merge patch must be backwards compatible such that patch requests valid in previous versions continue to be valid. That is, old patch formats sent by old clients to new servers with must continue to function correctly.

Previously valid patch requests do not need to keep the exact same behavior, but do need to behave correctly.

Example: if a patch request previously randomized the order of elements in a list and we want to provide a deterministic order, we must continue to support old patch format but we can make the ordering deterministic for the old format.

Client version skew

Because the server does not publish which patch versions it supports, and it silently ignores patch directives that it does not recognize, new patches should behave correctly when sent to old servers that may not support all of the patch directives.

While the patch API must be backwards compatible, it must also be forward compatible for 1 version. This is needed because kubectl must support talking to older and newer server versions without knowing what parts of patch are supported on each, and generate patches that work correctly on both.

Strategies for introducing new patch behavior

1. Add optional semantic meaning to the existing patch format.

Note: Must not require new data or elements to be present that was not required before. Meaning must not break old interpretation of old patches.

Good Example:

Old format

• ordering of elements in patch had no meaning and the final ordering was arbitrary

New format

• ordering of elements in patch has meaning and the final ordering is deterministic based on the ordering in the patch

Bad Example:

Old format

• fields not present in a patch for Kind foo are ignored
• unmodified fields for Kind foo are optional in patch request

New format

• fields not present in a patch for Kind foo are cleared
• unmodified fields for Kind foo are required in patch request

This example won't work, because old patch formats will contain data that is now considered required. To support this, introduce a new directive to guard the new patch format.

2. Add support for new directives in the patch format

• Optional directives may be introduced to change how the patch is applied by the server - backwards compatible (old patch against newer server).

  • May control how the patch is applied
  • May contain patch information - such as elements to delete from a list
  • Must NOT impose new requirements on the old patch format

• New patch requests should be a superset of old patch requests - forwards compatible (newer patch against older server)

  • Old servers will ignore directives they do not recognize
  • Must include the full patch that would have been sent before the new directives were added.
  • Must NOT rely on the directive being supported by the server

Good Example:

Old format

• fields not present in a patch for Kind foo are ignored
• unmodified fields for Kind foo are optional in patch request

New format without directive

• Same as old

New format with directive

• fields not present in a patch for Kind foo are cleared
• unmodified fields for Kind foo are required in patch request

In this example, the behavior was unchanged when the directive was missing, retaining the old behavior for old patch requests.

Bad Example:

Old format

• fields not present in a patch for Kind foo are ignored
• unmodified fields for Kind foo are optional in patch request

New format with directive

• Same as old

New format without directive

• fields not present in a patch for Kind foo are cleared
• unmodified fields for Kind foo are required in patch request

In this example, the behavior was changed when the directive was missing, breaking compatibility.

Alternatives

The previous strategy is necessary because there is no notion of patch versions. Having the client negotiate the patch version with the server would allow changing the patch format, but at the cost of supporting multiple patch formats in the server and client. Using client provided directives to evolve how a patch is merged provides some limited support for multiple versions.