Add source API spec proposal
This commit is contained in:
stefanprodan
parent
96c7ad502c
commit
b4b85029fb
|
|
@ -0,0 +1,52 @@
|
|||
# Source Controller Proposal
|
||||
|
||||
## Context
|
||||
|
||||
The desired state of a cluster is made out of Kubernetes objects, these objects are expressed in `.yaml` format and
|
||||
are applied on the cluster by operators running inside the cluster. An operator's role is to fetch the Kubernetes
|
||||
objects, run transformations on them and reconcile the cluster state with the resulting manifest.
|
||||
|
||||
For an operator to acquire the resources that make up the desired state it needs to understand the communication
|
||||
protocol and the authentication scheme, verify the authenticity of a source and deal with rate limits and retries.
|
||||
In the FluxCD organization there are currently two operators that perform such operations. Both Flux and
|
||||
Helm Operator connect to Git repositories to fetch Kubernetes objects, they need to maintain an up-to-date mirror
|
||||
of one or several repos. Besides Git, Helm Operator needs to connect to Helm repositories hosted on public or
|
||||
private HTTPS servers.
|
||||
|
||||
## Motivation
|
||||
|
||||
Each Flux or Helm Operator instance maintains its own Git repository mirror even if all of them
|
||||
point to the same source. If the Git repository host becomes unavailable, the cluster state will diverge from the last
|
||||
know desired state since the operators will stop the reconciliation due to pull errors.
|
||||
|
||||
Decoupling the Kubernetes objects acquisition from the reconciliation process with an in-cluster
|
||||
source manager would make Flux and Helm Operator resilient to outbound connectivity issues and would
|
||||
simplify the state machine(s) that these controllers operate.
|
||||
|
||||
Managing the source operations in a dedicated controller could allow Flux to compose the desire state of a cluster
|
||||
from multiple source.
|
||||
Further more the manifests transformation process could be performed by 3rd party tools
|
||||
(e.g. kustomize, jk, tanka, cue run by Tekton pipelines or Kubernetes Jobs)
|
||||
that would output the final state of the Kubernetes objects into an artifacts repository maintained by the source controller.
|
||||
|
||||
## Goals
|
||||
|
||||
The main goal is to define a set of Kubernetes objects that cluster admins and various automated operators
|
||||
can interact with to offload the sources (e.g. Git and Helm repositories)
|
||||
registration, authentication and resource fetching to a dedicated controller.
|
||||
|
||||
The controller implementation will watch for source objects in a cluster and act on them.
|
||||
The actions performed by the source controller could be:
|
||||
* validate source definitions
|
||||
* authenticate to sources and validate authenticity
|
||||
* detect source changes based on update policies (semver)
|
||||
* fetch resources on-demand and on-a-schedule
|
||||
* package the fetched resources into a well known format (tar.gz)
|
||||
* store the artifacts locally
|
||||
* make the artifacts addressable by their source identifier (sha, version, ts)
|
||||
* make the artifacts available in-cluster to interested 3rd parties
|
||||
* notify interested 3rd parties of source changes and availability (status conditions, events, hooks)
|
||||
|
||||
## API Specification
|
||||
|
||||
* [v1alpha1](v1alpha1/README.md)
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
# source.fluxcd.io/v1alpha1
|
||||
|
||||
The is the v1alpha1 API specification for defining the desired state sources of Kubernetes clusters.
|
||||
|
||||
Source kinds:
|
||||
* [GitRepository](gitrepositories.md)
|
||||
* [HelmRepository](helmrepositories.md)
|
||||
|
||||
Implementations:
|
||||
* source-controller [v0.0.1-alpha.1](https://github.com/fluxcd/source-controller/releases)
|
||||
|
|
@ -0,0 +1,212 @@
|
|||
# Git Repositories
|
||||
|
||||
The `GitReposiory` API defines a source for artifacts coming from Git.
|
||||
|
||||
## Specification
|
||||
|
||||
Git repository spec:
|
||||
|
||||
```go
|
||||
// GitRepositorySpec defines the desired state of GitRepository
|
||||
type GitRepositorySpec struct {
|
||||
// +kubebuilder:validation:Pattern="^(http|https|ssh)://"
|
||||
|
||||
// The repository URL, can be a HTTP or SSH address.
|
||||
Url string `json:"url"`
|
||||
|
||||
// The secret name containing the Git credentials
|
||||
// +optional
|
||||
SecretRef *v1.LocalObjectReference `json:"secretRef,omitempty"`
|
||||
|
||||
// The git reference to checkout and monitor for changes.
|
||||
// +optional
|
||||
Reference *GitRepositoryRef `json:"ref,omitempty"`
|
||||
|
||||
// The interval at which to check for repository updates.
|
||||
Interval metav1.Duration `json:"interval"`
|
||||
}
|
||||
|
||||
// GitRepositoryRef defines the git ref used for pull and checkout operations
|
||||
type GitRepositoryRef struct {
|
||||
// The git branch to checkout, defaults to ('master').
|
||||
// +optional
|
||||
Branch string `json:"branch"`
|
||||
|
||||
// The git tag to checkout, takes precedence over branch.
|
||||
// +optional
|
||||
Tag string `json:"tag"`
|
||||
|
||||
// The git tag semver expression, takes precedence over tag.
|
||||
// +optional
|
||||
SemVer string `json:"semver"`
|
||||
}
|
||||
```
|
||||
|
||||
Git repository status:
|
||||
|
||||
```go
|
||||
// GitRepositoryStatus defines the observed state of GitRepository
|
||||
type GitRepositoryStatus struct {
|
||||
// +optional
|
||||
Conditions []RepositoryCondition `json:"conditions,omitempty"`
|
||||
|
||||
// LastUpdateTime is the timestamp corresponding to the last status
|
||||
// change of this repository.
|
||||
// +optional
|
||||
LastUpdateTime *metav1.Time `json:"lastUpdateTime,omitempty"`
|
||||
|
||||
// URI for the artifacts of the last successful repository sync.
|
||||
// +optional
|
||||
Artifacts string `json:"artifacts,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Git repository status conditions:
|
||||
|
||||
```go
|
||||
// RepositoryCondition contains condition information for a repository
|
||||
type RepositoryCondition struct {
|
||||
// Type of the condition, currently ('Ready').
|
||||
Type RepositoryConditionType `json:"type"`
|
||||
|
||||
// Status of the condition, one of ('True', 'False', 'Unknown').
|
||||
Status corev1.ConditionStatus `json:"status"`
|
||||
|
||||
// LastTransitionTime is the timestamp corresponding to the last status
|
||||
// change of this condition.
|
||||
// +optional
|
||||
LastTransitionTime *metav1.Time `json:"lastTransitionTime,omitempty"`
|
||||
|
||||
// Reason is a brief machine readable explanation for the condition's last
|
||||
// transition.
|
||||
// +optional
|
||||
Reason string `json:"reason,omitempty"`
|
||||
|
||||
// Message is a human readable description of the details of the last
|
||||
// transition, complementing reason.
|
||||
// +optional
|
||||
Message string `json:"message,omitempty"`
|
||||
}
|
||||
|
||||
// RepositoryConditionType represents an repository condition value
|
||||
type RepositoryConditionType string
|
||||
|
||||
const (
|
||||
// RepositoryConditionReady represents the fact that a given repository condition
|
||||
// is in ready state.
|
||||
RepositoryConditionReady RepositoryConditionType = "Ready"
|
||||
)
|
||||
```
|
||||
|
||||
## Spec examples
|
||||
|
||||
Public repository:
|
||||
|
||||
```yaml
|
||||
apiVersion: source.fluxcd.io/v1alpha1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: podinfo
|
||||
namespace: default
|
||||
annotations:
|
||||
# force sync trigger
|
||||
source.fluxcd.io/syncAt: "2020-04-06T15:39:52+03:00"
|
||||
spec:
|
||||
interval: 1m
|
||||
url: https://github.com/stefanprodan/podinfo
|
||||
ref:
|
||||
branch: master
|
||||
tag: "3.2.0"
|
||||
semver: ">= 3.2.0 <3.3.0"
|
||||
```
|
||||
|
||||
HTTPS authentication:
|
||||
|
||||
```yaml
|
||||
apiVersion: source.fluxcd.io/v1alpha1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: podinfo
|
||||
namespace: default
|
||||
spec:
|
||||
url: https://github.com/stefanprodan/podinfo
|
||||
secretRef:
|
||||
name: https-credentials
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: https-credentials
|
||||
namespace: default
|
||||
type: Opaque
|
||||
data:
|
||||
username: <BASE64>
|
||||
password: <BASE64>
|
||||
```
|
||||
|
||||
SSH authentication:
|
||||
|
||||
```yaml
|
||||
apiVersion: source.fluxcd.io/v1alpha1
|
||||
kind: GitRepository
|
||||
metadata:
|
||||
name: podinfo
|
||||
namespace: default
|
||||
spec:
|
||||
url: ssh://git@github.com:stefanprodan/podinfo
|
||||
secretRef:
|
||||
name: ssh-credentials
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: ssh-credentials
|
||||
namespace: default
|
||||
type: Opaque
|
||||
data:
|
||||
identity: <BASE64>
|
||||
identity.pub: <BASE64>
|
||||
know_hosts: <BASE64>
|
||||
```
|
||||
|
||||
Example of generating the SSH credentials secret:
|
||||
|
||||
```bash
|
||||
ssh-keygen -q -N "" -f ./identity
|
||||
ssh-keyscan github.com > ./know_hosts
|
||||
|
||||
kubectl create secret generic ssh-credentials \
|
||||
--from-file=./identity \
|
||||
--from-file=./identity.pub \
|
||||
--from-file=./know_hosts
|
||||
```
|
||||
|
||||
## Status examples
|
||||
|
||||
Successful sync:
|
||||
|
||||
```yaml
|
||||
status:
|
||||
artifacts: http://source-controller.source-system/repositories/podinfo-default/5e747d3e088cd7a34ace4abc8cf7f3c3696e402f.tar.gz
|
||||
conditions:
|
||||
- lastTransitionTime: "2020-04-07T06:59:23Z"
|
||||
message: 'Fetched artifacts are available at
|
||||
/data/repositories/podinfo-default/5e747d3e088cd7a34ace4abc8cf7f3c3696e402f.tar.gz'
|
||||
reason: GitCloneSucceed
|
||||
status: "True"
|
||||
type: Ready
|
||||
lastUpdateTime: "2020-04-07T06:59:23Z"
|
||||
```
|
||||
|
||||
Failed sync:
|
||||
|
||||
```yaml
|
||||
status:
|
||||
conditions:
|
||||
- lastTransitionTime: "2020-04-06T06:48:59Z"
|
||||
message: 'git clone error ssh: handshake failed: ssh: unable to authenticate,
|
||||
attempted methods [none publickey], no supported methods remain'
|
||||
reason: GitCloneFailed
|
||||
status: "False"
|
||||
type: Ready
|
||||
```
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Helm Repositories
|
||||
|
||||
The `HelmReposiory` and `HelmChart` API defines a source for artifacts coming from Helm repositories.
|
||||
|
||||
## Specification
|
||||
|
||||
TODO
|
||||
|
||||
## Spec examples
|
||||
|
||||
TODO
|
||||
|
||||
## Status examples
|
||||
|
||||
TODO
|
||||
Loading…
Reference in New Issue