diff --git a/docs/master/release/release-process.md b/docs/master/release/release-process.md new file mode 100644 index 00000000..5b6bc5ae --- /dev/null +++ b/docs/master/release/release-process.md @@ -0,0 +1,294 @@ +# Release Process + +This document is meant to be a complete end-to-end guide for how to release new versions of software +for Crossplane and its related projects. + +## tl;dr Process Overview + +All the details are available in the sections below, but we'll start this guide with a very high +level sequential overview for how to run the release process. + +1. **feature freeze**: Merge all completed features into master branches of all repos to begin + "feature freeze" period +1. **RC tag**: Run tag pipeline to tag the release candidate (RC) build in master for each repo +1. **branch**: Create a new release branch using the GitHub UI for each repo +1. **release stacks**: Run the release process for each **stack** that we maintain + 1. **test** Test builds from the release branch, fix any critical bugs that are found + 1. **version**: Update all version information in the stack metadata, docs, and integration + tests in the release branch + 1. **tag**: Run the tag pipeline to tag the release branch with an official semver + 1. **build/publish**: Run build pipeline to publish build with official semver +1. **docs/examples**: In Crossplane's release branch, update all examples, docs, and integration + tests to reference the new versions +1. **tag**: Run the tag pipeline to tag Crossplane's release branch with an official semver +1. **build/publish**: Run build pipeline to publish Crossplane build from release branch with + official semver +1. **verify**: Verify all artifacts have been published successfully, perform sanity testing +1. **release notes**: Publish well authored and complete release notes on GitHub +1. **promote**: Run promote pipelines on all repos to promote releases to desired channel(s) +1. **announce**: Announce the release on Twitter, Slack, etc. + +## Detailed Process + +This section will walk through the release process in more fine grained and prescriptive detail. + +### Scope + +This document will cover the release process for all of the repositories that the Crossplane team +maintains and publishes regular versioned artifacts from. This set of repositories covers both core +Crossplane and the set of Stacks that Crossplane currently maintains: + +* [`crossplane`](https://github.com/crossplaneio/crossplane) +* [`stack-gcp`](https://github.com/crossplaneio/stack-gcp) +* [`stack-aws`](https://github.com/crossplaneio/stack-aws) +* [`stack-azure`](https://github.com/crossplaneio/stack-azure) +* [`stack-rook`](https://github.com/crossplaneio/stack-rook) + + +The release process for Stacks is almost identical to that of core Crossplane because they use the +same [shared build logic](https://github.com/upbound/build/). The steps in this guide will apply to +all repositories listed above unless otherwise mentioned. + +### Feature Freeze + +Feature freeze should be performed on all repos. In order to start the feature freeze period, the +following conditions should be met: + +* All expected features should be ["complete"](#857) and merged into master. This includes user + guides, examples, API documentation via [crossdocs](https://github.com/negz/crossdocs/), and test + updates. +* All issues in the [milestone](https://github.com/crossplaneio/crossplane/milestones) should be + closed +* Sanity testing has been performed on `master` + +After these conditions are met, the feature freeze begins by creating the RC tag and the release +branch. + +### Release Candidate (RC) Tag + +The first step of feature freeze is to create the RC tag for the `HEAD` commit in `master`. This +tag serves as an indication of when the release was branched from master and is also important for +generating future versions of `master` builds since that [versioning +process](https://github.com/upbound/build/blob/master/makelib/common.mk#L182-L196) is based on `git +describe --tags`. + +> **NOTE:** The `tag` pipeline does not yet support extra tags in the version number, such as +`v0.5.0-rc`. [#330](https://github.com/crossplaneio/crossplane/issues/330) will be resolved when +this functionality is available. In the meantime, manually tagging and pushing to the repo +required. + +To accomplish this, run the `tag` pipeline for each repo on the `master` branch. You will be +prompted to enter the `version` for the tag and the `commit` hash to tag. It's possible to leave the +`commit` field blank to default to tagging `HEAD`. The `version` should be the release number that +we are branching for, plus a trailing tag to indicate it is an RC. For example: + +```console +v0.5.0-rc +``` + +After the tag pipeline has succeeded, verify in the [GitHub +UI](https://github.com/crossplaneio/crossplane/tags) that the tag was successfully applied to the +correct commit. + +### Create Release Branch + +Creating the release branch can be done within the [GitHub UI][github-ui-create-branch]. Basically, +you just use the branch selector drop down and type in the name of the new release branch, e.g. +`release-0.5`. Release branch names always follow the convention of `release-[minor-semver]`. + +If this is the first ever release branch being created in a repo (uncommon), you should also set up +branch protection rules for the `release-*` pattern. You can find existing examples in the +[Crossplane repo settings](https://github.com/crossplaneio/crossplane/settings/branches). + +At this point, the `HEAD` commit in the release branch will be our release candidate. The build +pipeline will automatically be started due to the create branch event, so we can start to perform +testing on this build. Note that it should be the exact same as what is currently in `master` since +they are using the same commit and have the same tag. Also note that this is not the official +release build since we have not made the official release tag yet (e.g. `v0.5.0`). + +The `master` branch can now be opened for new features since we have a safe release branch to +continue bug fixes and improvements for the release itself. Essentially, `master` is free to now +diverge from the release branch. + +#### Bug Fixes in Release Branch + +During our testing of the release candidate, we may find issues or bugs that we triage and decide we +want to fix before the release goes out. In order to fix a bug in the release branch, the following +process is recommended: + +1. Make the bug fix into `master` first through the normal PR process + 1. If the applicable code has already been removed from `master` then simply fix the bug + directly in the release branch by opening a PR directly against the release branch +1. Backport the fix by performing a cherry-pick of the fix's commit hash (**not** the merge commit) + from `master` into the release branch. For example, to backport a fix from master to `v0.5.0`, + something like the following should be used: + + ```console + git fetch --all + git checkout -b release-0.5 upstream/master + git cherry-pick -x + ``` + +1. Open a PR with the cherry-pick commit targeting the release-branch + +After all bugs have been fixed and backported to the release branch, we can move on to releasing the +stacks that the Crossplane community owns. Note that core Crossplane has still not been released +yet. + +### Draft Release Notes + +We're getting close to starting the official release, so you should take this opportunity to draft +up the release notes. You can create a [new release draft +here](https://github.com/crossplaneio/crossplane/releases/new). Make sure you select "This is a +pre-release" and hit "Save draft" when you are ready to share and collect feedback. Do **not** hit +"Publish release" yet. + +You can see and follow the template and structure from [previous +releases](https://github.com/crossplaneio/crossplane/releases). + +### Stack Release Process + +At this point, we should have a release branch created in all repos with all issues resolved that +were triaged to be release blockers. This section will walk through how to release the Stacks +themselves and does not directly apply to core Crossplane. + +In the **release branch** for each Stack, you should update the version tag or metadata in: + +* `app.yaml` - `version` +* `install.yaml` - `image` +* `integration_tests.sh` - `STACK_IMAGE` +* `*.resource.yaml` - docs links in markdown + * Not all of these `*.resource.yaml` files have links that need to be updated, they are infrequent + and inconsistent + +### Stack Tag Pipeline + +Now that the Stacks are all tested and their version metadata has been updated, it's time to tag the +release branch with the official version tag. You can do this by running the `tag` pipeline on the +release branch of each Stack: + +* [`stack-gcp` tag + pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fstack-gcp-pipelines%2Fstack-gcp-tag/branches) +* [`stack-aws` tag + pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fstack-aws-pipelines%2Fstack-aws-tag/branches/) +* [`stack-azure` tag + pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fstack-azure-pipelines%2Fstack-azure-tag/branches/) +* [`stack-rook` tag + pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fstack-rook-pipelines%2Fstack-rook-tag/branches/) + +Run the tag pipeline by clicking the Run button in the Jenkins UI in the correct release branch's +row. You will be prompted for the version you are tagging, e.g., `v0.5.0` as well as the commit +hash. The hash is optional and if you leave it blank it will default to `HEAD` of the branch, which +is what you want. + +> **Note:** The first time you run a pipeline on a new branch, you won't get prompted for the values +> to input. The build will quickly fail and then you can run (not replay) it a second time to be +> prompted. This is a Jenkins bug that is tracked by +> [#41929](https://issues.jenkins-ci.org/browse/JENKINS-41929) and has been open for almost 3 years, +> so don't hold your breath. + +After the tag pipeline has been run and the release branch has been tagged, you can run the normal +build pipeline on the release branch. This will kick off the official release build and upon +success, all release artifacts will be officially published. + +After the release build succeeds, verify that the correctly versioned Stack images have been pushed +to Docker Hub. + +### Core Crossplane release + +Back in the core Crossplane repository, we need to update the release branch docs and examples to +point to the freshly released stack versions. + +* Documentation, such as [Installation + instructions](https://github.com/crossplaneio/crossplane/blob/master/docs/install-crossplane.md#installing-cloud-provider-stacks), + and [Stack + guides](https://github.com/crossplaneio/crossplane/blob/master/docs/stacks-guide-gcp.md). + * searching for `:master` will help a lot here + * `kubectl -k` statements will contain `?ref=master`, which should be updated to the release + branch name instead, e.g. `?ref=release-0.4` +* Examples, such as [`StackInstall` yaml + files](https://github.com/crossplaneio/crossplane/tree/master/cluster/examples/stacks) +* [Helm chart + defaults](https://github.com/crossplaneio/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) + for stack versions + +### Tag Core Crossplane + +Similar to running the `tag` pipelines for each stack, now it's time to run the [`tag` +pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fcrossplane%2Fcrossplane-tag/branches) +for core Crossplane. In fact, the [instructions](#stack-tag-pipeline) are exactly the same: + +* Run the `tag` pipeline on the release branch +* Enter the version and commit hash (leave blank for `HEAD`) +* The first time you run on a new release branch, you won't be prompted and the build will fail, + just run (not replay) a second time + +### Build and Release Core Crossplane + +After tagging is complete, ensure the [normal build +pipeline](https://jenkinsci.upbound.io/blue/organizations/jenkins/crossplaneio%2Fcrossplane%2Fbuild/branches) +is run on the release branch. This will be the official release build with an official version +number and all of its release artifacts will be published. + +After the pipeline runs successfully, you should verify that all artifacts have been published to: + +* [Docker Hub](https://hub.docker.com/repository/docker/crossplane/crossplane) +* [S3 releases bucket](https://releases.crossplane.io/) +* [Helm chart repository](https://charts.crossplane.io/) +* [Docs website](https://crossplane.io/docs/latest) + +### Promotion + +If everything looks good with the official versioned release that we just published, we can go ahead +and run the `promote` pipeline for all repos. This is a very quick pipeline that doesn't rebuild +anything, it simply makes metadata changes to the published release to also include the last release +in the channel of your choice. Currently, we only support the `alpha` channel. + +For each repo, run the `promote` pipeline on the release branch and input the version you would like +to promote (e.g. `v0.5.0`) and the channel you'd like to promote it to (e.g. `alpha`). The first +time you run this pipeline on a new release branch, you will not be prompted for values, so the +pipeline will fail. Just run (not replay) it a second time to be prompted. + +After the `promote` pipeline has succeeded, verify on DockerHub and the Helm chart repository that +the release has been promoted to the right channel. + +### Publish Release Notes + +Now that the release has been published and verified, you can publish the [release +notes](https://github.com/crossplaneio/crossplane/releases) that you drafted earlier. After +incorporating all feedback, you can now click on the "Publish release" button. + +This will send an email notification with the release notes to all watchers of the repo. + +### Announce Release + +We have completed the entire release, so it's now time to announce it to the world. Using the +[@crossplane_io](https://twitter.com/crossplane_io) Twitter account, tweet about the new release and +blog. You'll see examples from the previous releases, such as this tweet for +[v0.4](https://twitter.com/crossplane_io/status/1189307636350705664). + +Post a link to this tweet on the Slack #announcements channel, then copy a link to that and post it +in the #general channel. + +### Patch Releases + +We also have the ability to run patch releases to update previous releases that have already been +published. These patch releases are always run from the last release branch, we do **not** create a +new release branch for a patch release. + +The basic flow is **very** similar to a normal release, but with a few less steps. Please refer to +details for each step in the sections above. + +* Fix any bugs in `master` first and then `cherry-pick -x` to the release branch + * If `master` has already removed the relevant code then make your fix directly in the release + branch +* After all testing on the release branch look good and any docs/examples/tests have been updated + with the new version number, run the `tag` pipeline on the release branch with the new patch + version (e.g. `v0.5.1`) +* Run the normal build pipeline on the release branch to build and publish the release +* Publish release notes +* Run promote pipeline to promote the patch release to the `alpha` channel + + +[github-ui-create-branch]: +https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-and-deleting-branches-within-your-repository