Merge pull request #8436 from BenTheElder/no-godeps
delete defunct godeps docs
This commit is contained in:
		
						commit
						c9f4e7e868
					
				|  | @ -1,268 +1,2 @@ | ||||||
| **Note**: Kubernetes now manages dependencies using go modules. | **Note**: Kubernetes now manages dependencies using go modules. | ||||||
| See [current documentation for working with dependencies](./vendor.md) for master branch development. | See [current documentation for working with dependencies](./vendor.md) for master branch development. | ||||||
| This document only applies to Kubernetes 1.14.x and earlier, |  | ||||||
| and should be removed once Kubernetes 1.14.x is no longer supported. |  | ||||||
| 
 |  | ||||||
| # Using godep to manage dependencies |  | ||||||
| 
 |  | ||||||
| This document is intended to show a way for managing `vendor/` tree dependencies |  | ||||||
| in Kubernetes. If you do not need to manage vendored dependencies, you probably |  | ||||||
| do not need to read this. |  | ||||||
| 
 |  | ||||||
| ## Background |  | ||||||
| 
 |  | ||||||
| As a tool, `godep` leaves much to be desired.  It builds on `go get`, and adds |  | ||||||
| the ability to pin dependencies to exact git version. The `go get` tool itself |  | ||||||
| doesn't have any concept of versions, and tends to blow up if it finds a git |  | ||||||
| repo synced to anything but `master`, but that is exactly the state that |  | ||||||
| `godep` leaves repos.  This is a recipe for frustration when people try to use |  | ||||||
| the tools. |  | ||||||
| 
 |  | ||||||
| This doc will focus on predictability and reproducibility. |  | ||||||
| 
 |  | ||||||
| ## Justifications for an update |  | ||||||
| 
 |  | ||||||
| Before you update a dependency, take a moment to consider why it should be  |  | ||||||
| updated. Valid reasons include: |  | ||||||
|  1. We need new functionality that is in a later version. |  | ||||||
|  2. New or improved APIs in the dependency significantly improve Kubernetes code. |  | ||||||
|  3. Bugs were fixed that impact Kubernetes. |  | ||||||
|  4. Security issues were fixed even if they don't impact Kubernetes yet. |  | ||||||
|  5. Performance, scale, or efficiency was meaningfully improved. |  | ||||||
|  6. We need dependency A and there is a transitive dependency B. |  | ||||||
|  7. Kubernetes has an older level of a dependency that is precluding being able |  | ||||||
| to work with other projects in the ecosystem. |  | ||||||
| 
 |  | ||||||
| ## Theory of operation |  | ||||||
| 
 |  | ||||||
| The `go` toolchain assumes a global workspace that hosts all of your Go code. |  | ||||||
| 
 |  | ||||||
| The `godep` tool operates by first "restoring" dependencies into your `$GOPATH`. |  | ||||||
| This reads the `Godeps.json` file, downloads all of the dependencies from the |  | ||||||
| internet, and syncs them to the specified revisions.  You can then make |  | ||||||
| changes - sync to different revisions or edit Kubernetes code to use new |  | ||||||
| dependencies (and satisfy them with `go get`).  When ready, you tell `godep` to |  | ||||||
| "save" everything, which it does by walking the Kubernetes code, finding all |  | ||||||
| required dependencies, copying them from `$GOPATH` into the `vendor/` directory, |  | ||||||
| and rewriting `Godeps.json`. |  | ||||||
| 
 |  | ||||||
| This does not work well, when combined with a global Go workspace.  Instead, we |  | ||||||
| will set up a private workspace for this process. |  | ||||||
| 
 |  | ||||||
| The Kubernetes build process uses this same technique, and offers a tool called |  | ||||||
| `run-in-gopath.sh` which sets up and switches to a local, private workspace, |  | ||||||
| including setting up `$GOPATH` and `$PATH`.  If you wrap commands with this |  | ||||||
| tool, they will use the private workspace, which will not conflict with other |  | ||||||
| projects and is easily cleaned up and recreated. |  | ||||||
| 
 |  | ||||||
| To see this in action, you can run an interactive shell in this environment: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| # Run a shell, but don't run your own shell initializations. |  | ||||||
| hack/run-in-gopath.sh bash --norc --noprofile |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| ## Restoring deps |  | ||||||
| 
 |  | ||||||
| To extract and download dependencies into `$GOPATH` we provide a script: |  | ||||||
| `hack/godep-restore.sh`.  If you run this tool, it will restore into your own |  | ||||||
| `$GOPATH`.  If you wrap it in `run-in-gopath.sh` it will restore into your |  | ||||||
| `_output/` directory. |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh hack/godep-restore.sh |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| This script will try to optimize what it needs to download, and if it seems the |  | ||||||
| dependencies are all present already, it will return very quickly. |  | ||||||
| 
 |  | ||||||
| If there's ever any doubt about the correctness of your dependencies, you can |  | ||||||
| simply `make clean` or `rm -rf _output`, and run it again. |  | ||||||
| 
 |  | ||||||
| Now you should have a clean copy of all of the Kubernetes dependencies. |  | ||||||
| 
 |  | ||||||
| Downloading dependencies might take a while, so if you want to see progress |  | ||||||
| information use the `-v` flag: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh hack/godep-restore.sh -v |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| ## Making changes |  | ||||||
| 
 |  | ||||||
| The most common things people need to do with deps are add and update them. |  | ||||||
| These are similar but different. |  | ||||||
| 
 |  | ||||||
| ### Adding a dep |  | ||||||
| 
 |  | ||||||
| For the sake of examples, consider that we have discovered a wonderful Go |  | ||||||
| library at `example.com/go/frob`.  The first thing you need to do is get that |  | ||||||
| code into your workspace: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh go get -d example.com/go/frob |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| This will fetch, but not compile (omit the `-d` if you want to compile it now), |  | ||||||
| the library into your private `$GOPATH`.  It will pull whatever the default |  | ||||||
| revision of that library is, typically the `master` branch for git repositories. |  | ||||||
| If this is not the revision you need, you can change it, for example to |  | ||||||
| `v1.0.0`: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh bash -c 'git -C $GOPATH/src/example.com/go/frob checkout v1.0.0' |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| Now that the code is present, you can start to use it in Kubernetes code. |  | ||||||
| Because it is in your private workspace's `$GOPATH`, it might not be part of |  | ||||||
| your own `$GOPATH`, so tools like `goimports` might not find it.  This is an |  | ||||||
| unfortunate side-effect of this process.  You can either add the whole private |  | ||||||
| workspace to your own `$GOPATH` or you can `go get` the library into your own |  | ||||||
| `$GOPATH` until it is properly vendored into Kubernetes. |  | ||||||
| 
 |  | ||||||
| Another possible complication is a dep that uses `gopdep` itself.  In that case, |  | ||||||
| you need to restore its dependencies, too: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh bash -c 'cd $GOPATH/src/example.com/go/frob && godep restore' |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| If the transitive deps collide with Kubernetes deps, you may have to manually |  | ||||||
| resolve things.  This is where the ability to run a shell in this environment |  | ||||||
| comes in handy: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh bash --norc --noprofile |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| ### Updating a dep |  | ||||||
| 
 |  | ||||||
| Sometimes we already have a dep, but the version of it is wrong.  Because of the |  | ||||||
| way that `godep` and `go get` interact (badly) it's generally easiest to hit it |  | ||||||
| with a big hammer: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh bash -c 'rm -rf $GOPATH/src/example.com/go/frob' |  | ||||||
| hack/run-in-gopath.sh go get -d example.com/go/frob |  | ||||||
| hack/run-in-gopath.sh bash -c 'git -C $GOPATH/src/example.com/go/frob checkout v2.0.0' |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| This will remove the code, re-fetch it, and sync to your desired version. |  | ||||||
| 
 |  | ||||||
| ### Removing a dep |  | ||||||
| 
 |  | ||||||
| This happens almost for free.  If you edit Kubernetes code and remove the last |  | ||||||
| use of a given dependency, you only need to restore and save the deps, and the |  | ||||||
| `godep` tool will figure out that you don't need that dep any more: |  | ||||||
| 
 |  | ||||||
| ## Saving deps |  | ||||||
| 
 |  | ||||||
| Now that you have made your changes - adding, updating, or removing the use of a |  | ||||||
| dep - you need to rebuild the dependency database and make changes to the |  | ||||||
| `vendor/` directory. |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh hack/godep-save.sh |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| This will run through all of the primary targets for the Kubernetes project, |  | ||||||
| calculate which deps are needed, and rebuild the database.  It will also |  | ||||||
| regenerate other metadata files which the project needs, such as BUILD files and |  | ||||||
| the LICENSE database. |  | ||||||
| 
 |  | ||||||
| Commit the changes before updating deps in staging repos. |  | ||||||
| 
 |  | ||||||
| ## Saving deps in staging repos |  | ||||||
| 
 |  | ||||||
| Kubernetes stores some code in a directory called `staging` which is handled |  | ||||||
| specially, and is not covered by the above.  If you modified any code under |  | ||||||
| staging, or if you changed a dependency of code under staging (even |  | ||||||
| transitively), you'll also need to update deps there: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| ./hack/update-staging-godeps.sh |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| Then commit the changes generated by the above script. |  | ||||||
| 
 |  | ||||||
| ## Commit messages |  | ||||||
| 
 |  | ||||||
| Terse messages like "Update foo.org/bar to 0.42" are problematic |  | ||||||
| for maintainability.  Please include in your commit message the |  | ||||||
| detailed reason why the dependencies were modified. |  | ||||||
| 
 |  | ||||||
| Too commonly dependency changes have a ripple effect where something |  | ||||||
| else breaks unexpectedly.  The first instinct during issue triage |  | ||||||
| is to revert a change.  If the change was made to fix some other |  | ||||||
| issue and that issue was not documented, then a revert simply |  | ||||||
| continues the ripple by fixing one issue and reintroducing another |  | ||||||
| which then needs refixed.  This can needlessly span multiple days |  | ||||||
| as CI results bubble in and subsequent patches fix and refix and |  | ||||||
| rerefix issues.  This may be avoided if the original modifications |  | ||||||
| recorded artifacts of the change rationale. |  | ||||||
| 
 |  | ||||||
| ## Sanity checking |  | ||||||
| 
 |  | ||||||
| After all of this is done, `git status` should show you what files have been |  | ||||||
| modified and added/removed.  Make sure to sanity-check them with `git diff`, and |  | ||||||
| to `git add` and `git rm` them, as needed.  It is commonly advised to make one |  | ||||||
| `git commit` which includes just the dependencies and Godeps files, and |  | ||||||
| another `git commit` that includes changes to Kubernetes code to use (or stop |  | ||||||
| using) the new/updated/removed dependency.  These commits can go into a single |  | ||||||
| pull request. |  | ||||||
| 
 |  | ||||||
| Before sending your PR, it's a good idea to sanity check that your |  | ||||||
| Godeps.json file and the contents of `vendor/ `are ok: |  | ||||||
| 
 |  | ||||||
| ```sh |  | ||||||
| hack/run-in-gopath.sh hack/verify-godeps.sh |  | ||||||
| ``` |  | ||||||
| 
 |  | ||||||
| All this script will do is a restore, followed by a save, and then look for |  | ||||||
| changes.  If you followed the above instructions, it should be clean.  If it is |  | ||||||
| not, you get to figure out why. |  | ||||||
| 
 |  | ||||||
| ## Manual updates |  | ||||||
| 
 |  | ||||||
| It is sometimes expedient to manually fix the `Godeps.json` file to |  | ||||||
| minimize the changes. However, without great care this can lead to failures |  | ||||||
| with the verifier scripts.  The kubernetes codebase does "interesting things" |  | ||||||
| with symlinks between `vendor/` and `staging/` to allow multiple Go import |  | ||||||
| paths to coexist in the same git repo. |  | ||||||
| 
 |  | ||||||
| The verifiers, including `hack/verify-godeps.sh` *must* pass for every pull |  | ||||||
| request. |  | ||||||
| 
 |  | ||||||
| ## Reviewing and approving dependency changes |  | ||||||
| 
 |  | ||||||
| Particular attention to detail should be exercised when reviewing and approving |  | ||||||
| PRs that add/remove/update dependencies. Importing a new dependency should bring |  | ||||||
| a certain degree of value as there is a maintenance overhead for maintaining |  | ||||||
| dependencies into the future. |  | ||||||
| 
 |  | ||||||
| When importing a new dependency, be sure to keep an eye out for the following: |  | ||||||
| - Is the dependency maintained? |  | ||||||
| - Does the dependency bring value to the project? Could this be done without |  | ||||||
|   adding a new dependency? |  | ||||||
| - Is the target dependency the original source, or a fork? |  | ||||||
| - Is there already a dependency in the project that does something similar? |  | ||||||
| - Does the dependency have a license that is compatible with the Kubernetes |  | ||||||
|   project? |  | ||||||
| 
 |  | ||||||
| Additionally: |  | ||||||
| - Look at the godeps file. Check that the only changes are what the PR claims |  | ||||||
|   them to be.  |  | ||||||
| - Check if there is a tagged release we can vendor instead of a random hash |  | ||||||
| - Scan the imported code for things like init() functions |  | ||||||
| - Look at the Kubernetes code changes and make sure they are appropriate |  | ||||||
|   (e.g. renaming imports or similar). You do not need to do feature code review. |  | ||||||
| - If this is all good, approve, but don't LGTM, unless you also do code review |  | ||||||
|   or unless it is trivial (e.g. moving from k/k/pkg/utils -> k/utils). |  | ||||||
| 
 |  | ||||||
| All new dependency licenses should be reviewed by @kubernetes/dep-approvers to ensure that they |  | ||||||
| are compatible with the Kubernetes project license. It is also important to note |  | ||||||
| and flag if a license has changed when updating a dependency, so that these can |  | ||||||
| also be reviewed. |  | ||||||
| 
 |  | ||||||
| For reference, whitelisted licenses as per the CNCF Whitelist Policy are |  | ||||||
| mentioned [here](https://git.k8s.io/sig-release/licensing/README.md#licenses-for-dependencies). |  | ||||||
|  |  | ||||||
		Loading…
	
		Reference in New Issue