For back-compat, if we find both an old and new env var for the same
flag, prefer the old. This matters because the docker image sets
GITSYNC_ROOT but some users still set GIT_SYNC_ROOT.
Env-flags are "flags" that can only be set by env var (see caveat below).
All of the real flags have a corresponding env-flag (kind of, but not
really). The real goal was to deprecate `--password` but keep the env
var as a documented interface.
This does that (though --password still works) and updates the usage and
manual.
This allows some future work to follow the pattern. We do not register
every CLI flag as an env-flag because the help text would be
duplicative. This probably wants a wrapper API that allows declaring of
abstract flags, with CLI, env, or both sources.
Caveat:
ACTUALLY, these still have a flag, but the flag is specially named and
hidden. This makes testing a little easier where passing flags is
handled well but env vars is not.
The --ssh-key-file flag can be specified more than once and the
GITSYNC_SSH_KEY_FILE env var will be parsed like PATH.
Also adds e2e coverage for wrong-key and for multiple keys.
This ensures we do not miss events. E.g.
before:
t0: hash changes to X
t1: send webhook(X), waiting for response
t2: hash changes to Y
t3: queue next webhook(Y) but can't send because previous is not done
t4: crash
t5: restart
t6: find repo at hash Y
no webhook(Y) was sent.
after:
t0: hash changes to X
t1: send webhook(X), waiting for response
t2: hash changes to Y
t3: queue next webhook(Y) but can't send because previous is not done
t4: crash
t5: restart
t6: find repo at hash Y
t7: send webhook(Y), waiting for response
The previous (v3) sync loop betrays my lack of understanding about git.
It tried to codify my archaic mental model (e.g. --branch and --rev
being disting things) and was ultimately a patchwork of corner-cases
evolved over a few years.
This commit is less of a "diff" and more of a "rewrite".
The new logic is simpler and more efficient. It does not `git clone`
ever. It does not differentiate the first sync from subsequent syncs.
It uses `git fetch` to get the exact SHA and then makes a worktree from
that.
The new `--ref` flag replaces both `--rev` and `--branch`, though it
will use those if specified. In fact, almost all of the e2e tests
passed without change - using --ref and --branch!
I will follow this commit up with more cleanups and e2es.
A new flag `--password-file` is added. This allows git-sync to read
password from file and this is considered as safer than reading from
env or flag directly.
Few more checks are added as well:
1. `--password` and `--password-file` can't be specified at the same
time.
1. If `--username` is specified, then one of `--password` or
`--password-file` must be specified.
Wanted to finally tackle #54, I sidestepped the problem of how to handle the volume of flags that might be required by instead specifying a sparsecheckout file.
The workflow as I've had has been...
- Do a local sparse checkout, add the files you want ignored (or included on if you did a cone pattern https://git-scm.com/docs/git-sparse-checkout#_cone_pattern_set)
- Grab your .git/info/sparecheckout file, and reserve it for later use with this new flag
It's not quite as easy as specifying it all from a CLI, but I think it's a reasonable first pass.
Here are some logs of it being run on https://github.com/SpencerMalone/logstash-output-prometheus:
```
test-repo % cat sparseconfig
!/*
!/*/
README.md
test-repo % docker run --rm -d \
-v $(pwd)/git-data:/tmp/git \
-v $(pwd):/test \
docker.io/registry/git-sync:tag__linux_amd64 \
--repo=https://github.com/SpencerMalone/logstash-output-prometheus.git \
--branch=master \
--sparse-checkout-file=/test/sparseconfig
41494548dd64caf0ff8f7b75e4d3a86014cfaefc40ff31b14ba19accf99aa82f
test-repo % ls git-data/db86200b1ab158ce9ad403d06de2301b15333601
README.md
```
As you can see, I ignored everything but the `README.md`, and sure enough only got that file in my final checkout.
The current git-sync process outputs the error information to standard
out, which is inaccessible from outside the container. Users have to
dump the logs using kubectl logs in order to check the error details in
the git-sync process. This commit exports the error details to a file,
which provides users the capability to check the errors directly from
other sidecar containers.
proposal: https://github.com/kubernetes/git-sync/issues/326
This allows arbitrary git configs to be passed in. For example:
`git config --global http.postBuffer 1048576000`
`git config --global http.sslCAInfo /path/to/cert/file`
`git config --global http.sslVerify false`
This flag takes a comma-separated list of `key:val` pairs. The key part
is passed to `git config` and must be a valid gitconfig section header
and variable name. The val part can be either a quoted or unquoted
value. For all values the following escape sequences are supported:
* `\n` => [newline]
* `\t` => [tab]
* `\"` => `"`
* `\,` => `,`
* `\\` => `\`
Within unquoted values, commas MUST be escaped. Within quoted values,
commas MAY be escaped, but are not required to be. Any other escape
sequence is an error.
Example:
`--git-config=foo.one:val1,foo.two:"quoted val",foo.three:12345`
This commit exposed a bug in runCommand() which modified its args when
they had an embedded space.
Add '--period' to replace '--wait', which is now obsolete.
Add '--sync-timeout' to replace '--timeout', which is now obsolete.
Both of these new flags take a Go-style time string, rather than a bare
number. For example "1s" for 1 second or "1m" for one minute.
The old flags have been kept and will take precedence if specified.
THIS IS A BREAKING CHANGE
Switch flags implementation to use pflag. This means that long flags
like `-username` must now use 2 dashes: `--username`.
The `-v` flag (verbose) used to accept `-v` or `--v`. Now it only
accepts `-v.
The `--help` and `-h` flags are new.
The `--man` flag is new (print a man-page like help message).
Several glog flags which used to be exposed (e.g. --logtostderr) are no
longer exposed. Logs always go to stderr.
Tim Hockin
Liam Wyllie
VICIWUOHA
demoth
Ofek Buchnik
Kubernetes Prow Robot
Haibing Zhou
XinYang
Nan Yu
Haiyan Meng
Xiang Dai
Andres Lowrie
Michael Grosser
unknowingknow