Commit Graph

72 Commits

Author SHA1 Message Date
Tim Hockin d1afc12e67 Handle a race between ls-remote and fetch
This is a port of PR #413.
2021-06-23 12:51:15 -07:00
Tim Hockin ab0fe416de Rename slow_git to slow_git_clone
Making room for slow_git_fetch upcoming.
2021-06-23 12:28:12 -07:00
Tim Hockin 530ac2899b Clean up worktree dirs during sync loop
Avoid "already exists" error.  This is a port of #412 but I changed
the e2e to use `docker pause`.
2021-06-22 15:07:16 -07:00
Tim Hockin 9b1bfc7a97 Make e2e pass on v4 2021-06-22 15:04:00 -07:00
Tim Hockin c6878026a1 unbreak e2e for github 2021-06-22 08:29:30 -07:00
Tim Hockin 421fc4e298 Add a testcase for github HTTPS 2021-05-28 14:18:58 -07:00
Tim Hockin 0222a95f29 Make sure all code files have headers 2021-05-27 08:46:00 -07:00
Tim Hockin 16c3bf7dd1 move test cleanup to the end 2021-05-26 14:31:16 -07:00
Tim Hockin a3d9b673a8 Port #372 to v4 dev branch (from SpencerMalone)
Wanted to finally tackle #54, I sidestepped the problem of how to handle the volume of flags that might be required by instead specifying a sparsecheckout file.

The workflow as I've had has been...
- Do a local sparse checkout, add the files you want ignored (or included on if you did a cone pattern https://git-scm.com/docs/git-sparse-checkout#_cone_pattern_set)
- Grab your .git/info/sparecheckout file, and reserve it for later use with this new flag

It's not quite as easy as specifying it all from a CLI, but I think it's a reasonable first pass.

Here are some logs of it being run on https://github.com/SpencerMalone/logstash-output-prometheus:

```
test-repo % cat sparseconfig
!/*
!/*/
README.md
test-repo % docker run --rm -d \
    -v $(pwd)/git-data:/tmp/git \
    -v $(pwd):/test \
    docker.io/registry/git-sync:tag__linux_amd64  \
        --repo=https://github.com/SpencerMalone/logstash-output-prometheus.git \
        --branch=master \
        --sparse-checkout-file=/test/sparseconfig
41494548dd64caf0ff8f7b75e4d3a86014cfaefc40ff31b14ba19accf99aa82f
test-repo % ls git-data/db86200b1ab158ce9ad403d06de2301b15333601
README.md
```

As you can see, I ignored everything but the `README.md`, and sure enough only got that file in my final checkout.
2021-04-23 09:01:05 -07:00
Nan Yu f2581cff93 Export the error details to an error file
The current git-sync process outputs the error information to standard
out, which is inaccessible from outside the container. Users have to
dump the logs using kubectl logs in order to check the error details in
the git-sync process. This commit exports the error details to a file,
which provides users the capability to check the errors directly from
other sidecar containers.

proposal: https://github.com/kubernetes/git-sync/issues/326
2021-04-09 11:33:16 -07:00
Kubernetes Prow Robot 13af14e3bf
Merge pull request #370 from thockin/exec-hook-after-symlink-v4
Move sync-hook to after symlink flip
2021-04-07 16:47:42 -07:00
Tim Hockin d6c1fd3c6b Move sync-hook to after symlink flip 2021-04-05 17:02:00 -07:00
Tim Hockin dcb918f846 WIP: startup tests 2021-03-29 16:48:50 -07:00
Kubernetes Prow Robot 7b594781f8
Merge pull request #344 from thockin/fix-pid1-exit-code-v4
Fix exit non-zero exit codes when running as pid1
2021-03-15 09:39:06 -07:00
Tim Hockin 45b28530a7 Fix exit non-zero exit codes when running as pid1
(v4 branch)

Prior to this we would swallow the exit code and always exit(0).
2021-03-15 09:28:26 -07:00
Tim Hockin 83b4dd21de Add --git-config flag
This allows arbitrary git configs to be passed in.  For example:

`git config --global http.postBuffer 1048576000`

`git config --global http.sslCAInfo /path/to/cert/file`

`git config --global http.sslVerify false`

This flag takes a comma-separated list of `key:val` pairs.  The key part
is passed to `git config` and must be a valid gitconfig section header
and variable name.  The val part can be either a quoted or unquoted
value.  For all values the following escape sequences are supported:
   * `\n` => [newline]
   * `\t` => [tab]
   * `\"` => `"`
   * `\,` => `,`
   * `\\` => `\`

Within unquoted values, commas MUST be escaped. Within quoted values,
commas MAY be escaped, but are not required to be.  Any other escape
sequence is an error.

Example:

`--git-config=foo.one:val1,foo.two:"quoted val",foo.three:12345`

This commit exposed a bug in runCommand() which modified its args when
they had an embedded space.
2021-03-15 09:20:02 -07:00
Tim Hockin e5a438e446 Change the symlink targets to just the SHA
This allows users to call readlink() on the link and learn the current
checked out SHA.
2021-03-11 22:11:35 -08:00
Tim Hockin ccde597b7d Always specify branch name in e2e
As git changes the default from "master" to "main", it's not
predictable.  Now the test owns the decision.

This was already merged on release-3.x.
2021-01-23 16:13:26 -08:00
Tim Hockin a285a770e2 Use docker to run helper servers in e2e
Reliably cleaning up leftover things like `nc` processes is surprisingly
difficult in pure shell.  e2e passes on the master branch now.
2021-01-23 15:59:24 -08:00
Tim Hockin 24c06a5b48 Normalize the root path
This makes sure there's never any confusion about whether it is an
absolute path or has symlinks or whatever.  Add e2e cases to cover.
2020-11-09 09:20:24 -08:00
Tim Hockin 0e802450ee Don't allow --dest to start with a dot
BREAKING CHANGE

It's useful to reserve names that start with a dot.
2020-11-09 08:36:10 -08:00
Tim Hockin fa0e8696e2 Change time-related flags to durations
Add '--period' to replace '--wait', which is now obsolete.

Add '--sync-timeout' to replace '--timeout', which is now obsolete.

Both of these new flags take a Go-style time string, rather than a bare
number. For example "1s" for 1 second or "1m" for one minute.

The old flags have been kept and will take precedence if specified.
2020-11-05 16:46:09 -08:00
Tim Hockin 20696689f3 Add help and manual flags, use pflag
THIS IS A BREAKING CHANGE

Switch flags implementation to use pflag.  This means that long flags
like `-username` must now use 2 dashes: `--username`.

The `-v` flag (verbose) used to accept `-v` or `--v`.  Now it only
accepts `-v.

The `--help` and `-h` flags are new.

The `--man` flag is new (print a man-page like help message).

Several glog flags which used to be exposed (e.g. --logtostderr) are no
longer exposed.  Logs always go to stderr.
2020-10-31 00:50:51 -07:00
Tim Hockin 9203d14142 Specify "master" in e2e
Some git revs now use "main" but "master" is still coded into git-sync.
When that is fixed we can undo this.
2020-10-31 00:42:46 -07:00
Tim Hockin 771011f0cc Move --v into e2e framework (always same) 2020-10-02 14:38:19 -07:00
unknowingknow 9661d3132e add e2e test for --sync-hook-command 2020-10-02 19:29:21 +09:00
Tim Hockin 1992a2bfab logtostderr defaults to true: don't set it in test 2020-09-28 16:31:33 -07:00
Tim Hockin 44e94a16b0 Fix webhook regression: allow -1 to ignore status 2020-09-25 11:17:56 -07:00
Tim Hockin a29fab5bfa Remove useless test case 2020-09-04 20:03:37 -07:00
alix.cook11 dcf169c10a
add option to NOT recursively clone submodules (#246)
* add option to NOT recursively clone submodules

* Add "shallow" and "off" modes to submodule clone

* update readme to reflect new flag

* wording and such feedback
2020-07-30 21:29:07 -07:00
Tim Hockin ef8f5cf703 Bump to debian-base 2.0.0 (buster)
This changed `git fetch` to require a `-f` the way we use it.

Also fix a flaky test case for SSH.
2020-01-31 09:20:54 -08:00
Tim Hockin 72deefcec3 Make health check meaningful 2020-01-09 17:20:11 -08:00
Tim Hockin 1a1dd1cbe0 e2e: Add test for SSH
This requires having a docker image for git-over-ssh.
2020-01-07 08:25:36 -08:00
Tim Hockin fbdeead461 e2e: quote variables 2019-12-30 15:32:52 -08:00
Tim Hockin 9303c920d7 e2e: Use docker labels for cleanup 2019-12-30 15:32:52 -08:00
Tim Hockin a6cce7b757 e2e: move code around 2019-12-30 15:32:52 -08:00
Tim Hockin b56ea3796a e2e: reinit repo for each testcase 2019-12-30 15:32:52 -08:00
Tim Hockin ece56840f8 Fix askpass tests 2019-12-30 15:32:52 -08:00
Chuanying Du 5c70f02656 Verify both username and password in the askpass_git test. 2019-12-19 16:48:23 -08:00
Chuanying Du 0851cc5916 use free port 2019-12-19 16:26:46 -08:00
Chuanying Du 0b0f0a62d5 use random port for nc 2019-12-18 22:30:42 -08:00
Chuanying 27864c32cd
Merge branch 'master' into master 2019-12-18 22:11:57 -08:00
Chuanying Du d8d9ff72b8 add e2e test for askpasswd_url 2019-12-18 21:07:01 -08:00
Chuanying Du b0bdc02e8b manually merge https://github.com/kubernetes/git-sync/pull/217 2019-12-18 20:05:51 -08:00
Chuanying Du 66244108ed fix tests 2019-12-18 17:57:28 -08:00
Chuanying Du d58a241648 add comments; use --one-time for testing 2019-12-18 17:13:37 -08:00
Chuanying Du ac70751767 Change git credential from cache to store and also add test for
username/password case.

For cache to store change:
* By default, cache only last 900 seconds, gitsync will break after
  that. See https://git-scm.com/docs/git-credential-cache.
* The test won't work with cache since the test don't have access to
  the default unix socket location; XDG_CACHE_HOME override also can
  pre-create a socket in advance.
* `store` put the credential into a file, much easier to debug than cache.
* Considering anyone have access to the pod already able to get the
  credential via environment variables or yaml configs, so put it in
  file won't make it less secure.

For the new password test:
1. askpass_git.sh provided to simulate a git with password challenge.
2. Need and only need to similate "clone" action, need to bypass other
  actions like config/credential setup.
3. See `credential fill` is the official git action to ask password,
  see https://git-scm.com/docs/git-credential.

This change resolved issue #196.
2019-12-18 09:55:01 -08:00
Tim Hockin 27b43e624b e2e: move flags and better errors 2019-11-19 07:44:02 -08:00
Tim Hockin 2b11e078b0 Remove default-value flags in e2e 2019-11-19 07:44:02 -08:00
Tim Hockin 5a73199cf7 all e2e should use file://repo 2019-11-19 07:44:02 -08:00