Commit Graph

154 Commits

Author SHA1 Message Date
Tim Hockin 9af6b7cfe2 e2e: Don't manually "docker kill"
It happens automatically on test cleanup
2022-06-24 08:45:05 -07:00
Tim Hockin a341fb62f9 Make ssh e2e test not use --one-time
There is functionality that needs to be tested beyond that.
2022-06-11 10:56:20 -07:00
Tim Hockin 2c6356cbcb Pass CLEANUP=0 to test_e2e to leave logs 2022-06-11 19:49:11 +02:00
Tim Hockin f8a532d9df Pass the whole environment to exechooks 2022-05-16 18:14:09 -07:00
Tim Hockin 535735607f Adapt testcases for v4 2022-03-11 08:41:57 -08:00
sed-i ff3d11c925 Add repo-sync test 2022-03-11 08:34:16 -08:00
Tim Hockin 8c5f33d5dd Make e2e treat args as tests-name regexes 2022-02-24 09:00:57 -08:00
Tim Hockin f4d124bded Add --git-gc flag to control GC on each sync
Values:
* "auto" - run `git gc --auto` (default, respects git gc.* configs)
* "always" - run `git gc`
* "aggressive" - run `git gc --aggressive` (may require a longer timeout)
* "off" - do not run `git gc` on each sync (good for --one-time use)
2022-02-24 08:00:10 -08:00
Tim Hockin 2538d293ba Set repoReady even when there wasn't a 1st clone
E.g. if the repo is already present (after a restart).
2022-02-12 17:38:36 -08:00
Kubernetes Prow Robot d3324e0b90
Merge pull request #478 from thockin/v4-link-abspath
Allow --dest to be an absolute path (v4)
2022-01-23 02:41:56 -08:00
Kubernetes Prow Robot 6dc0e44697
Merge pull request #472 from thockin/master
Don't try to remove the root if it appears corrupt (v4)
2022-01-23 01:15:56 -08:00
Tim Hockin 4fc67f5f5f Allow --dest to be an absolute path
This means it might live outside of --root.
2022-01-22 23:39:56 -08:00
Tim Hockin 248a53461d Don't try to remove the root if it appears corrupt
The `--root` is often a volume and can't be removed.  Instead, remove
the contents of it.  Adjust tests to hit this.
2022-01-22 14:28:09 -08:00
Tim Hockin 9eed6946b7 e2e cleanup
Fix some whitespace and names.
Make command-line and flag handling a bit cleaner.
2022-01-19 11:38:23 -08:00
ChrisERo 5490b721d8 fix: made tests more efficient by waiting less 2022-01-14 13:53:09 -05:00
ChrisERo 043c356c03 fix: fixed e2e tests
Resolved a few issues with e2e tests discovered after running
test_e2d.sh. Just finished setting up Linux (Fedora) environment in
which this script can be run.
2022-01-14 03:23:20 -05:00
ChrisERo 0246fb509f feat: added e2e --one-time tests for webhook 2022-01-10 20:06:25 -05:00
ChrisERo 8e098f8c2a fix: made exechook_once tests more efficient
* Removed multiple iterations of tests
* Reduced sleep time after GIT_SYNC command execution
2022-01-10 20:06:24 -05:00
ChrisERo a362b4a5f4 feat: added tests for this PR that test exechook paired with --one-time flag 2022-01-10 20:06:21 -05:00
Tim Hockin 7b32caee77 v4: use repo default branch instead of master 2021-11-01 23:05:48 -07:00
Tim Hockin 2b471ba2d7 E2E: build IFF running tests 2021-10-18 09:36:52 -07:00
Tim Hockin 6e52fb0d22 E2E: Add whitespace for reading 2021-10-18 09:36:52 -07:00
Tim Hockin 320a4a7e93 E2E: Move code
Reduce the diff between v3 and v4
2021-10-18 09:36:52 -07:00
Tim Hockin 27f2c905fc E2E: Rename test funcs 2021-10-18 09:36:52 -07:00
Tim Hockin dfcafc9ca0 E2E: Fix endless loop on error 2021-10-18 09:36:52 -07:00
Tim Hockin e2ed136aad E2E: make all test cases be funcs
Make all test cases functions so they can be called individually on the
CLI.  Call test_e2e.sh with `-?` to list tests.   Fix related crashes
(uses of $TESTCASE and so on).
2021-10-18 09:36:52 -07:00
Tim Hockin e06f9c6acb E2E: Make SIGINT handling more complete 2021-10-18 09:36:52 -07:00
Tim Hockin 7b3454e51b E2E: Use vars for branch names 2021-10-18 09:36:52 -07:00
Tim Hockin 286175cb94 E2E: always use '[[ ]]' 2021-10-18 09:36:52 -07:00
Tim Hockin a91777288a use branch revision during worktree add (v4)
This is a port of the v3 commit 2c3bb035f6
2021-10-18 09:32:57 -07:00
Tim Hockin 4d286fc44f Fix e2e cases to not destroy logs 2021-10-18 09:32:57 -07:00
Tim Hockin 0075df238c Make exechooks work like webhooks.
This deprecates the `--sync-hook-command` flag in favor of
`--exechook-command`, `--exechook-timeout`, and `--exechook-backoff`.
2021-08-16 23:21:20 -07:00
Tim Hockin d1afc12e67 Handle a race between ls-remote and fetch
This is a port of PR #413.
2021-06-23 12:51:15 -07:00
Tim Hockin ab0fe416de Rename slow_git to slow_git_clone
Making room for slow_git_fetch upcoming.
2021-06-23 12:28:12 -07:00
Tim Hockin 530ac2899b Clean up worktree dirs during sync loop
Avoid "already exists" error.  This is a port of #412 but I changed
the e2e to use `docker pause`.
2021-06-22 15:07:16 -07:00
Tim Hockin 9b1bfc7a97 Make e2e pass on v4 2021-06-22 15:04:00 -07:00
Tim Hockin c6878026a1 unbreak e2e for github 2021-06-22 08:29:30 -07:00
Tim Hockin 421fc4e298 Add a testcase for github HTTPS 2021-05-28 14:18:58 -07:00
Tim Hockin 0222a95f29 Make sure all code files have headers 2021-05-27 08:46:00 -07:00
Tim Hockin 16c3bf7dd1 move test cleanup to the end 2021-05-26 14:31:16 -07:00
Tim Hockin a3d9b673a8 Port #372 to v4 dev branch (from SpencerMalone)
Wanted to finally tackle #54, I sidestepped the problem of how to handle the volume of flags that might be required by instead specifying a sparsecheckout file.

The workflow as I've had has been...
- Do a local sparse checkout, add the files you want ignored (or included on if you did a cone pattern https://git-scm.com/docs/git-sparse-checkout#_cone_pattern_set)
- Grab your .git/info/sparecheckout file, and reserve it for later use with this new flag

It's not quite as easy as specifying it all from a CLI, but I think it's a reasonable first pass.

Here are some logs of it being run on https://github.com/SpencerMalone/logstash-output-prometheus:

```
test-repo % cat sparseconfig
!/*
!/*/
README.md
test-repo % docker run --rm -d \
    -v $(pwd)/git-data:/tmp/git \
    -v $(pwd):/test \
    docker.io/registry/git-sync:tag__linux_amd64  \
        --repo=https://github.com/SpencerMalone/logstash-output-prometheus.git \
        --branch=master \
        --sparse-checkout-file=/test/sparseconfig
41494548dd64caf0ff8f7b75e4d3a86014cfaefc40ff31b14ba19accf99aa82f
test-repo % ls git-data/db86200b1ab158ce9ad403d06de2301b15333601
README.md
```

As you can see, I ignored everything but the `README.md`, and sure enough only got that file in my final checkout.
2021-04-23 09:01:05 -07:00
Nan Yu f2581cff93 Export the error details to an error file
The current git-sync process outputs the error information to standard
out, which is inaccessible from outside the container. Users have to
dump the logs using kubectl logs in order to check the error details in
the git-sync process. This commit exports the error details to a file,
which provides users the capability to check the errors directly from
other sidecar containers.

proposal: https://github.com/kubernetes/git-sync/issues/326
2021-04-09 11:33:16 -07:00
Kubernetes Prow Robot 13af14e3bf
Merge pull request #370 from thockin/exec-hook-after-symlink-v4
Move sync-hook to after symlink flip
2021-04-07 16:47:42 -07:00
Tim Hockin d6c1fd3c6b Move sync-hook to after symlink flip 2021-04-05 17:02:00 -07:00
Tim Hockin dcb918f846 WIP: startup tests 2021-03-29 16:48:50 -07:00
Kubernetes Prow Robot 7b594781f8
Merge pull request #344 from thockin/fix-pid1-exit-code-v4
Fix exit non-zero exit codes when running as pid1
2021-03-15 09:39:06 -07:00
Tim Hockin 45b28530a7 Fix exit non-zero exit codes when running as pid1
(v4 branch)

Prior to this we would swallow the exit code and always exit(0).
2021-03-15 09:28:26 -07:00
Tim Hockin 83b4dd21de Add --git-config flag
This allows arbitrary git configs to be passed in.  For example:

`git config --global http.postBuffer 1048576000`

`git config --global http.sslCAInfo /path/to/cert/file`

`git config --global http.sslVerify false`

This flag takes a comma-separated list of `key:val` pairs.  The key part
is passed to `git config` and must be a valid gitconfig section header
and variable name.  The val part can be either a quoted or unquoted
value.  For all values the following escape sequences are supported:
   * `\n` => [newline]
   * `\t` => [tab]
   * `\"` => `"`
   * `\,` => `,`
   * `\\` => `\`

Within unquoted values, commas MUST be escaped. Within quoted values,
commas MAY be escaped, but are not required to be.  Any other escape
sequence is an error.

Example:

`--git-config=foo.one:val1,foo.two:"quoted val",foo.three:12345`

This commit exposed a bug in runCommand() which modified its args when
they had an embedded space.
2021-03-15 09:20:02 -07:00
Tim Hockin e5a438e446 Change the symlink targets to just the SHA
This allows users to call readlink() on the link and learn the current
checked out SHA.
2021-03-11 22:11:35 -08:00
Tim Hockin ccde597b7d Always specify branch name in e2e
As git changes the default from "master" to "main", it's not
predictable.  Now the test owns the decision.

This was already merged on release-3.x.
2021-01-23 16:13:26 -08:00
Tim Hockin a285a770e2 Use docker to run helper servers in e2e
Reliably cleaning up leftover things like `nc` processes is surprisingly
difficult in pure shell.  e2e passes on the master branch now.
2021-01-23 15:59:24 -08:00
Tim Hockin 24c06a5b48 Normalize the root path
This makes sure there's never any confusion about whether it is an
absolute path or has symlinks or whatever.  Add e2e cases to cover.
2020-11-09 09:20:24 -08:00
Tim Hockin 0e802450ee Don't allow --dest to start with a dot
BREAKING CHANGE

It's useful to reserve names that start with a dot.
2020-11-09 08:36:10 -08:00
Tim Hockin fa0e8696e2 Change time-related flags to durations
Add '--period' to replace '--wait', which is now obsolete.

Add '--sync-timeout' to replace '--timeout', which is now obsolete.

Both of these new flags take a Go-style time string, rather than a bare
number. For example "1s" for 1 second or "1m" for one minute.

The old flags have been kept and will take precedence if specified.
2020-11-05 16:46:09 -08:00
Tim Hockin 20696689f3 Add help and manual flags, use pflag
THIS IS A BREAKING CHANGE

Switch flags implementation to use pflag.  This means that long flags
like `-username` must now use 2 dashes: `--username`.

The `-v` flag (verbose) used to accept `-v` or `--v`.  Now it only
accepts `-v.

The `--help` and `-h` flags are new.

The `--man` flag is new (print a man-page like help message).

Several glog flags which used to be exposed (e.g. --logtostderr) are no
longer exposed.  Logs always go to stderr.
2020-10-31 00:50:51 -07:00
Tim Hockin 9203d14142 Specify "master" in e2e
Some git revs now use "main" but "master" is still coded into git-sync.
When that is fixed we can undo this.
2020-10-31 00:42:46 -07:00
Tim Hockin 771011f0cc Move --v into e2e framework (always same) 2020-10-02 14:38:19 -07:00
unknowingknow 9661d3132e add e2e test for --sync-hook-command 2020-10-02 19:29:21 +09:00
Tim Hockin 1992a2bfab logtostderr defaults to true: don't set it in test 2020-09-28 16:31:33 -07:00
Tim Hockin 44e94a16b0 Fix webhook regression: allow -1 to ignore status 2020-09-25 11:17:56 -07:00
Tim Hockin a29fab5bfa Remove useless test case 2020-09-04 20:03:37 -07:00
alix.cook11 dcf169c10a
add option to NOT recursively clone submodules (#246)
* add option to NOT recursively clone submodules

* Add "shallow" and "off" modes to submodule clone

* update readme to reflect new flag

* wording and such feedback
2020-07-30 21:29:07 -07:00
Tim Hockin ef8f5cf703 Bump to debian-base 2.0.0 (buster)
This changed `git fetch` to require a `-f` the way we use it.

Also fix a flaky test case for SSH.
2020-01-31 09:20:54 -08:00
Tim Hockin 72deefcec3 Make health check meaningful 2020-01-09 17:20:11 -08:00
Tim Hockin 1a1dd1cbe0 e2e: Add test for SSH
This requires having a docker image for git-over-ssh.
2020-01-07 08:25:36 -08:00
Tim Hockin fbdeead461 e2e: quote variables 2019-12-30 15:32:52 -08:00
Tim Hockin 9303c920d7 e2e: Use docker labels for cleanup 2019-12-30 15:32:52 -08:00
Tim Hockin a6cce7b757 e2e: move code around 2019-12-30 15:32:52 -08:00
Tim Hockin b56ea3796a e2e: reinit repo for each testcase 2019-12-30 15:32:52 -08:00
Tim Hockin ece56840f8 Fix askpass tests 2019-12-30 15:32:52 -08:00
Chuanying Du 5c70f02656 Verify both username and password in the askpass_git test. 2019-12-19 16:48:23 -08:00
Chuanying Du 0851cc5916 use free port 2019-12-19 16:26:46 -08:00
Chuanying Du 0b0f0a62d5 use random port for nc 2019-12-18 22:30:42 -08:00
Chuanying 27864c32cd
Merge branch 'master' into master 2019-12-18 22:11:57 -08:00
Chuanying Du d8d9ff72b8 add e2e test for askpasswd_url 2019-12-18 21:07:01 -08:00
Chuanying Du b0bdc02e8b manually merge https://github.com/kubernetes/git-sync/pull/217 2019-12-18 20:05:51 -08:00
Chuanying Du 66244108ed fix tests 2019-12-18 17:57:28 -08:00
Chuanying Du d58a241648 add comments; use --one-time for testing 2019-12-18 17:13:37 -08:00
Chuanying Du ac70751767 Change git credential from cache to store and also add test for
username/password case.

For cache to store change:
* By default, cache only last 900 seconds, gitsync will break after
  that. See https://git-scm.com/docs/git-credential-cache.
* The test won't work with cache since the test don't have access to
  the default unix socket location; XDG_CACHE_HOME override also can
  pre-create a socket in advance.
* `store` put the credential into a file, much easier to debug than cache.
* Considering anyone have access to the pod already able to get the
  credential via environment variables or yaml configs, so put it in
  file won't make it less secure.

For the new password test:
1. askpass_git.sh provided to simulate a git with password challenge.
2. Need and only need to similate "clone" action, need to bypass other
  actions like config/credential setup.
3. See `credential fill` is the official git action to ask password,
  see https://git-scm.com/docs/git-credential.

This change resolved issue #196.
2019-12-18 09:55:01 -08:00
Tim Hockin 27b43e624b e2e: move flags and better errors 2019-11-19 07:44:02 -08:00
Tim Hockin 2b11e078b0 Remove default-value flags in e2e 2019-11-19 07:44:02 -08:00
Tim Hockin 5a73199cf7 all e2e should use file://repo 2019-11-19 07:44:02 -08:00
Tim Hockin 405bfefdc2 e2e: don't pass one-time and wait 2019-11-19 07:44:02 -08:00
Tim Hockin 5e82e574ac e2e more random (I actually hit this) 2019-11-19 07:44:02 -08:00
Tim Hockin c94c8087de e2e --one-time tests can be sync 2019-11-19 07:44:02 -08:00
Tim Hockin bc0980b906 make e2e easier to read 2019-11-19 07:44:02 -08:00
Tim Hockin bde21410c6 Clean up e2e formatting 2019-11-19 07:44:02 -08:00
Michal Lula d1b97bae6c support of depth flag 2019-10-22 16:43:51 +02:00
Michal Lula 41c8572ad6 add tests 2019-10-11 17:58:47 +02:00
Tim Hockin 5c09cac67c Fix e2e to avoid undef var 2019-03-25 08:54:44 -07:00
Thomas Jackson 5676842f8c Address comments from PR 2019-01-23 12:30:19 -08:00
Thomas Jackson b2677cc434 Add e2e test for http handler
Tests for #135
2019-01-23 11:05:47 -08:00
Thomas Jackson 6d98f420e7 Add e2e test for webhook
Test for #131
2019-01-23 11:05:47 -08:00
Thomas Jackson 07f0e6dde2 Cleanup container from sync-loop-timeout
Otherwise any test running afterwards fails because it left the
container running.
2019-01-22 17:15:03 -08:00
Thomas Jackson b0c1234787 Add e2e tests for depth flag 2019-01-22 16:59:53 -08:00
George Angel a86ad5d950 add timeout tests
- custom `git` command option
- tests on slow running git commands
2019-01-04 22:32:53 +00:00
Tim Hockin 648d31b7f2 fix e2e script to work with os/arch tags 2019-01-04 11:36:19 -08:00
Tim Hockin ada75e9c6c Build a manifest list
Retool some of the build/test to produce a manifest-list rather than a
single image.
2018-11-09 14:20:23 -08:00
Tim Hockin 40e188fb26 Switch to rev-parse
This handles non-annotated tags, which were not handled well before.  It
does mean that we use the hash of the (annotated) tag object instead of
the commit, but that seems OK.

Added a test case.
2018-11-09 10:16:36 -08:00
Tim Hockin 296093b9df Clean up failed clones
If an initial clone crashes, it can leave the git-root in a bad state
such that git can't retry the clone.  This change forces it to clean up
the mess and retry.
2018-08-21 08:04:45 -07:00