Commit Graph

192 Commits

Author SHA1 Message Date
Aleksandar Markovski ff51ca92dc fix: infinite bad loop caused by unexpected worktree directory removal 2023-10-13 13:20:54 -07:00
Tim Hockin 162e543e05 Add --credential flag to spec multiple user/pass
* New flag is a JSON-encoded struct.
* Add test case for multiple HTTP passwd.
2023-10-08 14:46:41 -07:00
Tim Hockin 4ccf0633fb Deprecate --ssh - it's not really needed
As long as a repo uses SSH transport (user@host:path or ssh://...), SSH
is active.  This is true for the main repo or submodules.
2023-09-27 13:32:15 -07:00
Tim Hockin 32c3a0c2a7 e2e: Add assert_fail and simplify fail cases 2023-09-21 12:48:59 -07:00
Tim Hockin 627fe23a9a e2e: Always pre-write FUNCNAME into repo/file
This removes a few LOC from many cases
2023-09-21 09:09:58 -07:00
Tim Hockin 66e0ba3562 e2e: merge SSH fail and success cases 2023-09-20 14:30:28 -07:00
Tim Hockin a8d5675e1d Add e2e for password-file 2023-09-20 12:46:13 -07:00
Tim Hockin d4244a3424 e2e: reorder some flags for clarity 2023-09-20 09:31:15 -07:00
Tim Hockin 96b8e8a4fc Remove redundant username/password auth test
The HTTP test covers this without needing a hacky git command.
2023-09-20 09:31:15 -07:00
Tim Hockin 8310ee9ee8 Auth tests really only need to sync once
Except for URL tests which prove resilience to failure.
2023-09-20 09:31:15 -07:00
Tim Hockin 86725cb85e Reorder tests to put auth together
No other changes
2023-09-20 09:31:15 -07:00
Tim Hockin af6b96ab74 Add e2e for username/password over HTTP 2023-09-19 14:57:41 -07:00
Tim Hockin 11b8d151d0 Clean up submodule cases to be a bit more readable 2023-09-19 13:40:40 -07:00
Tim Hockin 55f71d55ed Add a test case for submodules with diff SSH keys 2023-09-19 13:31:19 -07:00
Tim Hockin 29b291e9c2 Add support for multiple SSH keys
The --ssh-key-file flag can be specified more than once and the
GITSYNC_SSH_KEY_FILE env var will be parsed like PATH.

Also adds e2e coverage for wrong-key and for multiple keys.
2023-09-19 13:31:19 -07:00
Tim Hockin 50de3aaeab Dump logs when e2e fails in CI 2023-09-19 13:20:09 -07:00
justinsb 91f41dc719 tests: don't rely on global git configuration in tests
Otherwise got:

```
test root is /tmp/git-sync-e2e.228068759

Author identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
```
2023-07-29 11:39:33 -04:00
Tim Hockin a56f58dbe1
Drop distroless and DIY
Distroless is stuck on debian 11 - we can do the same thing, more or
less in our own script.  Sad that we have to, but here we are.

The net result is a about 8MB smaller and passes e2e and passes trivy.
2023-07-28 15:31:16 -07:00
Karl Isenberg dd57eb0049
Handle errors from credential refresh
Previously, errors from askpass and credential storage were being
ignored, causing git clone/fetch to later error with hard-to-read
errors.

Now the error indicates the credential refresh as the problem, and
does not try to sync.
2023-07-28 12:09:47 -07:00
Tim Hockin 0ae6c31b2e DNM 2023-07-16 12:10:55 -07:00
Tim Hockin 08296a48b0
Make relative-path submodules work, via origin
The "origin" remote is implicitly used as the basis for relative-paths
in submodules.  It's very subtly documented, and I have no idea if there
are other places where it is used.  It seems git really expects it to
exist, so let's just do that.
2023-07-10 10:46:34 -07:00
Tim Hockin 1894192b0f Run hooks at startup
This ensures we do not miss events.  E.g.

before:
    t0: hash changes to X
    t1: send webhook(X), waiting for response
    t2: hash changes to Y
    t3: queue next webhook(Y) but can't send because previous is not done
    t4: crash
    t5: restart
    t6: find repo at hash Y

    no webhook(Y) was sent.

after:
    t0: hash changes to X
    t1: send webhook(X), waiting for response
    t2: hash changes to Y
    t3: queue next webhook(Y) but can't send because previous is not done
    t4: crash
    t5: restart
    t6: find repo at hash Y
    t7: send webhook(Y), waiting for response
2023-05-29 14:50:24 -07:00
Tim Hockin c3a49e9d46 Get rid of "must not start with ." logic
It serves very little purpose and isn't even correct as-is.
2023-05-14 11:11:24 -07:00
Sacha Viscaino 622ced3864 Add --stale-worktree-timeout option 2023-05-14 00:28:29 +01:00
Tim Hockin 58cab771fc e2e: Fix weirdly quoted strings
I have been meaning to do this forever.
2023-05-11 09:42:05 -07:00
Tim Hockin 12c1ece35f Support in-place updates from v3
A few small accomodations and we can clean up properly.
2023-05-08 15:51:23 -07:00
Tim Hockin d197740d85 Exercise the git "dubious ownership" path
To do this, we run the e2e test as a different user.  To do that, we
need git-sync to make sure that everything is group accessible.  To
clean up after the test, we need everything to be group writable.  To do
that, we add a new flag: `--group-write`.
2023-05-07 10:29:39 -07:00
Tim Hockin f287d63171 Fetch just once per run and when hash changes 2023-05-07 10:29:31 -07:00
Tim Hockin 6cd8b5a1cf Add an absPath type and use it for git.root 2023-05-05 13:30:11 -07:00
Tim Hockin 7e6b429362 Make all env var names GITSYNC_FOO
Still support GIT_SYNC_FOO for compat.
2023-05-04 08:45:25 -07:00
Tim Hockin 5cc7160985 Clean up all old worktrees 2023-05-04 08:45:13 -07:00
Tim Hockin 10af97f51d Move worktrees to .worktrees/*
This will make it easier to enumerate old worktrees and do better
cleanup.
2023-04-29 10:51:13 -07:00
Tim Hockin 988bfb7a01 e2e: handle log capture automatically
Same as test_git.sh
2023-04-29 10:44:07 -07:00
Tim Hockin e84a4cb4fd Make --webhook-success-status=0 the same as -1 2023-04-29 10:40:53 -07:00
Tim Hockin cbedbc0ca5 Add test for repo size and change default GC
--git-gc=always seems to be the right tradeoff.
2023-04-28 14:07:54 -07:00
Tim Hockin dc56d5d6bf Tidy up log levels - logs read better now 2023-04-28 14:07:42 -07:00
Tim Hockin f6678b98e8 Change the default depth to 1
For most users this is functionally equivalent and more efficient.  For
users who really need the full history, set this to 0.
2023-04-28 14:07:35 -07:00
Tim Hockin ee6664748f Total overhaul of sync loop
The previous (v3) sync loop betrays my lack of understanding about git.
It tried to codify my archaic mental model (e.g. --branch and --rev
being disting things) and was ultimately a patchwork of corner-cases
evolved over a few years.

This commit is less of a "diff" and more of a "rewrite".

The new logic is simpler and more efficient.  It does not `git clone`
ever.  It does not differentiate the first sync from subsequent syncs.
It uses `git fetch` to get the exact SHA and then makes a worktree from
that.

The new `--ref` flag replaces both `--rev` and `--branch`, though it
will use those if specified.  In fact, almost all of the e2e tests
passed without change - using --ref and --branch!

I will follow this commit up with more cleanups and e2es.
2023-04-21 17:25:25 -07:00
Tim Hockin f95f3c24f7 e2e: print failing line 2023-04-06 17:45:36 -07:00
Tim Hockin f4d067c987 e2e: compare numbers numerically, not lexically
Thanks, bash, for supporting `<` and `>` and making them NOT mean the
obvious thing.

This shows up when you set RUNS=10:
`[[ 2 < 10 ]]` is false
`(( 2 < 10 ))` is true
2023-04-06 17:45:26 -07:00
Tim Hockin c1b8617385 e2e: s/REV/SHA 2023-04-06 17:45:16 -07:00
Kubernetes Prow Robot 1edf40bc2f
Merge pull request #682 from thockin/v4_e2e_init_test_cleanup
e2e: rename init tests, remove default-val args
2023-02-17 04:19:56 -08:00
Tim Hockin 75a256ee24
e2e: rename init tests, remove default-val args
Also don't pass --rev=HEAD when that's not part of the test.  It's the
default value.
2023-02-16 13:58:12 -08:00
Tim Hockin c5dcd87551
e2e: move helper scripts to _test_tools/ 2023-02-15 12:22:29 -08:00
Timothy Rule 2b3f1bcdd7
Add cli option sync-on-signal to trigger git-sync with a signal. (#664)
Operation:
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal SIGHUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal HUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal 1

Signals can be sent to docker containers with cmd:
docker kill -signal SIGHUP <Container ID>
2023-02-15 12:00:22 -08:00
Tim Hockin bbd0f56d11
e2e: assert metrics in tests 2023-02-10 21:12:56 -08:00
Tim Hockin 14422f8bac
e2e: set MAXWAIT to change wait times 2023-02-10 16:05:52 -08:00
Tim Hockin 759b8ebdfe
e2e: print path to logs on failure 2023-02-10 15:45:55 -08:00
Tim Hockin 950e391018
e2e: set RUNS to repeat tests 2023-02-10 15:45:54 -08:00
Tim Hockin b3ee55446e
e2e: allow file:// repos (missing piece) 2023-02-06 17:51:04 -08:00