1894192b0f
Run hooks at startup
...
This ensures we do not miss events. E.g.
before:
t0: hash changes to X
t1: send webhook(X), waiting for response
t2: hash changes to Y
t3: queue next webhook(Y) but can't send because previous is not done
t4: crash
t5: restart
t6: find repo at hash Y
no webhook(Y) was sent.
after:
t0: hash changes to X
t1: send webhook(X), waiting for response
t2: hash changes to Y
t3: queue next webhook(Y) but can't send because previous is not done
t4: crash
t5: restart
t6: find repo at hash Y
t7: send webhook(Y), waiting for response
2023-05-29 14:50:24 -07:00
c3a49e9d46
Get rid of "must not start with ." logic
...
It serves very little purpose and isn't even correct as-is.
2023-05-14 11:11:24 -07:00
622ced3864
Add --stale-worktree-timeout option
2023-05-14 00:28:29 +01:00
58cab771fc
e2e: Fix weirdly quoted strings
...
I have been meaning to do this forever.
2023-05-11 09:42:05 -07:00
12c1ece35f
Support in-place updates from v3
...
A few small accomodations and we can clean up properly.
2023-05-08 15:51:23 -07:00
d197740d85
Exercise the git "dubious ownership" path
...
To do this, we run the e2e test as a different user. To do that, we
need git-sync to make sure that everything is group accessible. To
clean up after the test, we need everything to be group writable. To do
that, we add a new flag: `--group-write`.
2023-05-07 10:29:39 -07:00
f287d63171
Fetch just once per run and when hash changes
2023-05-07 10:29:31 -07:00
6cd8b5a1cf
Add an absPath type and use it for git.root
2023-05-05 13:30:11 -07:00
7e6b429362
Make all env var names GITSYNC_FOO
...
Still support GIT_SYNC_FOO for compat.
2023-05-04 08:45:25 -07:00
5cc7160985
Clean up all old worktrees
2023-05-04 08:45:13 -07:00
10af97f51d
Move worktrees to .worktrees/*
...
This will make it easier to enumerate old worktrees and do better
cleanup.
2023-04-29 10:51:13 -07:00
988bfb7a01
e2e: handle log capture automatically
...
Same as test_git.sh
2023-04-29 10:44:07 -07:00
e84a4cb4fd
Make --webhook-success-status=0 the same as -1
2023-04-29 10:40:53 -07:00
cbedbc0ca5
Add test for repo size and change default GC
...
--git-gc=always seems to be the right tradeoff.
2023-04-28 14:07:54 -07:00
dc56d5d6bf
Tidy up log levels - logs read better now
2023-04-28 14:07:42 -07:00
f6678b98e8
Change the default depth to 1
...
For most users this is functionally equivalent and more efficient. For
users who really need the full history, set this to 0.
2023-04-28 14:07:35 -07:00
ee6664748f
Total overhaul of sync loop
...
The previous (v3) sync loop betrays my lack of understanding about git.
It tried to codify my archaic mental model (e.g. --branch and --rev
being disting things) and was ultimately a patchwork of corner-cases
evolved over a few years.
This commit is less of a "diff" and more of a "rewrite".
The new logic is simpler and more efficient. It does not `git clone`
ever. It does not differentiate the first sync from subsequent syncs.
It uses `git fetch` to get the exact SHA and then makes a worktree from
that.
The new `--ref` flag replaces both `--rev` and `--branch`, though it
will use those if specified. In fact, almost all of the e2e tests
passed without change - using --ref and --branch!
I will follow this commit up with more cleanups and e2es.
2023-04-21 17:25:25 -07:00
f95f3c24f7
e2e: print failing line
2023-04-06 17:45:36 -07:00
f4d067c987
e2e: compare numbers numerically, not lexically
...
Thanks, bash, for supporting `<` and `>` and making them NOT mean the
obvious thing.
This shows up when you set RUNS=10:
`[[ 2 < 10 ]]` is false
`(( 2 < 10 ))` is true
2023-04-06 17:45:26 -07:00
c1b8617385
e2e: s/REV/SHA
2023-04-06 17:45:16 -07:00
1edf40bc2f
Merge pull request #682 from thockin/v4_e2e_init_test_cleanup
...
e2e: rename init tests, remove default-val args
2023-02-17 04:19:56 -08:00
75a256ee24
e2e: rename init tests, remove default-val args
...
Also don't pass --rev=HEAD when that's not part of the test. It's the
default value.
2023-02-16 13:58:12 -08:00
c5dcd87551
e2e: move helper scripts to _test_tools/
2023-02-15 12:22:29 -08:00
2b3f1bcdd7
Add cli option sync-on-signal to trigger git-sync with a signal. ( #664 )
...
Operation:
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal SIGHUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal HUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal 1
Signals can be sent to docker containers with cmd:
docker kill -signal SIGHUP <Container ID>
2023-02-15 12:00:22 -08:00
bbd0f56d11
e2e: assert metrics in tests
2023-02-10 21:12:56 -08:00
14422f8bac
e2e: set MAXWAIT to change wait times
2023-02-10 16:05:52 -08:00
759b8ebdfe
e2e: print path to logs on failure
2023-02-10 15:45:55 -08:00
950e391018
e2e: set RUNS to repeat tests
2023-02-10 15:45:54 -08:00
b3ee55446e
e2e: allow file:// repos (missing piece)
2023-02-06 17:51:04 -08:00
75ba836f80
e2e: fix test-tool image names
2023-02-06 17:51:03 -08:00
2f7335868e
Fix cases of syncing different SHAs back to back
...
Prior to this, it would fail if the 2nd SHA wasn't in the local repo.
Now it doesn't care what the local SHA for rev is, it only cares what is
checked out at HEAD.
2022-12-25 14:02:12 -08:00
a761413d81
Merge pull request #649 from thockin/e2e_fix_broken
...
e2e: fix 2 broken tests
2022-11-30 03:28:54 -08:00
59e2d9e97c
e2e: fix exechook_fail_retry
...
Previous e2e perf changes caused this to fail.
2022-11-24 17:50:03 -08:00
a492289ebf
e2e: fix git submodules for file://
...
I guess upstream git intentionally broke this because of a CVE.
Also fix wait_for_sync to actually exit early when possible.
2022-11-23 23:43:27 -08:00
bda1f84614
Add --touch-file flag and use it in tests
...
This is mostly designed to make testing easier instead of sleep.
2022-10-30 17:01:55 -07:00
2edd4e5d9e
Merge pull request #632 from thockin/error-file-abs-path
...
Make --error-file allow abs paths
2022-09-11 08:47:23 -07:00
d0b7661c58
Merge pull request #630 from thockin/e2e_bad_branch
...
e2e: move err tests and add bad-branch
2022-09-11 07:45:23 -07:00
c32a042213
Make --error-file allow abs paths
...
This also enforces th previously unenforced "must not start with a
period" rule.
2022-09-05 14:54:10 -07:00
cbf034f87b
Merge pull request #629 from thockin/silence_e2e_noise
...
e2e: Silence noise from git init
2022-09-03 03:16:27 -07:00
fcb577b9fb
e2e: move err tests and add bad-branch
2022-09-02 13:45:43 -07:00
8e28337f3c
e2e: Silence noise from git init
2022-09-02 13:45:29 -07:00
bcd4629159
Disable local git configs
2022-09-02 13:45:19 -07:00
d4c60936ed
e2e: Allow N tests to fail and catch errexit
...
Prior to this, `errexit` was not active inside test functions (see
comments in this commit).
Now, multiple tests can fail - it does not abort on the first one.
2022-08-30 14:50:37 -07:00
6e03c52fc3
Merge pull request #615 from thockin/fetch_if_depth_and_rev_not_found
...
Support shallow sync when the rev is not in-range
2022-08-30 02:51:01 -07:00
eb33e7cfcb
--max-sync-failures -> --max-failures
...
Deprecate but retain the old flag and env.
2022-08-21 10:48:20 -07:00
ff0a73f5ad
Support shallow sync when the rev is not in-range
...
If I ask for `--depth 1` and a branch, it's fine.
If I ask for `--depth 1` and a tag, and that tag is not within 1 commit
of the branch that was cloned, it will give an error. Oddly, if the
initial `clone` was OK, and subsequent syncs drift, it is OK, because of
how we `fetch`. But if it is too far away at the beginning, kaboom.
This betrays that the current model of `--branch` and `--rev` is really
broken, and should be revamped. For now, I did something simple - if
the rev can't be found, try a fetch. A "real" fix is more involved.
Also add tests.
2022-08-20 15:41:33 -07:00
35f0720083
e2e: basename the link when comparing
2022-08-14 15:51:09 -07:00
48b006568d
e2e: also trap ERR
2022-07-26 11:27:56 -07:00
e7d6471ddd
Merge pull request #576 from thockin/v4_e2e_password_not_one_time
...
v4: e2e: Make password test not use --one-time
2022-07-07 23:33:49 -07:00
9e1fc987cf
Merge pull request #574 from thockin/v4_no_XDG_CONFIG_HOME
...
v4: e2e: don't set XDG_CONFIG_HOME
2022-07-07 05:19:34 -07:00
344f3ecd09
Merge pull request #570 from thockin/v4_e2e_CLEANUP_docker_rm
...
v4: e2e: $CLEANUP decides whether to rm containers
2022-07-07 04:43:34 -07:00
37eba714c5
Merge pull request #568 from thockin/v4_ncsvr_http_bug
...
v4: e2e: Fix ncsvr to wait for input on HTTP
2022-07-07 04:27:34 -07:00
56918b0055
e2e: don't set XDG_CONFIG_HOME
2022-07-03 16:53:09 -07:00
6898074c8a
Merge pull request #564 from thockin/v4_e2e_rebuild_less
...
v4: e2e: rebuild the container less often
2022-07-03 14:31:22 -07:00
20f0ab8a9b
e2e: Make password test not use --one-time
2022-07-03 11:38:23 -07:00
e579cf18b2
e2e: $CLEANUP decides whether to rm containers
2022-07-03 11:36:48 -07:00
ec25d1a3f5
e2e: Fix ncsvr to wait for input on HTTP
...
This caused occasional e2e flakes when the server responded before the
client request had been sent.
2022-07-03 11:36:05 -07:00
4b9431f9bc
Don't run apt-get in e2e build if we can avoid
2022-07-03 11:35:09 -07:00
10abfd561b
e2e: rebuild container less often
2022-07-03 11:35:09 -07:00
27ac1642c9
Rename tests for better filtering
2022-07-03 11:14:46 -07:00
23540b9d94
Beef up askpass_url support
...
* Don't need to get the password at startup (it happens in the sync loop
anyway)
* Add tests for bad password and flaky URL
2022-07-02 15:43:27 -07:00
d8da63cf48
Merge pull request #557 from thockin/v4_e2e_v6
...
v4: e2e with -v=6
2022-07-02 13:11:24 -07:00
58bb41e82f
Merge pull request #538 from thockin/v4-dont-docker-kill
...
v4: e2e: Don't manually "docker kill"
2022-07-02 12:59:24 -07:00
2a4ca7794c
e2e with -v=6
2022-07-02 10:59:47 -07:00
52d2e075c6
Merge pull request #542 from thockin/v4-passwd_e2e_split
...
v4: Split password test to 2 - right and wrong passwd
2022-07-02 03:53:24 -07:00
00d81fd3f9
Split password test to 2 - right and wrong passwd
2022-06-24 08:46:43 -07:00
5b7df08826
e2e: Reformat ncsvr scripts for readability
2022-06-24 08:45:17 -07:00
9af6b7cfe2
e2e: Don't manually "docker kill"
...
It happens automatically on test cleanup
2022-06-24 08:45:05 -07:00
a341fb62f9
Make ssh e2e test not use --one-time
...
There is functionality that needs to be tested beyond that.
2022-06-11 10:56:20 -07:00
2c6356cbcb
Pass CLEANUP=0 to test_e2e to leave logs
2022-06-11 19:49:11 +02:00
f8a532d9df
Pass the whole environment to exechooks
2022-05-16 18:14:09 -07:00
535735607f
Adapt testcases for v4
2022-03-11 08:41:57 -08:00
ff3d11c925
Add repo-sync test
2022-03-11 08:34:16 -08:00
8c5f33d5dd
Make e2e treat args as tests-name regexes
2022-02-24 09:00:57 -08:00
f4d124bded
Add --git-gc flag to control GC on each sync
...
Values:
* "auto" - run `git gc --auto` (default, respects git gc.* configs)
* "always" - run `git gc`
* "aggressive" - run `git gc --aggressive` (may require a longer timeout)
* "off" - do not run `git gc` on each sync (good for --one-time use)
2022-02-24 08:00:10 -08:00
2538d293ba
Set repoReady even when there wasn't a 1st clone
...
E.g. if the repo is already present (after a restart).
2022-02-12 17:38:36 -08:00
d3324e0b90
Merge pull request #478 from thockin/v4-link-abspath
...
Allow --dest to be an absolute path (v4)
2022-01-23 02:41:56 -08:00
6dc0e44697
Merge pull request #472 from thockin/master
...
Don't try to remove the root if it appears corrupt (v4)
2022-01-23 01:15:56 -08:00
4fc67f5f5f
Allow --dest to be an absolute path
...
This means it might live outside of --root.
2022-01-22 23:39:56 -08:00
248a53461d
Don't try to remove the root if it appears corrupt
...
The `--root` is often a volume and can't be removed. Instead, remove
the contents of it. Adjust tests to hit this.
2022-01-22 14:28:09 -08:00
9eed6946b7
e2e cleanup
...
Fix some whitespace and names.
Make command-line and flag handling a bit cleaner.
2022-01-19 11:38:23 -08:00
5490b721d8
fix: made tests more efficient by waiting less
2022-01-14 13:53:09 -05:00
043c356c03
fix: fixed e2e tests
...
Resolved a few issues with e2e tests discovered after running
test_e2d.sh. Just finished setting up Linux (Fedora) environment in
which this script can be run.
2022-01-14 03:23:20 -05:00
0246fb509f
feat: added e2e --one-time tests for webhook
2022-01-10 20:06:25 -05:00
8e098f8c2a
fix: made exechook_once tests more efficient
...
* Removed multiple iterations of tests
* Reduced sleep time after GIT_SYNC command execution
2022-01-10 20:06:24 -05:00
a362b4a5f4
feat: added tests for this PR that test exechook paired with --one-time flag
2022-01-10 20:06:21 -05:00
7b32caee77
v4: use repo default branch instead of master
2021-11-01 23:05:48 -07:00
2b471ba2d7
E2E: build IFF running tests
2021-10-18 09:36:52 -07:00
6e52fb0d22
E2E: Add whitespace for reading
2021-10-18 09:36:52 -07:00
320a4a7e93
E2E: Move code
...
Reduce the diff between v3 and v4
2021-10-18 09:36:52 -07:00
27f2c905fc
E2E: Rename test funcs
2021-10-18 09:36:52 -07:00
dfcafc9ca0
E2E: Fix endless loop on error
2021-10-18 09:36:52 -07:00
e2ed136aad
E2E: make all test cases be funcs
...
Make all test cases functions so they can be called individually on the
CLI. Call test_e2e.sh with `-?` to list tests. Fix related crashes
(uses of $TESTCASE and so on).
2021-10-18 09:36:52 -07:00
e06f9c6acb
E2E: Make SIGINT handling more complete
2021-10-18 09:36:52 -07:00
7b3454e51b
E2E: Use vars for branch names
2021-10-18 09:36:52 -07:00
286175cb94
E2E: always use '[[ ]]'
2021-10-18 09:36:52 -07:00
a91777288a
use branch revision during worktree add (v4)
...
This is a port of the v3 commit 2c3bb035f6
2021-10-18 09:32:57 -07:00
4d286fc44f
Fix e2e cases to not destroy logs
2021-10-18 09:32:57 -07:00
0075df238c
Make exechooks work like webhooks.
...
This deprecates the `--sync-hook-command` flag in favor of
`--exechook-command`, `--exechook-timeout`, and `--exechook-backoff`.
2021-08-16 23:21:20 -07:00
d1afc12e67
Handle a race between ls-remote and fetch
...
This is a port of PR #413 .
2021-06-23 12:51:15 -07:00
Tim Hockin
Sacha Viscaino
Kubernetes Prow Robot
Timothy Rule
yoyehan
sed-i
ChrisERo