Commit Graph

734 Commits

Author SHA1 Message Date
Tim Hockin 20ed7df894 Makefile: set GOOS,GOARCH defaults to kill warning 2023-04-29 10:39:07 -07:00
Tim Hockin cbedbc0ca5 Add test for repo size and change default GC
--git-gc=always seems to be the right tradeoff.
2023-04-28 14:07:54 -07:00
Tim Hockin dc56d5d6bf Tidy up log levels - logs read better now 2023-04-28 14:07:42 -07:00
Tim Hockin f6678b98e8 Change the default depth to 1
For most users this is functionally equivalent and more efficient.  For
users who really need the full history, set this to 0.
2023-04-28 14:07:35 -07:00
Tim Hockin 76e91a2304 Fine tune git command verbosity flags 2023-04-28 14:07:18 -07:00
Tim Hockin 07df59eccc Move code (no edits), document methods 2023-04-28 14:07:05 -07:00
Tim Hockin ee6664748f Total overhaul of sync loop
The previous (v3) sync loop betrays my lack of understanding about git.
It tried to codify my archaic mental model (e.g. --branch and --rev
being disting things) and was ultimately a patchwork of corner-cases
evolved over a few years.

This commit is less of a "diff" and more of a "rewrite".

The new logic is simpler and more efficient.  It does not `git clone`
ever.  It does not differentiate the first sync from subsequent syncs.
It uses `git fetch` to get the exact SHA and then makes a worktree from
that.

The new `--ref` flag replaces both `--rev` and `--branch`, though it
will use those if specified.  In fact, almost all of the e2e tests
passed without change - using --ref and --branch!

I will follow this commit up with more cleanups and e2es.
2023-04-21 17:25:25 -07:00
Tim Hockin bb0128b883 Don't ignore invalid env vars 2023-04-06 17:46:19 -07:00
Tim Hockin b36c6851a2 Use a private gitconfig file
This was changing my own ~/.gitconfig whenever run by hand outside of a
container.  Oops.
2023-04-06 17:46:05 -07:00
Tim Hockin 026c6814c8 Add execution time to pkg/cmd runs 2023-04-06 17:45:54 -07:00
Tim Hockin f95f3c24f7 e2e: print failing line 2023-04-06 17:45:36 -07:00
Tim Hockin f4d067c987 e2e: compare numbers numerically, not lexically
Thanks, bash, for supporting `<` and `>` and making them NOT mean the
obvious thing.

This shows up when you set RUNS=10:
`[[ 2 < 10 ]]` is false
`(( 2 < 10 ))` is true
2023-04-06 17:45:26 -07:00
Tim Hockin c1b8617385 e2e: s/REV/SHA 2023-04-06 17:45:16 -07:00
Kubernetes Prow Robot cadd8b06e6
Merge pull request #684 from saschagrunert/debian-base
Bump debian-base build image to v1.4.3
2023-02-25 14:38:19 -08:00
Kubernetes Prow Robot abb41da1e9
Merge pull request #683 from KimHyeonwoo/patch-1
fix typo
2023-02-25 14:36:20 -08:00
Sascha Grunert c410fd4a62
Bump debian-base build image to v1.4.3
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2023-02-22 12:35:40 +01:00
Tommy Hyeonwoo Kim 664fb74867
fix typo 2023-02-21 19:47:26 +09:00
Kubernetes Prow Robot 8fec7fa756
Merge pull request #680 from thockin/v4_git_tests
Add a test to exercise git
2023-02-17 11:50:02 -08:00
Kubernetes Prow Robot 1edf40bc2f
Merge pull request #682 from thockin/v4_e2e_init_test_cleanup
e2e: rename init tests, remove default-val args
2023-02-17 04:19:56 -08:00
Kubernetes Prow Robot 9cee1d5ba8
Merge pull request #681 from thockin/v4_log_deprecated_flags
Add logs when we use deprecated flags
2023-02-17 04:17:57 -08:00
Kubernetes Prow Robot ee60157c67
Merge pull request #679 from thockin/v4_scripts_to_test_tools
e2e: move helper scripts to _test_tools/
2023-02-17 04:15:58 -08:00
Tim Hockin 31e2a2669d
Add logs when we use deprecated flags 2023-02-16 14:44:25 -08:00
Tim Hockin 75a256ee24
e2e: rename init tests, remove default-val args
Also don't pass --rev=HEAD when that's not part of the test.  It's the
default value.
2023-02-16 13:58:12 -08:00
Tim Hockin 3133c38442
Add a test to exercise git
I keep having to cross-check whether things work the way I think they
work.  I am codifying them into a test.
2023-02-16 12:52:58 -08:00
Tim Hockin c5dcd87551
e2e: move helper scripts to _test_tools/ 2023-02-15 12:22:29 -08:00
Kubernetes Prow Robot e025726d0d
Merge pull request #678 from jmhbnz/master
Replace deprecated k8s registry references
2023-02-15 12:06:20 -08:00
Timothy Rule 2b3f1bcdd7
Add cli option sync-on-signal to trigger git-sync with a signal. (#664)
Operation:
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal SIGHUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal HUP
git-sync --repo https://github.com/kubernetes/kubernetes --sync-on-signal 1

Signals can be sent to docker containers with cmd:
docker kill -signal SIGHUP <Container ID>
2023-02-15 12:00:22 -08:00
Kubernetes Prow Robot f7ff396716
Merge pull request #673 from thockin/v4_e2e_runs_maxwait_metrics
v4: e2e improvements
2023-02-14 13:31:52 -08:00
Kubernetes Prow Robot 681dacd6b7
Merge pull request #676 from thockin/v4_deref_tags
v4: Deref tags on ls-remote
2023-02-14 11:49:52 -08:00
James Blair 104563fc47
Replace deprecated k8s registry references.
Problem: Previously all of Kubernetes' image hosting has been out of gcr.io. There were significant egress costs associated with this when images were pulled from entities outside gcp.  Refer to https://github.com/kubernetes/k8s.io/wiki/New-Registry-url-for-Kubernetes-(registry.k8s.io)

Solution: As highlighted at KubeCon NA 2022 k8s infra SIG update, the replacement for k8s.gcr.io which is registry.k8s.io is now ready for mainstream use and the old k8s.gcr.io has been formally deprecated and projects are requested to migrate off it. This commit migrates remaining references for kubernetes/git-sync to registry.k8s.io.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-02-14 16:26:30 +13:00
Tim Hockin 2d61e6a5ea Default http-metrics to false
After the change to require http-bind, this default doesn't make sense.
My local branch had other commits, which hid this error.
2023-02-11 10:38:17 -08:00
Kubernetes Prow Robot 4707356327
Merge pull request #674 from thockin/v4_http_bind
Require --http-bind when using other http flags
2023-02-11 01:05:29 -08:00
Kubernetes Prow Robot 052bbf43ca
Merge pull request #675 from thockin/v4_log_commands_depth
Log commands we run with original caller
2023-02-11 01:01:31 -08:00
Tim Hockin 139352ecee
Deref tags on ls-remote
A previous commit (2f7335868e) introduced
a quiet bug which results in the "update needed" condition triggering
every loop.  e2e passed for me by sheer luck of winning races.  Until it
didn't.  Walking thru with the debugger to figure it out, I realized
this issue.

The short story: `ls-remote` for a tag gets us the SHA of the tag, but
`rev-parse HEAD` gets us the SHA of the commit to which that tag is
attached.  Those are never equal, so we detect "update needed" every
loop.

Now we ask `ls-remote` for the rev and the dereferenced rev.  If that
rev is a branch, the deref does nothing.  If that rev is a tag it
produces both results.  ls-remote does its own sort, so the deref (if
found) comes after the non-deref.  This means that, in both cases, the
last line is the one we want.
2023-02-10 21:18:34 -08:00
Tim Hockin f5057b7fa8
Log commands we run with original caller 2023-02-10 21:18:28 -08:00
Tim Hockin 8abd849e2c
Require --http-bind when using other http flags
Also use the word "set" for bool flags, rather than "specified" (easier
to read.
2023-02-10 21:18:21 -08:00
Tim Hockin bbd0f56d11
e2e: assert metrics in tests 2023-02-10 21:12:56 -08:00
Tim Hockin 14422f8bac
e2e: set MAXWAIT to change wait times 2023-02-10 16:05:52 -08:00
Tim Hockin 759b8ebdfe
e2e: print path to logs on failure 2023-02-10 15:45:55 -08:00
Tim Hockin 950e391018
e2e: set RUNS to repeat tests 2023-02-10 15:45:54 -08:00
Kubernetes Prow Robot 1253f82cd9
Merge pull request #670 from thockin/master
Fix e2e on v4
2023-02-10 02:13:29 -08:00
Kubernetes Prow Robot 4501732a72
Merge pull request #669 from thockin/v4-prom
v4: Update prometheus client (CVE)
2023-02-10 02:11:30 -08:00
Tim Hockin b3ee55446e
e2e: allow file:// repos (missing piece) 2023-02-06 17:51:04 -08:00
Tim Hockin 75ba836f80
e2e: fix test-tool image names 2023-02-06 17:51:03 -08:00
Tim Hockin 6e68266403
Update prometheus client
go get github.com/prometheus/client_golang@v1.14.0
go mod tidy
go mod vendor
2023-02-06 17:49:03 -08:00
Tim Hockin 3ce1ec617d Update README on master to point to v3 2023-01-25 10:17:22 -08:00
Kubernetes Prow Robot d9120f7a40
Merge pull request #656 from thockin/fix-back-to-back-rev-syncs
Fix cases of syncing different SHAs back to back
2023-01-08 02:43:27 -08:00
yoyehan 2f7335868e
Fix cases of syncing different SHAs back to back
Prior to this, it would fail if the 2nd SHA wasn't in the local repo.
Now it doesn't care what the local SHA for rev is, it only cares what is
checked out at HEAD.
2022-12-25 14:02:12 -08:00
Kubernetes Prow Robot 503b9dae45
Merge pull request #653 from thockin/dockerfile-volume-cleanup
Update Dockerfile docs and default for volumes
2022-12-22 22:53:26 -08:00
Tim Hockin bbb58ad858
Update Dockerfile docs and default for volumes 2022-12-22 10:51:22 -08:00