Commit Graph

192 Commits

Author SHA1 Message Date
Tim Hockin f1e698e225 Support repo change between invocations
main logic from PR 499 (v3) from sed-i
2022-03-11 08:32:26 -08:00
Tim Hockin f4d124bded Add --git-gc flag to control GC on each sync
Values:
* "auto" - run `git gc --auto` (default, respects git gc.* configs)
* "always" - run `git gc`
* "aggressive" - run `git gc --aggressive` (may require a longer timeout)
* "off" - do not run `git gc` on each sync (good for --one-time use)
2022-02-24 08:00:10 -08:00
Tim Hockin a37a758e69 Set some gc.* git config
In particular, this sets `gc.autoDetach` to "false".
2022-02-23 16:53:19 -08:00
Tim Hockin 2538d293ba Set repoReady even when there wasn't a 1st clone
E.g. if the repo is already present (after a restart).
2022-02-12 17:38:36 -08:00
Tim Hockin 76854fc806 Set $GITSYNC_HASH in exechook 2022-02-06 16:49:34 -08:00
Tim Hockin ec169cb7d3 small manual text fix 2022-02-01 11:07:11 -08:00
Tim Hockin f24435fa47 Fix bug caused in previous PR
The .git to check is not the root/.git but the link/.git - tests pass now.
2022-01-24 11:14:24 -08:00
Kubernetes Prow Robot d3324e0b90
Merge pull request #478 from thockin/v4-link-abspath
Allow --dest to be an absolute path (v4)
2022-01-23 02:41:56 -08:00
Kubernetes Prow Robot 6dc0e44697
Merge pull request #472 from thockin/master
Don't try to remove the root if it appears corrupt (v4)
2022-01-23 01:15:56 -08:00
Tim Hockin 4fc67f5f5f Allow --dest to be an absolute path
This means it might live outside of --root.
2022-01-22 23:39:56 -08:00
Tim Hockin 248a53461d Don't try to remove the root if it appears corrupt
The `--root` is often a volume and can't be removed.  Instead, remove
the contents of it.  Adjust tests to hit this.
2022-01-22 14:28:09 -08:00
Tim Hockin c143bfd31a Small cleanup on one-time x hooks 2022-01-19 11:48:50 -08:00
Kubernetes Prow Robot d8bd262b87
Merge pull request #466 from ChrisERo/master
fix: resolve issue #463
2022-01-19 11:24:05 -08:00
ChrisERo 394b2c7149 fix: perform hook null check before waiting
Last 2 commits introduced fatal null-pointer exception when waiting on
HookRunner object when not set, e.g. waiting for WebhookRunner when no
webhook is defined. Fixed bug and got test_e2e.sh to run successfully.
2022-01-15 15:19:31 -05:00
ChrisERo 8827bf7489 refactor: imporved efficiency and code style
* changed sendCompletedOnceMessageIfApplicable so that it kills thread
  after executing
* moved logic for waiting on HookRunner into function defined in
  HookRunner itself
2022-01-14 13:41:59 -05:00
ChrisERo 0df6e005d3 refactor: fixed in-line documentation and added line-wraps where needed 2022-01-10 20:06:23 -05:00
ChrisERo f67669662b fix: corrected comment for hook channels to refflect true functionality 2022-01-10 20:06:22 -05:00
ChrisERo d2a4320e6c fix: execute hook check only on --one-time
Execute hook check only if --one-time  is true and corrected log spelling of webhook and exechook
2022-01-10 20:06:22 -05:00
ChrisERo 3b5ed549eb fix: addressed several concers raised by thockin
- Renamed [x]HookChannel to [x]hookChannel for consistency
- Made hookChannels buffered chanels of size 1
- Added clarifying documentation and renamed functions expalining how I was
    - ensuring that hooks execute at least once, assuming that main thread does not crash
    - make sure chanels are not written to more than once
2022-01-10 20:06:06 -05:00
ChrisERo 9e6348c3b5 fix: resolve issue number 463
Resolved original issue by introducing a boolean chanel by which exechook runner can communicate with main thread.
Then introduced and used webhook executed-at-least-once chanel and added documentation explaining sections of of code
only executed when git-sync pulls for first time.
2022-01-10 20:05:53 -05:00
Nan Yu 25a13c5288 Expose the ssh diagnostic message
When git-sync fails to clone the git project, the ssh diagnostic message
is suppressed. It is sometimes helpful for debugging the cloning issue.
One example of the ssh dianostic message is
```
ssh: Could not resolve hostname ${REPO_HOST}: Name or service not known
```

This commit removes the `-q` flag from the ssh command to expose such
message.
2022-01-06 16:34:56 -08:00
Tim Hockin c6c19dc61a Avoid writing to /etc/passwd unless needed
Port #461 by mac-chaffee to v4 branch.
2021-12-23 09:04:18 -08:00
Tim Hockin 4603f559f3 Replace glogr with funcr
No difference in flags.  Log output will now be JSON.

example:

{"logger":"","ts":"2021-11-24 11:19:46.340238","caller":{"file":"main.go","line":1031},"level":0,"msg":"cloning repo","origin":"https://github.com/thockin/git-sync","path":"/tmp/gt"}
2021-11-24 11:18:07 -08:00
Tim Hockin 7b32caee77 v4: use repo default branch instead of master 2021-11-01 23:05:48 -07:00
Tim Hockin a91777288a use branch revision during worktree add (v4)
This is a port of the v3 commit 2c3bb035f6
2021-10-18 09:32:57 -07:00
Tim Hockin 05f0476d63 Fix env var names 2021-08-17 08:28:15 -07:00
Tim Hockin dc02fd98c0 Make logging and exec members of the git struct
Otherwise they are floating global pointers, which sucks in tests.
2021-08-16 23:21:26 -07:00
Tim Hockin 0075df238c Make exechooks work like webhooks.
This deprecates the `--sync-hook-command` flag in favor of
`--exechook-command`, `--exechook-timeout`, and `--exechook-backoff`.
2021-08-16 23:21:20 -07:00
Tim Hockin 18947d3e37 Small nits I found with password-fil while porting
Forgot the manual entry for it, tweaked some words.
2021-07-05 15:39:56 -07:00
Haibing Zhou 6076daf3c4 Add option to read password from file
A new flag `--password-file` is added. This allows git-sync to read
password from file and this is considered as safer than reading from
env or flag directly.

Few more checks are added as well:

1. `--password` and `--password-file` can't be specified at the same
time.
1. If `--username` is specified, then one of `--password` or
`--password-file` must be specified.
2021-07-05 14:34:03 -07:00
Tim Hockin d1afc12e67 Handle a race between ls-remote and fetch
This is a port of PR #413.
2021-06-23 12:51:15 -07:00
Tim Hockin 530ac2899b Clean up worktree dirs during sync loop
Avoid "already exists" error.  This is a port of #412 but I changed
the e2e to use `docker pause`.
2021-06-22 15:07:16 -07:00
Tim Hockin 16c89cec6f Don't use global variables (flags)
'
2021-06-22 15:04:00 -07:00
Tim Hockin 476fba3d3e Use plflag, not flag
Make it harder to accidentally mess this up again.
2021-06-22 15:04:00 -07:00
Brian Kennedy a87c78665d Improve error message for askpass.
When endpoint returns non-200 status, include the body in the error
message since it can contain useful information for debugging.  Also
defer closing the response body ReadCloser as this may have leaked in
the past.
2021-06-15 13:26:32 -07:00
Tim Hockin 0222a95f29 Make sure all code files have headers 2021-05-27 08:46:00 -07:00
Tim Hockin f96cdbca17 Small error string cleanup 2021-05-26 17:04:06 -07:00
Nan Yu c05d17c50c Grant read access for the error file to all users 2021-05-26 14:35:29 -07:00
Tim Hockin 295cfd442d Clean up global flRoot use 2021-05-21 17:08:49 -07:00
Nan Yu 583ae46ec6 Create the root directory if it doesn't exist
The `git clone` command will create the root directory if it doesn't
exist, but if `git clone` fails, the root directory needs to be present
so that we can write the error to a file under the directory.
2021-05-21 16:51:21 -07:00
Kubernetes Prow Robot 1c2a18b0cb
Merge pull request #381 from thockin/master
Add support for sparse-checkout (v4 vbranch)
2021-04-26 07:39:02 -07:00
Tim Hockin a3d9b673a8 Port #372 to v4 dev branch (from SpencerMalone)
Wanted to finally tackle #54, I sidestepped the problem of how to handle the volume of flags that might be required by instead specifying a sparsecheckout file.

The workflow as I've had has been...
- Do a local sparse checkout, add the files you want ignored (or included on if you did a cone pattern https://git-scm.com/docs/git-sparse-checkout#_cone_pattern_set)
- Grab your .git/info/sparecheckout file, and reserve it for later use with this new flag

It's not quite as easy as specifying it all from a CLI, but I think it's a reasonable first pass.

Here are some logs of it being run on https://github.com/SpencerMalone/logstash-output-prometheus:

```
test-repo % cat sparseconfig
!/*
!/*/
README.md
test-repo % docker run --rm -d \
    -v $(pwd)/git-data:/tmp/git \
    -v $(pwd):/test \
    docker.io/registry/git-sync:tag__linux_amd64  \
        --repo=https://github.com/SpencerMalone/logstash-output-prometheus.git \
        --branch=master \
        --sparse-checkout-file=/test/sparseconfig
41494548dd64caf0ff8f7b75e4d3a86014cfaefc40ff31b14ba19accf99aa82f
test-repo % ls git-data/db86200b1ab158ce9ad403d06de2301b15333601
README.md
```

As you can see, I ignored everything but the `README.md`, and sure enough only got that file in my final checkout.
2021-04-23 09:01:05 -07:00
Tim Hockin a8a7ffb5a2 Sort the flags in the manual output 2021-04-21 21:48:55 -07:00
Nan Yu 53f3fa17a0 Export the error details to an error file
The current git-sync process outputs the error information to standard
out, which is inaccessible from outside the container. Users have to
dump the logs using kubectl logs in order to check the error details in
the git-sync process. This commit exports the error details to a file,
which provides users the capability to check the errors directly from
other sidecar containers.

proposal: https://github.com/kubernetes/git-sync/issues/326
2021-04-21 16:42:28 -07:00
Nan Yu f2581cff93 Export the error details to an error file
The current git-sync process outputs the error information to standard
out, which is inaccessible from outside the container. Users have to
dump the logs using kubectl logs in order to check the error details in
the git-sync process. This commit exports the error details to a file,
which provides users the capability to check the errors directly from
other sidecar containers.

proposal: https://github.com/kubernetes/git-sync/issues/326
2021-04-09 11:33:16 -07:00
Kubernetes Prow Robot 13af14e3bf
Merge pull request #370 from thockin/exec-hook-after-symlink-v4
Move sync-hook to after symlink flip
2021-04-07 16:47:42 -07:00
Tim Hockin d6c1fd3c6b Move sync-hook to after symlink flip 2021-04-05 17:02:00 -07:00
Tim Hockin dcb918f846 WIP: startup tests 2021-03-29 16:48:50 -07:00
Kubernetes Prow Robot 7b594781f8
Merge pull request #344 from thockin/fix-pid1-exit-code-v4
Fix exit non-zero exit codes when running as pid1
2021-03-15 09:39:06 -07:00
Tim Hockin 45b28530a7 Fix exit non-zero exit codes when running as pid1
(v4 branch)

Prior to this we would swallow the exit code and always exit(0).
2021-03-15 09:28:26 -07:00