Commit Graph

894 Commits

Author SHA1 Message Date
Kubernetes Prow Robot faba6051f4
Merge pull request #892 from rul/fix-e2e-sc2086
test_e2e.sh: quote variables to prevent globbing and word splitting
2024-06-19 11:54:42 -07:00
Raúl Benencia 20c26145ce test_e2e.sh: quote variables to prevent globbing and word splitting
Fixes SC2086.
2024-06-19 10:24:29 -07:00
Raúl Benencia 390f63cc6e test_e2e.sh: fully quote string sent to assert functions
Fixes SC2086. Some strings don't require to be fully quoted, but doing
so anyway to keep code consistency.
2024-06-19 10:13:58 -07:00
Tim Hockin 189a690a5b Add -? to manual 2024-06-13 14:40:14 -07:00
Tim Hockin 28394608e8 Add the idea of "env-flags"
Env-flags are "flags" that can only be set by env var (see caveat below).
All of the real flags have a corresponding env-flag (kind of, but not
really).  The real goal was to deprecate `--password` but keep the env
var as a documented interface.

This does that (though --password still works) and updates the usage and
manual.

This allows some future work to follow the pattern.  We do not register
every CLI flag as an env-flag because the help text would be
duplicative.  This probably wants a wrapper API that allows declaring of
abstract flags, with CLI, env, or both sources.

Caveat:

ACTUALLY, these still have a flag, but the flag is specially named and
hidden.  This makes testing a little easier where passing flags is
handled well but env vars is not.
2024-06-13 14:40:14 -07:00
Tim Hockin aa0f015606 Fix e2e, broken by a docker update I think
github actions fails with an error about "--ip can only be used on
user-defined subnets".  It looks like `--ip` never worked properly, but
wasn't a hard error before.

This is a simpler alternative to
11f475229692da93dff4d8a9337c5fc9cc48e51a (included below), which tried
using docker networks.  It seems to work but is complicated and can leak
resources.  Needs more work.

Instead, this commit just swaps out the `nc` response script
on the fly, rather than restarting `nc` and trying to get the same IP.

```diff
commit 11f475229692da93dff4d8a9337c5fc9cc48e51a
Good "git" signature for thockin@google.com with ED25519 key SHA256:PfQ0rwNUgsu5aRmerT0vkihWn/S3MXY3uoCPUiMdPrg
Author: Tim Hockin <thockin@google.com>
Date:   Wed Jun 12 20:12:54 2024 -0700

    debug test fail

    github actions fails with an error about "--ip can only be used on
    user-defined subnets"

diff --git a/test_e2e.sh b/test_e2e.sh
index d6ad730..b10e895 100755
--- a/test_e2e.sh
+++ b/test_e2e.sh
@@ -117,7 +117,7 @@ function assert_file_lines_ge() {

 function assert_metric_eq() {
     local val
-    val="$(curl --silent "http://localhost:$HTTP_PORT/metrics" \
+    val="$(curl --silent "http://$GITSYNC_IP:$HTTP_PORT/metrics" \
         | grep "^$1 " \
         | awk '{print $NF}')"
     if [[ "${val}" == "$2" ]]; then
@@ -138,6 +138,9 @@ function assert_fail() {
     )
 }

+DOCKER_SUBNET="192.168.0.0/24"
+GITSYNC_IP="192.168.0.254"
+
 # Helper: run a docker container.
 function docker_run() {
     RM="--rm"
@@ -148,6 +151,7 @@ function docker_run() {
         -d \
         ${RM} \
         --label git-sync-e2e="$RUNID" \
+        --network "e2e_$RUNID" \
         "$@"
     sleep 2 # wait for it to come up
 }
@@ -158,7 +162,8 @@ function docker_ip() {
         echo "usage: $0 <id>"
         return 1
     fi
-    docker inspect "$1" | jq -r .[0].NetworkSettings.IPAddress
+    docker inspect "$1" \
+        | jq -r ".[0].NetworkSettings.Networks.e2e_$RUNID.IPAddress"
 }

 function docker_kill() {
@@ -278,7 +283,8 @@ function GIT_SYNC() {
         -i \
         ${RM} \
         --label git-sync-e2e="$RUNID" \
-        --network="host" \
+        --network "e2e_$RUNID" \
+        --ip "$GITSYNC_IP" \
         -u git-sync:$(id -g) `# rely on GID, triggering "dubious ownership"` \
         -v "$ROOT":"$ROOT":rw \
         -v "$REPO":"$REPO":ro \
@@ -308,6 +314,9 @@ function remove_containers() {
         | while read CTR; do
             docker kill "$CTR" >/dev/null
         done
+    docker network prune -f \
+        --filter label=git-sync-e2e \
+        >/dev/null
 }

 #
@@ -2515,7 +2524,7 @@ function e2e::expose_http() {
     # do nothing, just wait for the HTTP to come up
     for i in $(seq 1 5); do
         sleep 1
-        if curl --silent --output /dev/null http://localhost:$HTTP_PORT; then
+        if curl --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT"; then
             break
         fi
         if [[ "$i" == 5 ]]; then
@@ -2524,23 +2533,23 @@ function e2e::expose_http() {
     done

     # check that health endpoint fails
-    if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 503 ]] ; then
-        fail "health endpoint should have failed: $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT)"
+    if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 503 ]] ; then
+        fail "health endpoint should have failed: $(curl --write-out %{http_code} --silent --output /dev/null http://$GITSYNC_IP:$HTTP_PORT)"
     fi
     wait_for_sync "${MAXWAIT}"

     # check that health endpoint is alive
-    if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 200 ]] ; then
+    if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 200 ]] ; then
         fail "health endpoint failed"
     fi

     # check that the metrics endpoint exists
-    if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT/metrics) -ne 200 ]] ; then
+    if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT/metrics") -ne 200 ]] ; then
         fail "metrics endpoint failed"
     fi

     # check that the pprof endpoint exists
-    if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT/debug/pprof/) -ne 200 ]] ; then
+    if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT/debug/pprof/") -ne 200 ]] ; then
         fail "pprof endpoint failed"
     fi
 }
@@ -2568,7 +2577,7 @@ function e2e::expose_http_after_restart() {
     # do nothing, just wait for the HTTP to come up
     for i in $(seq 1 5); do
         sleep 1
-        if curl --silent --output /dev/null http://localhost:$HTTP_PORT; then
+        if curl --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT"; then
             break
         fi
         if [[ "$i" == 5 ]]; then
@@ -2579,7 +2588,7 @@ function e2e::expose_http_after_restart() {
     sleep 2 # wait for first loop to confirm synced

     # check that health endpoint is alive
-    if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 200 ]] ; then
+    if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 200 ]] ; then
         fail "health endpoint failed"
     fi
     assert_link_exists "$ROOT/link"
@@ -3503,6 +3512,12 @@ function run_test() {
         set -o errexit
         set -o nounset
         set -o pipefail
+        docker network prune -f \
+            --filter label=git-sync-e2e \
+            >/dev/null
+        docker network create "e2e_$RUNID" \
+            --subnet "$DOCKER_SUBNET" \
+            --label git-sync-e2e="$RUNID"
         "$@"
     )
     eval "$retvar=$?"
```
2024-06-13 12:30:11 -07:00
Tim Hockin 6ef292034e Log tool versions in e2e 2024-06-13 12:30:11 -07:00
Tim Hockin aa230f92f8 Retool the demo
Rather than hugo, which I really don't know much about and don't care to
maintain, it now is just a trivial HTTP server (python) serving content
from this repo.
2024-06-12 19:13:13 -07:00
Kubernetes Prow Robot e027ef64ac
Merge pull request #883 from thockin/master
Clean up flag-related messages
2024-06-10 14:14:46 -07:00
Tim Hockin 9535f4fe7e Add curl to the image
This allows exechooks to call curl.
2024-06-09 11:24:03 -07:00
Tim Hockin fbc717e620
Clean up flag-related messages 2024-06-09 10:54:19 -07:00
Tim Hockin 750e20e6da Set git `safe.directory` in tool containers
Otherwise it hit the "dubious ownership" case.
2024-06-09 00:57:47 -07:00
Tim Hockin b5b0558d58 Bump golangci-lint and fix lint 2024-06-09 00:57:47 -07:00
Tim Hockin 9042405ea0 Bump github workflows to 1.22 2024-06-09 00:57:47 -07:00
Tim Hockin b4dfc82490 Log the git version 2024-06-09 00:42:51 -07:00
Tim Hockin cd1d050a67 Log the binary version 2024-06-09 00:42:51 -07:00
Tim Hockin 4a4c8f8d11 Don't log hidden flags 2024-06-09 00:42:51 -07:00
Tim Hockin b8e5c80ec4 Bump go to 1.22 2024-06-08 18:29:50 -07:00
Tim Hockin aa2ac24c25 Support -? as an alias for --help
Also print errors before and after usage, like pflag.
2024-06-08 14:10:38 -07:00
Kubernetes Prow Robot 5e40d477f9
Merge pull request #876 from kubernetes/dependabot/github_actions/golangci/golangci-lint-action-6
Bump golangci/golangci-lint-action from 5 to 6
2024-05-13 09:09:28 -07:00
dependabot[bot] 9da53c5406
Bump golangci/golangci-lint-action from 5 to 6
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 5 to 6.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-13 14:00:15 +00:00
Kubernetes Prow Robot df639fddbf
Merge pull request #872 from kubernetes/dependabot/github_actions/golangci/golangci-lint-action-5
Bump golangci/golangci-lint-action from 4 to 5
2024-04-29 08:12:36 -07:00
dependabot[bot] 179d0e6ebe
Bump golangci/golangci-lint-action from 4 to 5
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 4 to 5.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-29 13:07:47 +00:00
Kubernetes Prow Robot 9d546c38fa
Merge pull request #870 from demoth/patch-1
README.md - fix usage comments
2024-04-13 13:40:35 -07:00
Kubernetes Prow Robot e6957283d4
Merge pull request #871 from dims/Bump-bookworm-image-to-1.0.2
Bump base bookworm image to 1.0.2
2024-04-12 13:12:42 -07:00
Davanum Srinivas 4b78431723
Bump base bookworm image to 1.0.2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2024-04-10 11:03:36 -04:00
demoth 5425f59147
README.md - fix usage comments 2024-04-10 11:56:34 +02:00
Tim Hockin f521da5548 Add e2e for ssh:// URLs 2024-04-04 16:40:32 -07:00
Kubernetes Prow Robot e412c5f9ee
Merge pull request #868 from yyvess/fix/git_ssh_auth
fix(auth): Fix ssh authentication
2024-04-04 09:17:54 -07:00
Yves Galante 28b8f36746
fix(auth): Fix ssh authentication
Signed-off-by: Yves Galante <yyvess@gmail.com>
2024-04-04 18:01:02 +02:00
Kubernetes Prow Robot e8305c895f
Merge pull request #867 from sdowell/bump-protobuf
chore: bump google.golang.org/protobuf
2024-03-14 12:50:53 -07:00
Sam Dowell aca13550d4 chore: bump google.golang.org/protobuf
https://github.com/advisories/GHSA-8r3f-844c-mc37
2024-03-14 11:44:10 -07:00
Tim Hockin 772ae53548 Use bash instead of dash
It's actually slightly smaller.
2024-03-12 08:43:50 -07:00
Kubernetes Prow Robot 3f817b201d
Merge pull request #863 from kubernetes/dependabot/github_actions/golangci/golangci-lint-action-4
Bump golangci/golangci-lint-action from 3 to 4
2024-02-12 09:52:13 -08:00
dependabot[bot] 1f2fb61c61
Bump golangci/golangci-lint-action from 3 to 4
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-12 13:08:35 +00:00
Kubernetes Prow Robot 33e781eae4
Merge pull request #862 from jmichalek132/patch-1
chore: fix missing unit duration in the k8s example
2024-02-08 08:59:31 -08:00
Juraj Michálek 0afb5a27af
chore: fix missing unit duration in the k8s example 2024-02-08 15:19:11 +01:00
Tim Hockin 4c4bd3b310 Fix staging script
Better error handling in case of failed commands.

Match "vdso" and "vdso64".
2024-02-05 14:44:03 -08:00
Tim Hockin 5e4ceb1b73 Add '-F none' to e2e git SSH command for weird environments 2024-01-16 17:25:05 -08:00
Tim Hockin 1e1f846018 Bump to go 1.21 2024-01-16 16:59:41 -08:00
Tim Hockin d5c61f7ad1 Try not to log credentials in repo URL
This extracts the username and password from the URL and replaces it
with the bare URL, using the usual credential mechanism for user/pass.
2023-12-14 10:12:26 -08:00
Kubernetes Prow Robot 65e0953196
Merge pull request #850 from kubernetes/dependabot/github_actions/actions/setup-go-5
Bump actions/setup-go from 4 to 5
2023-12-12 18:49:47 +01:00
dependabot[bot] 3779d16285
Bump actions/setup-go from 4 to 5
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-11 13:08:00 +00:00
Tim Hockin 083d8dda85 Simplify inner loop: just fetch $ref
Old way:
  - ls-remote $ref $ref^{} and parse
  - compare to current
  - if changed, fetch
  - update worktree

New way:
  - fetch $ref
  - compare to current
  - if change, update worktree
2023-12-08 18:25:00 -08:00
Tim Hockin ad92ba62c4 Make abspath Split return an abspath 2023-12-08 18:04:24 -08:00
Tim Hockin 9a806c09cb Bug: links are relative to linkdir, not rootdir 2023-12-08 18:04:24 -08:00
Tim Hockin 718a938cd4 Log worktree path 2023-12-08 18:04:24 -08:00
Tim Hockin 9e1d0c5936 Document the change in default target 2023-11-12 16:20:26 -06:00
Tim Hockin 1fdc98e3b0 Document that short SHAs dont work 2023-11-12 16:20:26 -06:00
Tim Hockin a5cec7a036 Log less verbose flags in less verbose mode 2023-11-01 13:26:34 -07:00