The GitHub app e2e test requires a GitHub app to be created and
installed, and also requires a few environment variables to be set.
This commit updates the GitHub action workflow by providing the
environment variables which can be set via GitHub Secret. GitHub
Secrests cannot start with `GITHUB_`. Hence, this commit prepends
`TEST_` to the env variables.
It also updates how GitHub app private key file is set. It can be set by
either `TEST_GITHUB_APP_PRIVATE_KEY` or
`TEST_GITHUB_APP_PRIVATE_KEY_FILE`.
This introduces several changes to make the e2e test image logic more
robust:
- Rename IMAGE to GIT_SYNC_E2E_IMAGE to minimize accidental misuse
- Ensure that the image built by make container is the same as the image
used by docker run
- Do not build the container if a custom image is specified
As an example, with the prior logic the following command always fails:
IMAGE="" make test
...
Unable to find image 'gcr.io/k8s-staging-git-sync/git-sync:latest' locally
Because the docker run command pulls a different image from the one
built by make container.
Env-flags are "flags" that can only be set by env var (see caveat below).
All of the real flags have a corresponding env-flag (kind of, but not
really). The real goal was to deprecate `--password` but keep the env
var as a documented interface.
This does that (though --password still works) and updates the usage and
manual.
This allows some future work to follow the pattern. We do not register
every CLI flag as an env-flag because the help text would be
duplicative. This probably wants a wrapper API that allows declaring of
abstract flags, with CLI, env, or both sources.
Caveat:
ACTUALLY, these still have a flag, but the flag is specially named and
hidden. This makes testing a little easier where passing flags is
handled well but env vars is not.
github actions fails with an error about "--ip can only be used on
user-defined subnets". It looks like `--ip` never worked properly, but
wasn't a hard error before.
This is a simpler alternative to
11f475229692da93dff4d8a9337c5fc9cc48e51a (included below), which tried
using docker networks. It seems to work but is complicated and can leak
resources. Needs more work.
Instead, this commit just swaps out the `nc` response script
on the fly, rather than restarting `nc` and trying to get the same IP.
```diff
commit 11f475229692da93dff4d8a9337c5fc9cc48e51a
Good "git" signature for thockin@google.com with ED25519 key SHA256:PfQ0rwNUgsu5aRmerT0vkihWn/S3MXY3uoCPUiMdPrg
Author: Tim Hockin <thockin@google.com>
Date: Wed Jun 12 20:12:54 2024 -0700
debug test fail
github actions fails with an error about "--ip can only be used on
user-defined subnets"
diff --git a/test_e2e.sh b/test_e2e.sh
index d6ad730..b10e895 100755
--- a/test_e2e.sh
+++ b/test_e2e.sh
@@ -117,7 +117,7 @@ function assert_file_lines_ge() {
function assert_metric_eq() {
local val
- val="$(curl --silent "http://localhost:$HTTP_PORT/metrics" \
+ val="$(curl --silent "http://$GITSYNC_IP:$HTTP_PORT/metrics" \
| grep "^$1 " \
| awk '{print $NF}')"
if [[ "${val}" == "$2" ]]; then
@@ -138,6 +138,9 @@ function assert_fail() {
)
}
+DOCKER_SUBNET="192.168.0.0/24"
+GITSYNC_IP="192.168.0.254"
+
# Helper: run a docker container.
function docker_run() {
RM="--rm"
@@ -148,6 +151,7 @@ function docker_run() {
-d \
${RM} \
--label git-sync-e2e="$RUNID" \
+ --network "e2e_$RUNID" \
"$@"
sleep 2 # wait for it to come up
}
@@ -158,7 +162,8 @@ function docker_ip() {
echo "usage: $0 <id>"
return 1
fi
- docker inspect "$1" | jq -r .[0].NetworkSettings.IPAddress
+ docker inspect "$1" \
+ | jq -r ".[0].NetworkSettings.Networks.e2e_$RUNID.IPAddress"
}
function docker_kill() {
@@ -278,7 +283,8 @@ function GIT_SYNC() {
-i \
${RM} \
--label git-sync-e2e="$RUNID" \
- --network="host" \
+ --network "e2e_$RUNID" \
+ --ip "$GITSYNC_IP" \
-u git-sync:$(id -g) `# rely on GID, triggering "dubious ownership"` \
-v "$ROOT":"$ROOT":rw \
-v "$REPO":"$REPO":ro \
@@ -308,6 +314,9 @@ function remove_containers() {
| while read CTR; do
docker kill "$CTR" >/dev/null
done
+ docker network prune -f \
+ --filter label=git-sync-e2e \
+ >/dev/null
}
#
@@ -2515,7 +2524,7 @@ function e2e::expose_http() {
# do nothing, just wait for the HTTP to come up
for i in $(seq 1 5); do
sleep 1
- if curl --silent --output /dev/null http://localhost:$HTTP_PORT; then
+ if curl --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT"; then
break
fi
if [[ "$i" == 5 ]]; then
@@ -2524,23 +2533,23 @@ function e2e::expose_http() {
done
# check that health endpoint fails
- if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 503 ]] ; then
- fail "health endpoint should have failed: $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT)"
+ if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 503 ]] ; then
+ fail "health endpoint should have failed: $(curl --write-out %{http_code} --silent --output /dev/null http://$GITSYNC_IP:$HTTP_PORT)"
fi
wait_for_sync "${MAXWAIT}"
# check that health endpoint is alive
- if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 200 ]] ; then
+ if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 200 ]] ; then
fail "health endpoint failed"
fi
# check that the metrics endpoint exists
- if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT/metrics) -ne 200 ]] ; then
+ if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT/metrics") -ne 200 ]] ; then
fail "metrics endpoint failed"
fi
# check that the pprof endpoint exists
- if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT/debug/pprof/) -ne 200 ]] ; then
+ if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT/debug/pprof/") -ne 200 ]] ; then
fail "pprof endpoint failed"
fi
}
@@ -2568,7 +2577,7 @@ function e2e::expose_http_after_restart() {
# do nothing, just wait for the HTTP to come up
for i in $(seq 1 5); do
sleep 1
- if curl --silent --output /dev/null http://localhost:$HTTP_PORT; then
+ if curl --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT"; then
break
fi
if [[ "$i" == 5 ]]; then
@@ -2579,7 +2588,7 @@ function e2e::expose_http_after_restart() {
sleep 2 # wait for first loop to confirm synced
# check that health endpoint is alive
- if [[ $(curl --write-out %{http_code} --silent --output /dev/null http://localhost:$HTTP_PORT) -ne 200 ]] ; then
+ if [[ $(curl --write-out %{http_code} --silent --output /dev/null "http://$GITSYNC_IP:$HTTP_PORT") -ne 200 ]] ; then
fail "health endpoint failed"
fi
assert_link_exists "$ROOT/link"
@@ -3503,6 +3512,12 @@ function run_test() {
set -o errexit
set -o nounset
set -o pipefail
+ docker network prune -f \
+ --filter label=git-sync-e2e \
+ >/dev/null
+ docker network create "e2e_$RUNID" \
+ --subnet "$DOCKER_SUBNET" \
+ --label git-sync-e2e="$RUNID"
"$@"
)
eval "$retvar=$?"
```
Old way:
- ls-remote $ref $ref^{} and parse
- compare to current
- if changed, fetch
- update worktree
New way:
- fetch $ref
- compare to current
- if change, update worktree
Tim Hockin
AlanZhang2002
Nan Yu
Liam Wyllie
Sam Dowell
Raúl Benencia
Aleksandar Markovski