ekklesiaandClaude Opus 5.5
278e66049f
CI / verify (pull_request) Successful in 50s
CI / infrastructure-smoke (pull_request) Successful in 14s
CI / migration-smoke (pull_request) Successful in 1m49s
CI / image-build (pull_request) Successful in 19s
CI / deploy-smoke (pull_request) Successful in 41s
PR #51's migration-smoke failed 0.7 s after the build: the suite pinned PostgreSQL to an ephemeral port but inherited S3_PORT=8333 from .env.example, so concurrent runs on one runner (Gitea ignores workflow concurrency) contended for one host port. Compose's own error was discarded, so the failure said only that the services "did not become ready". Isolation, from an audit of every script CI runs: - test-migrations-integration.sh: S3_PORT=0, like POSTGRES_PORT=0. - test-migrations-integration.sh and smoke-infrastructure.sh: project names gain a random suffix; run id plus PID alone is not unique when job containers share low PIDs. - smoke-deploy.sh: the edge /24 is chosen from those no network on the daemon uses, and the first `up` chooses again when a concurrent run wins a race for the same subnet (Docker's address-pool overlap), up to five times. - CI image-build: the image tag includes the run id and attempt, so a pull request and a push of the same commit cannot remove each other's image. - Already isolated: every other published port (ephemeral), the TLS suite's container and network (random), smoke staging directories and image tags (random). Diagnostics: scripts/lib/compose-diagnostics.sh prints the failed `up` command's output, `docker compose ps`, and the last 40 log lines of each service, all redacted (supplied literals, credential-looking NAME=value pairs, URL user information, Authorization headers) with terminal control sequences stripped. All three scripts use it at every readiness check; the infrastructure smoke's unredacted full-log dump is replaced by it. tests/ci-scripts.test.ts pins ephemeral ports, unique project names, the subnet choice, the CI image tag, the diagnostics, and the redaction itself (run through bash). Co-Authored-By: Claude Opus 5.5 <[email protected]>
73 lines
3.3 KiB
Bash
73 lines
3.3 KiB
Bash
#!/usr/bin/env bash
|
|
# Why a disposable Compose project did not become ready: `docker compose ps` and the last log lines
|
|
# of every service, so a CI failure says what went wrong (a port already allocated, a container that
|
|
# exited, a health check that never passed) instead of only that something did.
|
|
#
|
|
# source scripts/lib/compose-diagnostics.sh
|
|
# COMPOSE_DIAGNOSTICS_SECRETS=("$password" "$secret_key") # literal values to mask, optional
|
|
# compose_report_not_ready "$up_output" "${compose[@]}"
|
|
#
|
|
# Everything printed passes through `compose_diagnostics_redact` first. The projects these scripts
|
|
# start hold only disposable, generated test credentials, but output is redacted anyway: every
|
|
# literal in COMPOSE_DIAGNOSTICS_SECRETS, the value of any `NAME=value` or `NAME: value` whose name
|
|
# looks like a credential, the user-information part of any URL, and the rest of any Authorization
|
|
# header. Terminal control sequences are stripped. Output goes to standard error.
|
|
|
|
COMPOSE_DIAGNOSTICS_SECRETS=()
|
|
COMPOSE_DIAGNOSTICS_TAIL_LINES=40
|
|
|
|
# compose_diagnostics_redact: standard input to standard output, with credentials masked.
|
|
compose_diagnostics_redact() {
|
|
local secrets=""
|
|
if ((${#COMPOSE_DIAGNOSTICS_SECRETS[@]} > 0)); then
|
|
secrets="$(printf '%s\n' "${COMPOSE_DIAGNOSTICS_SECRETS[@]}")"
|
|
fi
|
|
# Literal values first, by plain substring rather than as patterns. Very short values are skipped:
|
|
# masking every occurrence of a two-letter word would only make the output unreadable.
|
|
COMPOSE_DIAGNOSTICS_LITERALS="$secrets" awk '
|
|
BEGIN { count = split(ENVIRON["COMPOSE_DIAGNOSTICS_LITERALS"], secret, "\n") }
|
|
{
|
|
line = $0
|
|
for (i = 1; i <= count; i++) {
|
|
if (length(secret[i]) < 6) {
|
|
continue
|
|
}
|
|
masked = ""
|
|
while ((at = index(line, secret[i])) > 0) {
|
|
masked = masked substr(line, 1, at - 1) "[redacted]"
|
|
line = substr(line, at + length(secret[i]))
|
|
}
|
|
line = masked line
|
|
}
|
|
print line
|
|
}
|
|
' | sed -E \
|
|
-e 's/\x1b\[[0-9;?]*[A-Za-z]//g' \
|
|
-e 's#([Aa][Uu][Tt][Hh][Oo][Rr][Ii][Zz][Aa][Tt][Ii][Oo][Nn][[:space:]]*[:=]).*#\1 [redacted]#' \
|
|
-e 's#([A-Za-z0-9_.-]*([Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd]|[Pp][Aa][Ss][Ss][Ww][Dd]|[Ss][Ee][Cc][Rr][Ee][Tt]|[Tt][Oo][Kk][Ee][Nn]|[_.-][Kk][Ee][Yy])[A-Za-z0-9_.-]*"?[[:space:]]*[=:][[:space:]]*)("[^"]*"|[^[:space:],;"]+)#\1[redacted]#g' \
|
|
-e 's#([A-Za-z][A-Za-z0-9+.-]*://)[^/@[:space:]]+@#\1[redacted]@#g'
|
|
}
|
|
|
|
# compose_report_not_ready <up output> <compose command...>: the failed command's own output, then
|
|
# the project's containers and the tail of each service's logs, all redacted.
|
|
compose_report_not_ready() {
|
|
local up_output="$1"
|
|
shift
|
|
local service services
|
|
{
|
|
if [[ -n "$up_output" ]]; then
|
|
echo "--- docker compose up (last 20 lines)"
|
|
printf '%s\n' "$up_output" | tail -n 20 | compose_diagnostics_redact
|
|
fi
|
|
echo "--- docker compose ps"
|
|
"$@" ps --all 2>&1 | compose_diagnostics_redact
|
|
services="$("$@" config --services 2>/dev/null || true)"
|
|
for service in $services; do
|
|
echo "--- docker compose logs ${service} (last ${COMPOSE_DIAGNOSTICS_TAIL_LINES} lines)"
|
|
"$@" logs --no-color --tail "$COMPOSE_DIAGNOSTICS_TAIL_LINES" "$service" 2>&1 |
|
|
compose_diagnostics_redact
|
|
done
|
|
echo "--- end of diagnostics"
|
|
} >&2
|
|
}
|