Files
Ekklesia/scripts/test-migrations-integration.sh
T
ekklesiaandClaude Opus 5.5 278e66049f
CI / verify (pull_request) Successful in 50s
CI / infrastructure-smoke (pull_request) Successful in 14s
CI / migration-smoke (pull_request) Successful in 1m49s
CI / image-build (pull_request) Successful in 19s
CI / deploy-smoke (pull_request) Successful in 41s
fix(ci): isolate concurrent runs on a shared Docker daemon and explain readiness failures
PR #51's migration-smoke failed 0.7 s after the build: the suite pinned
PostgreSQL to an ephemeral port but inherited S3_PORT=8333 from
.env.example, so concurrent runs on one runner (Gitea ignores workflow
concurrency) contended for one host port. Compose's own error was
discarded, so the failure said only that the services "did not become
ready".

Isolation, from an audit of every script CI runs:
- test-migrations-integration.sh: S3_PORT=0, like POSTGRES_PORT=0.
- test-migrations-integration.sh and smoke-infrastructure.sh: project names
  gain a random suffix; run id plus PID alone is not unique when job
  containers share low PIDs.
- smoke-deploy.sh: the edge /24 is chosen from those no network on the
  daemon uses, and the first `up` chooses again when a concurrent run wins
  a race for the same subnet (Docker's address-pool overlap), up to five
  times.
- CI image-build: the image tag includes the run id and attempt, so a pull
  request and a push of the same commit cannot remove each other's image.
- Already isolated: every other published port (ephemeral), the TLS
  suite's container and network (random), smoke staging directories and
  image tags (random).

Diagnostics: scripts/lib/compose-diagnostics.sh prints the failed `up`
command's output, `docker compose ps`, and the last 40 log lines of each
service, all redacted (supplied literals, credential-looking NAME=value
pairs, URL user information, Authorization headers) with terminal control
sequences stripped. All three scripts use it at every readiness check; the
infrastructure smoke's unredacted full-log dump is replaced by it.

tests/ci-scripts.test.ts pins ephemeral ports, unique project names, the
subnet choice, the CI image tag, the diagnostics, and the redaction itself
(run through bash).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-03 12:15:01 -05:00

152 lines
6.0 KiB
Bash
Executable File

#!/usr/bin/env bash
set -Eeuo pipefail
repository_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$repository_root"
# shellcheck source=scripts/lib/compose-job-network.sh
source "$repository_root/scripts/lib/compose-job-network.sh"
# shellcheck source=scripts/lib/compose-diagnostics.sh
source "$repository_root/scripts/lib/compose-diagnostics.sh"
command -v docker >/dev/null || {
echo "Docker is required for the PostgreSQL migration integration suite." >&2
exit 1
}
command -v pnpm >/dev/null || {
echo "pnpm is required for the PostgreSQL migration integration suite." >&2
exit 1
}
if ! docker compose version >/dev/null 2>&1; then
echo "Docker Compose is required for the PostgreSQL migration integration suite." >&2
exit 1
fi
if ! compose_job_network_initialize; then
echo "The current Docker job container could not be identified." >&2
exit 1
fi
# Unique per invocation, not only per run: several runs (and several jobs of one run) can share a
# runner's Docker daemon at once, and the shell's PID is often identical across job containers.
raw_project="ekklesia-migration-${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-0}-${GITHUB_JOB:-job}-$$-${RANDOM}${RANDOM}"
project="${raw_project//[^a-zA-Z0-9_-]/-}"
project="${project,,}"
export POSTGRES_DB="ekklesia_migration_test"
export POSTGRES_USER="ekklesia_migration_test"
export POSTGRES_PASSWORD="temporary-$project-${RANDOM}-${RANDOM}"
# Both host ports are ephemeral: concurrent runs on one Docker daemon must never contend for a
# fixed one. `.env.example` pins the storage port for local development, so it is overridden here
# exactly like the database's. The bound ports are discovered below.
export POSTGRES_PORT="0"
export S3_PORT="0"
compose=(
docker compose
--project-name "$project"
--env-file .env.example
--file infrastructure/compose.yaml
)
cleanup() {
local exit_status=$?
trap - EXIT INT TERM
if ! compose_job_network_disconnect; then
echo "PostgreSQL migration test network cleanup failed." >&2
if ((exit_status == 0)); then
exit_status=1
fi
fi
if ! "${compose[@]}" down --volumes --remove-orphans --timeout 10 >/dev/null 2>&1; then
echo "PostgreSQL migration test infrastructure cleanup failed." >&2
if ((exit_status == 0)); then
exit_status=1
fi
fi
exit "$exit_status"
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
if ! "${compose[@]}" config --quiet >/dev/null 2>&1; then
echo "PostgreSQL migration test infrastructure configuration is invalid." >&2
exit 1
fi
# PostgreSQL and the S3-compatible object-storage service are both brought up from the repository's
# own pinned Compose file, so the live suites exercise exactly the services a developer runs locally
# and CI already knows how to provision. SeaweedFS is the intended deployment target (ADR 0004), and
# proving the storage contract against a mock instead would have hidden the provider quirks this
# suite exists to catch.
# Masked in any diagnostics below, along with anything else that looks like a credential.
COMPOSE_DIAGNOSTICS_SECRETS=(
"$POSTGRES_PASSWORD"
"$(grep -E '^S3_ACCESS_KEY=' .env.example | cut -d= -f2-)"
"$(grep -E '^S3_SECRET_KEY=' .env.example | cut -d= -f2-)"
)
if ! up_output="$(
"${compose[@]}" up --detach --wait --wait-timeout 180 postgres object-storage 2>&1
)"; then
echo "Disposable PostgreSQL and object storage did not become ready." >&2
compose_report_not_ready "$up_output" "${compose[@]}"
exit 1
fi
if [[ "$COMPOSE_JOB_NETWORK_RUNNING_IN_CONTAINER" == "true" ]]; then
if ! compose_job_network_connect postgres "${compose[@]}"; then
echo "The migration test job could not join the isolated PostgreSQL network." >&2
exit 1
fi
if ! compose_job_network_connect object-storage "${compose[@]}"; then
echo "The migration test job could not join the isolated object-storage network." >&2
exit 1
fi
database_host="postgres"
database_port="5432"
storage_endpoint="http://object-storage:8333"
else
if ! postgres_binding="$("${compose[@]}" port postgres 5432 2>/dev/null)"; then
echo "The disposable PostgreSQL loopback port could not be determined." >&2
exit 1
fi
if [[ ! "$postgres_binding" =~ ^127\.0\.0\.1:([0-9]+)$ ]]; then
echo "The disposable PostgreSQL port is not bound safely to loopback." >&2
exit 1
fi
database_host="127.0.0.1"
database_port="${BASH_REMATCH[1]}"
if ((database_port < 1 || database_port > 65535)); then
echo "The disposable PostgreSQL loopback port is invalid." >&2
exit 1
fi
if ! storage_binding="$("${compose[@]}" port object-storage 8333 2>/dev/null)"; then
echo "The disposable object-storage loopback port could not be determined." >&2
exit 1
fi
if [[ ! "$storage_binding" =~ ^127\.0\.0\.1:([0-9]+)$ ]]; then
echo "The disposable object-storage port is not bound safely to loopback." >&2
exit 1
fi
storage_port="${BASH_REMATCH[1]}"
if ((storage_port < 1 || storage_port > 65535)); then
echo "The disposable object-storage loopback port is invalid." >&2
exit 1
fi
storage_endpoint="http://127.0.0.1:$storage_port"
fi
# The TLS suite provisions its own disposable PostgreSQL rather than a Compose service, so it
# cannot call `compose_job_network_connect`. It receives the same topology decision this script
# already made, instead of detecting the topology a second time.
# The storage credentials come from `.env.example`, the same file `--env-file` already hands the
# Compose services, so the suite signs requests with exactly what the service was started with.
EKKLESIA_TEST_JOB_CONTAINER="$(compose_job_network_job_container)" \
EKKLESIA_MIGRATION_TEST_DATABASE_URL="postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@$database_host:$database_port/$POSTGRES_DB" \
EKKLESIA_TEST_STORAGE_ENDPOINT="$storage_endpoint" \
EKKLESIA_TEST_STORAGE_ACCESS_KEY="$(grep -E '^S3_ACCESS_KEY=' .env.example | cut -d= -f2-)" \
EKKLESIA_TEST_STORAGE_SECRET_KEY="$(grep -E '^S3_SECRET_KEY=' .env.example | cut -d= -f2-)" \
pnpm exec vitest run --config vitest.migrations.integration.config.ts