test(ci): the integration job must clean up its stacks even when killed

The platform leaked a whole CI stack for two days: a job the runner
kills never runs its cleanup trap. The platform's rule now applies here:
the job that starts Docker stacks (integration) has its own
EKKLESIA_CI_JOB_KEY, sweeps stale CI stacks first, removes its own in a
final if: always() step, and labels the image it builds.

ci/check-cleanup.sh checks that, and the lint job runs it. It fails
today on all five counts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-05 20:58:04 -05:00
co-authored by Claude Opus 5.5
parent 914034dda0
commit cfa4628f8c
2 changed files with 48 additions and 0 deletions
+3
View File
@@ -61,6 +61,9 @@ jobs:
working-directory: build/workspace
run: pnpm --filter @ekklesia/signage lint
- name: Check that CI cleans up every stack it starts
run: bash ci/check-cleanup.sh
typecheck:
name: typecheck
runs-on: ubuntu-latest
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Checks that Signage's CI cleans up every stack it starts, even when the runner kills a job (the
# platform's rule: docs/development/security.md, "Disposable stacks never outlive their job").
#
# bash ci/check-cleanup.sh
#
# The `integration` job is the one that starts Docker stacks. It must have its own
# EKKLESIA_CI_JOB_KEY, sweep stale CI stacks before it starts anything, and remove its own in a final
# `if: always()` step; and the image ci/integration.sh builds must carry the CI labels the sweep finds.
set -Eeuo pipefail
root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
workflow="$root/.gitea/workflows/ci.yml"
failures=0
check() {
if eval "$1"; then
echo "ok $2"
else
echo "FAIL $2"
failures=$((failures + 1))
fi
}
# The integration job's lines, from its key to the next job (or the end of the file).
job="$(awk '/^ integration:$/ { inside = 1; print; next } inside && /^ [a-z][a-z-]*:$/ { exit } inside { print }' "$workflow")"
line_of() { grep -n -F -- "$1" <<<"$job" | head -1 | cut -d: -f1 || true; }
stale="$(line_of 'ci-sweep.sh --stale')"
run="$(line_of 'bash ci/integration.sh')"
own="$(line_of 'ci-sweep.sh --job')"
last_step="$(grep -n -- " - name:" <<<"$job" | tail -1 | cut -d: -f1 || true)"
check '[[ -n "$job" ]]' "the workflow has an integration job"
check 'grep -q "EKKLESIA_CI_JOB_KEY:" <<<"$job"' "the integration job names its stacks with its own key"
check '[[ -n "$stale" && -n "$run" ]] && ((stale < run))' "it sweeps stale CI stacks before it starts anything"
check '[[ -n "$own" && -n "$run" ]] && ((own > run)) && ((own > last_step))' \
"its last step removes its own stacks"
check 'sed -n "${last_step},\$p" <<<"$job" | grep -q "if: always()"' "that last step always runs"
check 'grep -q "EKKLESIA_CI_LABEL" "$root/ci/integration.sh" && grep -q "EKKLESIA_CI_CREATED_LABEL" "$root/ci/integration.sh"' \
"the image ci/integration.sh builds carries the CI labels"
((failures == 0)) || {
echo "check-cleanup: ${failures} check(s) failed." >&2
exit 1
}