63fda0f8d6dcdca8d97d41c07fea43f95c3626d0
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
94fa7a17a8 |
fix(ci): confine the sweep self-test to its own run
CI / verify (pull_request) Successful in 1m10s
CI / infrastructure-smoke (pull_request) Successful in 42s
CI / package-release (pull_request) Successful in 49s
CI / migration-smoke (pull_request) Successful in 2m21s
CI / image-build (pull_request) Successful in 59s
CI / deploy-smoke (same-origin) (pull_request) Successful in 52s
CI / deploy-smoke (split) (pull_request) Successful in 2m35s
CI / package-publication (pull_request) Successful in 1m3s
scripts/test-ci-sweep.sh proves the stale sweep by running `ci-sweep.sh --stale --max-age-seconds 0` on the runner's shared Docker daemon. With a zero threshold that removes every `ekklesia-ci-` stack on the daemon, not just the test's own: any job running at that moment loses its containers, networks, and volumes. It reported "CI sweep checks passed" while doing it. Core's infrastructure-smoke job runs the self-test on every run, and twice tore down Signage's integration stack mid-scenario (Zayin/Signage run 288: its first attempt, and the second one during connectivity-blip). `ci-sweep.sh --stale` now takes `--prefix P`, which narrows the sweep to stacks whose names begin with P and must itself begin with the CI prefix, so it can only narrow a sweep, never widen one. The self-test confines every stale sweep to its own fake run and proves that another run's stack survives even the zero threshold. The workflow's own sweeps are unchanged. Regression: a bystander stack from another run in the self-test, and a Docker-free guard that every stale sweep in the self-test is confined and the workflow's keep the default threshold. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||
|
|
62951975da |
fix(ci): nothing a CI job starts outlives it, even when the runner kills the job
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 28s
CI / migration-smoke (pull_request) Successful in 1m49s
CI / image-build (pull_request) Successful in 19s
CI / deploy-smoke (same-origin) (pull_request) Successful in 48s
CI / deploy-smoke (split) (pull_request) Successful in 53s
A migration-smoke stack (PostgreSQL and SeaweedFS) ran on the shared runner for two days. Every CI script removes its stack in an EXIT trap, but a trap never runs when the runner kills the job container: a cancelled job, or the runner's own time limit (Gitea 1.24 ignores timeout-minutes). Reproduced: SIGKILL of the infrastructure smoke mid-run leaves exactly that stack running. Cleanup now has three layers: - the script's own trap, unchanged; - a final `if: always()` step in every job that starts stacks (infrastructure-smoke, migration-smoke, image-build, deploy-smoke) runs `scripts/ci-sweep.sh --job`, which removes everything that job started at any age; - a first step in each of those jobs runs `scripts/ci-sweep.sh --stale`, which removes CI stacks older than three hours: what a killed job left before it could clean up. How stacks are found is defined once, in scripts/lib/ci-project.sh: - every CI Compose project is named `ekklesia-ci-<run>-<attempt>-<job key>-<kind>-<unique>`, and Compose labels its containers, networks and volumes with that name; - the deploy smoke labels everything it starts outside Compose: browser, kiosk and driver containers, the kiosk volume, built images; - images also record when CI built them. An image's own creation time can be its base's (BuildKit, metadata-only builds), and the sweep removed a brand-new image as "old" until it stopped trusting that. The job key is per matrix leg, so one topology's cleanup cannot remove the other's running stack. The sweep never touches anything outside the ekklesia-ci- prefix, disconnects rather than removes a job container attached to a stack's network, and refuses --job outside CI. Network creation times are read as RFC 3339; Docker's default form was unparseable. scripts/test-ci-sweep.sh proves all of that against real Docker, and CI runs it before relying on it. security.md and testing.md document the rule, and that it complements, not replaces, a runner-side reaper. Co-Authored-By: Claude Opus 5.5 <[email protected]> |
||
|
|
703d92436b |
test(ci): every CI stack is cleaned up, even when its job is killed
A migration-smoke stack (PostgreSQL and SeaweedFS) leaked on the shared runner and ran for two days, until it was removed by hand. Every CI script already tears its stack down in an EXIT/INT/TERM trap, but a trap never runs when the runner kills the job container: a cancelled job, or the runner's own time limit, since Gitea 1.24 ignores timeout-minutes. Static checks (tests/ci-scripts.test.ts): - every stack script names its project through one helper: the reserved ekklesia-ci- prefix, run, attempt, job key, and a unique suffix; - everything the deploy smoke creates outside Compose carries the CI label; - every workflow job that starts stacks sweeps stale CI stacks first, and removes its own in a final if: always() step. Each matrix leg has its own key, so one cannot remove the other's running stack. Behaviour against real Docker (scripts/test-ci-sweep.sh): - --job removes this job's containers, network, volume, image and staging directory, and leaves other jobs' and non-CI stacks alone; - --job disconnects, never removes, a job container on the stack's network, and refuses to run outside CI; - --stale spares young stacks and removes old ones, never anything outside the prefix. Fails today: there is no helper, no sweep, no label and no step. Co-Authored-By: Claude Opus 5.5 <[email protected]> |