17 Commits

Include renames

Author	SHA1	Message	Date
ekklesia	6e64b3c5f9	test(deploy): allow slower display uploads	2026-10-09 22:37:18 -05:00
ekklesia	fa27603133	test(deploy): exercise external module packages	2026-10-09 22:18:04 -05:00
ekklesiaandClaude Opus 5.5	d3c62e63bf	feat: publish the exact @ekklesia package train to the npm registry (ADR 0053) ... Publishes one package release train to the Gitea npm registry under River_of_Glory_Church, whose packages are publicly readable on purpose: exactly the .tgz files the package-release gate proved, and nothing else. Nothing is published by this change. - scripts/publish-package-train.ts validates the whole local train before any request (gate manifest hash, version, revision, unmodified source, the nine packages, every artifact's size, SHA-256, integrity, SHA256SUMS line, and inner package.json), then publishes in dependency order with `npm publish <file.tgz>` under a staging tag. Each package is reconciled first (absent: publish; same bytes: accept; other bytes: hard stop, version burned), then read back: metadata (name, version, integrity, dependency maps, normalized bin, tarball on the registry) and the tarball itself, byte for byte, also anonymously. `latest` moves only after all nine verify; reruns converge and never delete anything. It writes a publication record. The token comes from EKKLESIA_PACKAGES_PUBLISH_TOKEN only, through a temporary npm config that refers to the variable. - scripts/test-package-publication.sh proves it against a disposable Gitea 1.27.2 with public owners, a publisher team and an outsider: anonymous reads, refused unauthorized publishing, byte-identical round trips, the registry's own 409, partial-train recovery, the burned version hard stop, and dist-tag retry convergence. - CI: package-release keeps its proven release and hands it on as an artifact with its manifest hash; the new package-publication job publishes exactly that artifact to the disposable Gitea. Artifact actions v3 (node20): v4 refuses any server other than github.com. - .github/workflows/publish-packages.yml publishes to the real registry only by manual dispatch on main, after its own gate run, with the secret in the publish step alone. - docs: the public-read model, the workflow, the record, credentials, immutability policy, provider moves, and the first-publication steps. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-08 09:49:27 -05:00
ekklesiaandClaude Opus 5.5	9debe4e891	feat: the public @ekklesia package release train, built, reproduced, and proven (ADR 0053) ... Produces the exact public package tarballs of a platform release, validates them, and proves a standalone module consumes those artifacts end to end. Nothing is published and no credential is configured. - scripts/package-release.ts (pnpm packages:release) is the canonical release builder: from a clean commit it rebuilds the nine public packages from clean output, packs each, inspects every tarball, checks the release set and coordinated versions, and only then writes ekklesia-packages-<version>/ with the tarballs, SHA256SUMS, and a provider-neutral manifest (ekklesia-package-train.json: version, revision, and each package's file, size, SHA-256, npm integrity). - Packing uses a pack-only pnpm beforePacking hook (passed with --config.pnpmfile, never the workspace's install settings). pnpm inserts rewritten workspace: dependencies in async completion order, so two checkouts of one commit packed module-pack's package.json with keys in a different order; the hook restores the source key order (and the inspection enforces it). It also drops devDependencies, which named private packages outside the train. - scripts/test-package-release.sh (pnpm test:package-release, the CI package-release gate) builds the train twice, the second time from a separate worktree, requires byte-identical release directories, then runs the standalone consumer proof on the release: manifest checked, offline install from the tarballs only, lockfile integrity equal to the manifest's, frozen-lockfile reinstall, no Drizzle, one React, contracts only transitive, every @ekklesia package and module-pack at the release version, skipLibCheck: false, UI build, pack, and Core verification. - docs/development/releases.md documents the release order, the train, the artifacts, and the reproducibility findings; registry publication remains unimplemented. - ADR 0053 is Accepted: everything it decides that does not need a registry is now proven, and registry provisioning is on its own Deferred list. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-08 08:02:41 -05:00
ekklesiaandClaude Opus 5.5	85fae9e07e	feat: Module Package v1 carries the module's own license (ADR 0051) ... Every Module Package v1 now declares the module's license and carries its text. packageFormat stays 1; pre-ADR-0051 metadata fails closed. - Contracts: ekklesia.package.json requires an SPDX `license` expression, validated with spdx-expression-parse, with UNLICENSED and non-expressions refused. The `files` list must hold exactly one non-empty root LICENSE. The package contracts move to the @ekklesia/contracts/module-package subpath, so the SPDX parser and its data (spdx-exceptions and spdx-license-ids ship no licence files) never reach a module bundle through the SDK's re-export of the contracts root. - Packager: reads the source package.json's string SPDX declaration and never invents one. It requires an explicit --license-file (a non-empty regular file, no symlink, no parent search, no download) and copies its bytes unchanged to LICENSE, listed with size and SHA-256 and covered by the archive digest. THIRD_PARTY_NOTICES stays third-party only, and bundled Ekklesia code stays in `incorporates`. - Verifier: rejects missing or invalid license metadata, and a missing, unlisted, duplicated, or corrupted LICENSE, through the schema and the existing strict inventory. - Template: packaged as AGPL-3.0-or-later with the repository's canonical LICENSE, passed explicitly (Dockerfile, smoke, docs). Packing it twice gives byte-identical archives. Started by Codex; reviewed and completed in Claude Code (empty license-text refusal, schema non-empty LICENSE, symlink and empty-file tests). ADR 0051 is Accepted. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-07 18:51:03 -05:00
ekklesiaandClaude Opus 5.5	27d85e0162	fix(smoke): hand the verifier the package's archive and digest as one readable release pair ... The split deploy smoke failed in CI with "The SHA-256 file could not be read." BuildKit's local exporter creates its output directory (`--output type=local,dest=$work/module-package`) with mode 0700, owned by whoever ran the build. That directory was mounted into the verifier, which runs as the image's unprivileged `node` user (uid 1000). In CI the build runs as root, so the directory was root:root 0700 and the verifier could read neither file; the digest file is simply read first. On a workstation the build runs as uid 1000, the same uid as `node`, so it was readable by coincidence. scripts/lib/module-package-release.sh turns one build's output into the release pair: it requires exactly one archive and its .sha256 and nothing else, checks that the digest line names that archive, then creates a fresh directory that must not exist yet (0755) and copies both files into it (0644). The smoke stages and mounts only that directory; BuildKit's output directory is never mounted. Archive-digest verification stays mandatory: the verifier still reads the .sha256 file. Nothing in the verifier changed. Tests: the helper is driven in bash against a 0700 build output, and refuses a missing, malformed, or mismatched digest, stray output, a second archive, and an existing release directory, creating nothing. A static test pins the smoke to the release pair and the verifier's single mount with --sha256-file. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-07 13:10:58 -05:00
ekklesiaandClaude Opus 5.5	35ef31df9f	feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049) ... A separately versioned module is now distributed as one deterministic .tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by Core before anything is installed. This slice implements the portable `any` target, with the module template as the end-to-end proof. ADR 0049 is accepted. @ekklesia/contracts: the Module Package v1 contract. The strict ekklesia.package.json schema (packageFormat 1, identity, target, runtime, provenance, incorporated Ekklesia packages, strict transitional facts, an exact byte-wise-sorted file list with size and SHA-256), target names, and the generated runtime package.json. @ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir> --target any --out <dir> --source-revision <commit>`. - Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle included; only Node built-ins remain imports. Any warning, unresolved import, external dependency, or native or platform-specific package fails. - Copies browser artifacts as built, after proving them current, and the migrations; generates package.json and ekklesia.package.json; never ships src, tests, declarations, source maps, or workspace ranges. - THIRD_PARTY_NOTICES: every third-party package traced from the bundles' own inputs, credited once with its declared licence and every licence, copying, copyright, and notice file it ships. A missing licence fails; a missing text fails unless its standard text is complete without copyright lines (Apache-2.0, vendored, needed by drizzle-orm). - Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime 0, 0755/0644, no links. Byte-identical across runs and machines. - Writes only the archive and its .sha256; never deploys, links, or installs, so the workspace's install state is untouched. Core: verifyModulePackage and `verify-module-package.js <archive> [--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe or escaping paths, and a second top-level directory; checks metadata, every file, identity, target, Node.js range, SDK, and the transitional Drizzle agreement; runs the platform's module and migration discovery; loads the backend once in a forked process with an empty environment; only then extracts, with normalized modes, into a new directory. Errors are sanitized. @ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs and whether the built file is current. The nested stylesheet build now works from the module directory, so artifacts no longer depend on the caller's working directory. The split deploy smoke builds the package with the canonical packager (scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm deploy packaging), verifies it with the same generic image, installs the verified tree, removes it, and reinstalls the exact same archive. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-07 12:44:54 -05:00
ekklesiaandClaude Opus 5.5	e36b70d8d4	feat: one generic image, with installed modules in installation-owned storage (ADR 0048) ... The production image now contains zero feature modules and is the same for every installation. Installed modules live in module storage that one Core resolver selects from EKKLESIA_MODULES_DIR: the checkout's modules/ when unset, an absolute path that must exist when set (/var/lib/ekklesia/modules in the image). A relative, blank, missing, or unreadable configured directory fails closed with a sanitized error that never contains the value. - Host configuration reads EKKLESIA_MODULES_DIR. The API, migration plan, apply and validate, module validation, module enablement, and Doctor all resolve it the same way; the test-only API override still takes precedence. - Doctor receives Core's migrations directory explicitly instead of deriving it from the modules directory's parent. - The migration runner retains and reports the applied history of a module that is not installed, and plans and applies Core's and present modules' migrations around it. Core's history and every installed module's history stay strict: a missing or changed applied migration, or a downgrade, still fails. No down migrations, no data deletion. - deploy/Dockerfile copies no module and installs nothing for the module template; .dockerignore excludes modules/ and the build refuses a context that contains it; the runtime has an empty root-owned /var/lib/ekklesia/modules and sets EKKLESIA_MODULES_DIR. - deploy/compose.yaml bind-mounts EKKLESIA_MODULES_HOST_DIR read-only into api and migrate, with create_host_path disabled. - scripts/check-image.sh proves the image holds no modules and that a populated modules/ directory leaves its runtime file system identical; CI runs it. - The split deploy smoke now uses the same generic image: it boots with no modules, installs a self-contained test module packaged from the template (test-only), migrates and restarts, removes it (Core runs, planning reports the retained history, data survives), and restores it with its data. Docs: deploy README (module storage, install/update/remove/restore, the runtime modules target), .env.example, API configuration, module development guide, testing, workflow, security, and the architecture overview. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-07 08:38:14 -05:00
ekklesiaandClaude Opus 5.5	62951975da	fix(ci): nothing a CI job starts outlives it, even when the runner kills the job ... A migration-smoke stack (PostgreSQL and SeaweedFS) ran on the shared runner for two days. Every CI script removes its stack in an EXIT trap, but a trap never runs when the runner kills the job container: a cancelled job, or the runner's own time limit (Gitea 1.24 ignores timeout-minutes). Reproduced: SIGKILL of the infrastructure smoke mid-run leaves exactly that stack running. Cleanup now has three layers: - the script's own trap, unchanged; - a final `if: always()` step in every job that starts stacks (infrastructure-smoke, migration-smoke, image-build, deploy-smoke) runs `scripts/ci-sweep.sh --job`, which removes everything that job started at any age; - a first step in each of those jobs runs `scripts/ci-sweep.sh --stale`, which removes CI stacks older than three hours: what a killed job left before it could clean up. How stacks are found is defined once, in scripts/lib/ci-project.sh: - every CI Compose project is named `ekklesia-ci-<run>-<attempt>-<job key>-<kind>-<unique>`, and Compose labels its containers, networks and volumes with that name; - the deploy smoke labels everything it starts outside Compose: browser, kiosk and driver containers, the kiosk volume, built images; - images also record when CI built them. An image's own creation time can be its base's (BuildKit, metadata-only builds), and the sweep removed a brand-new image as "old" until it stopped trusting that. The job key is per matrix leg, so one topology's cleanup cannot remove the other's running stack. The sweep never touches anything outside the ekklesia-ci- prefix, disconnects rather than removes a job container attached to a stack's network, and refuses --job outside CI. Network creation times are read as RFC 3339; Docker's default form was unparseable. scripts/test-ci-sweep.sh proves all of that against real Docker, and CI runs it before relying on it. security.md and testing.md document the rule, and that it complements, not replaces, a runner-side reaper. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-06 02:01:01 +00:00
ekklesiaandClaude Opus 5.5	dcbd7613ee	test(deploy): load the same image twice over HTTP/2, and pin storage CORS ... Two checks on storage delivery through the proxy, in both topologies. The same signed image, twice, over a real HTTP/2 connection (node:http2; Node's fetch speaks HTTP/1.1, where the failure does not exist). The second load is conditional, as a browser's revalidation is, and must be a clean 304. Today Caddy reaches SeaweedFS over HTTP/2, SeaweedFS's 304 declares Content-Length: 327 and sends no body, and Caddy resets the stream. A browser reports that as ERR_HTTP2_PROTOCOL_ERROR on every reload of a cached files. image. CORS on delivery is the deployment's own: - exactly the API origin (and the Dashboard origin when split) is allowed, with Vary: Origin and nothing else; - any other origin gets no CORS header; - a preflight is answered by the proxy (204, GET and HEAD only) and never reaches the object store. Today SeaweedFS reflects every origin with PUT, POST and DELETE, and answers the preflight itself. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-06 01:19:17 +00:00
ekklesiaandClaude Opus 5.5	f50e8f4201	test(deploy): word the long-watch checks for any watch length ... The recycle step's checks spoke of "the realtime recycle" even when run with a watch shorter than the five-minute connection age, as module CI does. They now say what they check: live for the whole watch (plus "across a realtime recycle" when the watch is long enough), and never remounted. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-05 19:09:51 -05:00
ekklesiaandClaude Opus 5.5	ff8a3e8097	test(deploy): add opt-in display-device scenarios to the split smoke ... Reproduction harness for what the first production Signage deployment found. With EKKLESIA_SMOKE_DISPLAY_DEVICE=1, the split smoke starts a second headless Chromium (the "kiosk") whose profile directory is a Docker volume. It pairs that kiosk through the real flow, closes it cleanly, restarts it on the same profile, and then runs injected-failure scenarios: two pages on one credential, a connectivity blip, one refused viewer-session request, one refused realtime connection, a cross-origin image-caching probe, a full realtime recycle, and revocation. Every DevTools command and page-side request has its own deadline, and each step has a budget. A step that overruns prints the exact call it was waiting for. The proxy's "aborting with incomplete response" lines are listed afterwards by UTC time and path only. Off by default: several scenarios expect Signage and fail on today's v0.1.1, by design. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-05 18:25:12 -05:00
ekklesiaandClaude Opus 5.5	f63f80f2f0	feat: support split Dashboard and API origins (ADR 0045) ... An installation can now serve the Dashboard and the API on sibling hostnames (dash./api.) beside the default same-origin topology, from one image and with configuration alone. - The Dashboard learns its API origin at run time: the built page carries one empty <meta name="ekklesia-api-origin">, which the API fills with its identity base URL's origin at startup, before computing the page's ETag. The shell accepts only a canonical http(s) origin and otherwise fails closed without making a request. The build-time EKKLESIA_API_BASE_URL is gone from the bundle; the development server injects the same element. - The display host stays on the API's origin, so its device cookies, requests, and offline cache are unchanged. - The API trusts its own origin (trusted origins plus the identity base URL), as Better Auth already does, so the display page's mutations pass the origin check without operators listing it. - Doctor validates EKKLESIA_DASHBOARD_ORIGIN exactly as the API's startup does (it previously skipped it), reports the topology without printing hostnames, and warns when split hostnames are not siblings. - The module UI contract states the CORS header limit (safelisted headers plus Content-Type, safelisted response headers only). - Reference deployment: Caddyfile.split.example (Dashboard hostname serves / and its script; API hostname serves /api/*, /display/*, and the display script; each refuses the other's paths) and EKKLESIA_DASHBOARD_HOSTNAME in the proxy overlay and env examples. - The deploy smoke test runs either topology (CI runs both). The split run installs the module template as a test-only module, checks per-hostname routing and exact CORS, then drives a digest-pinned headless Chromium over the DevTools protocol, with no added dependency: sign-in on the Dashboard hostname, the session cookie host-only on the API hostname, the module UI imported cross-origin and its credentialed request succeeding, the display and its service worker on the API hostname, and sign-out. ADR 0045 is marked Accepted. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-03 14:47:36 -05:00
ekklesiaandClaude Opus 5.5	278e66049f	fix(ci): isolate concurrent runs on a shared Docker daemon and explain readiness failures ... PR #51's migration-smoke failed 0.7 s after the build: the suite pinned PostgreSQL to an ephemeral port but inherited S3_PORT=8333 from .env.example, so concurrent runs on one runner (Gitea ignores workflow concurrency) contended for one host port. Compose's own error was discarded, so the failure said only that the services "did not become ready". Isolation, from an audit of every script CI runs: - test-migrations-integration.sh: S3_PORT=0, like POSTGRES_PORT=0. - test-migrations-integration.sh and smoke-infrastructure.sh: project names gain a random suffix; run id plus PID alone is not unique when job containers share low PIDs. - smoke-deploy.sh: the edge /24 is chosen from those no network on the daemon uses, and the first `up` chooses again when a concurrent run wins a race for the same subnet (Docker's address-pool overlap), up to five times. - CI image-build: the image tag includes the run id and attempt, so a pull request and a push of the same commit cannot remove each other's image. - Already isolated: every other published port (ephemeral), the TLS suite's container and network (random), smoke staging directories and image tags (random). Diagnostics: scripts/lib/compose-diagnostics.sh prints the failed `up` command's output, `docker compose ps`, and the last 40 log lines of each service, all redacted (supplied literals, credential-looking NAME=value pairs, URL user information, Authorization headers) with terminal control sequences stripped. All three scripts use it at every readiness check; the infrastructure smoke's unredacted full-log dump is replaced by it. tests/ci-scripts.test.ts pins ephemeral ports, unique project names, the subnet choice, the CI image tag, the diagnostics, and the redaction itself (run through bash). Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-03 12:15:01 -05:00
ekklesiaandClaude Opus 5.5	167b4c4f14	feat(deploy): add the reference deployment Compose project ... Implement ADR 0044's reference deployment as a base Compose file plus overlays, so an installation that already runs a proxy can use the base alone. - compose.yaml: api, postgres, object-storage, and the operator-only migrate (profile ops, plan by default). No published port. api and migrate run init, read-only, tmpfs /tmp, cap_drop ALL, no-new-privileges. postgres is only on the internal data network. - TLS everywhere from one installation-private CA: PostgreSQL TLS 1.3 only, with pg_hba rejecting every non-TLS connection, and the API on verify-full via EKKLESIA_DATABASE_CA (PEM text). SeaweedFS serves S3 over HTTPS (ADR 0030 requires https in production), trusted by the API through NODE_EXTRA_CA_CERTS. No Core change. - compose.proxy.yaml + Caddyfile.example: Caddy pinned by digest, ACME HTTP-01 (DNS-01 documented), routes /, /api/*, /display/*, /assets/* to api and refuses everything else; the storage hostname preserves Host. The API trusts exactly the proxy's fixed /32 on an edge network whose dynamic range excludes the fixed addresses; Caddy overwrites X-Forwarded-For and strips caller-supplied forwarding headers. - compose.tunnel.yaml: cloudflared pinned by digest, token from its own env file; Caddy honors CF-Connecting-IP only from its fixed address. - Four per-service root-owned env files, examples with no values; scripts/init-tls.sh (idempotent, never overwrites a key) and scripts/gen-secrets.sh (prints only). - CI: pnpm deploy:config in ci:verify for all three combinations, and a deploy-smoke job (scripts/smoke-deploy.sh) proving readiness, the Dashboard, signed storage delivery, the client-address chain, initial owner sign-in/out, verify-full database TLS, and Doctor through the proxy, against disposable services. tests/deploy.test.ts pins the structural guarantees. - ADR 0044 (still Proposed) amended to match: API-to-storage HTTPS, DNS-01 needs a custom Caddy build and tunnel-only needs DNS-01, the tunnel's origin TLS settings live in the Cloudflare dashboard, port 80 serves HTTP-01, and secrets are one env file per service. Co-Authored-By: Claude Opus 5.5 <[email protected]>	2026-10-01 20:59:38 -05:00
ekklesiaandClaude Opus 5	27088fc054	fix: run the TLS integration harness under a containerized job ... The `migration-smoke` job fails on the Gitea runner at `docker port ekklesia-database-tls-<pid> 5432` with `no public port '5432' published`; the later invalid-URL and `ECONNREFUSED 127.0.0.1:5432` failures all cascade from it. Gitea executes the job inside a container and drives sibling containers through the runner's Docker daemon. The harness held two host-only assumptions, both reproduced in a nested job container and fixed here. Bind-mount source paths are resolved by the daemon against the runner host's filesystem. The certificate directory is created in the job container's `/tmp`, which does not exist there, so the daemon silently creates an empty directory, the entrypoint's `cp` fails, the container exits, and `docker port` reports no published port for an exited container. The material is now transferred with `docker cp` into the created-but-not-yet-started container, which crosses the daemon boundary correctly in both topologies, and only the server leaf and its key are transferred — both CA private keys stay in the harness directory. A published port lands on the runner host's loopback, a different network namespace from the job container's; measured, a container the daemon published at `127.0.0.1:32907` is `ECONNREFUSED` at that address from inside the job container. Under a containerized job the harness now publishes nothing and joins both containers to one isolated user-defined network, reaching PostgreSQL by container DNS on 5432. Direct-host runs keep a random port pinned to loopback via Docker's `127.0.0.1::5432` form. `compose_job_network_job_container` exports the topology decision the integration script already made, so there is still exactly one detector. Every TLS assertion is preserved, and the certificate identity is generated from the host actually connected to — `localhost` directly, the container's DNS name otherwise. The mismatch and chain-only controls use the TLS container's address on the isolated network rather than a hardcoded one, and the certificate carries no IP SAN, so verify-full still fails with `ERR_TLS_CERT_ALTNAME_INVALID`. Correcting one measurement in passing: chain-only verification does not skip the identity check. For an IP host `pg` omits `servername` and hands `tls.connect` a bare socket with no host, and `net.Socket._host` is null for an IP literal, so Node checks the fixed fallback name `localhost` — never the address dialled. A certificate whose only SAN is `DNS:example.invalid`, reached chain-only over `127.0.0.1`, is rejected with `Host: localhost. is not in the cert's altnames`. The direct-host control therefore only ever passed because its SAN was `DNS:localhost`. The SAN set now always includes that fallback name so the control proves the same gap in both topologies, and the stale claim is corrected in the transport doc comment. No production behavior changes. Teardown also removes the container's anonymous volume, the isolated network, and the job container's attachment, after success and failure alike. Co-Authored-By: Claude Opus 5 <[email protected]>	2026-07-29 21:42:26 -05:00
ekklesia	831a48187c	test: add PostgreSQL migration integration coverage	2026-07-21 09:37:15 -05:00