be5ed7853d3268164db71307561e5c809c02ce38
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ekklesia
|
fc6c8045c4 | feat: add native module realtime transport | ||
|
ekklesiaandClaude Opus 5
|
7ff063b87a |
feat: deliver module objects directly to browsers through opaque read URLs
ADR 0030 left exactly one way to get an image to a browser: read the
object into the module's process and write it back out of a module HTTP
route. Every byte of every asset travelled storage -> Ekklesia -> browser.
For the slideshow this platform exists to replace that means Ekklesia in
the path of every image on every display, permanently, for bytes that
never change — and it would make Ekklesia responsible for re-implementing
byte ranges, without which video seeking cannot work at all.
A module can now ask a logical bucket for a temporary browser URL:
const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");
The contract names intent, not mechanism. There is no signing, expiry
format, endpoint, or bucket in it — a host with no S3 anywhere could
satisfy the same method, which is what keeps a future single-module
standalone host able to implement it. The URL is opaque to module code and
a bearer capability to whoever holds it, so a module authorizes the
request in its own route before asking: storage bounds reach, never
identity. Lifetimes run one minute to six hours, fifteen by default, and
out-of-range values are refused rather than clamped, so "expires forever"
is unreachable. Logical keys stay canonical state; URLs are generated on
demand and never persisted or logged.
The provider endpoint and the browser delivery origin are separated
because measurement showed they cannot be collapsed: an S3-compatible
signature covers the request Host, so a URL signed for one origin and
rewritten to another is rejected. The browser-facing origin is therefore a
signing input (S3_DELIVERY_ENDPOINT), defaulting to S3_ENDPOINT — so the
standard deployment configures nothing and delivery just works. The scheme
is not signed, so a reverse proxy may terminate TLS provided it passes the
original Host through; that is pinned by a test rather than described and
hoped for.
Read URLs are stable within a window of one third of their lifetime.
Signing is a pure function of the clock, so naive re-signing yields a new
URL per second — a guaranteed browser cache miss, and the reference client
keys its <img> by URL, so an unchanged image would be torn down and
re-downloaded continuously. Quantizing the signing timestamp makes
repeated calls byte-identical while leaving at least two-thirds of the
lifetime; Cache-Control is set at signing time to the guaranteed
remainder, because the pinned SeaweedFS image sends none of its own.
Verified against live SeaweedFS: byte ranges answered directly with 206
and a correct Content-Range, so no range proxy is needed in Core; signing
is local (200 URLs in 66ms), so no batch API is needed; unsigned requests
for the same object are refused, so delivery never depends on a public
bucket; tampered signatures, swapped keys, and swapped buckets all
rejected. Signed write and delete URLs are deliberately not built.
Co-Authored-By: Claude Opus 5 <[email protected]>
|
||
|
ekklesiaandClaude Opus 5
|
b9ca0d560d |
feat: add host-provided module object storage
`manifest.storage.buckets` has existed since manifest version 1 and meant nothing — discovery validated it and no code read it. This turns it into a real capability, which is what stands between Announcements and the module system: that product currently fetches a hardcoded LAN S3 endpoint, parses ListObjectsV2 XML by hand, and hands the browser direct object URLs. - A module declares a logical bucket and receives a scoped facade at `context.storage` — list, head, get, put, delete. The contract names no provider: no endpoint, region, credential, or physical bucket is reachable from it, which is what lets the same module run under a future standalone host over its own or a shared S3-compatible service. - Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the deployment target (ADR 0004), never an API name. Probing the pinned image found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would have failed every second boot. A live test pins it. - Logical → physical mapping is deterministic and collision-safe: an installation prefix, the module and bucket ids joined by `--` (which cannot occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide), and a digest over the full untruncated identity. Two modules declaring `assets` always get separate namespaces; neither can see the physical name. - Declaring a bucket is now a requirement: with no provider configured the module is `blocked · storage_unavailable` through ADR 0028's existing resolver, propagating to required dependents. An installation with no object storage stays a complete deployment. - Binding availability is not provider health. Eligibility asks whether storage is configured, never whether it answered; bucket creation is lazy on first use so a storage outage cannot stop the platform booting; a running module whose store blinks stays running and retries. Nothing deletes a bucket. - Credentials are deployment-supplied, never persisted, never routed through module configuration, and never visible to modules, module UI, diagnostics, or API responses. Provider errors translate to a closed code set carrying no endpoint, bucket, request id, or XML. - Retires ADR 0029's transitional `bucket` configuration key. No manifest version change. The template declares no bucket, so it still runs without S3. See ADR 0030. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
baa689b2a7 |
feat: add the module configuration and secrets contract
Modules had no way to be told anything. Announcements — the first real product Ekklesia will absorb — has zero environment variables and compiles a LAN IP address into its source, so this is the contract standing between it and the module system. - A module declares what it needs in its manifest, using six bounded types (string, integer, boolean, enum, url, secret) with no nesting and no regular expressions. Static declaration is what lets the platform validate configuration *before* deciding whether a module may start: an enabled module missing a required value is reported `blocked · configuration_missing · <key>` rather than taking startup down from inside its own `start` hook, and Doctor reports the same while still importing no module code. That forces `manifestVersion: 5`, a hard cutover. - Ordinary values persist in one Core table (migration 0005), one row per (module_id, key), stored as text until validated against the declaring module's current manifest. Nothing is coerced; an invalid stored value blocks the module rather than falling back to the default. A key the manifest no longer declares is retained and ignored. - Secrets are never persisted. There is no encryption at rest in this repository, so a `secret` key is deployment-supplied and mapped in by the host under a name the module never sees. A secret declaration has no `default` field, the write API refuses secret material, and the HTTP view has no `value` field for one — absent by construction, not filtered. - Modules receive a frozen `context.config` on the registration, runtime, and HTTP handler contexts: the same object identity, per module, beside `services` rather than inside it (services is deliberately one shared instance). No environment access, no database reference, no way to reach another module's values. - Configuration decides eligibility through ADR 0028's existing resolver: three new blocked reasons, each with a distinct operator fix, propagating to required dependents through the same mechanism as a disabled module. - Changes take effect at the next restart; both resources report restartRequired. New restricted permission `core.modules.configuration.manage`, separate from lifecycle, at the sibling prefix `/api/v1/module-configuration`. - Migration participation stays independent of configuration validity. Endpoints and credentials for shared infrastructure stay platform concerns; a module declares which bucket it needs, not an S3 endpoint. See ADR 0029. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
7729c99e24 |
feat: add the minimal operational module lifecycle
Ekklesia had no notion of a module being on or off: the accepted module set was exactly "whatever discovery found on disk", and `ModuleLifecycleStage` named nine states nothing produced or consumed. This replaces that vocabulary with two axes that real code produces — persistent, administrator-owned configuration (`enabled`/`disabled`) and per-boot runtime observation (`running`/`disabled`/`blocked`) — and wires them through startup. - One Core table (`ekklesia_core_module_state`, migration 0004) holding only `module_id` and `enabled`. A module with no record is enabled by default, so deploying a module is still sufficient to run it and discovery never needs an initialization write. A record for an undeployed module is retained and ignored, so redeploying restores the decision. - `resolveModuleEligibility` is pure: a disabled module transitively blocks every module that *requires* it (module and capability edges alike) with a structured reason naming the provider; optional dependencies never block; and a missing, incompatible, or cyclic dependency stays discovery's concern. - `startApi` starts only the eligible set and builds the UI publication registry from that same list, so routes, artifacts, browser inventory, and Dashboard navigation all follow one authoritative set. A disabled module is never imported, so there is no per-request enablement check anywhere. - Startup stays all-or-nothing, deliberately; there is consequently no `failed` runtime state. `pnpm modules:enable`/`modules:disable` is the out-of-band recovery path for a module that breaks startup. - Changes take effect at the next start, and the read model reports configuration, runtime, and `restartRequired` separately rather than claiming a module stopped when it has not. - Migration participation and permission composition still span the deployed and discovered inventories respectively, unchanged. - New restricted platform permission `core.modules.lifecycle.manage` gates both reading and changing enablement, at a sibling prefix (`/api/v1/module-lifecycle`) that cannot collide with the module route namespace. - `pnpm doctor` gains a `modules.lifecycle` check so "why is this module unavailable?" is answerable without reading logs. See ADR 0028. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
2983789779 |
fix: harden migration command execution
Two problems in the migration commands: no timeouts, and SQL split on literal bytes. `migrations:plan` and `migrations:apply` constructed their connection with neither a connection deadline nor an operation deadline — a real server reported `statement_timeout = 0` on the migration connection. Both now default to 10 seconds for the connection deadline, the client query timeout, and PostgreSQL's own statement_timeout, applied at the one shared construction boundary so a future call site cannot silently omit them. History reads, advisory-lock queries, transaction control, migration SQL, and history writes are all covered. The existing lock-wait deadline, the Pass 2 TLS policy, and Doctor's and the bootstrap CLI's own timeouts are untouched, and no environment variable or flag was added. Explicit client fields do not win under the pinned driver. [email protected].0 builds parameters as `Object.assign({}, config, parse(config.connectionString))`, so the URL overrides passed options — verified directly, where an explicit statement_timeout of 10000 alongside `?statement_timeout=0` produced a server-reported 0, while a clean URL produced 10s. A DATABASE_URL carrying connect_timeout, query_timeout, statement_timeout, or an `options` value setting statement_timeout is therefore refused with the existing fixed configuration error, before any client is constructed and without echoing the URL, parameter, or value. Unrelated parameters are left alone. The runner split each migration on the literal `--> statement-breakpoint` bytes. Those bytes inside a string literal, a dollar-quoted function body, a quoted identifier, or a comment tore the migration into fragments that were no longer valid SQL — each case verified to produce two fragments where one was required. Each migration's exact SQL is now passed to the client once, with no values and no prepared-statement name, inside the existing transaction, and its history row inserted only after that batch succeeds. One transaction still spans all pending migrations; advisory locking, checksum validation, approval preflight, ordering, rollback, cleanup, and sanitized error identity are unchanged. The port method is renamed executeStatement to executeMigrationSql so the contract says what it receives. PostgreSQL parses the batch because it owns the only correct parser. A homegrown splitter would have to track single quotes, dollar quoting with arbitrary tags, quoted identifiers, line comments, nested block comments, and escape syntax to answer a question the server already answers, and a parser dependency would add supply-chain surface for the same reason. Drizzle's marker lines are ordinary `--` comments when the batch is sent whole. No custom SQL parser exists here. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
490b85b07c |
fix: bound module lifecycle shutdown
Nothing bounded shutdown, and a module that failed to start was never cleaned up. `startModules` appended to its rollback set only after `start()` resolved, so a hook that acquired a connection, registered a listener, or recorded partial initialization and then threw had its own `stop()` skipped, while every previously completed module was rolled back correctly. A module now joins the rollback set the moment its start hook is invoked: its own stop hook runs first, then earlier modules in reverse order. The original start issue stays first in the reported issues, and a failure stopping the failing module is appended without preventing the rest of the rollback. The consequence for module authors is explicit and documented — `stop()` may run after a partially completed `start()` and must be safe and idempotent about resources it may or may not hold. Neither stage of shutdown had a deadline. `shutdown()` awaited `server.close()` with no bound, so one open connection could hold it forever, and each stop or rollback hook was awaited without a bound, so a hook that never settled kept the pool open indefinitely. Both now have fixed, non-configurable defaults: 10 seconds per module hook and 30 seconds for the graceful HTTP drain. A hook that reaches its deadline records MODULE_STOP_TIMEOUT or MODULE_ROLLBACK_TIMEOUT and the sequence continues with the next module, so the whole stop sequence is finite in the started module count rather than one global ten-second bound. Timers that lose the race are cleared, the losing promise gets a permanent rejection handler so a late throw cannot become an unhandled rejection, and a timed-out hook is never awaited again. At the drain deadline the platform closes idle and then active connections through Node's own public closeIdleConnections/closeAllConnections, after server.close(), touching no private field, and stops waiting for the close callback. It then runs the bounded module stop sequence and closes the pool, so a drain timeout or a stop failure never skips later cleanup. Both are reported together as an ApiShutdownError carrying only fixed booleans and the already sanitized module error; a forced drain is a shutdown failure, not a success. Stated honestly rather than overstated: a deadline bounds how long the platform waits, not how long module code runs. An in-process promise cannot be canceled, and closing a socket does not prove a handler promise ended. What is guaranteed is that shutdown stops waiting and the pool closes afterwards, so late database work fails. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
532ce3803b |
fix: preserve authorization security history
Two security records did not survive user deletion, because both hung off a cascading foreign key to the live Better Auth user row. Deleting a user with authorization history deleted that history. A subject with a create and a revoke audit row had zero rows afterwards: the record of what they had been granted, by whom, and when it was revoked disappeared at exactly the moment it mattered most. `ekklesia_core_authorization_audit.subject_user_id` is now a retained, non-null historical snapshot with no foreign key — a plain identifier that may name a user row which no longer exists. No name, email, or other profile data is copied into authorization tables. The Drizzle relation to the user table is dropped too, because modelling a retained snapshot as a live relation misdescribes the data. Live rows still cascade, as intended: deleting a user removes their grants and platform-owner row, so nothing continues authorizing a principal that no longer exists. Platform-owner bootstrap's single use was equally fragile. "Already consumed" meant "a row exists in ekklesia_core_platform_owner", and that table cascades too — so deleting the bootstrapped owner's user row removed the owner row, its audit evidence, and with them the only thing preventing the operation that establishes a platform's first administrator from being run again. A dedicated `ekklesia_core_platform_owner_bootstrap` singleton now carries that authority. Its id is pinned by a check constraint to the literal 'singleton', so the primary key makes a second row structurally impossible rather than merely unwritten by convention; it has no user foreign key, and Core never deletes from it. Bootstrap claims that record inside the same transaction as the owner row and its audit entry, so consumption is tied to success: a missing target user or any other failure rolls the sentinel back and leaves the one legitimate first use available. The existing transaction-scoped advisory lock still serializes concurrent first attempts to exactly one success. The operational consequence is deliberate and permanent: deleting the last platform owner does not reactivate bootstrap, and neither does revoking one. Core migration 0002_authorization_integrity creates the singleton table, drops the cascading audit-subject constraint, and backfills consumed state for any installation that already holds a platform-owner row — active or revoked — so an existing deployment cannot gain a fresh first use by upgrading. It records no subject, because a backfill cannot know which of several historical owner rows the original use targeted and guessing would fabricate history. A never-owned installation is untouched and keeps its one first use. One upgrade limitation is irrecoverable and is documented rather than papered over: if an operator manually deleted the bootstrapped owner's user row before this migration, the old cascade already erased the owner row and its audit evidence, so nothing remains to detect. At this checkpoint no supported Better Auth or API user-deletion path can produce that state; it requires direct database access. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
6b6b77c105 |
fix: enforce production transport policy
Every PostgreSQL connection this repository opened was plaintext, in every environment: no connection site set `ssl` at all, so credentials, session tokens, and row data crossed the network in the clear and nothing in configuration, startup, or `pnpm doctor` said so. The public identity URL boundary was also looser than it appeared. `core/src/database/transport.ts` is now the one typed policy. It has exactly two modes: `verify-full` (TLS required, certificate chain verified, and server identity verified against the configured host — the production default) and `disable` (the development/test default, refused outright in production). There is deliberately no encryption-only `require` mode, no chain-only `verify-ca`, and no `rejectUnauthorized: false` escape hatch anywhere in the configuration surface; every loose or aliased spelling is rejected in both environments, so a typo never quietly becomes a weaker mode. Production has no plaintext exception at all, not even for a loopback database — a connection string cannot prove the database is process-local. Verification is certificate *and* hostname. The policy supplies an explicit `checkServerIdentity` pinned to the host parsed from DATABASE_URL, because the pinned driver sets TLS `servername` only for non-IP hosts and Node performs no identity check when it is unset — so `rejectUnauthorized: true` alone accepts a certificate from any host the trusted CA ever signed when connecting by IP. Both halves are proven against a real TLS-enabled PostgreSQL, including that the same connection without the pinned callback succeeds. Every CA, certificate, and key is generated at test runtime and deleted afterwards. A DATABASE_URL may no longer carry its own TLS parameters. The pinned driver merges parsed connection-string values over explicitly passed options, so a `?sslmode=disable` in the URL would silently replace the policy's verified `ssl` object; that now fails with a stable code, reported by both `pnpm doctor` and the migration CLIs rather than as "failed unexpectedly". One shared environment reader keeps `process.env` out of Core's reusable constructors, and every executable that opens a connection resolves the policy identically: the API (validated before any pool, Better Auth instance, module runtime, or listener exists), both migration commands, the bootstrap CLI, and Doctor's own connection. Doctor reports it as a blocking failure, never a warning, naming only the effective mode. Separately, the identity base URL and every trusted origin must now be exactly one canonical origin — the WHATWG origin serialization must equal the supplied string — so a path, trailing slash, credentials, explicit default port, or non-canonical case is rejected rather than normalized into something that no longer matches what a browser sends. Production's loopback rejection now classifies the address instead of comparing against three literals, covering `.localhost`, all of 127.0.0.0/8, 0.0.0.0, `::`, expanded and IPv4-mapped forms. This is a configuration-shape guarantee only: it does not resolve DNS or prove reachability, and TLS protects the connection, not the exposure. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesiaandClaude Opus 5
|
7a0f624fc3 |
fix: distinguish identity session failures
Principal resolution could not tell a broken session store from a genuinely anonymous request, and did unnecessary database work for callers that never presented a session at all. Both are corrected here. Any cookie previously triggered the ambiguity check, so `Cookie: x=y` on an otherwise anonymous request caused an identity database round trip an unauthenticated caller could drive with a header they chose themselves. `hasConfiguredSessionCookie` now parses the Cookie header per RFC 6265 (split on `;`, name is the text before the first `=`, trimmed) and compares the whole name exactly — never `includes`, a substring test, or an unanchored regular expression — so `better-auth.session_token_x`, `x_better-auth.session_token`, a differently-cased spelling, and a percent-encoded lookalike are all correctly not the session cookie. A duplicated name still reports true: ambiguity fails safe towards checking, and reporting true never grants access on its own. The cookie name is not a second hard-coded copy. One `betterAuthOptions` object now feeds both `betterAuth()` and Better Auth's own supported `getCookies` helper, so the detected name — including the `__Secure-` prefix an https baseURL implies — is derived from the exact configuration Better Auth itself uses, and drift is structurally impossible rather than merely tested. The readiness check itself proved almost nothing: `select 1` still succeeded against a dropped session table, a drifted or removed column, or a revoked SELECT privilege, so Better Auth's ambiguous null was reported as a genuinely anonymous request. `probeSessionStore` now plans a real read of the session relation and the exact columns the adapter depends on, under `where false`: PostgreSQL resolves the relation, every column, and the caller's privilege at plan time while matching zero rows, so no session row, token, or user id can be read, logged, or enumerated by the check. It uses the injected pool, never a second connection, and stays bounded by the existing timeout. A missing relation, dropped column, revoked privilege, unreachable connection, and query timeout now all produce the same sanitized 503 with no relation name, column name, SQL, driver text, host, or stack — proven against real PostgreSQL, since that is the only place a broken store looks exactly like an anonymous request. Co-Authored-By: Claude Opus 5 <[email protected]> |
||
|
ekklesia
|
c38b572778 |
docs: document module database access
Add ADR 0016 recording the accepted module database access design (ModuleDatabase, the frozen ModuleRuntimeServices boundary, one pool/one facade/one services-object identity, explicit migrations, and the trust model), with narrow cross-reference corrections to ADR 0006 and ADR 0015 where their original text is now inaccurate. Sweep docs/development/api.md, docs/development/authorization.md, docs/module-development/README.md, docs/development/testing.md, and docs/development/identity.md for the new database boundary and remaining Phase 4C staleness; correct the stale "Phase 4C is next" roadmap paragraph in platform-architecture.md and the matching gap in README's current-status narrative. Extend the module template with a permissioned GET /records route reading through context.services.database, using the canonical createModuleTable helper, with matching template tests. |
||
|
ekklesiaandClaude Sonnet 5
|
1285c90618 |
docs: record module HTTP architecture
Phase 5B, pass 3: adds ADR 0015 recording the durable module HTTP
routing and production runtime lifecycle architecture pass 1 and pass
2 implemented, extending ADR 0006 and ADR 0014 without superseding
either. Adds narrow cross-reference notes to both, and to the ADR
index, pointing at ADR 0015 for the work they deferred, while
preserving their historical text and accepted status.
Corrects stale forward-looking "pass 2"/future-tense language left
over from pass 1 in core/src/modules/runtime.ts and
packages/contracts/src/module-http.ts (comment-only; production
behavior unchanged), and a shutdown-lifecycle description in
docs/development/testing.md that predated the module-runtime-stop
step pass 2 added between draining the HTTP server and closing the
pool.
Full audit against origin/main...HEAD found the architecture,
contract, startup ordering, routing, authorization/origin
enforcement, and PostgreSQL integration coverage from passes 1-2
sound; the one concrete defect found (Vary-header merge producing an
invalid "*, Origin, Cookie" combination and case-sensitive duplicate
tokens) was already fixed with focused regression tests in a separate
commit (
|
||
|
ekklesia
|
d9b73fd81a | fix: harden identity HTTP transport | ||
|
ekklesia
|
91d8f5946f | feat: expose core identity transport | ||
|
ekklesiaandClaude Sonnet 5
|
679a5d225c |
feat: establish core identity foundation
Adds Core-owned identity contracts, a Core-owned Better Auth database schema (ekklesia_core_user/session/account/verification), a first-class Core-owned migration source that coexists with module-owned migrations under the exact existing migration system, and an explicit Better Auth construction boundary. No authentication endpoint or sign-in UI is exposed yet (ADR 0010). Co-Authored-By: Claude Sonnet 5 <[email protected]> |
||
|
ekklesia
|
5104f2e954 | fix: build workspace packages before migration tests | ||
|
ekklesia
|
831a48187c | test: add PostgreSQL migration integration coverage |