docs: document module database access
Add ADR 0016 recording the accepted module database access design (ModuleDatabase, the frozen ModuleRuntimeServices boundary, one pool/one facade/one services-object identity, explicit migrations, and the trust model), with narrow cross-reference corrections to ADR 0006 and ADR 0015 where their original text is now inaccurate. Sweep docs/development/api.md, docs/development/authorization.md, docs/module-development/README.md, docs/development/testing.md, and docs/development/identity.md for the new database boundary and remaining Phase 4C staleness; correct the stale "Phase 4C is next" roadmap paragraph in platform-architecture.md and the matching gap in README's current-status narrative. Extend the module template with a permissioned GET /records route reading through context.services.database, using the canonical createModuleTable helper, with matching template tests.
This commit is contained in:
ekklesia
+75
-27
@@ -215,11 +215,15 @@ abuse-control design.
|
||||
permission registry (Core's own declarations plus every module's) — `prepareStartupModuleInventory`,
|
||||
`apps/api/src/platform-permissions.ts`.
|
||||
2. Construct the one shared PostgreSQL pool.
|
||||
3. Construct the Core identity runtime, authorization store, and evaluator.
|
||||
4. Start the module runtime from that exact same discovered inventory
|
||||
(`startDiscoveredModuleRuntime`) — modules are never rediscovered for this step.
|
||||
5. Create the Hono application, mounting the started module runtime's published routes.
|
||||
6. Bind the HTTP server.
|
||||
3. Construct the module database facade and the frozen `ModuleRuntimeServices` object from that same
|
||||
pool (see [Module database access](#module-database-access)).
|
||||
4. Construct the Core identity runtime, authorization store, and evaluator.
|
||||
5. Start the module runtime from that exact same discovered inventory
|
||||
(`startDiscoveredModuleRuntime`) — modules are never rediscovered for this step — passing it the
|
||||
services object every module's `start`/`stop` hook receives.
|
||||
6. Create the Hono application, mounting the started module runtime's published routes and passing
|
||||
the identical services object to every module HTTP handler.
|
||||
7. Bind the HTTP server.
|
||||
|
||||
Discovery and permission composition happen before any pool, Better Auth instance, module
|
||||
entrypoint import, or HTTP server exists, so a discovery or composition failure (an invalid
|
||||
@@ -257,9 +261,11 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
|
||||
(same permission check, response body stripped); a module cannot separately author `HEAD`.
|
||||
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and
|
||||
body all behave normally, and the full query string remains available) plus a frozen context of
|
||||
exactly `moduleId`, the authorized `EkklesiaPrincipal`, and frozen, Hono-decoded path parameters
|
||||
(an empty frozen object for a route with none) — never a Hono `Context`, a database connection,
|
||||
the authorization store, storage, jobs, configuration, or secrets.
|
||||
exactly `moduleId`, the authorized `EkklesiaPrincipal`, frozen, Hono-decoded path parameters (an
|
||||
empty frozen object for a route with none), and `services` — the same frozen
|
||||
`ModuleRuntimeServices` object the module's own `start`/`stop` hooks received (see
|
||||
[Module database access](#module-database-access)) — never a Hono `Context`, the authorization
|
||||
store, storage, jobs, configuration, or secrets.
|
||||
- **Handler failure.** A handler that throws, rejects, or returns anything other than a `Response`
|
||||
becomes a fixed `500 {"code":"MODULE_HANDLER_FAILED"}` with `Cache-Control: no-store` — never the
|
||||
caught value, a stack, the route's path or permission id, or the module id. A `Response` a
|
||||
@@ -276,6 +282,46 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
|
||||
registry: installing, removing, or editing a module's routes takes effect only on the next API
|
||||
restart. Removing a module makes its former routes return the ordinary `404 {"code":"NOT_FOUND"}`.
|
||||
|
||||
## Module database access
|
||||
|
||||
`context.services.database` gives a module's `start`/`stop` lifecycle hooks and every authorized
|
||||
HTTP handler a typed Drizzle query surface over its own tables — never during `register`, which
|
||||
stays declarative and database-free. See
|
||||
[Module development](module-development/README.md#database-access) for the module-authoring
|
||||
contract; this section covers only how `apps/api` composes and wires it.
|
||||
|
||||
- **`ModuleDatabase`.** A restricted, typed view over `NodePgDatabase`
|
||||
(`packages/contracts/src/module-database.ts`), exposing exactly `select`, `selectDistinct`,
|
||||
`selectDistinctOn`, `insert`, `update`, `delete`, `execute`, and `transaction`, each preserving
|
||||
Drizzle's own table-aware argument and result inference. It exposes no `$client`, no internal
|
||||
session/dialect state, no schema-bound relational `query` builder, and no connection string,
|
||||
pool, or raw `pg` client — not merely hidden by the TypeScript type, but physically absent from
|
||||
the constructed object at runtime (`core/src/modules/database.ts`'s `createModuleDatabaseView`
|
||||
builds a frozen object with each operation explicitly bound, rather than handing out Drizzle's
|
||||
own instance).
|
||||
- **One pool, one facade, one services object.** `apps/api/src/database.ts`'s `createApiDatabase`
|
||||
wraps the one shared PostgreSQL pool in three independent typed Drizzle views — identity,
|
||||
authorization, and now `module` (a `ModuleDatabase`). `startApi` builds one frozen
|
||||
`ModuleRuntimeServices` object (currently just `{ database }`) from that pool and passes the
|
||||
*same* object to every module's `start` and `stop` hook and to every module HTTP handler for the
|
||||
lifetime of one running API — never a fresh object per hook or per request, and never a second
|
||||
pool or facade.
|
||||
- **No database access without authorization.** A handler's context — and therefore
|
||||
`services.database` — is unreachable until `requirePermission` (and, for a state-changing
|
||||
method, `requireTrustedOrigin`) has already succeeded; a denied request, a rejected-Origin
|
||||
request, and an `OPTIONS` preflight never invoke a module handler at all, so they never touch a
|
||||
module's database.
|
||||
- **Errors.** A handler-thrown database error is sanitized through the exact same
|
||||
`MODULE_HANDLER_FAILED` boundary described above — there is no separate, database-specific public
|
||||
error contract. A module that deliberately catches its own database failure may still return any
|
||||
valid `Response` of its own choosing.
|
||||
- **Migrations stay explicit.** Constructing the facade performs no query and no migration.
|
||||
Module schema and migration changes still follow the existing generation, validation, policy, and
|
||||
deployment flow (see [Module development](module-development/README.md#database-access)); nothing
|
||||
in module discovery, registration, start, or an HTTP request infers a schema or creates a table
|
||||
automatically. See [ADR 0016](../architecture/decisions/0016-module-database-access.md) for the
|
||||
complete design and its rejected alternatives.
|
||||
|
||||
## Database connection limits
|
||||
|
||||
`apps/api/src/server.ts` configures the one shared PostgreSQL pool (`createApiDatabase`,
|
||||
@@ -287,10 +333,10 @@ a query is already running on one. The operation timeout additionally sets `pg`'
|
||||
and `statement_timeout`, so a hung or unexpectedly slow query is aborted rather than holding a pool
|
||||
connection (and its caller) indefinitely. An invalid timeout (non-integer, zero, negative, or
|
||||
larger than PostgreSQL's own maximum) is rejected as configuration error before any pool is
|
||||
constructed, sanitized the same way an invalid connection string already is. Both the identity and
|
||||
authorization domains read from this one pool through independent typed Drizzle views — see
|
||||
[Authorization](authorization.md#shared-database-composition) for why there is exactly one pool,
|
||||
not two.
|
||||
constructed, sanitized the same way an invalid connection string already is. The identity,
|
||||
authorization, and module domains all read from this one pool through independent typed Drizzle
|
||||
views — see [Authorization](authorization.md#shared-database-composition) for why there is exactly
|
||||
one pool, not two.
|
||||
|
||||
## Graceful shutdown
|
||||
|
||||
@@ -300,19 +346,22 @@ shared promise the first time it is called: every caller, concurrent or sequenti
|
||||
exact same operation, and that operation runs exactly once no matter how many times `shutdown()`
|
||||
is called. In order: it stops the HTTP listener from accepting new connections and lets in-flight
|
||||
requests finish; then stops the module runtime (so no module resource stops while a route handler
|
||||
might still be in flight); then closes the one shared PostgreSQL pool (identity and authorization
|
||||
both read from it — see [Authorization](authorization.md#shared-database-composition)). Unlike a
|
||||
startup-failure cleanup path (below), a module-stop failure here is not swallowed — it is a
|
||||
genuine result a caller should observe — but pool cleanup always still runs first, so a stop
|
||||
failure never leaves the pool open. The executable entry point's `SIGINT`/`SIGTERM` wiring is
|
||||
extracted into `installShutdownSignalHandlers(running, exit)` — the same function production
|
||||
might still be in flight) — a module's `stop` hook may still use `services.database` here, since
|
||||
the pool has not yet closed; then closes the one shared PostgreSQL pool (identity, authorization,
|
||||
and the module database facade all read from it — see
|
||||
[Authorization](authorization.md#shared-database-composition)). Unlike a startup-failure cleanup
|
||||
path (below), a module-stop failure here is not swallowed — it is a genuine result a caller should
|
||||
observe — but pool cleanup always still runs first, so a stop failure never leaves the pool open,
|
||||
and no module code can run against it afterward. The executable entry point's `SIGINT`/`SIGTERM`
|
||||
wiring is extracted into `installShutdownSignalHandlers(running, exit)` — the same function production
|
||||
composition uses, but independently testable with a fake `running`/`exit` so a test can trigger "a
|
||||
signal arrived" without sending a real process signal or invoking the real `process.exit`.
|
||||
|
||||
A startup failure after the database pool was already created always closes that pool before the
|
||||
failure propagates: identity/authorization construction failing (no module runtime was started
|
||||
yet); the server failing to bind its listen address; or an app-construction failure, both of which
|
||||
also stop the module runtime first, since it may have already started successfully. A module
|
||||
failure propagates: module-services/identity/authorization construction failing (no module runtime
|
||||
was started yet — the frozen `ModuleRuntimeServices` object is built in the same step, from the
|
||||
same pool); the server failing to bind its listen address; or an app-construction failure, both of
|
||||
which also stop the module runtime first, since it may have already started successfully. A module
|
||||
load/register/start failure uses Core's own existing rollback (already-started modules stop, in
|
||||
reverse order, before the failure propagates) and then closes the pool — there is no separate
|
||||
runtime handle to stop in that case, since `startDiscoveredModuleRuntime` never returned one.
|
||||
@@ -354,10 +403,9 @@ Cookie: better-auth.session_token=<redacted, now revoked>
|
||||
{"kind":"anonymous"}
|
||||
```
|
||||
|
||||
## Next phase
|
||||
## What's next
|
||||
|
||||
Phase 4C, pass 3: module discovery/Module SDK permission registration, roles/groups/teams/tenants,
|
||||
an owner-management HTTP endpoint, service accounts/API tokens for non-browser automation, and
|
||||
reversible relationship-derived access itself. See [Authorization](authorization.md) for what this
|
||||
pass already added (permission registry, platform-owner bootstrap, explicit-grant HTTP API). Not a
|
||||
Dashboard Surface, which still depends on an authorization model that is only now getting started.
|
||||
See the repository root [README](../../README.md#current-status) and
|
||||
[platform architecture](../architecture/platform-architecture.md#current-implementation-status)
|
||||
for what has landed across every phase and what remains before Surface work begins. This document
|
||||
tracks `apps/api`'s own HTTP, module-runtime, and database-composition contracts as they evolve.
|
||||
|
||||
@@ -396,15 +396,19 @@ Every response sets `Cache-Control: no-store`.
|
||||
|
||||
`apps/api/src/database.ts`'s `createApiDatabase(databaseUrl, options)` is the one place `apps/api`
|
||||
touches PostgreSQL connection setup. It constructs exactly one validated, timeout-bounded
|
||||
`pg.Pool` (`createValidatedPostgresPool`, `@ekklesia/core/database`) and wraps it with two
|
||||
independent typed Drizzle views — `createIdentityDrizzleView(pool)` and
|
||||
`createAuthorizationDrizzleView(pool)` — rather than two separate pools. `close()` on the returned
|
||||
handle is the only thing that ever calls `pool.end()`; neither domain view owns or closes the pool
|
||||
itself. `apps/api/src/server.ts` calls `createApiDatabase` once during `startApi()`, builds the
|
||||
identity runtime from `.identity` and the `AuthorizationStore`/`AuthorizationEvaluator` from
|
||||
`.authorization` off that same handle, and closes the handle exactly once — on graceful shutdown,
|
||||
and on any startup failure after the pool was already constructed (mirroring the existing
|
||||
identity-only failure-cleanup path from [API](api.md#graceful-shutdown)).
|
||||
`pg.Pool` (`createValidatedPostgresPool`, `@ekklesia/core/database`) and wraps it with three
|
||||
independent typed Drizzle views — `createIdentityDrizzleView(pool)`,
|
||||
`createAuthorizationDrizzleView(pool)`, and `createModuleDatabaseView(pool)` (the `ModuleDatabase`
|
||||
facade module code queries through `context.services.database`; see
|
||||
[API](api.md#module-database-access) and
|
||||
[Module development](../module-development/README.md#database-access)) — rather than three
|
||||
separate pools. `close()` on the returned handle is the only thing that ever calls `pool.end()`;
|
||||
no domain view owns or closes the pool itself. `apps/api/src/server.ts` calls `createApiDatabase`
|
||||
once during `startApi()`, builds the identity runtime from `.identity`, the
|
||||
`AuthorizationStore`/`AuthorizationEvaluator` from `.authorization`, and the frozen
|
||||
`ModuleRuntimeServices` from `.module` off that same handle, and closes the handle exactly once —
|
||||
on graceful shutdown, and on any startup failure after the pool was already constructed (mirroring
|
||||
the existing identity-only failure-cleanup path from [API](api.md#graceful-shutdown)).
|
||||
|
||||
The existing per-domain factories (`createIdentityDrizzleDatabase`,
|
||||
`createAuthorizationDrizzleDatabase`), which each construct and own a private pool, are
|
||||
@@ -413,9 +417,9 @@ unaffected and remain available for standalone/testable use — the CLI bootstra
|
||||
the authorization schema and has no reason to share a pool with anything else.
|
||||
|
||||
`apps/api/tests/lifecycle.test.ts` proves this composition explicitly: exactly one
|
||||
`new Pool(...)` call for a full `startApi()` lifecycle (covering both identity and authorization
|
||||
usage), and exactly one `pool.end()` call regardless of how many concurrent `shutdown()` callers
|
||||
there are or whether the failure happened during startup.
|
||||
`new Pool(...)` call for a full `startApi()` lifecycle (covering identity, authorization, and
|
||||
module database usage), and exactly one `pool.end()` call regardless of how many concurrent
|
||||
`shutdown()` callers there are or whether the failure happened during startup.
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
@@ -169,9 +169,10 @@ rejected the same sanitized way an invalid connection string already is.
|
||||
## What is not operational yet
|
||||
|
||||
No email verification, password recovery, invitation flow, email delivery, social or OAuth
|
||||
provider, passkey, magic link, two-factor authentication, or roles/permissions/platform-owner model
|
||||
provider, passkey, magic link, two-factor authentication, or roles/groups/teams/tenants model
|
||||
exists. Public registration, sign-in, principal resolution, and sign-out over HTTP **are**
|
||||
operational as of Phase 4B — see [API](api.md) for the transport and routes, and ADR 0011 for why
|
||||
public registration is intentional and grants no elevation. The next expected slice is Phase 4C:
|
||||
Core authorization and an explicit, auditable platform-owner bootstrap command — not a Dashboard
|
||||
Surface, which still depends on an authorization model that does not exist yet.
|
||||
public registration is intentional and grants no elevation. Core's authorization foundation
|
||||
(platform owners, explicit grants, an immutable permission registry) landed in Phase 4C and is
|
||||
covered separately — see [Authorization](authorization.md) — not a Dashboard Surface, which still
|
||||
depends on an authorization model that only now exists and has not yet been built on.
|
||||
|
||||
@@ -97,6 +97,23 @@ database failure and asserts, against actual `console.error`/`warn`/`log` output
|
||||
Auth's own internal error logging — configured explicitly in `core/src/identity/runtime.ts` — never
|
||||
emits a raw exception, SQL, connection detail, or the rejected request's origin.
|
||||
|
||||
A third dedicated suite (`apps/api/tests/module-database.integration.ts`, included in the same
|
||||
`pnpm test:migrations:integration` run) proves `context.services.database` against a real,
|
||||
installed test module (`core/tests/fixtures/module-database-integration/db-facade/` — never the
|
||||
repository's live `modules/` directory, and invisible to normal discovery/Doctor/`modules:validate`
|
||||
unless a test explicitly points discovery at it): the module's own table does not exist until its
|
||||
migration is explicitly applied, and constructing the pool or facade never creates it; `start`,
|
||||
`stop`, and an authorized HTTP handler all receive the identical `ModuleRuntimeServices` object,
|
||||
proven by reference equality; a real authorized mutation persists and a later read observes it,
|
||||
gated by the same permission/trusted-Origin enforcement every other module route uses; a real
|
||||
committed transaction persists and a real rollback (a deliberate unique-constraint violation)
|
||||
leaves no partial mutation; a module deliberately catching a real PostgreSQL failure returns its
|
||||
own response untouched, while an uncaught one sanitizes to the existing
|
||||
`MODULE_HANDLER_FAILED` boundary with no leaked SQL, error text, or path; and a `stop` hook's write
|
||||
completes and is independently verifiable, through the harness's own administrative connection —
|
||||
never the API's own pool — before that pool closes. See
|
||||
[ADR 0016](../architecture/decisions/0016-module-database-access.md).
|
||||
|
||||
CI runs the same command in the dedicated `migration-smoke` job. After this change is merged, add
|
||||
the following pull-request status context to `main` branch protection:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user