feat: add native module realtime transport
CI / verify (pull_request) Successful in 37s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Successful in 1m7s

This commit is contained in:
ekklesia
2026-08-23 18:47:18 -05:00
parent fcc531a39c
commit fc6c8045c4
53 changed files with 3959 additions and 108 deletions
+97 -22
View File
@@ -76,7 +76,8 @@ No real values are shown here — configure per environment and never commit the
| `GET /api/v1/principal` | Ekklesia | Returns the minimal `EkklesiaPrincipal` contract as JSON. `Cache-Control: no-store`. |
| `GET\|POST /api/v1/grants*` | Ekklesia | Explicit authorization grant management, gated by `core.authorization.grants.manage`. Listing is bounded keyset pagination; creation is unscoped-only and conflicts (`409`) on an equivalent effective grant; revocation clears every unrevoked equivalent. See [Authorization](authorization.md#explicit-grant-http-api). |
| `/api/v1/modules/<module-id>/*` | Installed modules | Every route an installed module registered during its `register` hook, gated by that route's own declared permission. See [Module HTTP routes](#module-http-routes). |
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's API lives, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
| `GET /api/v1/module-realtime/<module-id>/<channel>` | Installed modules | Snapshot-first Server-Sent Events for one registered module-local channel, gated by that channel's declared permission. See [Module realtime](#module-realtime). |
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's HTTP and realtime roots live, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
| `GET /api/v1/dashboard/navigation` | Ekklesia | The resolved Dashboard navigation: the same mountable modules, grouped, labelled, and ordered as this installation has configured them. Requires `core.modules.ui.read`. See [Dashboard navigation](#dashboard-navigation). |
| `PUT\|DELETE /api/v1/dashboard/navigation/{groups,placements}/*` | Ekklesia | Organize the Dashboard: create/rename/order/delete a group, and place a module. Requires `core.dashboard.navigation.manage` (baseline `restricted`) and a trusted `Origin`. See [Dashboard navigation](#dashboard-navigation). |
| `GET /api/v1/module-ui/<module-id>/<version>` | Ekklesia | Publishes an installed module's validated `entrypoints.ui` artifact at a browser-loadable URL. No authentication; only the declared artifact, never another file. See [Module UI asset publication](#module-ui-asset-publication). |
@@ -400,9 +401,9 @@ account.
6. Start the module runtime from the eligible subset (`startDiscoveredModuleRuntime`) — modules are
never rediscovered for this step — passing it the services object every module's `start`/`stop`
hook receives.
7. Create the Hono application, mounting the started module runtime's published routes, publishing
UI for that same eligible set, and passing the identical services object to every module HTTP
handler.
7. Create the Hono application, mounting the started module runtime's HTTP routes and realtime
channels, publishing UI for that same eligible set, and passing the identical scoped runtime
objects to module HTTP handlers.
8. Bind the HTTP server.
Discovery and permission composition happen before any pool, Better Auth instance, module
@@ -779,13 +780,12 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
check and before permission evaluation — `0` means no body is accepted at all. An oversized
request receives the platform's stable `413 {"code":"REQUEST_BODY_TOO_LARGE"}` and never reaches
authorization, the handler, or the module database.
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and
body all behave normally, and the full query string remains available) plus a frozen context of
exactly `moduleId`, the authorized `EkklesiaPrincipal`, frozen, Hono-decoded path parameters (an
empty frozen object for a route with none), and `services` — the same frozen
`ModuleRuntimeServices` object the module's own `start`/`stop` hooks received (see
[Module database access](#module-database-access)) — never a Hono `Context`, the authorization
store, storage, jobs, configuration, or secrets.
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and body
all behave normally, and the full query string remains available) plus a frozen context of exactly
`moduleId`, the authorized `EkklesiaPrincipal`, frozen Hono-decoded path parameters (an empty
frozen object for a route with none), `services`, and the module's scoped configuration, storage,
and realtime publisher. It never receives a Hono `Context`, authorization/grant stores, jobs, or
another module's scoped objects. Core does not serialize configuration or secrets into responses.
- **Handler failure.** A handler that throws, rejects, or returns anything other than a `Response`
becomes a fixed `500 {"code":"MODULE_HANDLER_FAILED"}` with `Cache-Control: no-store` — never the
caught value, a stack, the route's path or permission id, or the module id. A `Response` a
@@ -802,6 +802,69 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
registry: installing, removing, or editing a module's routes takes effect only on the next API
restart. Removing a module makes its former routes return the ordinary `404 {"code":"NOT_FOUND"}`.
## Module realtime
An eligible module may declare a module-local state channel during `register`. Ekklesia mounts one
exact route for each successfully started declaration:
```text
GET /api/v1/module-realtime/<module-id>/<channel>
```
The path is host-private. Module UI receives the module's server-produced realtime base in its mount
descriptor and calls `context.realtime.subscribe("state", handlers)`; it never builds this URL,
chooses a module id, supplies a permission, or sees an EventSource. Browser commands remain ordinary
module HTTP requests, retaining their body, origin, authorization, and response semantics.
The response is a Server-Sent Events stream. It begins with exactly one authoritative `snapshot`
event and then carries `update` events:
```text
event: snapshot
data: {"sequence":1,"payload":{"mode":"on","currentSlide":0}}
event: update
data: {"sequence":2,"payload":{"mode":"on","currentSlide":1}}
```
Core atomically captures the latest state while installing the subscriber, so there is no separate
GET-then-subscribe gap. Sequence is monotonically increasing per channel for this process only; there
is no `id` field, `Last-Event-ID` replay, durable history, or exactly-once promise. A reconnect is a
new subscription and begins with the latest snapshot. Slow clients retain only the newest pending
update, so sequence gaps are expected and intermediate states may be coalesced.
- **Authentication and authorization.** The SSE GET uses the existing session cookie and the same
principal resolver, permission registry, and evaluator as module HTTP. The channel's declaration
fixes the permission before any request exists. Permission evaluation completes before a Core
subscriber is allocated. There is no query-string token or second realtime auth system.
- **Reauthorization.** A stream has a host-owned maximum age of five minutes. The host closes it at
that boundary; native EventSource reconnects and the ordinary cookie and permission checks run
again. Grant revocation, grant expiry, and session changes therefore do not leave an accepted
stream authorized forever.
- **Bounds.** Core accepts at most 16 channels per module, 100 subscribers per channel, and 64 KiB
of serialized JSON per publication. Publishing never awaits a browser. One latest snapshot and
one pending update per subscriber are the only state retained.
- **Transport behavior.** Responses use `text/event-stream; charset=utf-8`, `Cache-Control:
no-store`, `Vary: Origin, Cookie`, `X-Accel-Buffering: no`, and a 15-second comment keepalive.
Keepalives consume no module sequence and never reach module UI. A reverse proxy must forward
streaming bodies incrementally and permit that idle interval. No proxy product is part of the
module contract.
- **Methods and errors.** Hono normally lets `HEAD` reuse GET handlers; this route explicitly returns
`405` before allocating a stream because discarding an infinite body could leak a subscriber.
Subscriber exhaustion is the fixed `503 {"code":"REALTIME_SUBSCRIBER_LIMIT"}`; other stopped or
unavailable runtime states are `503 {"code":"REALTIME_UNAVAILABLE"}`. Raw errors, paths, URLs,
payloads, permissions, secrets, and stacks are never reflected.
- **Lifecycle.** Disabled and blocked modules have no route because their entrypoint never runs and
their channel is never mounted. Shutdown first refuses and closes realtime subscriptions so they
cannot hold HTTP drain open. Ordinary in-flight module HTTP handlers may finish and publish during
that drain. Core then stops modules in reverse dependency order, deactivating each module's scoped
publisher immediately before its own stop hook; a captured publisher fails predictably afterward.
Realtime carries opaque module state. If a module includes a direct object-delivery URL, the exact
serialized string passes through unchanged: Core neither re-signs it per subscriber nor extends its
expiry. Image, video, and audio bytes remain object-storage traffic, never realtime payloads. See
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md).
## Dashboard navigation
```text
@@ -829,6 +892,7 @@ How this installation presents its mountable modules
"version": "1.0.0",
"ui": { "url": "/api/v1/module-ui/livestream/1.0.0" },
"api": { "baseUrl": "/api/v1/modules/livestream" },
"realtime": { "baseUrl": "/api/v1/module-realtime/livestream" },
"permissions": ["livestream.streams.read"]
}
}
@@ -908,21 +972,22 @@ root of the same resource the artifact route below serves: one prefix, two repre
"version": "0.1.0",
"ui": { "url": "/api/v1/module-ui/example/0.1.0" },
"api": { "baseUrl": "/api/v1/modules/example" },
"realtime": { "baseUrl": "/api/v1/module-realtime/example" },
"permissions": ["example.records.read"]
}
]
}
```
- **Five fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
- **Six fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
package name, server/schema/migrations entrypoint, capability, dependency, or storage bucket. And
no `name`, `icon`, `description`, `category`, `folder`, or `order`: those are navigation metadata,
and [ADR 0021](../architecture/decisions/0021-modules-as-hostable-products.md) keeps Dashboard
grouping an administrator's presentation configuration rather than a module declaration.
- **Both URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl` and `api.baseUrl` from
`moduleApiBaseUrl`, so each URL shape is constructed in exactly one place in the repository. A
browser never rebuilds `/api/v1/module-ui/<id>/<version>` or `/api/v1/modules/<id>`, which is what
lets either change without touching host or module code.
- **All three URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl`, `api.baseUrl`
from `moduleApiBaseUrl`, and `realtime.baseUrl` from `moduleRealtimeBaseUrl`, so each URL shape is
constructed in exactly one place in the repository. A browser never rebuilds any platform module
prefix, which lets one change without touching host or module code.
- **The module set is the publication registry**, not a rescan: modules that survived discovery and
declare `entrypoints.ui`, captured after the module runtime started. A module with no UI is absent.
Order is the registry's own deterministic discovery order.
@@ -1114,15 +1179,25 @@ safe, and the network boundary remains the deployment's own responsibility. See
executable entry point and tests) returns `{app, server, shutdown}`. `shutdown()` memoizes one
shared promise the first time it is called: every caller, concurrent or sequential, awaits the
exact same operation, and that operation runs exactly once no matter how many times `shutdown()`
is called. In order: it stops the HTTP listener from accepting new connections and lets in-flight
requests finish; then stops the module runtime (so no module resource stops while a route handler
might still be in flight) — a module's `stop` hook may still use `services.database` here, since
the pool has not yet closed; then closes the one shared PostgreSQL pool (identity, authorization,
and the module database facade all read from it — see
is called. In order: it refuses new realtime subscribers and closes every existing stream; then
stops the HTTP listener from accepting new connections and lets finite in-flight requests finish;
then stops the module runtime in reverse dependency order; then closes storage and the one shared
PostgreSQL pool (identity, authorization, and the module database facade all read from it — see
[Authorization](authorization.md#shared-database-composition)). Unlike a startup-failure cleanup
path (below), a module-stop failure here is not swallowed — it is a genuine result a caller should
observe — but pool cleanup always still runs first, so a stop failure never leaves the pool open,
and no module code can run against it afterward. The executable entry point's `SIGINT`/`SIGTERM`
and no module code can run against it afterward.
Closing streams and disabling publishers are deliberately separate. Streams are long-lived HTTP
responses and would otherwise force every healthy shutdown to its drain deadline. Publishers remain
active during the finite HTTP drain so an already-running module command can commit and publish its
result without turning a successful request into a false `500`. As reverse-order module stop reaches
each module, Core disables that module's publisher immediately before its stop hook; dependencies
remain live while their dependents stop, and a stale publisher fails predictably once its own module
begins stopping. A module's `stop` hook may still use `services.database` and storage here, since
those providers have not yet closed.
The executable entry point's `SIGINT`/`SIGTERM`
wiring is extracted into `installShutdownSignalHandlers(running, exit)` — the same function production
composition uses, but independently testable with a fake `running`/`exit` so a test can trigger "a
signal arrived" without sending a real process signal or invoking the real `process.exit`.