feat: add native module realtime transport
This commit is contained in:
ekklesia
+97
-22
@@ -76,7 +76,8 @@ No real values are shown here — configure per environment and never commit the
|
||||
| `GET /api/v1/principal` | Ekklesia | Returns the minimal `EkklesiaPrincipal` contract as JSON. `Cache-Control: no-store`. |
|
||||
| `GET\|POST /api/v1/grants*` | Ekklesia | Explicit authorization grant management, gated by `core.authorization.grants.manage`. Listing is bounded keyset pagination; creation is unscoped-only and conflicts (`409`) on an equivalent effective grant; revocation clears every unrevoked equivalent. See [Authorization](authorization.md#explicit-grant-http-api). |
|
||||
| `/api/v1/modules/<module-id>/*` | Installed modules | Every route an installed module registered during its `register` hook, gated by that route's own declared permission. See [Module HTTP routes](#module-http-routes). |
|
||||
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's API lives, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
|
||||
| `GET /api/v1/module-realtime/<module-id>/<channel>` | Installed modules | Snapshot-first Server-Sent Events for one registered module-local channel, gated by that channel's declared permission. See [Module realtime](#module-realtime). |
|
||||
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's HTTP and realtime roots live, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
|
||||
| `GET /api/v1/dashboard/navigation` | Ekklesia | The resolved Dashboard navigation: the same mountable modules, grouped, labelled, and ordered as this installation has configured them. Requires `core.modules.ui.read`. See [Dashboard navigation](#dashboard-navigation). |
|
||||
| `PUT\|DELETE /api/v1/dashboard/navigation/{groups,placements}/*` | Ekklesia | Organize the Dashboard: create/rename/order/delete a group, and place a module. Requires `core.dashboard.navigation.manage` (baseline `restricted`) and a trusted `Origin`. See [Dashboard navigation](#dashboard-navigation). |
|
||||
| `GET /api/v1/module-ui/<module-id>/<version>` | Ekklesia | Publishes an installed module's validated `entrypoints.ui` artifact at a browser-loadable URL. No authentication; only the declared artifact, never another file. See [Module UI asset publication](#module-ui-asset-publication). |
|
||||
@@ -400,9 +401,9 @@ account.
|
||||
6. Start the module runtime from the eligible subset (`startDiscoveredModuleRuntime`) — modules are
|
||||
never rediscovered for this step — passing it the services object every module's `start`/`stop`
|
||||
hook receives.
|
||||
7. Create the Hono application, mounting the started module runtime's published routes, publishing
|
||||
UI for that same eligible set, and passing the identical services object to every module HTTP
|
||||
handler.
|
||||
7. Create the Hono application, mounting the started module runtime's HTTP routes and realtime
|
||||
channels, publishing UI for that same eligible set, and passing the identical scoped runtime
|
||||
objects to module HTTP handlers.
|
||||
8. Bind the HTTP server.
|
||||
|
||||
Discovery and permission composition happen before any pool, Better Auth instance, module
|
||||
@@ -779,13 +780,12 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
|
||||
check and before permission evaluation — `0` means no body is accepted at all. An oversized
|
||||
request receives the platform's stable `413 {"code":"REQUEST_BODY_TOO_LARGE"}` and never reaches
|
||||
authorization, the handler, or the module database.
|
||||
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and
|
||||
body all behave normally, and the full query string remains available) plus a frozen context of
|
||||
exactly `moduleId`, the authorized `EkklesiaPrincipal`, frozen, Hono-decoded path parameters (an
|
||||
empty frozen object for a route with none), and `services` — the same frozen
|
||||
`ModuleRuntimeServices` object the module's own `start`/`stop` hooks received (see
|
||||
[Module database access](#module-database-access)) — never a Hono `Context`, the authorization
|
||||
store, storage, jobs, configuration, or secrets.
|
||||
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and body
|
||||
all behave normally, and the full query string remains available) plus a frozen context of exactly
|
||||
`moduleId`, the authorized `EkklesiaPrincipal`, frozen Hono-decoded path parameters (an empty
|
||||
frozen object for a route with none), `services`, and the module's scoped configuration, storage,
|
||||
and realtime publisher. It never receives a Hono `Context`, authorization/grant stores, jobs, or
|
||||
another module's scoped objects. Core does not serialize configuration or secrets into responses.
|
||||
- **Handler failure.** A handler that throws, rejects, or returns anything other than a `Response`
|
||||
becomes a fixed `500 {"code":"MODULE_HANDLER_FAILED"}` with `Cache-Control: no-store` — never the
|
||||
caught value, a stack, the route's path or permission id, or the module id. A `Response` a
|
||||
@@ -802,6 +802,69 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
|
||||
registry: installing, removing, or editing a module's routes takes effect only on the next API
|
||||
restart. Removing a module makes its former routes return the ordinary `404 {"code":"NOT_FOUND"}`.
|
||||
|
||||
## Module realtime
|
||||
|
||||
An eligible module may declare a module-local state channel during `register`. Ekklesia mounts one
|
||||
exact route for each successfully started declaration:
|
||||
|
||||
```text
|
||||
GET /api/v1/module-realtime/<module-id>/<channel>
|
||||
```
|
||||
|
||||
The path is host-private. Module UI receives the module's server-produced realtime base in its mount
|
||||
descriptor and calls `context.realtime.subscribe("state", handlers)`; it never builds this URL,
|
||||
chooses a module id, supplies a permission, or sees an EventSource. Browser commands remain ordinary
|
||||
module HTTP requests, retaining their body, origin, authorization, and response semantics.
|
||||
|
||||
The response is a Server-Sent Events stream. It begins with exactly one authoritative `snapshot`
|
||||
event and then carries `update` events:
|
||||
|
||||
```text
|
||||
event: snapshot
|
||||
data: {"sequence":1,"payload":{"mode":"on","currentSlide":0}}
|
||||
|
||||
event: update
|
||||
data: {"sequence":2,"payload":{"mode":"on","currentSlide":1}}
|
||||
```
|
||||
|
||||
Core atomically captures the latest state while installing the subscriber, so there is no separate
|
||||
GET-then-subscribe gap. Sequence is monotonically increasing per channel for this process only; there
|
||||
is no `id` field, `Last-Event-ID` replay, durable history, or exactly-once promise. A reconnect is a
|
||||
new subscription and begins with the latest snapshot. Slow clients retain only the newest pending
|
||||
update, so sequence gaps are expected and intermediate states may be coalesced.
|
||||
|
||||
- **Authentication and authorization.** The SSE GET uses the existing session cookie and the same
|
||||
principal resolver, permission registry, and evaluator as module HTTP. The channel's declaration
|
||||
fixes the permission before any request exists. Permission evaluation completes before a Core
|
||||
subscriber is allocated. There is no query-string token or second realtime auth system.
|
||||
- **Reauthorization.** A stream has a host-owned maximum age of five minutes. The host closes it at
|
||||
that boundary; native EventSource reconnects and the ordinary cookie and permission checks run
|
||||
again. Grant revocation, grant expiry, and session changes therefore do not leave an accepted
|
||||
stream authorized forever.
|
||||
- **Bounds.** Core accepts at most 16 channels per module, 100 subscribers per channel, and 64 KiB
|
||||
of serialized JSON per publication. Publishing never awaits a browser. One latest snapshot and
|
||||
one pending update per subscriber are the only state retained.
|
||||
- **Transport behavior.** Responses use `text/event-stream; charset=utf-8`, `Cache-Control:
|
||||
no-store`, `Vary: Origin, Cookie`, `X-Accel-Buffering: no`, and a 15-second comment keepalive.
|
||||
Keepalives consume no module sequence and never reach module UI. A reverse proxy must forward
|
||||
streaming bodies incrementally and permit that idle interval. No proxy product is part of the
|
||||
module contract.
|
||||
- **Methods and errors.** Hono normally lets `HEAD` reuse GET handlers; this route explicitly returns
|
||||
`405` before allocating a stream because discarding an infinite body could leak a subscriber.
|
||||
Subscriber exhaustion is the fixed `503 {"code":"REALTIME_SUBSCRIBER_LIMIT"}`; other stopped or
|
||||
unavailable runtime states are `503 {"code":"REALTIME_UNAVAILABLE"}`. Raw errors, paths, URLs,
|
||||
payloads, permissions, secrets, and stacks are never reflected.
|
||||
- **Lifecycle.** Disabled and blocked modules have no route because their entrypoint never runs and
|
||||
their channel is never mounted. Shutdown first refuses and closes realtime subscriptions so they
|
||||
cannot hold HTTP drain open. Ordinary in-flight module HTTP handlers may finish and publish during
|
||||
that drain. Core then stops modules in reverse dependency order, deactivating each module's scoped
|
||||
publisher immediately before its own stop hook; a captured publisher fails predictably afterward.
|
||||
|
||||
Realtime carries opaque module state. If a module includes a direct object-delivery URL, the exact
|
||||
serialized string passes through unchanged: Core neither re-signs it per subscriber nor extends its
|
||||
expiry. Image, video, and audio bytes remain object-storage traffic, never realtime payloads. See
|
||||
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md).
|
||||
|
||||
## Dashboard navigation
|
||||
|
||||
```text
|
||||
@@ -829,6 +892,7 @@ How this installation presents its mountable modules
|
||||
"version": "1.0.0",
|
||||
"ui": { "url": "/api/v1/module-ui/livestream/1.0.0" },
|
||||
"api": { "baseUrl": "/api/v1/modules/livestream" },
|
||||
"realtime": { "baseUrl": "/api/v1/module-realtime/livestream" },
|
||||
"permissions": ["livestream.streams.read"]
|
||||
}
|
||||
}
|
||||
@@ -908,21 +972,22 @@ root of the same resource the artifact route below serves: one prefix, two repre
|
||||
"version": "0.1.0",
|
||||
"ui": { "url": "/api/v1/module-ui/example/0.1.0" },
|
||||
"api": { "baseUrl": "/api/v1/modules/example" },
|
||||
"realtime": { "baseUrl": "/api/v1/module-realtime/example" },
|
||||
"permissions": ["example.records.read"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- **Five fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
|
||||
- **Six fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
|
||||
package name, server/schema/migrations entrypoint, capability, dependency, or storage bucket. And
|
||||
no `name`, `icon`, `description`, `category`, `folder`, or `order`: those are navigation metadata,
|
||||
and [ADR 0021](../architecture/decisions/0021-modules-as-hostable-products.md) keeps Dashboard
|
||||
grouping an administrator's presentation configuration rather than a module declaration.
|
||||
- **Both URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl` and `api.baseUrl` from
|
||||
`moduleApiBaseUrl`, so each URL shape is constructed in exactly one place in the repository. A
|
||||
browser never rebuilds `/api/v1/module-ui/<id>/<version>` or `/api/v1/modules/<id>`, which is what
|
||||
lets either change without touching host or module code.
|
||||
- **All three URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl`, `api.baseUrl`
|
||||
from `moduleApiBaseUrl`, and `realtime.baseUrl` from `moduleRealtimeBaseUrl`, so each URL shape is
|
||||
constructed in exactly one place in the repository. A browser never rebuilds any platform module
|
||||
prefix, which lets one change without touching host or module code.
|
||||
- **The module set is the publication registry**, not a rescan: modules that survived discovery and
|
||||
declare `entrypoints.ui`, captured after the module runtime started. A module with no UI is absent.
|
||||
Order is the registry's own deterministic discovery order.
|
||||
@@ -1114,15 +1179,25 @@ safe, and the network boundary remains the deployment's own responsibility. See
|
||||
executable entry point and tests) returns `{app, server, shutdown}`. `shutdown()` memoizes one
|
||||
shared promise the first time it is called: every caller, concurrent or sequential, awaits the
|
||||
exact same operation, and that operation runs exactly once no matter how many times `shutdown()`
|
||||
is called. In order: it stops the HTTP listener from accepting new connections and lets in-flight
|
||||
requests finish; then stops the module runtime (so no module resource stops while a route handler
|
||||
might still be in flight) — a module's `stop` hook may still use `services.database` here, since
|
||||
the pool has not yet closed; then closes the one shared PostgreSQL pool (identity, authorization,
|
||||
and the module database facade all read from it — see
|
||||
is called. In order: it refuses new realtime subscribers and closes every existing stream; then
|
||||
stops the HTTP listener from accepting new connections and lets finite in-flight requests finish;
|
||||
then stops the module runtime in reverse dependency order; then closes storage and the one shared
|
||||
PostgreSQL pool (identity, authorization, and the module database facade all read from it — see
|
||||
[Authorization](authorization.md#shared-database-composition)). Unlike a startup-failure cleanup
|
||||
path (below), a module-stop failure here is not swallowed — it is a genuine result a caller should
|
||||
observe — but pool cleanup always still runs first, so a stop failure never leaves the pool open,
|
||||
and no module code can run against it afterward. The executable entry point's `SIGINT`/`SIGTERM`
|
||||
and no module code can run against it afterward.
|
||||
|
||||
Closing streams and disabling publishers are deliberately separate. Streams are long-lived HTTP
|
||||
responses and would otherwise force every healthy shutdown to its drain deadline. Publishers remain
|
||||
active during the finite HTTP drain so an already-running module command can commit and publish its
|
||||
result without turning a successful request into a false `500`. As reverse-order module stop reaches
|
||||
each module, Core disables that module's publisher immediately before its stop hook; dependencies
|
||||
remain live while their dependents stop, and a stale publisher fails predictably once its own module
|
||||
begins stopping. A module's `stop` hook may still use `services.database` and storage here, since
|
||||
those providers have not yet closed.
|
||||
|
||||
The executable entry point's `SIGINT`/`SIGTERM`
|
||||
wiring is extracted into `installShutdownSignalHandlers(running, exit)` — the same function production
|
||||
composition uses, but independently testable with a fake `running`/`exit` so a test can trigger "a
|
||||
signal arrived" without sending a real process signal or invoking the real `process.exit`.
|
||||
|
||||
@@ -7,7 +7,9 @@ document's guarantees.
|
||||
Related: [ADR 0006](../architecture/decisions/0006-versioned-capabilities-and-compiled-module-runtime.md)
|
||||
(compiled module runtime), [ADR 0016](../architecture/decisions/0016-module-database-access.md)
|
||||
(module database access), [ADR 0020](../architecture/decisions/0020-module-runtime-and-migration-execution-hardening.md)
|
||||
(module routing/lifecycle and migration execution), and
|
||||
(module routing/lifecycle and migration execution),
|
||||
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md)
|
||||
(native module realtime), and
|
||||
[Module development](../module-development/README.md).
|
||||
|
||||
## Installed modules are trusted compiled backend code
|
||||
@@ -44,6 +46,7 @@ cooperative. None of them constrains a module that is not:
|
||||
| `ekklesia.module.json` manifest and `manifestVersion` | Declared compatibility and dependency ordering |
|
||||
| Permission declarations and the composed registry | Which permissions *this* module's own routes may name, checked when Core mounts them |
|
||||
| Route validation, overlap rejection, body limits | Deterministic, unambiguous HTTP routing and request bounds |
|
||||
| Scoped realtime channels, publishers, and subscriptions | Correct module-local state wiring, permission enforcement, and finite fanout — not process isolation |
|
||||
| `Object.freeze` on contexts and services | Prevents accidental mutation; a module can still reach the underlying objects |
|
||||
| The `ModuleDatabase` facade | A typed, ergonomic surface — not a privilege reduction, since `execute` is arbitrary SQL |
|
||||
| Capability registry | Explicit inter-module wiring, not access control |
|
||||
@@ -316,6 +319,37 @@ see the URL it is fetching.
|
||||
an unsigned request for the same object is refused, and an integration test asserts it so that a
|
||||
future change cannot "fix" delivery by opening a bucket up.
|
||||
|
||||
### Realtime is permission-gated state delivery, not a trust boundary
|
||||
|
||||
A module may declare a local realtime channel protected by one of its own manifest permissions. Core
|
||||
closes both the server publisher and browser subscription over that module id: neither API accepts a
|
||||
caller-supplied module namespace, global topic, wildcard, broker address, origin, URL, permission, or
|
||||
credential. This prevents accidental cross-module wiring by a cooperating module. It does not
|
||||
sandbox hostile installed code; trusted same-process backend code could bypass the facade, and
|
||||
trusted same-realm UI could call browser globals directly.
|
||||
|
||||
Ekklesia's SSE endpoint resolves the ordinary session cookie and evaluates the channel's fixed
|
||||
permission before allocating a subscriber. There is no query-string token and no second realtime
|
||||
authentication path. A stream is closed after five minutes so native EventSource reconnects and
|
||||
session, grant-expiry, revocation, and baseline policy are evaluated again; authorization is not
|
||||
continuously queried between those boundaries. Disabled and dependency-blocked modules have no
|
||||
stream route at all. Public or device/display identity remains deliberately unsolved, so no channel
|
||||
is made anonymous merely to support a future kiosk.
|
||||
|
||||
Fanout is finite: 16 channels per module, 100 subscribers per channel, 64 KiB of validated serialized
|
||||
JSON per publication, one latest snapshot per channel, and one coalescing pending slot per client.
|
||||
A slow browser cannot block a publisher or create an unbounded queue. The endpoint rejects implicit
|
||||
`HEAD` before allocation, disconnect/cancel closes its Core subscription, and server shutdown closes
|
||||
streams before finite HTTP drain. Each scoped publisher becomes inert immediately before its own
|
||||
module's stop hook.
|
||||
|
||||
Core never adds configuration, secret values, database values, storage credentials, or delivery URLs
|
||||
to a realtime document. It transports only what trusted module code explicitly publishes. A module
|
||||
must therefore treat realtime like any other authorized response: include only data every principal
|
||||
holding the channel permission may receive. If it intentionally includes a signed delivery URL, that
|
||||
URL remains a bearer capability and is forwarded byte-for-byte; fixed transport errors never reflect
|
||||
payloads, URLs, permissions, runtime exceptions, paths, or stacks.
|
||||
|
||||
### Client-side permission checks are not a security boundary
|
||||
|
||||
A host may tell a module UI what permissions it believes the viewer holds
|
||||
|
||||
@@ -247,6 +247,34 @@ cannot appear unnoticed. See
|
||||
[ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md) and
|
||||
[ADR 0031](../architecture/decisions/0031-direct-object-delivery.md).
|
||||
|
||||
Phase 7A pass 5 adds realtime evidence at every boundary
|
||||
([ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md)). Contract and Core
|
||||
unit suites prove channel grammar, module/permission scoping, staged registration, mandatory initial
|
||||
snapshots, atomic snapshot-plus-subscribe, sequence ordering, payload/container limits, latest-state
|
||||
coalescing, subscriber limits, exact shutdown behavior, per-module publisher deactivation, sanitized
|
||||
errors, and that modules declaring no realtime continue unchanged. Browser SDK/shell suites prove
|
||||
that the public contract has no server dependency or transport vocabulary, descriptors and local
|
||||
channels are strictly contained, credentials are host-owned, reconnect requires a fresh snapshot,
|
||||
terminal EventSource failure cleans up listeners, and explicit unmount is idempotent.
|
||||
|
||||
`apps/api/tests/module-realtime-routes.test.ts` exercises permission-before-allocation, the
|
||||
snapshot/update wire envelope, lack of replay ids, stream headers, comment keepalives, forced
|
||||
five-minute reauthorization through a short test-only override, shutdown closure, sanitized
|
||||
subscriber failures, and the explicit `HEAD` refusal that prevents an infinite discarded body from
|
||||
retaining a subscriber. `apps/api/tests/module-realtime-live.test.ts` binds an actual Node
|
||||
HTTP server on an OS-assigned loopback port and drives an Announcements-shaped module through real
|
||||
discovery, registration, startup, two live SSE clients, fanout, disconnect cleanup, reconnect with a
|
||||
fresh latest snapshot, and shutdown. Every bind, fetch, stream read, cancellation, cleanup poll, and
|
||||
listener close is bounded; the test also pins an opaque signed delivery URL byte-for-byte. It uses no
|
||||
PostgreSQL or object-store double because neither belongs in realtime transport.
|
||||
|
||||
The live PostgreSQL API suite adds lifecycle composition evidence: an enabled module's realtime route
|
||||
is live while disabled and dependency-blocked modules have no route, just like their HTTP/UI
|
||||
surfaces. Its real SSE fetch/read/cancel path is bounded and migrations remain independent of runtime
|
||||
enablement. During realtime development, run the focused live Node suite repeatedly as well as
|
||||
`pnpm verify` and `pnpm test:migrations:integration`; consecutive runs are specifically intended to
|
||||
catch retained timers, listeners, subscribers, or loopback servers.
|
||||
|
||||
### Which kind of timing proof lives where
|
||||
|
||||
Pass 4 introduced several deadlines, and their tests deliberately differ in kind:
|
||||
|
||||
Reference in New Issue
Block a user