feat: add native module realtime transport
CI / verify (pull_request) Successful in 37s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Successful in 1m7s

This commit is contained in:
ekklesia
2026-08-23 18:47:18 -05:00
parent fcc531a39c
commit fc6c8045c4
53 changed files with 3959 additions and 108 deletions
@@ -74,6 +74,12 @@ administrator owns rather than something a module declares. A display field adde
become module-owned navigation before that design exists, so the temporary selector renders module
ids instead.
> **Cross-reference — see [ADR 0032](0032-native-module-realtime-transport.md).**
> Phase 7A pass 5 extends this original descriptor with a sixth,
> `realtime: { baseUrl: "/api/v1/module-realtime/example" }`, so the browser host can construct the
> transport-neutral `context.realtime` facade without inventing a platform URL. The five-field shape
> above remains the historical decision this ADR recorded; ADR 0032 owns the extension.
It is not a module management API. There is no create, update, enable, disable, install, or
configure, and no module that declares no UI. Ekklesia has no enable/disable state to report —
`ModuleLifecycleStage` remains vocabulary with no runtime state machine behind it — and reporting one
@@ -210,5 +210,6 @@ value it could branch on.
Signed **write** and **delete** URLs are not built. Uploads still go through a module route, where
the module can validate what it is accepting; handing a browser a capability to write into a
module's bucket needs its own decision about size, type, and quota, and does not belong in the pass
that established reads. Realtime URL refresh is not built either — ordinary API reloads are
sufficient, and a future realtime pass can refresh state and URLs together when it exists.
that established reads. [ADR 0032](0032-native-module-realtime-transport.md) now supplies the native
state channel on which a module can publish a replacement delivery URL. Core still does not inspect,
renew, or automatically publish expiring URLs; that refresh policy remains explicitly module-owned.
@@ -0,0 +1,204 @@
# 0032: Native module realtime is scoped snapshot state over SSE
- Status: Accepted
- Date: 2026-08-23
- Extends: [0006](0006-versioned-capabilities-and-compiled-module-runtime.md),
[0013](0013-module-permission-contract.md),
[0015](0015-module-http-transport.md),
[0021](0021-modules-as-hostable-products.md),
[0022](0022-module-ui-authoring-and-host-contract.md),
[0028](0028-module-operational-lifecycle.md),
[0031](0031-direct-object-delivery.md)
## Context
Ekklesia has a host-neutral module HTTP contract, but no way for a running module to deliver a state
change to its browser UI without polling or owning a second server. The first real consumer is
`The_River/Announcements` at commit `9aab486`: its server sends one complete `slideshow:state`
snapshot on connection and broadcasts the same complete shape after a slide tick, slide-list reload,
or mode change. The browser sends no application messages back, uses no acknowledgements, and on
reconnection needs current state rather than a history of missed transitions.
That application currently uses Socket.IO, hard-coded LAN origins, and MQTT mode control. Those are
properties of the old deployment, not requirements for a module contract. Browser commands already
have the authenticated, CSRF-protected module HTTP path from ADR 0015. MQTT is a legacy external
control integration that will be removed; it is not an Ekklesia internal transport.
The existing Node adapter could be extended with a WebSocket server and upgrade handling, but the API
does not have any of that wiring today. Its public module vocabulary is Fetch `Request`/`Response`,
and it already preserves streaming response bodies. Server-Sent Events fit that boundary without a
new bidirectional protocol, dependency, upgrade/authentication path, reconnect implementation, or
shutdown stack.
## Decision
### The public contract names state, not a network protocol
A module declares a small number of local realtime channels during `register`, each protected by one
permission its own manifest declares:
```ts
context.realtime.channel({
channel: "state",
permission: "announcements.slideshow.view",
});
```
Its `start` hook, running work, and HTTP handlers receive the same module-scoped publisher:
```ts
context.realtime.publish("state", slideshowState);
```
The module does not pass a module id, origin, URL, transport, credential, or permission when it
publishes. It cannot name another module's channel through this facade. `channel` is one validated
lowercase kebab-case identifier, not a hierarchical topic: there are no slashes, wildcards, global
namespace, arbitrary cross-module subscription, or broker semantics.
Registration remains declarative and I/O-free. Publication is a live operation and is therefore
absent from registration, exactly like database and storage access. HTTP handlers receive the
publisher because an ordinary authenticated command frequently changes the state that should be
shown immediately; modules must not capture a late-bound lifecycle context to achieve that.
`stop` receives the same context shape for lifecycle consistency, but its publisher has already been
made inert: stopped code cannot emit new live state.
The first Ekklesia host implementation is an authenticated `GET` SSE stream. SSE is private host
machinery: neither the server module contract nor `ModuleUiHostContext` names SSE, EventSource,
WebSocket, or Socket.IO. A standalone host may implement the exact contracts using another transport
without changing a module.
WebSocket is rejected for this slice. The real consumer has no persistent client-to-server messages,
while commands already fit normal module HTTP. Choosing WebSocket would add an upgrade-specific auth
path, ping/pong, reconnect wrapper, dependency, server wiring, and shutdown behavior without buying a
required capability. It can be reconsidered if a real module demonstrates genuine persistent
bidirectional traffic; that would be a new transport decision, not an invisible change here.
### Snapshot first, then latest-state updates
Every declared channel must publish one authoritative state value before its module's `start` hook
completes. If it does not, startup fails and the existing reverse-order rollback runs. The API is not
bound until every eligible module starts, so every reachable channel necessarily has a snapshot.
Core validates and serializes a publication once, assigns the channel's next monotonically increasing
process-local sequence, and stores that exact serialized value as its latest snapshot. Subscription
atomically registers the consumer and captures that snapshot, eliminating a separate `GET state` then
`subscribe` race. A new connection always receives `snapshot` first and then later `update` values.
The sequence detects ordering, duplicates, and gaps; it is not a durable cursor. It may reset when the
process restarts. The transport deliberately does not promise replay, history, exactly-once delivery,
or recovery through `Last-Event-ID`. A reconnect is a new authorized subscription and receives the
fresh latest snapshot, followed by future updates. The module remains the semantic authority and
reconstructs its state from its own persistence/runtime after restart; Core's in-memory copy is only
ephemeral transport state.
### JSON and resource bounds
Payloads are JSON-compatible data only. Core rejects functions, symbols, bigints, cycles, non-finite
numbers, unsupported object values, and anything whose UTF-8 serialized representation exceeds the
fixed platform ceiling. The byte budget is consumed while traversing strings, keys, containers, and
values, before building or stringifying an arbitrarily large clone. The first ceiling is 64 KiB per
publication: far beyond an Announcements state document, while making it unambiguous that media bytes
belong in object storage.
The channel count, subscribers per channel, and per-client pending work are all finite. Publishing is
synchronous and never awaits a browser. A consumer receives publications in increasing sequence while
it keeps up; if it is slow, it retains at most the newest pending state and intermediate updates are
coalesced. Sequence gaps make that behavior visible. This is correct for authoritative state and
prevents one display from blocking a module or creating an unbounded queue.
### Browser host contract
`ModuleUiHostContext` exposes a transport-neutral, module-scoped subscription method. Module UI names
only its local channel and receives snapshot/update/status callbacks plus an idempotent close handle.
The browser does not supply a module id, permission, origin, URL, cookie, or bearer token.
Ekklesia's server adds a root-relative realtime base to the existing module UI mount descriptor, just
as it already supplies the module HTTP base. The shell strictly validates that root-relative value,
resolves it against the configured API origin, appends only the validated local channel, and opens
native `EventSource` with credentials enabled. A standalone host supplies its own root and transport
adapter behind the same browser contract.
The browser reports only stable transport-neutral states (`connecting`, `connected`, `reconnecting`,
`closed`) and fixed safe errors. A recoverable EventSource transport error reports `reconnecting` and
uses the browser's native retry; a terminal `CLOSED` state removes every listener and reports
`closed` instead of promising a retry that will never occur. It does not expose EventSource objects,
raw server responses, socket exceptions, or URLs to module UI. This is a portability and correctness
boundary, not a sandbox: installed UI is trusted same-realm JavaScript and could use browser globals
directly. Server-side permission enforcement remains the security authority.
### Authentication and authorization
The SSE request uses the existing session cookie. Ekklesia evaluates the normal principal and the
channel's registered permission before allocating a subscriber, using the same permission registry
and evaluator as module HTTP. There is no query-string token and no second realtime auth system.
Unknown, disabled, blocked, stopped, or undeclared channels have no mounted route.
Authorization is checked when a stream is established. Ekklesia closes every stream after five
minutes so EventSource reconnects and authorization is re-evaluated; changing a grant does not
otherwise reach backward into an already-established connection. A deliberate public
display/device principal remains deferred. This transport is proven only with existing authenticated
principals and permission checks, and no channel becomes public as a shortcut.
### Lifecycle and cleanup
Only the exact eligible module set that successfully starts publishes channels. Disablement and
dependency blocking therefore remove realtime the same structural way they remove HTTP and UI: the
module is never imported or mounted. There is no hot unload; lifecycle changes remain
restart-required.
Before normal stop or rollback invokes a module's stop hook, Core deactivates that module's publisher.
Deactivation happens per module in reverse dependency order: a dependency remains operational while
its dependents finish stopping, but a stale captured publisher fails with a fixed runtime error once
its own stop begins. Browser cancellation, transport failure, module stop, startup rollback, server
shutdown, and connection-age expiry all remove subscriptions idempotently.
Server shutdown closes realtime streams before draining the HTTP listener. Otherwise a healthy
long-lived stream would keep the drain open until its deadline and turn ordinary shutdown into a
forced failure. Publishers remain active for finite requests already in flight; module stop hooks run
only after the listener has drained, and Core then deactivates each publisher immediately before its
own hook. This preserves both a command's publish-after-commit path and the guarantee that no ordinary
request handler runs concurrently with stop.
### Proxy and keepalive behavior
Ekklesia sends `text/event-stream`, `Cache-Control: no-store`, disables common reverse-proxy response
buffering where supported, and emits host-owned comment keepalives while a stream is idle. Keepalives
do not enter module code, consume a module sequence, or mutate state. Deployments and reverse proxies
must pass streaming responses incrementally and allow the configured idle interval; there is no
Caddy- or Cloudflare-specific API in the contract.
### Object delivery is unchanged
Realtime carries small state, logical identifiers, metadata, and — when a module intentionally
includes one — an opaque delivery URL. It never carries image, video, or audio bytes. Core transports
the module's serialized payload unchanged; it does not inspect object keys or call `createReadUrl` for
each publication, subscriber, reconnect, or keepalive.
That preserves ADR 0031's byte-stable/quantized URLs and browser cache keys. A module deliberately
refreshes an expiring URL by creating it once and publishing new state. Realtime never re-signs an
unchanged URL behind the module's back.
### Realtime is not Companion's global bus
A future Companion module may observe deliberately exported module capabilities and invoke explicit
module actions, and may bridge supported external protocols. It does not gain privileged access to
every module's browser stream. Browser realtime and external automation/control are separate
concerns; this contract has no cross-module subscription seam to repurpose.
## Consequences
The first real state consumer gets immediate useful state, ordered live updates, automatic browser
reconnect, permission enforcement, bounded memory, and deterministic cleanup without owning a port or
transport server. Modules with no realtime declarations behave exactly as before.
The contract deliberately loses intermediate states for a slow or disconnected client. Modules must
publish authoritative state, not use this channel as an audit log, work queue, payment ledger, or sole
record of a domain transition. A future event-oriented or durable facility would be a separate design.
SSE is one-way. Browser commands remain ordinary authenticated module HTTP, retaining its existing
body limits, trusted-origin checks, and request/response semantics. A demonstrated bidirectional need
may justify WebSocket later, without renaming the module-facing API.
This slice does not add the Announcements module, MQTT, Companion, a kiosk/display host, a public or
device principal, a standalone host, hot lifecycle changes, multi-node fanout, durable history, or a
message broker.
+1
View File
@@ -40,3 +40,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0029: Module configuration is manifest-declared, host-resolved, and secret-free at rest](0029-module-configuration-and-secrets.md) — Accepted
- [0030: Modules declare logical buckets; hosts provide object storage](0030-host-provided-module-object-storage.md) — Accepted
- [0031: Browsers fetch objects directly, through opaque temporary read URLs](0031-direct-object-delivery.md) — Accepted
- [0032: Native module realtime is scoped snapshot state over SSE](0032-native-module-realtime-transport.md) — Accepted
+44 -11
View File
@@ -70,7 +70,7 @@ Every module-facing contract in this platform is written so that **full Ekklesia
thing that can satisfy it**. The intended second implementation is a reusable single-module
standalone host — conceptually `@ekklesia/standalone-host` — which loads exactly one module and
supplies the same host contracts Ekklesia does: configuration, secrets, object storage, HTTP, module
lifecycle, and later realtime and display/UI hosting.
lifecycle, and realtime, with display/kiosk hosting still later.
**No such package exists yet, and this pass does not create one.** It is recorded here because it is
what several contract decisions are *for*, and because a change that quietly makes it unimplementable
@@ -84,6 +84,12 @@ the scoped facade, binding resolution, delivery URLs and their bounds, lifecycle
host-neutral, and a standalone host implements one interface to get all of it, including direct
browser delivery ([ADR 0031](decisions/0031-direct-object-delivery.md)).
Realtime is the second worked example. A module declares local state channels and publishes opaque
JSON through scoped contracts; its browser UI subscribes through another scoped contract. Neither
side names SSE, EventSource, a platform URL, an origin, a cookie, or a module namespace. Ekklesia's
host adapter uses permission-gated SSE, while a standalone host may choose another concrete
transport without changing the module. See [ADR 0032](decisions/0032-native-module-realtime-transport.md).
### Storage servers are shared infrastructure, never per-module
- A module hosted under Ekklesia uses **Ekklesia's** storage service, through its own bindings.
@@ -280,17 +286,17 @@ by its own esbuild script, that discovers installed module UIs and mounts them.
The missing piece was discovery. `GET /api/v1/module-ui` — the collection root of the same prefix the
artifact route already owns — returns, per publishable module, its id and version, the server-produced
artifact URL, the server-produced root of the module's own API, and the subset of that module's
declared permissions the caller actually holds. Nothing else: no filesystem paths, and no display or
navigation metadata, since ADR 0021 keeps Dashboard grouping an administrator's presentation
configuration. The module set is the publication registry, never a rescan, so the inventory and the
artifact route cannot disagree. It requires `core.modules.ui.read` (baseline `authenticated`) because
it is precisely the enumeration ADR 0025 withheld from the anonymous artifact route.
artifact URL, the server-produced roots of the module's own HTTP and realtime APIs, and the subset of
that module's declared permissions the caller actually holds. Nothing else: no filesystem paths, and
no display or navigation metadata, since ADR 0021 keeps Dashboard grouping an administrator's
presentation configuration. The module set is the publication registry, never a rescan, so the
inventory and artifact route cannot disagree. It requires `core.modules.ui.read` (baseline
`authenticated`) because it is precisely the enumeration ADR 0025 withheld from the anonymous
artifact route. ADR 0032 later extended ADR 0026's original descriptor with the realtime root.
Both URLs being server-produced is the load-bearing part. `moduleUiArtifactUrl` and the newly exported
`moduleApiBaseUrl` are each the single construction of their URL shape, so the browser is a consumer
of both — it never rebuilds `/api/v1/modules/<id>`, the prefix ADR 0022 forbids a module UI from
spelling, and neither prefix is pinned by having a second author.
All three URLs being server-produced is load-bearing. `moduleUiArtifactUrl`, `moduleApiBaseUrl`, and
`moduleRealtimeBaseUrl` are each the single construction of their URL shape, so the browser consumes
them rather than rebuilding a platform module prefix, and none is pinned by having a second author.
The shell builds a `ModuleUiHostContext` entirely from server-derived data, resolves a module's
relative API calls onto its own API root (refusing anything that escapes it), and hands the artifact
@@ -438,3 +444,30 @@ configures nothing. A reverse proxy in front of storage must pass the original `
unchanged; it may terminate TLS, because the scheme is not signed. Streaming through a module's own
route remains available for responses that must be computed or authorized per byte, and signed
**write** URLs are deliberately not offered.
Phase 7A pass 5 adds native module realtime
([ADR 0032](decisions/0032-native-module-realtime-transport.md)). Registration declares a bounded,
module-local channel and one permission owned by that module; live runtime and HTTP contexts receive
only that module's publisher. Every channel must publish an authoritative initial state before
startup completes. Core serializes opaque JSON once, assigns a process-local per-channel sequence,
and atomically installs each subscriber with the latest snapshot, eliminating a GET-then-subscribe
race. It retains only the latest channel state and one coalescing pending update per client, so a
slow browser never blocks module work or creates an unbounded queue. There is no history, replay,
broker namespace, or durable state.
Ekklesia's concrete host transport is one-way Server-Sent Events because the demonstrated
Announcements consumer sends complete state server-to-browser while browser commands already fit
authenticated module HTTP. The channel permission is evaluated through the existing cookie
principal and authorization system before allocation; the host closes streams after five minutes so
reconnect re-evaluates that authority. Native EventSource owns recoverable reconnects, each of which
starts with a fresh snapshot. The wire sends 15-second comment keepalives, disables common proxy
buffering, carries no query token or replay id, and never exposes transport objects through the
module UI SDK.
Shutdown closes long-lived subscriptions before finite HTTP drain, while leaving publishers usable
for requests already in flight. Reverse-order stop then deactivates each module's publisher
immediately before that module's own stop hook, so dependencies remain operational while dependents
clean up and captured publishers cannot outlive their module. Direct object-delivery URLs remain
opaque payload strings and are never re-signed per publication or subscriber. MQTT remains an
external-control concern for a future Companion boundary, not an internal module bus. Display/kiosk
hosting and its public/device principal are explicitly not part of this pass.
+97 -22
View File
@@ -76,7 +76,8 @@ No real values are shown here — configure per environment and never commit the
| `GET /api/v1/principal` | Ekklesia | Returns the minimal `EkklesiaPrincipal` contract as JSON. `Cache-Control: no-store`. |
| `GET\|POST /api/v1/grants*` | Ekklesia | Explicit authorization grant management, gated by `core.authorization.grants.manage`. Listing is bounded keyset pagination; creation is unscoped-only and conflicts (`409`) on an equivalent effective grant; revocation clears every unrevoked equivalent. See [Authorization](authorization.md#explicit-grant-http-api). |
| `/api/v1/modules/<module-id>/*` | Installed modules | Every route an installed module registered during its `register` hook, gated by that route's own declared permission. See [Module HTTP routes](#module-http-routes). |
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's API lives, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
| `GET /api/v1/module-realtime/<module-id>/<channel>` | Installed modules | Snapshot-first Server-Sent Events for one registered module-local channel, gated by that channel's declared permission. See [Module realtime](#module-realtime). |
| `GET /api/v1/module-ui` | Ekklesia | The browser host's module UI inventory: which module UIs may be mounted, where to fetch each, where each module's HTTP and realtime roots live, and the viewer's granted permissions for it. Requires `core.modules.ui.read` (baseline `authenticated`). See [Module UI inventory](#module-ui-inventory). |
| `GET /api/v1/dashboard/navigation` | Ekklesia | The resolved Dashboard navigation: the same mountable modules, grouped, labelled, and ordered as this installation has configured them. Requires `core.modules.ui.read`. See [Dashboard navigation](#dashboard-navigation). |
| `PUT\|DELETE /api/v1/dashboard/navigation/{groups,placements}/*` | Ekklesia | Organize the Dashboard: create/rename/order/delete a group, and place a module. Requires `core.dashboard.navigation.manage` (baseline `restricted`) and a trusted `Origin`. See [Dashboard navigation](#dashboard-navigation). |
| `GET /api/v1/module-ui/<module-id>/<version>` | Ekklesia | Publishes an installed module's validated `entrypoints.ui` artifact at a browser-loadable URL. No authentication; only the declared artifact, never another file. See [Module UI asset publication](#module-ui-asset-publication). |
@@ -400,9 +401,9 @@ account.
6. Start the module runtime from the eligible subset (`startDiscoveredModuleRuntime`) — modules are
never rediscovered for this step — passing it the services object every module's `start`/`stop`
hook receives.
7. Create the Hono application, mounting the started module runtime's published routes, publishing
UI for that same eligible set, and passing the identical services object to every module HTTP
handler.
7. Create the Hono application, mounting the started module runtime's HTTP routes and realtime
channels, publishing UI for that same eligible set, and passing the identical scoped runtime
objects to module HTTP handlers.
8. Bind the HTTP server.
Discovery and permission composition happen before any pool, Better Auth instance, module
@@ -779,13 +780,12 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
check and before permission evaluation — `0` means no body is accepted at all. An oversized
request receives the platform's stable `413 {"code":"REQUEST_BODY_TOO_LARGE"}` and never reaches
authorization, the handler, or the module database.
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and
body all behave normally, and the full query string remains available) plus a frozen context of
exactly `moduleId`, the authorized `EkklesiaPrincipal`, frozen, Hono-decoded path parameters (an
empty frozen object for a route with none), and `services` — the same frozen
`ModuleRuntimeServices` object the module's own `start`/`stop` hooks received (see
[Module database access](#module-database-access)) — never a Hono `Context`, the authorization
store, storage, jobs, configuration, or secrets.
- **Handler input.** A handler receives the raw Fetch `Request` (so `Request.url`, headers, and body
all behave normally, and the full query string remains available) plus a frozen context of exactly
`moduleId`, the authorized `EkklesiaPrincipal`, frozen Hono-decoded path parameters (an empty
frozen object for a route with none), `services`, and the module's scoped configuration, storage,
and realtime publisher. It never receives a Hono `Context`, authorization/grant stores, jobs, or
another module's scoped objects. Core does not serialize configuration or secrets into responses.
- **Handler failure.** A handler that throws, rejects, or returns anything other than a `Response`
becomes a fixed `500 {"code":"MODULE_HANDLER_FAILED"}` with `Cache-Control: no-store` — never the
caught value, a stack, the route's path or permission id, or the module id. A `Response` a
@@ -802,6 +802,69 @@ are mounted under `/api/v1/modules/<module-id>`, in the module runtime's publish
registry: installing, removing, or editing a module's routes takes effect only on the next API
restart. Removing a module makes its former routes return the ordinary `404 {"code":"NOT_FOUND"}`.
## Module realtime
An eligible module may declare a module-local state channel during `register`. Ekklesia mounts one
exact route for each successfully started declaration:
```text
GET /api/v1/module-realtime/<module-id>/<channel>
```
The path is host-private. Module UI receives the module's server-produced realtime base in its mount
descriptor and calls `context.realtime.subscribe("state", handlers)`; it never builds this URL,
chooses a module id, supplies a permission, or sees an EventSource. Browser commands remain ordinary
module HTTP requests, retaining their body, origin, authorization, and response semantics.
The response is a Server-Sent Events stream. It begins with exactly one authoritative `snapshot`
event and then carries `update` events:
```text
event: snapshot
data: {"sequence":1,"payload":{"mode":"on","currentSlide":0}}
event: update
data: {"sequence":2,"payload":{"mode":"on","currentSlide":1}}
```
Core atomically captures the latest state while installing the subscriber, so there is no separate
GET-then-subscribe gap. Sequence is monotonically increasing per channel for this process only; there
is no `id` field, `Last-Event-ID` replay, durable history, or exactly-once promise. A reconnect is a
new subscription and begins with the latest snapshot. Slow clients retain only the newest pending
update, so sequence gaps are expected and intermediate states may be coalesced.
- **Authentication and authorization.** The SSE GET uses the existing session cookie and the same
principal resolver, permission registry, and evaluator as module HTTP. The channel's declaration
fixes the permission before any request exists. Permission evaluation completes before a Core
subscriber is allocated. There is no query-string token or second realtime auth system.
- **Reauthorization.** A stream has a host-owned maximum age of five minutes. The host closes it at
that boundary; native EventSource reconnects and the ordinary cookie and permission checks run
again. Grant revocation, grant expiry, and session changes therefore do not leave an accepted
stream authorized forever.
- **Bounds.** Core accepts at most 16 channels per module, 100 subscribers per channel, and 64 KiB
of serialized JSON per publication. Publishing never awaits a browser. One latest snapshot and
one pending update per subscriber are the only state retained.
- **Transport behavior.** Responses use `text/event-stream; charset=utf-8`, `Cache-Control:
no-store`, `Vary: Origin, Cookie`, `X-Accel-Buffering: no`, and a 15-second comment keepalive.
Keepalives consume no module sequence and never reach module UI. A reverse proxy must forward
streaming bodies incrementally and permit that idle interval. No proxy product is part of the
module contract.
- **Methods and errors.** Hono normally lets `HEAD` reuse GET handlers; this route explicitly returns
`405` before allocating a stream because discarding an infinite body could leak a subscriber.
Subscriber exhaustion is the fixed `503 {"code":"REALTIME_SUBSCRIBER_LIMIT"}`; other stopped or
unavailable runtime states are `503 {"code":"REALTIME_UNAVAILABLE"}`. Raw errors, paths, URLs,
payloads, permissions, secrets, and stacks are never reflected.
- **Lifecycle.** Disabled and blocked modules have no route because their entrypoint never runs and
their channel is never mounted. Shutdown first refuses and closes realtime subscriptions so they
cannot hold HTTP drain open. Ordinary in-flight module HTTP handlers may finish and publish during
that drain. Core then stops modules in reverse dependency order, deactivating each module's scoped
publisher immediately before its own stop hook; a captured publisher fails predictably afterward.
Realtime carries opaque module state. If a module includes a direct object-delivery URL, the exact
serialized string passes through unchanged: Core neither re-signs it per subscriber nor extends its
expiry. Image, video, and audio bytes remain object-storage traffic, never realtime payloads. See
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md).
## Dashboard navigation
```text
@@ -829,6 +892,7 @@ How this installation presents its mountable modules
"version": "1.0.0",
"ui": { "url": "/api/v1/module-ui/livestream/1.0.0" },
"api": { "baseUrl": "/api/v1/modules/livestream" },
"realtime": { "baseUrl": "/api/v1/module-realtime/livestream" },
"permissions": ["livestream.streams.read"]
}
}
@@ -908,21 +972,22 @@ root of the same resource the artifact route below serves: one prefix, two repre
"version": "0.1.0",
"ui": { "url": "/api/v1/module-ui/example/0.1.0" },
"api": { "baseUrl": "/api/v1/modules/example" },
"realtime": { "baseUrl": "/api/v1/module-realtime/example" },
"permissions": ["example.records.read"]
}
]
}
```
- **Five fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
- **Six fields, each because a `ModuleUiHostContext` needs it.** No filesystem path, package root,
package name, server/schema/migrations entrypoint, capability, dependency, or storage bucket. And
no `name`, `icon`, `description`, `category`, `folder`, or `order`: those are navigation metadata,
and [ADR 0021](../architecture/decisions/0021-modules-as-hostable-products.md) keeps Dashboard
grouping an administrator's presentation configuration rather than a module declaration.
- **Both URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl` and `api.baseUrl` from
`moduleApiBaseUrl`, so each URL shape is constructed in exactly one place in the repository. A
browser never rebuilds `/api/v1/module-ui/<id>/<version>` or `/api/v1/modules/<id>`, which is what
lets either change without touching host or module code.
- **All three URLs are server-produced.** `ui.url` comes from `moduleUiArtifactUrl`, `api.baseUrl`
from `moduleApiBaseUrl`, and `realtime.baseUrl` from `moduleRealtimeBaseUrl`, so each URL shape is
constructed in exactly one place in the repository. A browser never rebuilds any platform module
prefix, which lets one change without touching host or module code.
- **The module set is the publication registry**, not a rescan: modules that survived discovery and
declare `entrypoints.ui`, captured after the module runtime started. A module with no UI is absent.
Order is the registry's own deterministic discovery order.
@@ -1114,15 +1179,25 @@ safe, and the network boundary remains the deployment's own responsibility. See
executable entry point and tests) returns `{app, server, shutdown}`. `shutdown()` memoizes one
shared promise the first time it is called: every caller, concurrent or sequential, awaits the
exact same operation, and that operation runs exactly once no matter how many times `shutdown()`
is called. In order: it stops the HTTP listener from accepting new connections and lets in-flight
requests finish; then stops the module runtime (so no module resource stops while a route handler
might still be in flight) — a module's `stop` hook may still use `services.database` here, since
the pool has not yet closed; then closes the one shared PostgreSQL pool (identity, authorization,
and the module database facade all read from it — see
is called. In order: it refuses new realtime subscribers and closes every existing stream; then
stops the HTTP listener from accepting new connections and lets finite in-flight requests finish;
then stops the module runtime in reverse dependency order; then closes storage and the one shared
PostgreSQL pool (identity, authorization, and the module database facade all read from it — see
[Authorization](authorization.md#shared-database-composition)). Unlike a startup-failure cleanup
path (below), a module-stop failure here is not swallowed — it is a genuine result a caller should
observe — but pool cleanup always still runs first, so a stop failure never leaves the pool open,
and no module code can run against it afterward. The executable entry point's `SIGINT`/`SIGTERM`
and no module code can run against it afterward.
Closing streams and disabling publishers are deliberately separate. Streams are long-lived HTTP
responses and would otherwise force every healthy shutdown to its drain deadline. Publishers remain
active during the finite HTTP drain so an already-running module command can commit and publish its
result without turning a successful request into a false `500`. As reverse-order module stop reaches
each module, Core disables that module's publisher immediately before its stop hook; dependencies
remain live while their dependents stop, and a stale publisher fails predictably once its own module
begins stopping. A module's `stop` hook may still use `services.database` and storage here, since
those providers have not yet closed.
The executable entry point's `SIGINT`/`SIGTERM`
wiring is extracted into `installShutdownSignalHandlers(running, exit)` — the same function production
composition uses, but independently testable with a fake `running`/`exit` so a test can trigger "a
signal arrived" without sending a real process signal or invoking the real `process.exit`.
+35 -1
View File
@@ -7,7 +7,9 @@ document's guarantees.
Related: [ADR 0006](../architecture/decisions/0006-versioned-capabilities-and-compiled-module-runtime.md)
(compiled module runtime), [ADR 0016](../architecture/decisions/0016-module-database-access.md)
(module database access), [ADR 0020](../architecture/decisions/0020-module-runtime-and-migration-execution-hardening.md)
(module routing/lifecycle and migration execution), and
(module routing/lifecycle and migration execution),
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md)
(native module realtime), and
[Module development](../module-development/README.md).
## Installed modules are trusted compiled backend code
@@ -44,6 +46,7 @@ cooperative. None of them constrains a module that is not:
| `ekklesia.module.json` manifest and `manifestVersion` | Declared compatibility and dependency ordering |
| Permission declarations and the composed registry | Which permissions *this* module's own routes may name, checked when Core mounts them |
| Route validation, overlap rejection, body limits | Deterministic, unambiguous HTTP routing and request bounds |
| Scoped realtime channels, publishers, and subscriptions | Correct module-local state wiring, permission enforcement, and finite fanout — not process isolation |
| `Object.freeze` on contexts and services | Prevents accidental mutation; a module can still reach the underlying objects |
| The `ModuleDatabase` facade | A typed, ergonomic surface — not a privilege reduction, since `execute` is arbitrary SQL |
| Capability registry | Explicit inter-module wiring, not access control |
@@ -316,6 +319,37 @@ see the URL it is fetching.
an unsigned request for the same object is refused, and an integration test asserts it so that a
future change cannot "fix" delivery by opening a bucket up.
### Realtime is permission-gated state delivery, not a trust boundary
A module may declare a local realtime channel protected by one of its own manifest permissions. Core
closes both the server publisher and browser subscription over that module id: neither API accepts a
caller-supplied module namespace, global topic, wildcard, broker address, origin, URL, permission, or
credential. This prevents accidental cross-module wiring by a cooperating module. It does not
sandbox hostile installed code; trusted same-process backend code could bypass the facade, and
trusted same-realm UI could call browser globals directly.
Ekklesia's SSE endpoint resolves the ordinary session cookie and evaluates the channel's fixed
permission before allocating a subscriber. There is no query-string token and no second realtime
authentication path. A stream is closed after five minutes so native EventSource reconnects and
session, grant-expiry, revocation, and baseline policy are evaluated again; authorization is not
continuously queried between those boundaries. Disabled and dependency-blocked modules have no
stream route at all. Public or device/display identity remains deliberately unsolved, so no channel
is made anonymous merely to support a future kiosk.
Fanout is finite: 16 channels per module, 100 subscribers per channel, 64 KiB of validated serialized
JSON per publication, one latest snapshot per channel, and one coalescing pending slot per client.
A slow browser cannot block a publisher or create an unbounded queue. The endpoint rejects implicit
`HEAD` before allocation, disconnect/cancel closes its Core subscription, and server shutdown closes
streams before finite HTTP drain. Each scoped publisher becomes inert immediately before its own
module's stop hook.
Core never adds configuration, secret values, database values, storage credentials, or delivery URLs
to a realtime document. It transports only what trusted module code explicitly publishes. A module
must therefore treat realtime like any other authorized response: include only data every principal
holding the channel permission may receive. If it intentionally includes a signed delivery URL, that
URL remains a bearer capability and is forwarded byte-for-byte; fixed transport errors never reflect
payloads, URLs, permissions, runtime exceptions, paths, or stacks.
### Client-side permission checks are not a security boundary
A host may tell a module UI what permissions it believes the viewer holds
+28
View File
@@ -247,6 +247,34 @@ cannot appear unnoticed. See
[ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md) and
[ADR 0031](../architecture/decisions/0031-direct-object-delivery.md).
Phase 7A pass 5 adds realtime evidence at every boundary
([ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md)). Contract and Core
unit suites prove channel grammar, module/permission scoping, staged registration, mandatory initial
snapshots, atomic snapshot-plus-subscribe, sequence ordering, payload/container limits, latest-state
coalescing, subscriber limits, exact shutdown behavior, per-module publisher deactivation, sanitized
errors, and that modules declaring no realtime continue unchanged. Browser SDK/shell suites prove
that the public contract has no server dependency or transport vocabulary, descriptors and local
channels are strictly contained, credentials are host-owned, reconnect requires a fresh snapshot,
terminal EventSource failure cleans up listeners, and explicit unmount is idempotent.
`apps/api/tests/module-realtime-routes.test.ts` exercises permission-before-allocation, the
snapshot/update wire envelope, lack of replay ids, stream headers, comment keepalives, forced
five-minute reauthorization through a short test-only override, shutdown closure, sanitized
subscriber failures, and the explicit `HEAD` refusal that prevents an infinite discarded body from
retaining a subscriber. `apps/api/tests/module-realtime-live.test.ts` binds an actual Node
HTTP server on an OS-assigned loopback port and drives an Announcements-shaped module through real
discovery, registration, startup, two live SSE clients, fanout, disconnect cleanup, reconnect with a
fresh latest snapshot, and shutdown. Every bind, fetch, stream read, cancellation, cleanup poll, and
listener close is bounded; the test also pins an opaque signed delivery URL byte-for-byte. It uses no
PostgreSQL or object-store double because neither belongs in realtime transport.
The live PostgreSQL API suite adds lifecycle composition evidence: an enabled module's realtime route
is live while disabled and dependency-blocked modules have no route, just like their HTTP/UI
surfaces. Its real SSE fetch/read/cancel path is bounded and migrations remain independent of runtime
enablement. During realtime development, run the focused live Node suite repeatedly as well as
`pnpm verify` and `pnpm test:migrations:integration`; consecutive runs are specifically intended to
catch retained timers, listeners, subscribers, or loopback servers.
### Which kind of timing proof lives where
Pass 4 introduced several deadlines, and their tests deliberately differ in kind:
+95 -11
View File
@@ -248,7 +248,7 @@ independent.
### What the host gives you
`mount` receives exactly four members, and nothing else is available:
`mount` receives exactly five members, and nothing else is available:
| Member | What it is |
| --- | --- |
@@ -256,6 +256,7 @@ independent.
| `api` | `request(path, init)` against **your own** module API — see below |
| `principal` | Who is viewing: `ModuleUiPrincipal`, either `{ kind: "anonymous" }` or `{ kind: "authenticated", userId }` |
| `permissions` | `has(permissionId)` — a presentation hint, never a security check |
| `realtime` | `subscribe(channel, handlers)` against **your own** module's live state — see [Realtime state](#realtime-state) |
Navigation, host theme and design tokens, module configuration, organization context, and
diagnostics reporting are **not** in the contract yet. They need designs that do not exist, and a
@@ -579,10 +580,12 @@ export default defineModuleRuntime({
ID and version are intentional build-artifact identity markers and must match the manifest. Core
rejects missing, duplicate, or undeclared capability registrations. A module may resolve only its
own capabilities and the capability dependencies declared in its manifest. The `register` context
is deliberately limited to `moduleId`, capability access, and the HTTP route registrar (below) —
never `services`. `start` and `stop` share one context shape and both additionally receive
`services` — the same frozen `ModuleRuntimeServices` object for the lifetime of one running API,
currently exposing only `database` — see [Database access](#database-access).
contains `moduleId`, capability access, the frozen configuration snapshot, and the HTTP and realtime
declaration registrars — never live `services` or storage. `start` and `stop` share one context shape:
capability access, configuration, `services`, storage, and a module-scoped realtime publisher. The
same scoped objects are also supplied to the module's HTTP handlers; see
[Database access](#database-access), [Object storage](#object-storage), and
[Realtime state](#realtime-state).
Entrypoint imports must have no top-level side effects. Modules are trusted backend plugins, and ESM
evaluation cannot be rolled back. Keep registration declarative; open resources in `start` and
@@ -734,12 +737,12 @@ method and path (or two paths that are identical once every parameter is treated
placeholder, such as `/events/:eventId` and `/events/:id` under `GET`) fails as a duplicate route.
The handler receives the standard `Request` plus a context of exactly `moduleId`, the caller's
already-authorized `EkklesiaPrincipal`, the immutable decoded path parameters, and `services` — the
same frozen `ModuleRuntimeServices` object the module's own `start`/`stop` hooks receive, currently
exposing only `database` (see [Database access](#database-access)) — never an authorization store,
grant/platform-owner state, storage, jobs, configuration, secrets, a framework-specific request
context, or a raw Better Auth session. The full query string remains available through
`Request.url`; there is no separate query or body abstraction.
already-authorized `EkklesiaPrincipal`, immutable decoded path parameters, `services`, the module's
frozen configuration snapshot, scoped storage, and scoped realtime publisher. It never receives an
authorization store, grant/platform-owner state, jobs, a framework-specific request context, or a
raw Better Auth session. Secret configuration exists only when the module explicitly declared that
key and the deployment supplied it; Core never serializes it. The full query string remains
available through `Request.url`; there is no separate query or body abstraction.
Core defines, validates, and collects routes into `StartedModuleRuntime`'s immutable route
snapshot — it never invokes a handler itself. `apps/api` mounts that exact snapshot into production
@@ -756,6 +759,87 @@ deliberately conservative, since even a public-baseline handler still receives a
principal and may personalize its response. See [API](../development/api.md#module-http-routes) for
the full production mounting, enforcement, and error-response contract.
## Realtime state
Realtime is a small state-delivery contract, not a broker. Declare each module-local channel during
`register`, using one permission owned by the same module:
```ts
export default defineModuleRuntime({
id: "announcements",
version: "0.1.0",
register({ realtime }) {
realtime.channel({
channel: "state",
permission: "announcements.slideshow.view",
});
},
start({ realtime }) {
realtime.publish("state", {
mode: "on",
currentSlide: 0,
previousSlide: -1,
slides: [],
});
},
});
```
Channel names are flat lowercase kebab case, at most 48 characters. There are no global topics,
wildcards, rooms, broker addresses, or module-id arguments: Core closes the registrar and publisher
over the module identity, so a cooperating module cannot accidentally publish into another module's
channel. A module may declare at most 16 channels. Each payload must be plain, finite JSON no larger
than 64 KiB after UTF-8 serialization; functions, class instances, sparse arrays, accessors,
non-finite numbers, cycles, and `toJSON` conversion are rejected with fixed errors.
Every declared channel must receive an initial publication before that module's `start` completes.
Core captures that latest state atomically when a subscriber is installed, then supplies later
publications with a monotonically increasing per-channel sequence. This avoids a separate
GET-then-subscribe race. State is ephemeral and best effort: Core keeps no history, restart resets
the sequence, and reconnect receives a fresh authoritative snapshot instead of replay. A slow client
never blocks `publish`; each subscription retains only the newest pending publication, so
intermediate updates may be coalesced. There are at most 100 subscribers per channel.
In browser UI, subscribe through the host context rather than opening a transport yourself:
```ts
const subscription = context.realtime.subscribe<SlideshowState>("state", {
onSnapshot(state, sequence) {
replaceSlideshow(state, sequence);
},
onUpdate(state, sequence) {
replaceSlideshow(state, sequence);
},
onStatus(status) {
showConnectionStatus(status);
},
onError(error) {
showSafeRealtimeError(error.code);
},
});
return () => subscription.close();
```
The public UI vocabulary contains no SSE, WebSocket, Socket.IO, URL, cookie, token, origin, or retry
setting. Ekklesia currently implements it with credentialed Server-Sent Events and native browser
reconnect; a standalone host can implement the same interface differently. The host-generated
endpoint is always scoped to this module, and the server performs the channel's normal permission
check before subscribing. Ekklesia closes each stream after five minutes so reconnect rechecks the
session and permission. Client permission hints remain presentation-only.
On shutdown, Core refuses new subscribers and closes existing subscriptions before HTTP drain, while
letting requests already in flight finish publishing. Reverse-order stop then disables each module's
publisher immediately before that module's own stop hook. Call `close()` during UI unmount as well;
it is synchronous and idempotent. Direct object-delivery URLs are opaque strings in a payload and are
forwarded unchanged.
Realtime does not make those bearer URLs live longer, refresh them, or authorize them. Public
display/device identity, durable replay, multi-node fanout, client-to-server commands, and
display/kiosk hosting remain future work. See
[ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md).
## Drizzle schema definition
A module with tables keeps the generator-facing declarations in `src/schema/tables.ts`: