Files
Ekklesia/docs/architecture/decisions/0028-module-operational-lifecycle.md
T
ekklesiaandClaude Opus 5 b9ca0d560d
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Failing after 2m20s
feat: add host-provided module object storage
`manifest.storage.buckets` has existed since manifest version 1 and meant
nothing — discovery validated it and no code read it. This turns it into a
real capability, which is what stands between Announcements and the module
system: that product currently fetches a hardcoded LAN S3 endpoint, parses
ListObjectsV2 XML by hand, and hands the browser direct object URLs.

- A module declares a logical bucket and receives a scoped facade at
  `context.storage` — list, head, get, put, delete. The contract names no
  provider: no endpoint, region, credential, or physical bucket is reachable
  from it, which is what lets the same module run under a future standalone
  host over its own or a shared S3-compatible service.
- Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the
  deployment target (ADR 0004), never an API name. Probing the pinned image
  found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's
  `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would
  have failed every second boot. A live test pins it.
- Logical → physical mapping is deterministic and collision-safe: an
  installation prefix, the module and bucket ids joined by `--` (which cannot
  occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide),
  and a digest over the full untruncated identity. Two modules declaring
  `assets` always get separate namespaces; neither can see the physical name.
- Declaring a bucket is now a requirement: with no provider configured the
  module is `blocked · storage_unavailable` through ADR 0028's existing
  resolver, propagating to required dependents. An installation with no object
  storage stays a complete deployment.
- Binding availability is not provider health. Eligibility asks whether
  storage is configured, never whether it answered; bucket creation is lazy on
  first use so a storage outage cannot stop the platform booting; a running
  module whose store blinks stays running and retries. Nothing deletes a
  bucket.
- Credentials are deployment-supplied, never persisted, never routed through
  module configuration, and never visible to modules, module UI, diagnostics,
  or API responses. Provider errors translate to a closed code set carrying no
  endpoint, bucket, request id, or XML.
- Retires ADR 0029's transitional `bucket` configuration key. No manifest
  version change. The template declares no bucket, so it still runs without S3.

See ADR 0030.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-23 08:45:53 -05:00

440 lines
26 KiB
Markdown

# 0028: Module operational lifecycle — enablement, eligibility, and what "running" means
- Status: Accepted
- Date: 2026-08-22
- Extends: [0006](0006-versioned-capabilities-and-compiled-module-runtime.md),
[0015](0015-static-module-http-routing-and-runtime-lifecycle.md),
[0025](0025-trusted-module-ui-asset-publication.md),
[0026](0026-minimal-ekklesia-module-ui-host.md),
[0027](0027-dashboard-navigation-and-flat-modules.md)
## Context
Everything upstream of this decision exists. Core discovers modules, validates manifests and
dependencies, orders them topologically, composes permissions, starts a runtime, mounts module HTTP
routes, publishes module-owned UI, and resolves a Dashboard arrangement over it.
What did not exist was any notion of a module being **on or off**. The accepted module set was
exactly "the modules found on disk at startup", and three separate places in the repository said so
in almost the same words:
> `ModuleLifecycleStage` (`@ekklesia/contracts`) names nine stages, but nothing implements a state
> machine over them: there is no installer, no enable/disable, no trust store, and no per-module
> status record.
> — ADR 0025
That vocabulary — `discovered`, `validated`, `configured`, `registered`, `migrated`, `initialized`,
`started`, `stopped`, `failed` — was aspiration written down as a contract. Nothing produced any of
those values, and nothing consumed them.
Before migrating a real product into the module system, the platform needs an answer to a small set
of operational questions that a church running this software will genuinely ask: *is this module
on? should it start? what happens to the things that depend on it? why is it not available?* This
ADR records the smallest model that answers them honestly.
Two constraints came from the repository rather than from principle.
**Startup is all-or-nothing, deliberately.** ADR 0015 and ADR 0020 built that guarantee carefully:
a module joins the rollback set the moment its `start` hook is invoked, capabilities and routes are
published only after every module registers and starts, and a failed start unwinds in reverse order
under bounded deadlines. ADR 0025 then *depends* on it — "the published set is also the running
set" — as does ADR 0026's inventory and ADR 0027's navigation.
**There is exactly one persistence precedent, and it is recent.** ADR 0027 added two Core-owned,
platform-global tables for Dashboard configuration, with the explicit rule that a module with no
stored row is still visible. That precedent shaped both the schema conventions and the default
policy below.
## Decision
### Two axes, never one status word
```text
configuration enabled / disabled persistent, administrator-owned
runtime running / disabled / blocked ephemeral, observed per boot
```
`MODULE_LIFECYCLE_STAGES` is **replaced** by these two, because a contract should name states that
something actually produces. Both are in `@ekklesia/contracts`; both are observable through
`GET /api/v1/module-lifecycle`.
They are kept apart because they answer different questions and change at different times.
Collapsing them into one word would force a lie in one direction or the other the moment an
administrator changes a setting on a process that is still running.
### `installed` means "a valid module package on disk that discovery accepted" — and nothing else
There is no installer, no installation registry, no package database, and no install source. This
pass deliberately did not create one. The honest chain today is:
```text
present on disk
↓ discovery validates manifest, package, entrypoints, dependencies, cycles
discovered
↓ platform-owned enablement configuration
enabled
↓ required-dependency resolution
eligible
↓ module runtime import, register, start
running
```
A future installer's job is to put a validated package on disk and record an enablement decision,
then reconcile on restart. Nothing in this contract assumes a source-control checkout, an image, or
a registry, so that installer remains possible without changing anything here.
### Enablement is host configuration, and a module can never see it
One Core table, `ekklesia_core_module_state` (migration `0004_module_lifecycle_state`), with
exactly `module_id`, `enabled`, and two audit timestamps. No install source, download URL, update
channel, signature, marketplace metadata, or JSON blob — those are the columns an installer would
need, and their absence is what stops one being built here by accident.
`module_id` carries **no foreign key**, for the same reason `ekklesia_core_dashboard_placement`
does not: modules are packages, not rows. A record for a module that is no longer deployed is
retained and ignored, so redeploying restores the decision rather than resetting it.
A module never declares, reads, or influences its enablement. Nothing in a manifest, a capability,
a permission, or a lifecycle hook can reach it, and a module is never told it is enabled — its host
either loads and starts it, or does not. **That is what keeps the same module implementation
reusable under a future standalone host**, whose loading decisions are its own. A module asking
"am I enabled in Ekklesia?" would be a module that only runs inside Ekklesia.
### A newly discovered module is enabled, and the default is the absence of a row
`enabled` has no column default. The platform's default is expressed by there being no record at
all, and resolved in `resolveModuleEligibility`.
This preserves the behavior every earlier phase had — deploying a module is sufficient to run it —
requires no initialization write when a module first appears, and mirrors ADR 0027's rule that a
module with no placement is still visible. Default-disabled was rejected: with no management UI, it
would mean every deployment ships a platform where nothing works until someone runs a command, and
it would make "install a module" a two-step operation for no safety gain, since the code is already
trusted the moment it is on disk (see **Trust is unchanged**).
An explicit `enabled: true` record is still meaningful and is retained: it records that somebody
looked and left it on. The read model reports `explicit` alongside `enabled` for exactly that
reason.
### Required dependencies block; optional dependencies never do
Disabling `storage` does not silently re-enable it because something needs it. It makes every
module that **requires** it ineligible, transitively:
```text
storage disabled
↓
media blocked · required_dependency_disabled · storage
↓
announcements blocked · required_dependency_blocked · media
```
Following the chain always terminates at a `..._disabled` entry, so an operator can walk from any
blocked module to the one decision that caused it.
A required **capability** dependency blocks identically, because it is one: a capability's only
provider is a module, and if that module does not run, nothing registers the capability. The reason
names the *providing module* — the thing an administrator can act on — with the capability id
alongside it.
An optional dependency never blocks. A module declaring one has already accepted that it may be
absent (discovery permits an optional dependency to not be installed at all), and "disabled" is the
same situation arriving a different way: `context.capabilities.resolve(...)` returns `undefined`,
exactly as it does when the provider was never installed. Required and optional behavior are tested
separately and deliberately.
A dependency that is **missing, version-incompatible, duplicated, or cyclic** has no eligibility
state at all. Discovery rejects the whole inventory before enablement is ever consulted, so the
dependency graph is validated in exactly one place.
Eligibility is a **filter over discovery's order, never a re-sort**. Blocking propagates along the
same edges discovery used to produce a topological order, so removing an ineligible module can
never leave a module ahead of a required dependency it still has.
### Startup atomicity is preserved, deliberately
Startup remains all-or-nothing **across the eligible set**. A module that fails to load, register,
or start still aborts API startup and unwinds through Core's existing reverse-order rollback.
This was the sharpest question in the pass, and changing it would have been a much larger decision
than this ADR should make by accident. ADR 0015 and ADR 0020 built that guarantee on purpose, and
ADR 0025, ADR 0026, and ADR 0027 all rest on "the published set is the running set". Moving to
per-module failure isolation means the permission registry, the route snapshot, the publication
registry, and the navigation view each have to be derived from a set that is only known *after*
partial failure — a real design, and a separate one.
The cost is stated plainly rather than hidden: **one broken module still takes down the whole
platform.** Enablement is what makes that recoverable, and this pass closes the one path that would
otherwise be a deadlock — the HTTP lifecycle API is served by the process a bad module takes down,
so `pnpm modules:disable -- --module <id>` exists as an out-of-band operator command, in the same
shape as `pnpm authorization:bootstrap-owner`: explicit, never HTTP-exposed, never inferred.
> **Cross-reference — see [ADR 0029](0029-module-configuration-and-secrets.md).**
> The blocked-reason vocabulary this ADR introduced has since gained three configuration reasons
> (`configuration_missing`, `configuration_invalid`, `secret_missing`), and `blocked` became a
> discriminated union so a configuration reason names a **key** where a dependency reason names a
> dependency. Everything else here is unchanged: startup atomicity, the absence of a `failed` state,
> the default-enabled policy, and the dependency propagation those new reasons flow through.
>
> **See also [ADR 0030](0030-host-provided-module-object-storage.md)**, which adds one further
> reason, `storage_unavailable`, for a module whose declared logical object-storage bucket this host
> cannot bind. It deliberately reports a *binding* problem and never a *health* one: a configured
> provider that is briefly unreachable remains an operational condition a module retries through,
> exactly as this ADR established for external resources generally.
There is consequently **no `failed` runtime state**, because no live process could ever report one.
Naming a state nothing can produce is exactly the mistake `MODULE_LIFECYCLE_STAGES` made. It is the
first thing a future isolated-startup pass adds.
### Runtime failure is never persisted as configuration
```text
enabled configuration → stored
failed to start during this boot runtime → not stored
```
A restart naturally retries every enabled module. There is no failure counter, no quarantine, no
backoff, and no "disabled because it crashed" write — a platform that disabled a module on its
behalf would be making an administrative decision nobody asked for, and the operator would have no
way to tell it apart from one they made.
### `running` means its `start` hook completed, and nothing more
It does **not** mean every external resource the module uses is reachable, now or ever. A module
that starts successfully and then cannot reach its object storage is a module with a health
problem, not a module with a lifecycle problem.
This pass adds no runtime health-probe contract, no readiness hook, and no per-module health
endpoint. A module that wants to report its own health does so through its own API and its own UI,
like any other domain concern. The limitation is recorded here rather than left to be discovered.
### Configuration changes take effect at the next start, and the API says so
There is no hot module installation, removal, or reloading (ADR 0015, unchanged), so `PUT` writes a
decision and changes nothing about the running process. The response reports all three facts:
```json
{ "id": "announcements", "version": "1.0.0",
"configured": { "enabled": false, "explicit": true },
"runtime": { "state": "running" },
"restartRequired": true }
```
Reporting `runtime: disabled` there would be a lie about a process still serving that module's
routes. `restartRequired` is computed by re-resolving eligibility against live configuration and
comparing with what the runtime **reports it actually started**, not with what eligibility
predicted — so it is symmetric (enabling a stopped module marks it too) and transitive (disabling a
dependency marks its dependents, because they really will stop).
Hot-unloading to make a button feel immediate was rejected: it would trade a real guarantee — that
a trusted module's ESM evaluation happens exactly once and can never be rolled back — for a
cosmetic one.
### Exposure follows eligibility, structurally rather than by a downstream check
`apps/api/src/server.ts` passes the **same eligible list** to `startDiscoveredModuleRuntime` and to
`createModuleUiPublicationRegistry`. There is one authoritative eligible set, and everything that
exposes a module consumes it:
- **HTTP routes.** A disabled module is never imported, so its `register` hook never runs and it
publishes no routes. There is no per-request "is this module enabled?" check anywhere, because
there is nothing mounted to check — an unmounted path gets the platform's ordinary `404`.
- **UI publication.** Its artifact URL returns the same `404` an uninstalled module does, even to a
caller who knows the exact id and version. ADR 0025's rule — *the existence of an artifact on disk
must not be what makes it servable* — now covers "discovered but disabled" as well.
- **Inventory and navigation.** Absent from `GET /api/v1/module-ui` and from
`GET /api/v1/dashboard/navigation`, by construction, since both derive from the publication
registry. Its Dashboard placement row is retained and simply renders nothing, exactly like an
uninstalled module's (ADR 0027).
- **Lifecycle hooks.** Never invoked, in either direction: shutdown runs no `stop` hook for a module
that never started.
A module whose enablement changed but whose process has not restarted stays fully exposed. It really
is still mounted and still serving; hiding it now would break a viewer mid-session for a change that
has not taken effect, and `restartRequired` is where that pending change is reported instead.
### Permission composition still spans the whole discovered inventory
Unchanged, deliberately. ADR 0015 fixed the startup order so that discovery and permission
composition complete **before any pool, Better Auth instance, or module entrypoint exists** — a
composition failure therefore leaves nothing to close. Filtering permissions by enablement would
require reading the database first and would invert that order for no safety gain.
A disabled module's permissions therefore stay registered and grantable. Nothing enforces them,
because it mounts no routes, and grants survive an enable/disable cycle intact — which is what an
administrator would expect, and lets a permission be granted ahead of enabling a module.
### Migration participation is deployment, not runtime state
```text
deployed schema/migration ownership ≠ runtime enabled state
```
Migration discovery, planning, and application continue to read the **deployed** inventory,
regardless of enablement. They are offline, Core-owned, explicitly operator-invoked commands and
must not depend on runtime configuration. A disabled module's tables must still be migrated, or
enabling it later would require a schema change in the middle of a restart. Nothing about the
planner, destructive-change approval, locking, checksums, or transactional guarantees changed.
### The URL is a sibling of the module namespace, not a child
```text
GET /api/v1/module-lifecycle
PUT /api/v1/module-lifecycle/<module-id>/enabled { "enabled": boolean }
```
The obvious `/api/v1/modules` and `/api/v1/modules/:id/enabled` are **wrong**, for the exact reason
ADR 0025 gave for `/api/v1/module-ui`. Core's route grammar accepts `enabled` as an ordinary static
segment, so a module named `announcements` may legitimately register `PUT /enabled` — mounted at
exactly `/api/v1/modules/announcements/enabled`. Publishing the platform's own lifecycle mutation
there would put Core and an installed module in one URL namespace with registration order deciding
which one answers, for a route that changes whether modules run. A sibling prefix cannot collide.
`PUT` against a module id not in the boot inventory is a `404`. Unlike an inert Dashboard placement,
an enablement record is operational configuration; accepting one for an arbitrary identifier would
make this an unbounded write surface for a delegable permission. The CLI enforces the identical rule
by running the same manifest-only discovery. *Existing* records for undeployed modules are still
retained and ignored.
The body is closed and structured — any key beyond `enabled` invalidates the whole request rather
than being dropped — matching ADR 0027's mutation contract exactly. There is no configuration blob.
### One restricted permission, gating read as well as write
`core.modules.lifecycle.manage`, baseline `restricted`. The read is not a milder version of the
write: this resource is the only place a *disabled* module, a *blocked* module, and the
required-dependency edges between them are visible at all. `core.modules.ui.read` deliberately
exposes the mountable set to every signed-in user; this exposes the parts of the installation that
are deliberately not running, which is not the same audience. A second, weaker read permission over
the same resource would be ceremony.
It is a **platform** permission, never a module's, and holding it confers no authority inside any
module. Mutations additionally require an exact trusted `Origin` and a bounded body before
authorization work begins, like every other Core mutation.
Neither response carries a filesystem path, package name, manifest path, entrypoint, caught error,
or stack. The only identifiers are module ids, capability ids, and fixed reason codes from closed
sets — all values the platform itself produced and validated.
### Startup fails closed when the configuration cannot be read
`startApi` reads enablement before importing any module entrypoint. If that read fails — an
unmigrated or unreachable database — startup **refuses**, with Core's sanitized
`MODULE_LIFECYCLE_READ_FAILED`. It cannot know what an administrator decided, and starting a module
they disabled is worse than not starting at all. This is a behavior change: previously the API would
bind a listener against an unmigrated database and fail every request instead. Nothing is created or
repaired on that path.
### Diagnostics answer "why is this module unavailable?" without the logs
`pnpm doctor` gains one check, `modules.lifecycle`, in its existing Modules section — not a second
diagnostics framework. It is skipped when discovery failed or `DATABASE_URL` is unset (configuration
lives in PostgreSQL, and reporting a guessed default is worse than skipping). A **disabled** module
is a `pass` with a detail line: it is a decision, not a problem. A **blocked** module is a
`warning`: an enabled module that cannot run is the anomaly worth surfacing. Every rendered value is
re-validated before interpolation, matching the file's existing `safeModuleDetails` discipline.
### Trust is unchanged
Installed modules remain trusted compiled code running in the API process with its PostgreSQL
authority. Enablement is an **operational** control, not a security boundary: a module's code is on
disk and could be started by any operator with filesystem access. Nothing here sandboxes, isolates,
or restricts a module that does run, and nothing here should be described as if it did.
## Validated against a real product
The Announcements repository (`The_River/Announcements`, commit `9aab486`) was walked against this
model as a reference consumer. It was not modified, and nothing about it entered Core.
It is a small Express + Socket.IO server that polls an S3-compatible bucket listing every 30
seconds, advances a slide index on a 7-second timer, broadcasts state to a full-screen React
client, and toggles a blank-screen `mode` from an MQTT topic. It has no database and no
authentication.
Three things it does made the two-axis model earn its keep.
**Its storage is unreachable regularly, and that is not a lifecycle failure.** `loadSlides()`
already catches its own fetch failure, logs, and retries on the next tick. Under this model the
module stays `running`, because its `start` hook completed; "slides could not be loaded right now"
is the module's own health, surfaced through its own UI. Had `running` been defined as "everything
it needs is reachable", a router reboot would have looked like a lifecycle event.
**Its `mode: on/off` toggle is emphatically not enablement.** It blanks the screen, is flipped
frequently by whoever is running a service, and must take effect instantly. Module enablement is an
administrative deployment decision that takes effect at the next restart. Mapping one onto the other
would have pushed a per-service operational toggle into Core's lifecycle and made it restart-based —
which is precisely the kind of conflation the separate configuration and runtime axes exist to
prevent. (Per the platform direction, that control belongs to a future Companion module speaking the
module API; Ekklesia Core will not use MQTT, and this ADR designs nothing around it.)
**Its long-lived timers are what `stop` is for.** Neither `setInterval` is ever cleared today. As a
module it must clear both in a `stop` hook that is safe after a partially completed `start` (ADR
0020) — a migration task, not a Core gap, and comfortably inside the platform's 10-second hook
deadline.
The rest of the walkthrough came out as designed: disabled means no routes, no UI, no navigation
entry, no timers, and no hooks; its migrations would still be discovered and applied; a future
required dependency being disabled would block it with a named reason; and nothing in its feature
code would ever ask whether it is enabled, so the same implementation stays deployable standalone.
## Alternatives
- **Implementing the nine `MODULE_LIFECYCLE_STAGES` names.** Rejected: most name a transition rather
than a state, several are unobservable, and shipping them would have repeated the original mistake
of publishing a vocabulary nothing produces.
- **A `failed` runtime state now.** Rejected while startup is all-or-nothing: no live process could
ever report it.
- **Per-module failure isolation in this pass.** Deferred deliberately. It is a genuine improvement
and a genuinely larger decision; four accepted ADRs depend on the current guarantee.
- **Default-disabled for a newly discovered module.** Rejected: it breaks every earlier phase's
behavior, requires an administrative step with no UI to perform it in, and buys no safety the
trusted-code model does not already concede.
- **A `disabled` column default.** Rejected: it would materialize a decision nobody made and destroy
the distinction between "at the default" and "explicitly set".
- **Persisting startup failures.** Rejected: it turns a transient runtime fact into administrative
configuration an operator did not write and cannot distinguish from their own.
- **Filtering permission composition by enablement.** Rejected: it inverts ADR 0015's deliberate
startup order for no gain, and would make grants vanish across an enable/disable cycle.
- **Filtering migration discovery by enablement.** Rejected: schema ownership is a deployment fact,
and a disabled module's tables must already exist when it is enabled.
- **Enable/disable at `/api/v1/modules/...`.** Rejected: it collides with the module route namespace,
where registration order would decide which handler answers.
- **A separate read permission.** Rejected: the read exposes what the write controls.
- **Hot enable/disable without a restart.** Rejected: ESM evaluation cannot be undone, and the
platform must not claim a module stopped when its code is still resident.
- **A generalized runtime health-probe SDK.** Rejected for this pass: "running" is deliberately a
narrow claim, and a health contract is a design of its own.
- **A module installer, package registry, or install-source columns.** Explicitly out of scope; the
schema deliberately has nowhere to put them.
## Consequences
Ekklesia has an operational lifecycle: an administrator can turn a module off, that decision
survives restarts, its dependents behave deterministically, and every surface that exposes a module
— routes, artifacts, inventory, navigation — follows the one eligible set rather than a filter
somewhere downstream.
`createApiApp` takes a required `moduleLifecycle` option, matching the pattern `dashboard`,
`moduleUi`, and `moduleRoutes` already follow. Core gains a fourth permission, which moved several
existing assertions. `startApi` now reads the database before starting modules, so it refuses to
start against an unmigrated one — stricter than before, and stated as such.
Two costs worth naming. Enablement is platform-global and will need a scope column when an
organization model exists, exactly like the Dashboard tables. And a broken module still takes the
whole platform down; the CLI makes that recoverable without a database edit, but it does not make it
pleasant, and closing it properly is the isolated-startup pass.
## Deferred
Not implemented or promised here: per-module startup failure isolation and a `failed` runtime state;
a module installer, marketplace, package registry, download/update/signature infrastructure, or
install-source metadata; hot loading, hot unloading, and filesystem watchers; a module-management
Dashboard UI or any administrator-facing affordance beyond the API and the CLI; automatic migration
application, and any coupling between enablement and migrations; a runtime health-probe or readiness
contract; module-owned configuration and secrets; per-module or per-organization scoping of
enablement; a "reset to default" operation that deletes an explicit record; and process isolation or
sandboxing of any kind.
The permission-id typing gap carried since ADR 0022 is unchanged: `permissions.has` takes a plain
string, and the wire arrays are `string[]`.