Zayin/Ekklesia
3
0
Fork 0
Code Issues 1 Pull Requests 3 Actions Packages Projects Releases 8 Wiki Activity

feat: add the module configuration and secrets contract
CI / verify (pull_request) Successful in 35s
Details
CI / infrastructure-smoke (pull_request) Successful in 21s
Details
CI / migration-smoke (pull_request) Successful in 46s
Details

Browse Source 
Modules had no way to be told anything. Announcements — the first real
product Ekklesia will absorb — has zero environment variables and compiles
a LAN IP address into its source, so this is the contract standing between
it and the module system.

- A module declares what it needs in its manifest, using six bounded types
  (string, integer, boolean, enum, url, secret) with no nesting and no
  regular expressions. Static declaration is what lets the platform validate
  configuration *before* deciding whether a module may start: an enabled
  module missing a required value is reported `blocked ·
  configuration_missing · <key>` rather than taking startup down from inside
  its own `start` hook, and Doctor reports the same while still importing no
  module code. That forces `manifestVersion: 5`, a hard cutover.
- Ordinary values persist in one Core table (migration 0005), one row per
  (module_id, key), stored as text until validated against the declaring
  module's current manifest. Nothing is coerced; an invalid stored value
  blocks the module rather than falling back to the default. A key the
  manifest no longer declares is retained and ignored.
- Secrets are never persisted. There is no encryption at rest in this
  repository, so a `secret` key is deployment-supplied and mapped in by the
  host under a name the module never sees. A secret declaration has no
  `default` field, the write API refuses secret material, and the HTTP view
  has no `value` field for one — absent by construction, not filtered.
- Modules receive a frozen `context.config` on the registration, runtime,
  and HTTP handler contexts: the same object identity, per module, beside
  `services` rather than inside it (services is deliberately one shared
  instance). No environment access, no database reference, no way to reach
  another module's values.
- Configuration decides eligibility through ADR 0028's existing resolver:
  three new blocked reasons, each with a distinct operator fix, propagating
  to required dependents through the same mechanism as a disabled module.
- Changes take effect at the next restart; both resources report
  restartRequired. New restricted permission
  `core.modules.configuration.manage`, separate from lifecycle, at the
  sibling prefix `/api/v1/module-configuration`.
- Migration participation stays independent of configuration validity.
  Endpoints and credentials for shared infrastructure stay platform
  concerns; a module declares which bucket it needs, not an S3 endpoint.

See ADR 0029.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-22 21:44:43 -05:00
co-authored by Claude Opus 5
parent fdaad58848
commit baa689b2a7
76 changed files with 6650 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/decisions/0028-module-operational-lifecycle.md
+7
View File
@@ -169,6 +169,13 @@ otherwise be a deadlock — the HTTP lifecycle API is served by the process a ba
so `pnpm modules:disable -- --module <id>` exists as an out-of-band operator command, in the same
shape as `pnpm authorization:bootstrap-owner`: explicit, never HTTP-exposed, never inferred.
> **Cross-reference — see [ADR 0029](0029-module-configuration-and-secrets.md).**
> The blocked-reason vocabulary this ADR introduced has since gained three configuration reasons
> (`configuration_missing`, `configuration_invalid`, `secret_missing`), and `blocked` became a
> discriminated union so a configuration reason names a **key** where a dependency reason names a
> dependency. Everything else here is unchanged: startup atomicity, the absence of a `failed` state,
> the default-enabled policy, and the dependency propagation those new reasons flow through.
There is consequently **no `failed` runtime state**, because no live process could ever report one.
Naming a state nothing can produce is exactly the mistake `MODULE_LIFECYCLE_STAGES` made. It is the
first thing a future isolated-startup pass adds.
docs/architecture/decisions/0029-module-configuration-and-secrets.md
+385
View File
@@ -0,0 +1,385 @@
# 0029: Module configuration is manifest-declared, host-resolved, and secret-free at rest
- Status: Accepted
- Date: 2026-08-22
- Extends: [0006](0006-versioned-capabilities-and-compiled-module-runtime.md),
[0015](0015-static-module-http-routing-and-runtime-lifecycle.md),
[0016](0016-module-database-access.md),
[0021](0021-modules-as-hostable-products.md),
[0028](0028-module-operational-lifecycle.md)
## Context
ADR 0028 gave the platform an operational lifecycle: a module is enabled or disabled, eligible or
blocked, running or not. It answered *whether* a module runs. It said nothing about *how it should
behave once it does*, and the repository had no answer at all — no configuration API, no settings
table, no schema, no concept.
That gap has a concrete shape. The Announcements product (`The_River/Announcements`, commit
`9aab486`) is the first real module Ekklesia will absorb, and it currently contains **zero
environment variables and zero configuration files**. Every deployment-specific value is a literal
in source:
```js
const PORT = 7532; // server/index.js:17
const slideSeconds = 7; // server/index.js:20
fetch("http://10.40.0.2:9000/announcements?list-type=2") // server/index.js:29
mqtt.connect("mqtt://10.40.0.2") // server/index.js:73
setInterval(() => loadSlides(), 30000) // server/index.js:127
const socket = io("http://10.40.0.2:7532"); // client/src/App.jsx:5
```
A module cannot become portable while a LAN IP address is compiled into it. But the fix is not "let
modules read `process.env`" — that would weld every module to one host's variable names and destroy
the standalone-hostability ADR 0021 exists to protect.
Two facts from the repository shaped what was possible.
**Nothing reads the environment except executables.** Every `process.env` access lives in
`apps/api/src/server.ts` or `core/src/cli/*`; reusable Core constructors take typed inputs and
`core/src/cli/database-transport-environment.ts` states the rule explicitly. Modules read the
environment nowhere, and that was worth keeping.
**There is no encryption at rest, and no key management.** No cipher, no KMS, no keyring, nothing.
The only secret-shaped values PostgreSQL holds are Better Auth's own hashed passwords and session
tokens, which Better Auth owns. That single fact decides the secrets question below, and it decides
it against the convenient answer.
## Decision
### Three categories, kept apart
```text
enabled / disabled lifecycle configuration (ADR 0028)
slideSeconds = 7 module configuration (this ADR)
currentSlide = 4, mode=off runtime / domain state (the module's own, never the platform's)
```
The lifecycle table gains no feature settings, and the configuration table holds nothing transient.
The test for the third category is not "does it change?" but **"does the module own its meaning?"**
Announcements' `mode` toggle blanks the screen, is flipped mid-service, and must take effect
instantly; it is domain state the module stores and exposes through its own route. A value that must
change without a restart is, by that fact, telling you it is domain state rather than configuration.
### Configuration is declared in the manifest, and that forces version 5
A module declares what it needs in `ekklesia.module.json`:
```json
{
"manifestVersion": 5,
"configuration": [
{ "key": "slideSeconds", "type": "integer", "default": 7, "minimum": 1, "maximum": 600 },
{ "key": "bucket", "type": "string", "required": true },
{ "key": "storageCredential", "type": "secret", "required": true }
]
}
```
Static declaration was chosen over runtime registration for one decisive reason: **the platform must
validate configuration before deciding whether a module may start.** An enabled module whose required
configuration is missing is reported `blocked · configuration_missing` — it is never imported, and it
never takes API startup down from inside its own `start` hook. Runtime registration cannot do that,
because the module's code would have to run first, and under the preserved all-or-nothing startup
model a throw at that point kills the platform. Doctor and `pnpm modules:validate` get the same
benefit: they report configuration health while remaining manifest-only, importing nothing.
Version 4's manifest object is `.strict()`, so adding a field is an incompatible shape change and the
repository's convention is a hard cutover per version (ADR 0006, 0014, 0021). Version 5 it is.
Upgrading a version 4 manifest is a one-character edit — `configuration` is optional and defaults to
empty, so a backend-only module needing no settings stays exactly as simple as it was. A separate
`ekklesia.configuration.json` was rejected: it would create a second static-metadata mechanism and a
second discovery path for something the manifest already is.
### Six types, and no seventh without a real module needing it
`string`, `integer`, `boolean`, `enum`, `url`, `secret`. No `object`, no `array`, no nesting, no
recursion, no regular expressions.
Each earns its place against a real need — a bucket name, a poll interval, a toggle, a fixed choice,
an endpoint, a credential. `number` was dropped for lack of one. Regular-expression constraints were
rejected outright: a module-supplied pattern evaluated by the platform is a ReDoS vector for no
benefit that `minLength`/`maxLength` does not already cover. `url` accepts only absolute
`http:`/`https:`, because a configuration value is frequently handed straight to `fetch` and the
difference between an endpoint and `file:///etc/passwd` must not be a parsing accident.
This is a validation vocabulary, not a form-builder DSL. A module needing structured data owns a
table for it (ADR 0016).
### `secret` is a type, not a flag — and Ekklesia never stores one
Strategy A, without apology: **ordinary configuration is persisted in PostgreSQL; secret values are
supplied by the deployment and never persisted at all.**
There is no encryption at rest in this repository. Storing credentials in plaintext because a column
is easy to add would be the single worst thing this pass could do, and dressing it up as a "secret
store" would be worse. The table has no column, flag, or convention that would make it look
acceptable, and the write API returns `CONFIGURATION_SECRET_NOT_WRITABLE` rather than accepting
material it cannot protect.
Making `secret` a distinct *type* rather than `string` + `secret: true` is what makes every
consequence structural instead of remembered:
- it has no `default` field to populate, so a checked-in default credential is unrepresentable;
- the resolver never reads one from storage, so a row planted directly in the database is ignored;
- the HTTP view has no `value` field for it, so a secret is **absent from responses by construction
rather than filtered by a branch** somebody could later delete.
The host maps a declared secret to a deployment value:
```text
EKKLESIA_MODULE_SECRET__<MODULE_ID>__<KEY>
EKKLESIA_MODULE_SECRET__ANNOUNCEMENTS__STORAGE_CREDENTIAL
```
**A module never sees that name.** It calls `config.getSecret("storageCredential")`. The mapping is
the host's, which is exactly what lets a standalone host read the same declared secret from YAML, a
file, or a secret manager without the module's feature code changing. Core's factory takes the
environment *as an argument* and never touches `process.env` itself, preserving the rule that only
executables read the environment.
There is deliberately **no** `context.secrets.getEnv("ANYTHING")`. The only reachable values are the
module's own declared keys, looked up by a source that requires both a module id and a key — so a
module cannot read `DATABASE_URL`, the identity signing secret, or another module's credential.
### A frozen snapshot, resolved once, before any module code runs
```text
stored values + deployment secrets + manifest declarations
↓ validate, apply defaults (one pass, one implementation)
frozen per-module snapshot ─────────────→ eligible?
↓
register · start · stop · HTTP handlers
```
`context.config` is a frozen object present on the registration context, the runtime context, and
every authorized HTTP handler context. The *same object identity* across all of them, so two hooks
can never disagree about a value, and no module ever queries a database for a setting.
It sits **beside** `services` rather than inside it, and that is load-bearing: `services` is
deliberately one shared instance every module receives — a property the runtime has a test for —
while configuration is per-module by definition. Putting it in `services` would have forced one of
those two properties to give.
It is available during `register` even though `services` is not. Registration must stay declarative
and I/O-free; an already-resolved in-memory snapshot reintroduces no I/O, whereas a live database
facade would.
Every accessor fails closed. An undeclared key, a declared key of a different type, and an absent
optional value are all `undefined` and deliberately indistinguishable, in the same style as
`PermissionRegistry.get`. A `required` key, or one with a `default`, is guaranteed present by the
time module code runs — the platform refuses to start the module otherwise.
### Validation rules, stated exactly
1. A stored value that validates against the current declaration wins.
2. A stored value that does **not** validate is `configuration_invalid` — never replaced by the
default. Silently falling back would run the module on a value nobody chose while the operator
believes theirs took effect.
3. Otherwise the declared `default` applies.
4. Otherwise a `required` key is `configuration_missing` (or `secret_missing`); an optional key is
simply absent.
Nothing is coerced. `"7"` is not `7`, `"yes"` is not `true`, `"07"` is not `7`. Exactly one spelling
of any value is ever accepted, so a stored row can never validate differently from the request that
created it, and two deployments can never read the same row differently.
**Schema evolution** needs no migration engine. A key a newer version adds takes its default. A key a
newer version made required blocks until set. A stored key the current manifest no longer declares is
**retained and ignored**, and reported as `unknownKeys` — deleting it would destroy an operator's
value on a temporary rollback, the same reasoning ADR 0027 applied to a Dashboard placement for an
uninstalled module.
### Configuration decides eligibility, through the lifecycle resolver that already exists
Three reasons join ADR 0028's two, each corresponding to a real check with a **different operational
fix** — the only justification for a distinct code:
| reason | fix |
| --- | --- |
| `configuration_missing` | set a value through the configuration API |
| `configuration_invalid` | correct the value |
| `secret_missing` | set an environment variable on the deployment |
`ModuleEligibility.blocked` becomes a discriminated union: a dependency reason always names a
dependency, a configuration reason always names a **key**. Neither can be accidentally omitted, and
no consumer has to handle a `blocked` record that explains nothing. A key is safe to name — it is
declared in a public manifest — while a value may be a credential or arbitrary operator text.
Validation runs *before* the dependency pass, so a module blocked by its own configuration blocks its
required dependents through the existing mechanism rather than a parallel one:
```text
storage blocked · configuration_missing · bucket
↓
media blocked · required_dependency_blocked · storage
↓
announcements blocked · required_dependency_blocked · media
```
A **disabled** module's configuration is not evaluated at all. Reporting a configuration problem for
something nobody asked to run would be noise, and claiming it is valid would assert something nothing
checked.
Startup atomicity is unchanged: still all-or-nothing across the eligible set, still no `failed`
runtime state.
### Changes take effect at the next restart, and the API says so
Modules hold a snapshot resolved at boot, so a `PUT` writes a decision and changes nothing about the
running process. Both the configuration resource and the lifecycle resource report `restartRequired`
by comparing stored state against what this process actually resolved — an observation, not a guess.
No hot reload, no watchers. A value that genuinely needs to change live is domain state, not
configuration (see the three categories above).
### One row per (module, key)
`ekklesia_core_module_configuration`: `module_id`, `key`, `value` (text), timestamps, primary key
`(module_id, key)`.
A `jsonb` blob per module was rejected for the reason ADR 0027 rejected a generic settings framework:
it would put every invariant in application code and make a single key unaddressable. `value` is
`text` holding the unparsed literal, which is what makes "a persisted value is never silently
trusted" structural — nothing can read it as a number until it has been validated against the
module's current declaration.
`key` is **data in a parameterized column, never an SQL identifier.** There is no dynamic SQL in this
domain, so an outlandish declared key can produce a strange row and nothing else. `module_id` carries
no foreign key, like the lifecycle and Dashboard tables, so a value for an undeployed module is
retained and redeploying restores it.
### `/api/v1/module-configuration/:moduleId`, a sibling prefix
The third application of ADR 0025's rule. `config` and `configuration` are ordinary lowercase static
segments under `moduleRoutePathSchema`, so a module named `announcements` may legitimately register
`PUT /config` and land on `/api/v1/modules/announcements/config`. Publishing the platform's own
mutation there would put Core and an installed module in one URL namespace with registration order
deciding which answers.
`GET` returns the declared schema, stored values, `unknownKeys`, and `restartRequired`. `PUT`
replaces the whole document with closed validation — an undeclared key, a wrong JSON type, a
constraint violation, or a secret key each fail with a distinct sanitized code and store nothing.
Constraint checking runs through **the same resolver the platform boots with**, over the exact
document about to be stored, so the API and startup can never disagree about validity.
An *incomplete* document is accepted (an operator may save partial work; the module reports
`configuration_missing` until they finish). An *invalid* one never is.
Both verbs require a new restricted permission, `core.modules.configuration.manage`, deliberately
**separate from** `core.modules.lifecycle.manage`: deciding whether a module runs and deciding how it
behaves are different authorities, and folding them together would grant authority nobody asked for
in whichever direction it was used. Mutations additionally require an exact trusted `Origin` and a
bounded body.
### Browser editing is deferred, and no secret ever reaches `ModuleUiContext`
`ModuleUiHostContext` is unchanged. A settings UI is not in this pass, and module UI receives no
configuration at all. When it does, it will receive non-secret values only — the server-only/browser
split already exists in the shape of the API, which never serves a secret to anyone.
### The object-storage boundary, stated before it is crossed
Announcements reads `http://10.40.0.2:9000/announcements`. It would be easy, and wrong, to let it
declare `storageEndpoint` and `storageAccessKey` as module configuration. That would make **every**
module separately configure S3, and the eventual platform storage service would arrive to find a
dozen competing endpoint settings already in production.
The boundary this ADR draws:
```text
module declares which bucket it needs → host supplies the storage service
```
not
```text
every module configures its own endpoint, region, and credentials
```
A storage *endpoint* and its credentials are **platform infrastructure**, and they belong to the
storage service Core does not have yet — not to Announcements. A **bucket name** is arguably module
configuration today, and is accepted as such transitionally, but note that `manifest.storage.buckets`
already declares logical buckets: when the storage service exists, the bucket binding should move
there and the configuration key should be retired. That overlap is recorded here deliberately so it
is resolved by decision rather than by accident.
## Validated against the real product
Every configuration-shaped literal in Announcements at commit `9aab486`, classified:
| value | classification | under Ekklesia |
| --- | --- | --- |
| `slideSeconds = 7` | module configuration | `integer`, default 7 |
| `30000` poll interval | module configuration | `pollIntervalSeconds`, default 30 |
| bucket `announcements` | module configuration (transitional — see above) | `string`, required |
| `10.40.0.2:9000` storage endpoint | **platform infrastructure** | future storage service, not module config |
| storage credentials | **secret** (none exist today — the bucket is read anonymously) | deployment-supplied |
| `PORT = 7532` | platform infrastructure | disappears; the host mounts routes |
| Socket.IO `origin: "*"` | platform infrastructure | Ekklesia owns CORS |
| `io("http://10.40.0.2:7532")` client URL | legacy | disappears; the host supplies the API base URL |
| `mqtt://10.40.0.2` + topic | **legacy integration to remove** | removed; a future Companion module speaks the module API |
| `mode = "on"` | **runtime/domain state** | the module's own state, not configuration |
| `currentSlide`, `slides[]` | runtime/domain state | not configuration |
| `1100` fade milliseconds | module-internal constant (coupled to CSS) | not platform configuration |
Two findings worth recording. **Announcements has no credentials at all today** — the bucket is read
anonymously over plain HTTP — so `storageCredential` is a future need, and the secret mechanism is
proven here with a fixture rather than by inventing a credential the product does not yet have. And
**`slideSeconds` being restart-required is a real operational wrinkle**: a church may want to tweak it
mid-service. The honest answer is the one above — if it must change live, it is domain state the
module owns, exactly like `mode`. This ADR does not paper over that by adding hot reload.
## Alternatives
- **Runtime/SDK schema registration.** Rejected: configuration must be validated before module code
runs, or an unconfigured module takes startup down instead of being reported blocked.
- **A separate `ekklesia.configuration.json`.** Rejected: a second static-metadata mechanism and a
second discovery path for something the manifest already is.
- **Extending manifest v4 additively.** Not available — the manifest object is `.strict()`, and the
repository's convention is a hard cutover per version.
- **Persisting secrets in PostgreSQL.** Rejected outright: there is no encryption at rest, and
plaintext credentials in a database are not made acceptable by a column named `secret`.
- **Secret *references* (`secretRef: "..."`) resolved from a vault.** Premature: there is no vault
to reference. The declared-key-plus-host-mapping model already gives a standalone host the same
freedom without inventing an indirection layer nothing implements.
- **`jsonb` document per module.** Rejected: unaddressable keys and invariants living only in
application code.
- **A live, database-backed `config.get()`.** Rejected: it would make every read fallible and
mid-request-mutable, and would reintroduce I/O into `register`.
- **Configuration inside `ModuleRuntimeServices`.** Rejected: `services` is deliberately one shared
object identity; configuration is per-module.
- **Hot configuration reload.** Rejected for the same reason ADR 0028 rejected hot enable/disable:
the platform must not claim a change took effect when the running process still holds the old one.
- **Reusing `core.modules.lifecycle.manage`.** Rejected: running a module and configuring it are
different authorities.
- **Regular-expression constraints, nested objects, arrays.** Rejected: a validation vocabulary, not
a UI DSL — and a module-supplied regex is a ReDoS vector.
- **Letting modules read `process.env` directly.** Rejected: it welds a module to one host's variable
names and destroys standalone hostability.
## Consequences
A module declares what it needs, an operator sets it, the platform validates it before anything runs,
and the module receives a frozen, host-neutral snapshot. Announcements has a clear path away from
every hardcoded value it currently contains, and none of that path requires it to know that Ekklesia
uses PostgreSQL.
`createApiApp` takes a required `moduleConfiguration` option. Core gains a fifth permission and a
sixth migration. The manifest moves to version 5, a hard cutover affecting the template and every
fixture. `startApi` now reads two tables before importing any module, sequentially rather than
concurrently so a broken deployment reports the same error code every time.
Costs worth naming. Configuration is platform-global and will need a scope column when an
organization model exists, like every other Core table. Ordinary configuration changes require a
restart, which is honest but not convenient, and modules with genuinely live settings must model them
as domain state. And there is still **no settings UI** — configuration is an API, not a screen.
## Deferred
Not implemented or promised here: a settings Dashboard UI, form generation, or any
administrator-facing affordance beyond the HTTP API; browser-visible configuration in
`ModuleUiContext`; secret persistence, a vault, encryption at rest, or key management; secret
rotation; a `secretRef` indirection; hot configuration reload or watchers; per-organization or
per-environment configuration scoping; configuration history, audit, or diff; a migration engine for
renamed keys; object storage, and the eventual move of bucket binding from configuration to
`manifest.storage.buckets`; and the Announcements migration itself.
docs/architecture/decisions/README.md
+1
View File
@@ -37,3 +37,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0026: The minimal Ekklesia module UI host](0026-minimal-ekklesia-module-ui-host.md) — Accepted
- [0027: Dashboard navigation is presentation configuration, and modules stay flat](0027-dashboard-navigation-and-flat-modules.md) — Accepted
- [0028: Module operational lifecycle — enablement, eligibility, and what "running" means](0028-module-operational-lifecycle.md) — Accepted
- [0029: Module configuration is manifest-declared, host-resolved, and secret-free at rest](0029-module-configuration-and-secrets.md) — Accepted
docs/architecture/platform-architecture.md
+27
View File
@@ -329,3 +329,30 @@ Migration discovery still spans the deployed inventory regardless of enablement,
composition still spans the whole discovered set. `running` means a module's `start` hook completed
and nothing more; there is no runtime health-probe contract. Modules remain trusted compiled code —
disabling one is an operational control, never a sandbox. No installer exists.
Phase 7A pass 2 adds the module configuration and secrets contract
([ADR 0029](decisions/0029-module-configuration-and-secrets.md)). A module declares what it needs
configured in its manifest — `manifestVersion: 5`, a hard cutover — using six types (`string`,
`integer`, `boolean`, `enum`, `url`, `secret`) with bounded constraints and no nesting. Static
declaration is what lets the platform validate configuration **before** deciding whether a module may
start, so an unconfigured module is reported `blocked · configuration_missing` rather than taking
startup down from inside its own `start` hook; discovery, `pnpm modules:validate`, and Doctor all read
it while remaining manifest-only.
Ordinary values are persisted in one Core table, one row per `(module_id, key)`, stored as unvalidated
text until checked against the declaring module's current manifest. **Secret values are never
persisted**: Ekklesia has no encryption at rest, so a `secret` key is supplied by the deployment and
mapped into the module's snapshot by the host, under a name the module never sees. A module receives a
frozen `context.config` on its registration, runtime, and HTTP handler contexts — the same object
identity across all of them — with no environment access, no database reference, and no way to reach
another module's values, which is what keeps the same module implementation runnable under a future
standalone host.
Configuration decides eligibility through ADR 0028's existing resolver rather than a parallel concept:
three new blocked reasons (`configuration_missing`, `configuration_invalid`, `secret_missing`) name
the offending key, and a configuration-blocked module blocks its required dependents exactly like a
disabled one. Changes take effect at the next restart and the API reports `restartRequired`; nothing
is hot-reloaded. Migration discovery remains independent of configuration validity, exactly as it is
independent of enabled state. Endpoints and credentials for shared infrastructure stay platform
concerns — a module declares which bucket it needs, not an S3 endpoint — leaving room for the object
storage service Core still does not have. There is no settings UI.