Ekklesia/docs/module-development/README.md

# Module development

An Ekklesia module is a backend capability package. It owns business behavior and data declarations;
it never owns components, styles, layouts, renderers, navigation, or other UI.

## Required package shape

Each immediate child of `modules/` must be an ESM workspace package with a unique package name, a
strict semantic version, and the module layer marker:

```text
modules/example/
├── drizzle/                  # when the module has migration history
│   ├── 0000_initial.sql
│   └── meta/
│       ├── 0000_snapshot.json
│       ├── _ekklesia.json
│       └── _journal.json
├── dist/schema/index.js  # when the module owns tables
├── dist/server/index.js
├── drizzle.config.ts     # when generating migrations
├── ekklesia.module.json
├── package.json
├── src/schema/index.ts   # when the module owns tables
├── src/schema/tables.ts  # pure Drizzle declarations used by Drizzle Kit
└── src/server/index.ts
```

The schema and migration paths are optional. A tableless module omits `entrypoints.schema`; a
module with no migration history omits `entrypoints.migrations` and needs no placeholder files.

```json
{
  "type": "module",
  "ekklesia": {
    "layer": "module"
  }
}
```

The package version must match the manifest and runtime-definition versions. Internal package
dependencies use `workspace:` ranges and supported public package entrypoints.

## Canonical manifest

`ekklesia.module.json` is inert and authoritative, so Core can validate every installed module
without executing module code:

```json
{
  "manifestVersion": 3,
  "id": "calendar",
  "name": "Calendar",
  "version": "0.1.0",
  "compatibility": {
    "platform": "^0.1.0",
    "moduleSdk": "^0.1.0"
  },
  "dependencies": {
    "modules": {
      "required": {
        "people": "^0.1.0"
      }
    },
    "capabilities": {
      "required": {
        "people.records": "^2.0.0"
      },
      "optional": {
        "media.items": ">=1.1.0 <2.0.0"
      }
    }
  },
  "capabilities": [
    {
      "id": "calendar.events",
      "version": "1.0.0"
    }
  ],
  "permissions": [
    {
      "id": "calendar.events.read",
      "baseline": "restricted",
      "description": "Read calendar events."
    }
  ],
  "storage": {
    "buckets": [
      {
        "id": "assets",
        "description": "Private event assets owned by Calendar."
      }
    ]
  },
  "entrypoints": {
    "server": "./dist/server/index.js",
    "schema": "./dist/schema/index.js",
    "migrations": "./drizzle"
  }
}
```

Module, capability, and permission identifiers are lowercase and namespaced. A provided capability
has its own exact semantic version because its public contract can evolve independently from the
module release. Capability dependencies use semantic-version ranges. Optional means the provider
may be absent; if it is installed, its capability version must be compatible.

`manifestVersion: 3` (`packages/contracts/src/module-manifest.ts`) is a required, exact literal — a
version 2 manifest, and any other version, fails validation outright rather than being silently
reinterpreted; there is no compatibility adapter or automatic migration between manifest versions.
Each `permissions` entry is a complete declaration, not a bare identifier string: `id` (must begin
with `<module-id>.`, exactly like a capability id), `baseline` (`public`, `authenticated`, or
`restricted` — see [Authorization](../development/authorization.md#permission-registry) for what
each baseline means to the evaluator), and a required, non-blank `description` up to 240 characters
for operators and audit review only — Core's authorization layer never reads or interprets it.
Declaring a permission here is necessary but not sufficient for it to be enforceable: the manifest
only proves the declaration is well-formed and namespaced correctly, and does not itself register
anything with Core's authorization registry (see
[Authorization](../development/authorization.md#module-permission-composition) for how Core composes
every discovered module's declarations, alongside its own, into the one platform permission
registry). A duplicate permission id within one manifest, or a permission id outside the module's
own namespace, fails validation the same way a duplicate or misnamed capability id does.

`apps/api` composes this registry once, at startup, from every module installed at that time (see
[Authorization](../development/authorization.md#api-startup-discovery-composition-and-enforcement));
installing, removing, or editing a module's manifest takes effect only on the next API restart, not
while it is running. Once installed and discovered, a module's declared permission is enforceable
and grantable through exactly the same mechanism as any Core-owned permission — the same
`requirePermission` middleware and the same `/api/v1/grants` explicit-grant HTTP API a platform
owner (or anyone they delegate `core.authorization.grants.manage` to) already uses. Mounting a
module's own HTTP routes, and creating relationship-derived grants automatically from a module's
own business logic, are both still later work; this only makes a declared permission *usable*
through Core's existing, generic surface.

Server and optional schema entrypoints must be compiled ESM `.js` or `.mjs` files. The optional
migration entrypoint is a directory. All entrypoints begin with `./`, remain inside the package,
and exist. Production never imports source TypeScript directly. Modules with no tables omit
`entrypoints.schema`; modules with no migration history omit `entrypoints.migrations`. Unknown
manifest properties are rejected, including presentation fields such as components or renderers.

Storage bucket IDs are logical names scoped to the module. Modules may request multiple buckets.
Core will later map them to deployment-specific physical S3 buckets and provide scoped access;
module code must not configure SeaweedFS or another S3 client directly.

## Runtime server definition

The source entrypoint exports one typed runtime definition and does not repeat the static manifest:

```ts
import { defineModuleRuntime } from "@ekklesia/module-sdk";

export default defineModuleRuntime({
  id: "calendar",
  version: "0.1.0",

  register({ capabilities }) {
    capabilities.provide("calendar.events", calendarEvents);
  },

  async start({ capabilities, services }) {
    const people = capabilities.resolve("people.records");
    await services.database.insert(events).values({ id: "seed", title: "First event" });
  },

  async stop() {
    // Release runtime resources here.
  }
});
```

ID and version are intentional build-artifact identity markers and must match the manifest. Core
rejects missing, duplicate, or undeclared capability registrations. A module may resolve only its
own capabilities and the capability dependencies declared in its manifest. The `register` context
is deliberately limited to `moduleId`, capability access, and the HTTP route registrar (below) —
never `services`. `start` and `stop` share one context shape and both additionally receive
`services` — the same frozen `ModuleRuntimeServices` object for the lifetime of one running API,
currently exposing only `database` — see [Database access](#database-access).

Entrypoint imports must have no top-level side effects. Modules are trusted backend plugins, and ESM
evaluation cannot be rolled back. Keep registration declarative; open resources in `start` and
close them in `stop`.

## HTTP routes

A module registers plain Fetch (`Request` in, `Response` out) route handlers during `register`,
through the `http` registrar `ModuleRegistrationContext` provides alongside `capabilities`:

```ts
import { defineModuleRuntime } from "@ekklesia/module-sdk";

export default defineModuleRuntime({
  id: "calendar",
  version: "0.1.0",

  register({ capabilities, http }) {
    capabilities.provide("calendar.events", calendarEvents);

    http.route({
      method: "GET",
      path: "/events/:eventId",
      permission: "calendar.events.read",
      async handler(request, context) {
        return Response.json({
          eventId: context.params.eventId,
          principal: context.principal.kind,
        });
      },
    });
  },
});
```

A route definition has exactly four fields — `method`, `path`, `permission`, `handler` — and nothing
else; an unknown field (a baseline, a mount prefix, CORS/origin policy, middleware, audit metadata,
authentication behavior, or another module's namespace) fails registration outright rather than
being ignored. `method` is one of `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, always uppercase; there is
no `OPTIONS` (the platform owns CORS preflight) and no wildcard/all-method registration.

`path` is local to the module, never the path a caller actually sends. The platform alone owns the
production mount prefix, `/api/v1/modules/<module-id>`: `"/"` is the module's own root, `"/events"`
means `/api/v1/modules/calendar/events`, and `"/events/:eventId"` is a parameterized route using the
same `:name` syntax a caller's request is matched against. A canonical path begins with `/`, a
non-root path never ends with `/`, and every segment is either a lowercase kebab-case static segment
or one `:parameterName` segment — no empty or repeated slashes, no query string or fragment, no
backslash or percent-encoding, no dot segments, and no wildcard/catch-all segment. A malformed path
is rejected outright, never silently normalized.

Every route — including a route meant to be publicly reachable — names exactly one `permission` the
owning module's own manifest declares; there is no way to register an unprotected, no-permission
route. A module cannot supply a baseline directly: the permission's trusted `public`, `authenticated`,
or `restricted` baseline always comes from the composed platform registry, exactly like any other
enforcement decision. Public access is therefore expressed by referencing a permission whose manifest
declaration has `baseline: "public"`, never by omitting a permission. A module may only reference a
permission its own manifest declares — never a Core-owned permission, and never another module's —
so it can never gate a route behind authority it does not itself own. The same local path may be
registered under more than one method, but registering the same module twice under the identical
method and path (or two paths that are identical once every parameter is treated as an anonymous
placeholder, such as `/events/:eventId` and `/events/:id` under `GET`) fails as a duplicate route.

The handler receives the standard `Request` plus a context of exactly `moduleId`, the caller's
already-authorized `EkklesiaPrincipal`, the immutable decoded path parameters, and `services` — the
same frozen `ModuleRuntimeServices` object the module's own `start`/`stop` hooks receive, currently
exposing only `database` (see [Database access](#database-access)) — never an authorization store,
grant/platform-owner state, storage, jobs, configuration, secrets, a framework-specific request
context, or a raw Better Auth session. The full query string remains available through
`Request.url`; there is no separate query or body abstraction.

Core defines, validates, and collects routes into `StartedModuleRuntime`'s immutable route
snapshot — it never invokes a handler itself. `apps/api` mounts that exact snapshot into production
at startup, under the `/api/v1/modules/<module-id>` prefix described above: every route is enforced
through the identical `requirePermission` middleware Core's own routes use, a state-changing method
(`POST`/`PUT`/`PATCH`/`DELETE`) additionally requires a trusted `Origin` before authorization ever
runs, and a handler that throws, rejects, or returns something other than a `Response` becomes a
fixed, sanitized `500 {"code":"MODULE_HANDLER_FAILED"}` — never the caught value, a stack, the
route's path, its permission id, or the module id. The platform remains the sole owner of CORS and
caching for every module route: a handler-supplied CORS-control header is stripped, and every
module response is forced to `Cache-Control: no-store` with `Vary: Origin, Cookie` (in addition to
any of the handler's own legitimate `Vary` values) until a dedicated cache-policy contract exists —
deliberately conservative, since even a public-baseline handler still receives an authorized
principal and may personalize its response. See [API](../development/api.md#module-http-routes) for
the full production mounting, enforcement, and error-response contract.

## Drizzle schema definition

A module with tables keeps the generator-facing declarations in `src/schema/tables.ts`:

```ts
import { integer, pgTableCreator, text } from "drizzle-orm/pg-core";

const moduleTable = pgTableCreator((localName) => `calendar_${localName}`);

export const events = moduleTable("events", {
  id: integer("id").primaryKey(),
  title: text("title").notNull()
});
```

The production schema entrypoint in `src/schema/index.ts` wraps those same objects with the Module
SDK ownership marker:

```ts
import { defineModuleSchema } from "@ekklesia/module-sdk";
import { events } from "./tables.js";

export { events } from "./tables.js";

export default defineModuleSchema({
  moduleId: "calendar",
  tables: { events }
});
```

The prefix callback converts the module ID to a snake-case namespace and prefixes every local table
name. Here the physical PostgreSQL name is `calendar_events`; `care-team` plus `members` becomes
`care_team_members`. Local names must be lowercase snake case, and the complete physical identifier
must not exceed 63 characters. The SDK's `createModuleTable` provides the same convenience when a
generator-facing pure declaration file is not needed; Core validates built metadata independently
of either helper.

Schema definitions use PostgreSQL tables in the platform-managed default/public schema. They must
be side-effect-free: do not connect to a database, read configuration, or perform I/O during import.
The `moduleId` is an ownership marker and must match the static manifest. Export physical base
tables, never query aliases.

Foreign keys may reference self or another exported table from the same module. They may not target
another module or an omitted private table. Every source column must belong to the declaring table,
and all columns in a compound target must belong to the same exported target table. Modules must
never import another module's tables, columns, relations, schema entrypoint, or migration files.
Store opaque identifiers and integrate through public capabilities, events, or APIs instead.

## Drizzle migrations

Module migrations use the stable Drizzle Kit `0.31.10` PostgreSQL format. Pin that exact version in
the module, set `migrations.prefix` to `index`, and generate from the module's own source schema:

```ts
import { defineConfig } from "drizzle-kit";

export default defineConfig({
  dialect: "postgresql",
  schema: "./src/schema/tables.ts",
  out: "./drizzle",
  migrations: { prefix: "index" },
  breakpoints: true
});
```

Keep `tables.ts` limited to Drizzle declarations so the stable generator can load it without
executing the ESM Module SDK wrapper. `src/schema/index.ts` remains the production compiled schema
entrypoint and wraps those same exported table objects with `defineModuleSchema`; tables are not
declared twice.

Run the module's `migration:generate` script with a lowercase snake-case name, review the generated
SQL and snapshot, and add the matching entry to `drizzle/meta/_ekklesia.json`. Every entry must
explicitly declare either `{"mode":"none"}` or
`{"mode":"approval-required","reason":"..."}`. Do not add an `approved` flag; approval will be
supplied by an external Core-owned provider and bound to module ID, identity, and checksum.

Migration identities are `<module-id>:<generated-tag>`. Core preserves Drizzle journal order inside
each module and dependency-first discovery order between modules. It calculates SHA-256 from the
exact SQL bytes, so changing whitespace or line endings changes the checksum. Never edit an
already-applied migration: Core compares the current exact checksum and metadata with its fixed
PostgreSQL history and refuses changed, missing, unknown, or out-of-prefix entries. Add a new forward
migration instead. Downgrade and uninstall migrations are not supported.

A module may migrate only its own objects. SQL is intentionally treated as opaque: offline
validation checks the artifact and metadata contract but cannot prove SQL ownership or classify
destructive behavior. Review those properties explicitly. Core is the only migration executor. It
follows Drizzle's exact `--> statement-breakpoint` delimiter and never naively splits semicolons or
falls back to non-transactional execution.

## Database access

`context.services.database` — available in `start`, `stop`, and every authorized route handler,
never in `register` — is a `ModuleDatabase`: a typed Drizzle query surface over exactly `select`,
`selectDistinct`, `selectDistinctOn`, `insert`, `update`, `delete`, `execute`, and `transaction`,
preserving Drizzle's own table-aware argument and result inference. It is the platform's only
supported way for module code to reach PostgreSQL:

```ts
import { defineModuleRuntime } from "@ekklesia/module-sdk";
import { events } from "./schema/tables.js";

export default defineModuleRuntime({
  id: "calendar",
  version: "0.1.0",

  register({ http }) {
    http.route({
      method: "GET",
      path: "/events",
      permission: "calendar.events.read",
      async handler(_request, context) {
        const rows = await context.services.database
          .select({ id: events.id, title: events.title })
          .from(events);
        return Response.json(rows);
      },
    });
  },
});
```

Import your module's own table objects (`src/schema/tables.js` — the same declarations
[Drizzle schema definition](#drizzle-schema-definition) already covers) exactly as you would with
an independently constructed Drizzle instance; the only difference is that `services.database`,
not a client you built yourself, executes the query. Never:

- Construct a `Pool`, a `Client`, or a Drizzle instance in module code.
- Read `DATABASE_URL` or any other connection configuration.
- Access `$client`, a session, or a dialect object — `ModuleDatabase` exposes none of them, and not
  merely at the type level: the constructed facade physically has no such property.
- Stash `services` (or `services.database`) in a module-global variable. The platform provides the
  identical object directly to `start`, `stop`, and every authorized handler; there is nothing to
  cache.
- Run or generate a migration from `register`, `start`, `stop`, or a route handler. Schema and
  migration changes remain [explicit deployment work](#drizzle-migrations) — generate, review,
  commit, and apply them the same way as any other module migration, *before* starting code that
  depends on the new shape. Nothing in discovery, registration, or startup infers a schema or
  creates a table automatically.

A handler's database error is sanitized through the same `500 {"code":"MODULE_HANDLER_FAILED"}`
boundary as any other handler failure (see [HTTP routes](#http-routes)) — there is no separate,
database-specific error contract. Catch a failure yourself if your route should return something
more specific; an uncaught one is sanitized automatically.

`ModuleDatabase` is an integration boundary for trusted, compiled backend module code, not a
sandbox: it exists so your module never has to construct or configure a PostgreSQL connection
itself, not to defend against malicious code, since a module already runs arbitrary code in this
process. It provides no tenant isolation and does not stop your own code from querying a table it
imports; `execute` in particular lets trusted code run arbitrary SQL, exactly as every other
operation already lets it construct arbitrary queries. Keep querying only your own module's tables
— the same convention [Drizzle schema definition](#drizzle-schema-definition) already establishes
for imports, just extended to runtime queries. See
[API](../development/api.md#module-database-access) for how `apps/api` composes and wires the
facade, and [ADR 0016](../architecture/decisions/0016-module-database-access.md) for the complete
design.

## Discovery and lifecycle

From the repository root, run:

```bash
pnpm install
pnpm modules:validate
pnpm migrations:validate
pnpm doctor
pnpm migrations:plan
pnpm verify
```

`pnpm modules:validate` builds the workspace and then atomically validates every installed module.
It rejects malformed manifests, duplicate IDs or package names, version mismatches, incompatible
platform or SDK ranges, missing required modules or capabilities, incompatible installed
dependencies, cycles, and missing, escaping, or non-JavaScript server or schema entrypoints.
Missing optional dependencies are allowed. Valid modules are ordered dependency-first with stable
identifier sorting.

After complete static discovery succeeds, validation loads optional schema entrypoints
dependency-first and rejects invalid definitions, unnamespaced or duplicate tables, custom
PostgreSQL schemas, and foreign keys outside the owning module. The aggregate is published only when
every schema is valid. This slice does not connect to PostgreSQL or execute migrations.

`pnpm migrations:validate` performs complete static discovery first, then reads every declared
migration directory without importing module code or connecting to PostgreSQL. It validates native
Drizzle journals and snapshot chains, Ekklesia policy metadata, ordering, inventory, paths, and
checksums atomically. Its concise report lists every installed module and migration in deterministic
order.

With `DATABASE_URL` set through the uncommitted root `.env`, `pnpm migrations:plan` performs a
read-only applied-history comparison. The plan can become stale. `pnpm migrations:apply` is a
separate, explicit database mutation: Core acquires its bounded session advisory lock, rereads and
validates history, preflights the whole destructive-approval set, and applies all pending modules in
dependency-first order in one transaction. If a later migration fails, every schema and history
change from that invocation rolls back; migrations committed by an earlier invocation remain.

An `approval-required` migration needs an approval from an external provider matching its module
ID, exact identity, and exact checksum. Module metadata cannot grant approval, and there is no
wildcard, force, or approve-all option. The default developer command supplies no approvals and
therefore refuses destructive pending migrations.

Runner behavior is unit-tested through a narrow database port. The separate
`pnpm test:migrations:integration` command now exercises the production adapter against disposable
PostgreSQL, including DDL rollback, advisory-lock contention, history behavior, approvals, and
sanitized failures. It remains outside the Docker-free `pnpm verify` gate; see the
[testing guide](../development/testing.md).

`pnpm doctor` builds first and presents the same production discovery results as a concise,
read-only dashboard. It lists discovered module IDs and versions in dependency order, reports
schema and migration counts, and shows safe migration identities with truncated checksums. Schema
discovery imports the compiled schema-definition entrypoint, so its no-I/O and no-configuration
contract still applies. Doctor does not import or start module server entrypoints, apply migrations,
or exercise runtime hooks; lifecycle behavior remains covered by automated tests. PostgreSQL
planning is skipped when `DATABASE_URL` is absent, and object storage is skipped until Core has a
production storage integration. A skip or pending migration is visible as `READY WITH WARNINGS`,
not misreported as a successful check.

Core completes discovery before importing any server entrypoint. It then loads and verifies every
runtime definition before running hooks, registers all modules dependency-first into an unpublished
staged capability registry and an unpublished staged HTTP route list side by side, and starts them in
the same order. Neither the capability registry nor the route list is published — visible on
`StartedModuleRuntime` — unless every module registers and starts successfully. Shutdown is
reverse-order. If a later module fails to start, Core stops already-started modules in reverse order
and reports any cleanup failures without leaking caught exceptions or machine paths. A failing
`start` hook must clean up its own partial work before rejecting.

## Start from the template

Copy `templates/module` to `modules/<your-module-id>` and follow its README. No central registry edit
is needed: the workspace glob, build, discovery, and runtime loading all find the new package.