Ekklesia/docs/module-development/README.md

# Module development

An Ekklesia module is a backend capability package. It owns business behavior and data declarations;
it never owns components, styles, layouts, renderers, navigation, or other UI.

## Required package shape

Each immediate child of `modules/` must be an ESM workspace package with a unique package name, a
strict semantic version, and the module layer marker:

```text
modules/example/
├── drizzle/                  # when the module has migration history
│   ├── 0000_initial.sql
│   └── meta/
│       ├── 0000_snapshot.json
│       ├── _ekklesia.json
│       └── _journal.json
├── dist/schema/index.js  # when the module owns tables
├── dist/server/index.js
├── drizzle.config.ts     # when generating migrations
├── ekklesia.module.json
├── package.json
├── src/schema/index.ts   # when the module owns tables
├── src/schema/tables.ts  # pure Drizzle declarations used by Drizzle Kit
└── src/server/index.ts
```

The schema and migration paths are optional. A tableless module omits `entrypoints.schema`; a
module with no migration history omits `entrypoints.migrations` and needs no placeholder files.

```json
{
  "type": "module",
  "ekklesia": {
    "layer": "module"
  }
}
```

The package version must match the manifest and runtime-definition versions. Internal package
dependencies use `workspace:` ranges and supported public package entrypoints.

## Canonical manifest

`ekklesia.module.json` is inert and authoritative, so Core can validate every installed module
without executing module code:

```json
{
  "manifestVersion": 3,
  "id": "calendar",
  "name": "Calendar",
  "version": "0.1.0",
  "compatibility": {
    "platform": "^0.1.0",
    "moduleSdk": "^0.1.0"
  },
  "dependencies": {
    "modules": {
      "required": {
        "people": "^0.1.0"
      }
    },
    "capabilities": {
      "required": {
        "people.records": "^2.0.0"
      },
      "optional": {
        "media.items": ">=1.1.0 <2.0.0"
      }
    }
  },
  "capabilities": [
    {
      "id": "calendar.events",
      "version": "1.0.0"
    }
  ],
  "permissions": [
    {
      "id": "calendar.events.read",
      "baseline": "restricted",
      "description": "Read calendar events."
    }
  ],
  "storage": {
    "buckets": [
      {
        "id": "assets",
        "description": "Private event assets owned by Calendar."
      }
    ]
  },
  "entrypoints": {
    "server": "./dist/server/index.js",
    "schema": "./dist/schema/index.js",
    "migrations": "./drizzle"
  }
}
```

Module, capability, and permission identifiers are lowercase and namespaced. A provided capability
has its own exact semantic version because its public contract can evolve independently from the
module release. Capability dependencies use semantic-version ranges. Optional means the provider
may be absent; if it is installed, its capability version must be compatible.

`manifestVersion: 3` (`packages/contracts/src/module-manifest.ts`) is a required, exact literal — a
version 2 manifest, and any other version, fails validation outright rather than being silently
reinterpreted; there is no compatibility adapter or automatic migration between manifest versions.
Each `permissions` entry is a complete declaration, not a bare identifier string: `id` (must begin
with `<module-id>.`, exactly like a capability id), `baseline` (`public`, `authenticated`, or
`restricted` — see [Authorization](../development/authorization.md#permission-registry) for what
each baseline means to the evaluator), and a required, non-blank `description` up to 240 characters
for operators and audit review only — Core's authorization layer never reads or interprets it.
Declaring a permission here is necessary but not sufficient for it to be enforceable: the manifest
only proves the declaration is well-formed and namespaced correctly, and does not itself register
anything with Core's authorization registry (see
[Authorization](../development/authorization.md#module-permission-composition) for how Core composes
every discovered module's declarations, alongside its own, into the one platform permission
registry). A duplicate permission id within one manifest, or a permission id outside the module's
own namespace, fails validation the same way a duplicate or misnamed capability id does.

`apps/api` composes this registry once, at startup, from every module installed at that time (see
[Authorization](../development/authorization.md#api-startup-discovery-composition-and-enforcement));
installing, removing, or editing a module's manifest takes effect only on the next API restart, not
while it is running. Once installed and discovered, a module's declared permission is enforceable
and grantable through exactly the same mechanism as any Core-owned permission — the same
`requirePermission` middleware and the same `/api/v1/grants` explicit-grant HTTP API a platform
owner (or anyone they delegate `core.authorization.grants.manage` to) already uses. Mounting a
module's own HTTP routes, and creating relationship-derived grants automatically from a module's
own business logic, are both still later work; this only makes a declared permission *usable*
through Core's existing, generic surface.

Server and optional schema entrypoints must be compiled ESM `.js` or `.mjs` files. The optional
migration entrypoint is a directory. All entrypoints begin with `./`, remain inside the package,
and exist. Production never imports source TypeScript directly. Modules with no tables omit
`entrypoints.schema`; modules with no migration history omit `entrypoints.migrations`. Unknown
manifest properties are rejected, including presentation fields such as components or renderers.

Storage bucket IDs are logical names scoped to the module. Modules may request multiple buckets.
Core will later map them to deployment-specific physical S3 buckets and provide scoped access;
module code must not configure SeaweedFS or another S3 client directly.

## Runtime server definition

The source entrypoint exports one typed runtime definition and does not repeat the static manifest:

```ts
import { defineModuleRuntime } from "@ekklesia/module-sdk";

export default defineModuleRuntime({
  id: "calendar",
  version: "0.1.0",

  register({ capabilities }) {
    capabilities.provide("calendar.events", calendarEvents);
  },

  async start({ capabilities }) {
    const people = capabilities.resolve("people.records");
    // Acquire runtime resources here.
  },

  async stop() {
    // Release runtime resources here.
  }
});
```

ID and version are intentional build-artifact identity markers and must match the manifest. Core
rejects missing, duplicate, or undeclared capability registrations. A module may resolve only its
own capabilities and the capability dependencies declared in its manifest. The runtime context is
deliberately limited to the module ID and capability access.

Entrypoint imports must have no top-level side effects. Modules are trusted backend plugins, and ESM
evaluation cannot be rolled back. Keep registration declarative; open resources in `start` and
close them in `stop`.

## Drizzle schema definition

A module with tables keeps the generator-facing declarations in `src/schema/tables.ts`:

```ts
import { integer, pgTableCreator, text } from "drizzle-orm/pg-core";

const moduleTable = pgTableCreator((localName) => `calendar_${localName}`);

export const events = moduleTable("events", {
  id: integer("id").primaryKey(),
  title: text("title").notNull()
});
```

The production schema entrypoint in `src/schema/index.ts` wraps those same objects with the Module
SDK ownership marker:

```ts
import { defineModuleSchema } from "@ekklesia/module-sdk";
import { events } from "./tables.js";

export { events } from "./tables.js";

export default defineModuleSchema({
  moduleId: "calendar",
  tables: { events }
});
```

The prefix callback converts the module ID to a snake-case namespace and prefixes every local table
name. Here the physical PostgreSQL name is `calendar_events`; `care-team` plus `members` becomes
`care_team_members`. Local names must be lowercase snake case, and the complete physical identifier
must not exceed 63 characters. The SDK's `createModuleTable` provides the same convenience when a
generator-facing pure declaration file is not needed; Core validates built metadata independently
of either helper.

Schema definitions use PostgreSQL tables in the platform-managed default/public schema. They must
be side-effect-free: do not connect to a database, read configuration, or perform I/O during import.
The `moduleId` is an ownership marker and must match the static manifest. Export physical base
tables, never query aliases.

Foreign keys may reference self or another exported table from the same module. They may not target
another module or an omitted private table. Every source column must belong to the declaring table,
and all columns in a compound target must belong to the same exported target table. Modules must
never import another module's tables, columns, relations, schema entrypoint, or migration files.
Store opaque identifiers and integrate through public capabilities, events, or APIs instead.

## Drizzle migrations

Module migrations use the stable Drizzle Kit `0.31.10` PostgreSQL format. Pin that exact version in
the module, set `migrations.prefix` to `index`, and generate from the module's own source schema:

```ts
import { defineConfig } from "drizzle-kit";

export default defineConfig({
  dialect: "postgresql",
  schema: "./src/schema/tables.ts",
  out: "./drizzle",
  migrations: { prefix: "index" },
  breakpoints: true
});
```

Keep `tables.ts` limited to Drizzle declarations so the stable generator can load it without
executing the ESM Module SDK wrapper. `src/schema/index.ts` remains the production compiled schema
entrypoint and wraps those same exported table objects with `defineModuleSchema`; tables are not
declared twice.

Run the module's `migration:generate` script with a lowercase snake-case name, review the generated
SQL and snapshot, and add the matching entry to `drizzle/meta/_ekklesia.json`. Every entry must
explicitly declare either `{"mode":"none"}` or
`{"mode":"approval-required","reason":"..."}`. Do not add an `approved` flag; approval will be
supplied by an external Core-owned provider and bound to module ID, identity, and checksum.

Migration identities are `<module-id>:<generated-tag>`. Core preserves Drizzle journal order inside
each module and dependency-first discovery order between modules. It calculates SHA-256 from the
exact SQL bytes, so changing whitespace or line endings changes the checksum. Never edit an
already-applied migration: Core compares the current exact checksum and metadata with its fixed
PostgreSQL history and refuses changed, missing, unknown, or out-of-prefix entries. Add a new forward
migration instead. Downgrade and uninstall migrations are not supported.

A module may migrate only its own objects. SQL is intentionally treated as opaque: offline
validation checks the artifact and metadata contract but cannot prove SQL ownership or classify
destructive behavior. Review those properties explicitly. Core is the only migration executor. It
follows Drizzle's exact `--> statement-breakpoint` delimiter and never naively splits semicolons or
falls back to non-transactional execution.

## Discovery and lifecycle

From the repository root, run:

```bash
pnpm install
pnpm modules:validate
pnpm migrations:validate
pnpm doctor
pnpm migrations:plan
pnpm verify
```

`pnpm modules:validate` builds the workspace and then atomically validates every installed module.
It rejects malformed manifests, duplicate IDs or package names, version mismatches, incompatible
platform or SDK ranges, missing required modules or capabilities, incompatible installed
dependencies, cycles, and missing, escaping, or non-JavaScript server or schema entrypoints.
Missing optional dependencies are allowed. Valid modules are ordered dependency-first with stable
identifier sorting.

After complete static discovery succeeds, validation loads optional schema entrypoints
dependency-first and rejects invalid definitions, unnamespaced or duplicate tables, custom
PostgreSQL schemas, and foreign keys outside the owning module. The aggregate is published only when
every schema is valid. This slice does not connect to PostgreSQL or execute migrations.

`pnpm migrations:validate` performs complete static discovery first, then reads every declared
migration directory without importing module code or connecting to PostgreSQL. It validates native
Drizzle journals and snapshot chains, Ekklesia policy metadata, ordering, inventory, paths, and
checksums atomically. Its concise report lists every installed module and migration in deterministic
order.

With `DATABASE_URL` set through the uncommitted root `.env`, `pnpm migrations:plan` performs a
read-only applied-history comparison. The plan can become stale. `pnpm migrations:apply` is a
separate, explicit database mutation: Core acquires its bounded session advisory lock, rereads and
validates history, preflights the whole destructive-approval set, and applies all pending modules in
dependency-first order in one transaction. If a later migration fails, every schema and history
change from that invocation rolls back; migrations committed by an earlier invocation remain.

An `approval-required` migration needs an approval from an external provider matching its module
ID, exact identity, and exact checksum. Module metadata cannot grant approval, and there is no
wildcard, force, or approve-all option. The default developer command supplies no approvals and
therefore refuses destructive pending migrations.

Runner behavior is unit-tested through a narrow database port. The separate
`pnpm test:migrations:integration` command now exercises the production adapter against disposable
PostgreSQL, including DDL rollback, advisory-lock contention, history behavior, approvals, and
sanitized failures. It remains outside the Docker-free `pnpm verify` gate; see the
[testing guide](../development/testing.md).

`pnpm doctor` builds first and presents the same production discovery results as a concise,
read-only dashboard. It lists discovered module IDs and versions in dependency order, reports
schema and migration counts, and shows safe migration identities with truncated checksums. Schema
discovery imports the compiled schema-definition entrypoint, so its no-I/O and no-configuration
contract still applies. Doctor does not import or start module server entrypoints, apply migrations,
or exercise runtime hooks; lifecycle behavior remains covered by automated tests. PostgreSQL
planning is skipped when `DATABASE_URL` is absent, and object storage is skipped until Core has a
production storage integration. A skip or pending migration is visible as `READY WITH WARNINGS`,
not misreported as a successful check.

Core completes discovery before importing any server entrypoint. It then loads and verifies every
runtime definition before running hooks, registers all modules dependency-first into an unpublished
staged capability registry, and starts them in the same order. Shutdown is reverse-order. If a later
module fails to start, Core stops already-started modules in reverse order and reports any cleanup
failures without leaking caught exceptions or machine paths. A failing `start` hook must clean up
its own partial work before rejecting.

## Start from the template

Copy `templates/module` to `modules/<your-module-id>` and follow its README. No central registry edit
is needed: the workspace glob, build, discovery, and runtime loading all find the new package.