feat: add module schema discovery and validation
CI / verify (pull_request) Successful in 24s
CI / infrastructure-smoke (pull_request) Successful in 20s

This commit is contained in:
ekklesia
2026-07-13 21:35:21 -05:00
parent 2465d9c1f7
commit 80b56b72b1
31 changed files with 1584 additions and 34 deletions
@@ -0,0 +1,88 @@
# 0007: Module-owned Drizzle schemas and table isolation
- Status: Accepted
- Date: 2026-07-13
## Context
Ekklesia needs to collect database declarations from independently installed modules without
giving Core feature-specific knowledge or allowing modules to couple through private database
structures. Schema declarations must be inspectable before any future migration or database
startup work, and their physical names must remain globally collision-resistant in a shared
PostgreSQL deployment.
The module manifest already has an optional `entrypoints.schema` field. Production module loading
already requires compiled ESM artifacts and complete static discovery before executable code is
imported.
## Decision
A module that owns tables declares a compiled ESM `.js` or `.mjs` schema entrypoint in
`ekklesia.module.json`, normally `./dist/schema/index.js`. The entrypoint default-exports a
`defineModuleSchema` definition containing only a module ID ownership marker and a table record.
The ID must match the already-discovered manifest. The schema entrypoint contains declarations
only, performs no I/O, opens no database connection, and has no top-level side effects beyond
constructing Drizzle metadata.
All module tables use Drizzle's PostgreSQL dialect in the platform-managed default/public
PostgreSQL schema. Modules cannot select custom PostgreSQL schemas. A module owns only the table
objects exported by its own schema definition. Schema definitions contain physical base tables,
not query aliases that could disguise another table under an owned name.
Physical table names use this mandatory namespace:
```text
<module ID with hyphens converted to underscores>_<lowercase snake-case local name>
```
For example, module `care-team` creates local table `members` as `care_team_members`.
Identifiers must be lowercase ASCII snake case and no longer than PostgreSQL's 63-character
identifier limit. `createModuleTable(moduleId)` applies this transformation for authors, while Core
validates the resulting physical names independently. Every table object and every physical table
name may appear only once in the aggregate.
Foreign keys may target self-referential or other exported tables owned by the same module. Every
source column must belong to the declaring table, and every target column in a simple or compound
key must belong to one exported target table. A foreign key may not target a table owned by another
module, even when a module or capability dependency exists. A target with the same namespace that
is omitted from the owning module's schema definition is also invalid.
Modules may not import another module's schema entrypoint, tables, columns, relations, or other
private code. Cross-module data relationships use opaque identifiers and public capabilities,
events, or APIs. The repository boundary checker enforces literal source/package imports, and Core
schema validation independently rejects cross-module foreign-key metadata.
Core schema discovery first completes ordinary module discovery for the entire installed graph. If
that fails, no schema entrypoint is imported. It then imports optional schema entrypoints in
dependency-first module order, validates every definition and table, and publishes a frozen
aggregate only if all modules succeed. Module contributions and table records are explicitly
sorted, and failures are structured, deterministic, and stripped of caught exception messages,
stacks, URLs, causes, and machine-specific paths.
Atomicity here means no schema code runs before complete static discovery and no partial aggregate
is published. ESM evaluation itself cannot be undone, which is why schema entrypoints must remain
side-effect-free.
## Alternatives
- Source TypeScript entrypoints were rejected because production schema discovery must consume the
same built artifacts that deployment uses.
- PostgreSQL schema-per-module isolation was rejected for this slice because table prefixes already
provide explicit ownership without adding schema search-path and migration-filtering complexity.
- Cross-module foreign keys were rejected because they couple module installation, upgrades,
deletion, and migration ordering through private storage details.
- Inferring tables from all named module exports was rejected because an explicit default
definition provides one reviewable ownership boundary and deterministic aggregation.
- A central table registry was rejected because cloning a module into `modules/` must remain enough
to install its declarations.
## Consequences
Module authors must use names short enough for the complete namespaced identifier to fit within 63
characters. Modules with no tables omit `entrypoints.schema`. Custom PostgreSQL schemas, views,
enums, sequences, and richer relation metadata can receive explicit contracts later rather than
being accepted implicitly now.
This decision discovers and validates Drizzle table metadata only. It does not create a database,
execute migrations, compare live schema state, or define upgrade, downgrade, uninstall, or
destructive-change policy. Those migration decisions remain a later Phase 3 slice.
+1
View File
@@ -15,3 +15,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0004: Replace unmaintained MinIO baseline](0004-replace-unmaintained-minio.md) — Accepted
- [0005: Static module manifests and atomic discovery](0005-static-module-manifests.md) — Superseded
- [0006: Versioned capabilities and compiled module runtime](0006-versioned-capabilities-and-compiled-module-runtime.md) — Accepted
- [0007: Module-owned Drizzle schemas and table isolation](0007-module-owned-drizzle-schemas.md) — Accepted
+3 -2
View File
@@ -62,8 +62,9 @@ Implemented now: the Phase 1 repository and infrastructure foundation plus publi
static versioned manifests, independent capability versions and dependency ranges, automatic
filesystem discovery, compiled server loading, runtime-definition verification, staged capability
registration, dependency-ordered startup and reverse shutdown, rollback, and a backend-only module
template.
template. Optional compiled schema entrypoints now contribute validated, namespaced module-owned
Drizzle tables to a deterministic aggregate without connecting to a database.
Target architecture, not yet implemented: Drizzle schema aggregation and migrations, Better Auth,
Target architecture, not yet implemented: migration generation or execution, Better Auth,
permission enforcement, physical storage-bucket provisioning, Surface SDK, richer public resource
and action contracts, API server, dashboard, Builder, and reference feature modules.
+4 -3
View File
@@ -32,7 +32,8 @@ These rules map platform ownership to filesystem and package boundaries. The exe
- Workspace package names are unique and shared packages declare a recognized architectural role.
- Circular source and workspace-package dependencies are rejected.
- Cross-module data access uses services, events, capabilities, or opaque identifiers rather than
direct table imports and foreign keys.
direct table imports and foreign keys. Modules may not import another module's Drizzle tables,
columns, relations, schema entrypoint, or migration internals.
The boundary test suite includes intentionally invalid fixture graphs. This proves that the guard
rejects every currently machine-checkable repository rule even before production packages fill
@@ -40,8 +41,8 @@ every directory. The UI guard rejects component, view, template, stylesheet, sha
common UI-framework dependencies inside modules. This deny list is defense in depth; package
review remains required when adopting a new frontend technology.
Cross-module table ownership, foreign keys, and schema namespace rules require Drizzle metadata
and are intentionally deferred to the Phase 3 schema validator.
The Phase 3 schema validator complements source boundaries by checking table ownership, foreign-key
targets, and physical table namespaces in compiled Drizzle metadata.
## Generated code
+3 -1
View File
@@ -31,7 +31,9 @@ the root verification command discovers them automatically.
`pnpm modules:validate` builds the workspace, then scans `modules/*` without a central registry. A
missing or empty modules directory is valid and reports zero installed modules. Installed modules
must produce the compiled JavaScript entrypoints declared by their static manifests.
must produce the compiled JavaScript entrypoints declared by their static manifests. Optional
schema entrypoints are loaded and validated for table ownership, namespacing, duplicates, and
foreign-key isolation; this command does not connect to PostgreSQL or run migrations.
CI runs `pnpm ci:verify`, which adds Compose configuration validation to `pnpm verify`. A separate
CI job runs the live infrastructure smoke test so code-only local verification does not require
+56 -6
View File
@@ -10,12 +10,16 @@ strict semantic version, and the module layer marker:
```text
modules/example/
├── dist/schema/index.js # when the module owns tables
├── dist/server/index.js
├── ekklesia.module.json
├── package.json
├── src/schema/index.ts # when the module owns tables
└── src/server/index.ts
```
The schema paths are optional. A tableless module omits them and omits `entrypoints.schema`.
```json
{
"type": "module",
@@ -74,7 +78,8 @@ without executing module code:
]
},
"entrypoints": {
"server": "./dist/server/index.js"
"server": "./dist/server/index.js",
"schema": "./dist/schema/index.js"
}
}
```
@@ -84,9 +89,10 @@ has its own exact semantic version because its public contract can evolve indepe
module release. Capability dependencies use semantic-version ranges. Optional means the provider
may be absent; if it is installed, its capability version must be compatible.
Server entrypoints must be compiled ESM `.js` or `.mjs` files, begin with `./`, remain inside the
package, and exist. Production never imports source TypeScript directly. Unknown manifest
properties are rejected, including presentation fields such as components or renderers.
Server and optional schema entrypoints must be compiled ESM `.js` or `.mjs` files, begin with `./`,
remain inside the package, and exist. Production never imports source TypeScript directly. Modules
with no tables omit `entrypoints.schema`. Unknown manifest properties are rejected, including
presentation fields such as components or renderers.
Storage bucket IDs are logical names scoped to the module. Modules may request multiple buckets.
Core will later map them to deployment-specific physical S3 buckets and provide scoped access;
@@ -127,6 +133,44 @@ Entrypoint imports must have no top-level side effects. Modules are trusted back
evaluation cannot be rolled back. Keep registration declarative; open resources in `start` and
close them in `stop`.
## Drizzle schema definition
A module with tables exports a declaration-only schema definition from `src/schema/index.ts`:
```ts
import { createModuleTable, defineModuleSchema } from "@ekklesia/module-sdk";
import { integer, text } from "drizzle-orm/pg-core";
const moduleTable = createModuleTable("calendar");
export const events = moduleTable("events", {
id: integer("id").primaryKey(),
title: text("title").notNull()
});
export default defineModuleSchema({
moduleId: "calendar",
tables: { events }
});
```
`createModuleTable` converts the module ID to a snake-case namespace and prefixes every local table
name. Here the physical PostgreSQL name is `calendar_events`; `care-team` plus `members` becomes
`care_team_members`. Local names must be lowercase snake case, and the complete physical identifier
must not exceed 63 characters. Core validates the built metadata even when a module does not use
the helper.
Schema definitions use PostgreSQL tables in the platform-managed default/public schema. They must
be side-effect-free: do not connect to a database, read configuration, or perform I/O during import.
The `moduleId` is an ownership marker and must match the static manifest. Export physical base
tables, never query aliases.
Foreign keys may reference self or another exported table from the same module. They may not target
another module or an omitted private table. Every source column must belong to the declaring table,
and all columns in a compound target must belong to the same exported target table. Modules must
never import another module's tables, columns, relations, schema entrypoint, or migration files.
Store opaque identifiers and integrate through public capabilities, events, or APIs instead.
## Discovery and lifecycle
From the repository root, run:
@@ -140,8 +184,14 @@ pnpm verify
`pnpm modules:validate` builds the workspace and then atomically validates every installed module.
It rejects malformed manifests, duplicate IDs or package names, version mismatches, incompatible
platform or SDK ranges, missing required modules or capabilities, incompatible installed
dependencies, cycles, and missing, escaping, or non-JavaScript server entrypoints. Missing optional
dependencies are allowed. Valid modules are ordered dependency-first with stable identifier sorting.
dependencies, cycles, and missing, escaping, or non-JavaScript server or schema entrypoints.
Missing optional dependencies are allowed. Valid modules are ordered dependency-first with stable
identifier sorting.
After complete static discovery succeeds, validation loads optional schema entrypoints
dependency-first and rejects invalid definitions, unnamespaced or duplicate tables, custom
PostgreSQL schemas, and foreign keys outside the owning module. The aggregate is published only when
every schema is valid. This slice does not connect to PostgreSQL or execute migrations.
Core completes discovery before importing any server entrypoint. It then loads and verifies every
runtime definition before running hooks, registers all modules dependency-first into an unpublished