feat: add module migration contracts and validation

2026-07-21 08:09:30 -05:00
parent 37f54edd89
commit 88c28d75c9
30 changed files with 2947 additions and 48 deletions
@@ -0,0 +1,151 @@
+# 0008: Module-owned migration contracts and offline validation
+
+- Status: Accepted
+- Date: 2026-07-21
+
+## Context
+
+ADR 0007 gives each module exclusive ownership of its namespaced Drizzle tables, but it does not
+define how those tables evolve. Ekklesia needs a cloneable module to carry its migration history
+without a central registry, and Core needs to inspect every installed history deterministically
+before a future runner is allowed to touch PostgreSQL.
+
+Drizzle Kit already generates PostgreSQL SQL files, a journal, and schema snapshots. Replacing that
+format would separate module authors from supported Drizzle tooling. At the same time, Drizzle's
+journal does not express Ekklesia's destructive-change policy or module ownership boundary.
+
+## Decision
+
+### Ownership and packaging
+
+Each module owns its migration history and may create, alter, or remove only database objects that
+it owns. A dependency on another module does not grant permission to migrate that module's tables,
+indexes, constraints, sequences, types, or other private database objects. Cross-module foreign
+keys and imports of another module's private schema or migrations remain prohibited by ADR 0007.
+
+A module with migrations declares the optional manifest field
+`entrypoints.migrations`, normally as `./drizzle`. It points to a packaged directory inside the
+module, not executable code. A module with no migrations omits the field. Static module discovery
+must validate the directory for every installed module before migration files are inspected. The
+directory is included in the module package so cloning a module into `modules/`, installing, and
+building remains sufficient; no central migration registry is introduced.
+
+### Supported Drizzle format and workflow
+
+This slice pins Drizzle Kit `0.31.10`, matching the stable pre-v1 format compatible with the
+repository's Drizzle ORM `0.45.2`. A module configures PostgreSQL, its source schema, an output
+directory, index-prefixed migrations, and statement breakpoints. Authors generate migrations with
+the module's `migration:generate` script, review the generated SQL and snapshots, add the required
+Ekklesia policy entry, and run `pnpm migrations:validate` from the repository root.
+
+The accepted directory format is:
+
+```text
+drizzle/
+├── 0000_initial.sql
+└── meta/
+    ├── 0000_snapshot.json
+    ├── _ekklesia.json
+    └── _journal.json
+```
+
+`_journal.json` and indexed snapshots are the native Drizzle Kit PostgreSQL version-7 metadata.
+Every journal entry has a corresponding `<tag>.sql` file and `<index>_snapshot.json`. Extra SQL or
+snapshot files and unsupported file types are invalid. Files and the `meta` directory must be real
+entries within the declared directory; symbolic links are rejected rather than followed.
+
+Ekklesia adds strict `meta/_ekklesia.json` policy metadata:
+
+```json
+{
+  "version": 1,
+  "moduleId": "example",
+  "entries": [
+    {
+      "tag": "0000_initial",
+      "destructive": { "mode": "none" }
+    }
+  ]
+}
+```
+
+The sidecar has exactly one entry for each journal entry in the same order. A destructive migration
+uses `{"mode":"approval-required","reason":"..."}`. The source tree never records an
+`approved` boolean: a future approval record must be external to the module and bound to the
+module ID, migration identity, and checksum so editing the migration invalidates approval.
+
+### Identity, checksums, and ordering
+
+Within a module, `_journal.json` is authoritative for ordering. Indices start at zero, are
+contiguous, and match each indexed tag; timestamps strictly increase; snapshot IDs form the native
+Drizzle predecessor chain. The validator does not repair or re-sort an invalid journal.
+
+Between modules, Core uses the dependency-first order produced by complete static module
+discovery, with its deterministic identifier tie-breaking. It then preserves journal order within
+each module.
+
+A migration's stable identity is `<module ID>:<Drizzle tag>`, for example
+`example:0000_initial`. Core calculates a lowercase SHA-256 checksum from the exact SQL file bytes.
+Line endings, whitespace, breakpoint comments, and final newlines therefore affect the checksum.
+SQL must be non-empty valid UTF-8, but it is not normalized. The source metadata does not duplicate
+the calculated checksum. When execution exists, Core's applied database history is the authority
+for comparing the stored applied checksum with current packaged bytes and rejecting edits to an
+already-applied migration.
+
+### Destructive changes and SQL limits
+
+Migrations are forward-only for now. There are no downgrade or uninstall migrations. A migration
+declared `approval-required` must never be silently applied; offline validation reports that state,
+while the future runner will require an explicit external approval contract before execution.
+This slice neither grants approval nor executes SQL.
+
+The ownership and destructive declarations are enforceable author contracts and review inputs,
+not claims that the offline validator can prove arbitrary SQL safe. SQL comments, quoted and
+schema-qualified identifiers, procedural bodies, dynamic SQL, extension behavior, and
+`search_path` changes make simplistic regular-expression classification incomplete and
+misleading. The validator deliberately treats SQL as opaque bytes and does not claim to detect
+cross-module access or destructive statements. Stronger enforcement, if added, requires a real
+PostgreSQL-aware model and must state its limits.
+
+### Atomic validation and future execution
+
+Offline migration discovery begins only after complete static module discovery succeeds. It opens
+no PostgreSQL connection, imports no schema or server entrypoint, and executes no SQL. It validates
+the full inventory, metadata, ordering, identities, paths, snapshots, policy, and checksums for all
+modules, then publishes one frozen aggregate. Any issue prevents publication of every module's
+result. Errors contain deterministic codes, module IDs, and logical metadata paths only; caught
+exception messages, SQL text, URLs, secrets, machine paths, causes, stacks, and partial results are
+not exposed.
+
+Migration execution remains exclusively controlled by Core. A later ADR and slice will define the
+database history schema, pending/applied comparison, advisory locking, transaction boundaries,
+approval lookup, failure recovery, and execution semantics.
+
+The selected format follows Drizzle Kit's stable
+[`_journal.json` and snapshot handling](https://github.com/drizzle-team/drizzle-orm/blob/drizzle-kit%400.31.10/drizzle-kit/src/utils.ts),
+its [migration generation flow](https://github.com/drizzle-team/drizzle-orm/blob/drizzle-kit%400.31.10/drizzle-kit/src/cli/commands/migrate.ts),
+and Drizzle ORM's [SQL reading and SHA-256 behavior](https://github.com/drizzle-team/drizzle-orm/blob/0.45.2/drizzle-orm/src/migrator.ts).
+
+## Alternatives
+
+- A custom migration language was rejected because it would abandon supported Drizzle generation
+  and metadata without improving runtime safety.
+- Expected checksums in packaged migration policy metadata were rejected because they duplicate the
+  artifact being checked and do not prove whether an edited migration was already applied. Applied
+  database history will provide that authority.
+- Timestamp-only or filename-only identity was rejected because independently installed modules
+  can legitimately use the same generated tag and timestamp.
+- Lexicographically sorting migration files was rejected because Drizzle's journal is the native
+  ordered history and must be internally consistent.
+- Automatically classifying SQL with regular expressions was rejected because it would imply an
+  ownership and destructive-change guarantee the implementation cannot provide.
+- Module-controlled execution was rejected because ordering, history, locking, approval, and
+  failure behavior are platform-wide invariants owned by Core.
+
+## Consequences
+
+Module authors commit generated SQL, native Drizzle metadata, and the Ekklesia policy sidecar. They
+must review ownership and destructive impact explicitly. Modules with no database history remain
+valid and need no placeholder directory. Offline validation gives CI a stable inventory and exact
+checksums without requiring PostgreSQL, but intentionally does not prove SQL semantics or predict
+whether a migration will succeed against a particular live database.