feat: add transactional module migration runner
CI / verify (pull_request) Successful in 19s
CI / infrastructure-smoke (pull_request) Successful in 20s

This commit is contained in:
ekklesia
2026-07-21 08:52:41 -05:00
parent dc37e59035
commit 0b282865cb
22 changed files with 2589 additions and 47 deletions
@@ -0,0 +1,167 @@
# 0009: Transactional module migration execution
- Status: Accepted
- Date: 2026-07-21
## Context
ADR 0008 defines deterministic, module-owned Drizzle migration artifacts and offline validation.
It deliberately stops before connecting to PostgreSQL. Ekklesia now needs one Core-controlled path
that compares that immutable inventory with applied database history and either reports a plan or
applies every pending migration safely.
Migration execution is a platform-wide coordination problem. Independently installed modules can
share one database, their dependencies determine current execution order, and a concurrent deploy
must not race another runner. Destructive declarations also need approval from outside the module
artifact so editing SQL invalidates an earlier approval. These invariants cannot be delegated to
module entrypoints or Drizzle Kit configuration.
## Decision
### Core owns connectivity and execution
Only Core opens the PostgreSQL connection, reads applied migration history, acquires the migration
lock, executes SQL, and records success. Modules contribute only the validated artifacts defined by
ADR 0008. Migration execution is not a module lifecycle hook and does not occur automatically when
the runtime starts.
Core uses the pinned `pg` 8.22.0 driver. A run uses one explicitly managed client session because
PostgreSQL transactions and session-level advisory locks must remain on the same connection. The
client is released in every success and failure path.
### Fixed applied-history contract
Core owns the fixed table `public.ekklesia_core_migrations`. Modules may neither migrate nor query
it as private module state. Its columns are:
| Column | PostgreSQL type | Contract |
| --- | --- | --- |
| `identity` | `text` | Unique exact `<module-id>:<tag>` identity |
| `module_id` | `text` | Owning module ID |
| `migration_tag` | `text` | Drizzle journal tag |
| `migration_index` | `integer` | Module-local contiguous journal index |
| `checksum` | `text` | Lowercase SHA-256 of the exact packaged SQL bytes |
| `applied_order` | `bigint` identity | Primary key and database-assigned insertion order |
| `applied_at` | `timestamp with time zone` | Database application time, defaulting to `clock_timestamp()` |
The table also has unique constraints for `(module_id, migration_index)` and
`(module_id, migration_tag)`. A missing table means no migrations have been applied. Planning does
not create it. Applying creates it inside the migration transaction only when at least one
migration is pending, so a no-op apply does not mutate the database.
Packaged SQL remains the source of migration identity and exact contents; the database history is
the authority for what was successfully applied. Every history row must identify a migration in
the current validated inventory and match its module, tag, index, and checksum. Unknown identities,
duplicate or inconsistent metadata, and checksum mismatches stop planning and applying.
Within each module, applied history must be an exact prefix of its current Drizzle journal. A gap
or a later migration without all predecessors is invalid. Historical rows are not required to be
one prefix of today's global aggregate: dependencies and installed module sets can change, so
valid historical migrations from different modules may interleave. `applied_order` records that
historical interleaving for audit rather than redefining module-local order. Its values must be
unique and increasing but need not be contiguous because PostgreSQL identity sequences can retain
gaps after a rollback. An applied migration
which is absent from the current inventory is still rejected; removing an installed history
requires a future explicit platform policy.
Migrations remain forward-only. There are no downgrade or uninstall operations.
### Read-only planning and locked applying
`migrations:plan` completes static module and migration discovery, reads history, validates it,
and reports each migration as applied, pending, or awaiting destructive approval. It does not
acquire the apply lock or write to PostgreSQL. Its result is informational and can become stale as
soon as another process applies migrations.
`migrations:apply` completes the same atomic discovery before opening a database session. It then
acquires a session advisory lock and rereads history while holding that lock before it calculates
the authoritative pending set. The lock uses PostgreSQL's two-key advisory-lock namespace with the
stable signed 32-bit keys `1162562380` (`EKKL`) and `1296648018` (`MIGR`). Core polls
`pg_try_advisory_lock` for at most ten seconds rather than waiting indefinitely. A timeout fails
without executing SQL. Core explicitly releases the lock in cleanup and closes the session as a
second release guarantee, including after transaction, approval, or execution failures.
### One transaction for the whole pending batch
After history and approval preflight succeed, Core applies the complete dependency-first pending
aggregate in one PostgreSQL transaction. This matches Drizzle ORM 0.45.2's PostgreSQL migration
transaction boundary. The history table is created if needed, each migration is executed in
aggregate order, and its history row is inserted only after that migration succeeds. A later
failure rolls back every schema change and history insert from the current invocation. Migrations
committed by earlier invocations remain applied.
Core matches Drizzle's generated statement-breakpoint behavior: it splits only on the exact
literal `--> statement-breakpoint` marker. It never splits SQL on semicolons or tries to parse,
classify, reorder, or repair arbitrary SQL. Transaction-incompatible SQL fails the transaction;
there is no non-transactional fallback and no regular-expression safety claim. A module author
must express a compatible forward migration or wait for a future explicitly designed execution
mode.
### Destructive approval contract
Before beginning the transaction, Core resolves approvals from an externally supplied approval
provider and preflights the entire pending batch. Every migration declared
`approval-required` needs an exact approval triple:
```text
module ID + migration identity + exact SHA-256 checksum
```
All three values must match. Wildcards, module-wide grants, a command-line force flag, an
approve-all mode, and an `approved` value in the module sidecar are not accepted. The default
provider supplies no approvals, so destructive migrations are blocked until a deployment composes
a trusted external provider. Because all approvals are checked before the transaction begins, a
missing or stale approval causes no migration mutation.
### Deterministic and sanitized reporting
The runner consumes only the fully validated, frozen migration aggregate and publishes frozen,
deterministically ordered plans or results. Errors use stable categories and safe inventory
identifiers. They do not expose SQL contents, connection URLs, credentials, driver exception
messages, machine-specific paths, causes, stacks, or partial execution results. Cleanup failures
are reported without hiding the primary failure and without copying underlying exception text.
The developer commands read `DATABASE_URL` from the environment. A local uncommitted `.env` may
supply it, but commands never print it. This slice verifies database behavior through a narrow
in-memory database port and unit tests. A real PostgreSQL integration suite for lock contention,
DDL rollback, history creation, and driver behavior is deliberately deferred rather than claimed
by the current verification gate.
The transaction and single-client choices follow node-postgres's
[transaction guidance](https://node-postgres.com/features/transactions). The non-blocking lock and
explicit release behavior follow PostgreSQL's
[advisory-lock functions](https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADVISORY-LOCKS).
The batch transaction and exact breakpoint delimiter match Drizzle ORM 0.45.2's
[PostgreSQL dialect](https://github.com/drizzle-team/drizzle-orm/blob/0.45.2/drizzle-orm/src/pg-core/dialect.ts)
and [migration reader](https://github.com/drizzle-team/drizzle-orm/blob/0.45.2/drizzle-orm/src/migrator.ts).
## Alternatives
- Running migrations during module startup was rejected because database mutation must remain an
explicit deployment operation and complete discovery must precede all execution.
- A transaction per module or migration was rejected because a failure would publish a partially
applied invocation and diverge from the pinned Drizzle PostgreSQL runner.
- Transaction-level advisory locks were rejected because the pending set must be protected while
history is reread and approvals are checked before the transaction begins.
- An indefinitely blocking advisory lock was rejected because deploy automation needs a bounded,
deterministic failure mode.
- Trusting only a timestamp or the newest history row was rejected because installed modules can
interleave and exact identity and checksum comparison is required.
- A source-controlled approval flag or force option was rejected because the migration author
could grant their own approval and an edited artifact could reuse it.
- Parsing SQL with regular expressions was rejected for the reasons in ADR 0008 and because it
would imply safety guarantees the runner does not provide.
- Automatically falling back to non-transactional execution was rejected because it would weaken
rollback guarantees exactly when a migration is least compatible with them.
## Consequences
Operators receive an explicit read-only plan and a serialized transactional apply command. A
successful invocation records exact immutable history, while a failed invocation leaves none of
that invocation's schema or history changes committed. Destructive migrations are unusable until
a trusted approval provider is composed, by design.
The all-or-nothing batch can hold locks for longer than per-migration transactions and excludes
operations PostgreSQL forbids in a transaction. The current unit-tested database port gives fast,
deterministic coverage but cannot prove real server behavior. The next infrastructure-focused
slice must add live PostgreSQL tests before production operation is considered complete.
+1
View File
@@ -17,3 +17,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0006: Versioned capabilities and compiled module runtime](0006-versioned-capabilities-and-compiled-module-runtime.md) — Accepted
- [0007: Module-owned Drizzle schemas and table isolation](0007-module-owned-drizzle-schemas.md) — Accepted
- [0008: Module-owned migration contracts and offline validation](0008-module-owned-migration-contracts.md) — Accepted
- [0009: Transactional module migration execution](0009-transactional-module-migrations.md) — Accepted
+6 -3
View File
@@ -65,9 +65,12 @@ registration, dependency-ordered startup and reverse shutdown, rollback, and a b
template. Optional compiled schema entrypoints now contribute validated, namespaced module-owned
Drizzle tables to a deterministic aggregate without connecting to a database. Optional module-owned
Drizzle migration directories now produce an atomic dependency-ordered offline inventory with
validated metadata, explicit destructive-change declarations, and exact SHA-256 checksums.
validated metadata, explicit destructive-change declarations, and exact SHA-256 checksums. Core can
now compare that inventory with its fixed PostgreSQL history table, produce a read-only plan, and
apply the complete pending batch under a session advisory lock and one transaction. Destructive
migrations require an exact externally supplied approval bound to module, identity, and checksum.
Target architecture, not yet implemented: transactional migration execution and applied history,
Better Auth,
Target architecture, not yet implemented: production approval-provider composition, live
PostgreSQL migration-runner integration coverage, Better Auth,
permission enforcement, physical storage-bucket provisioning, Surface SDK, richer public resource
and action contracts, API server, dashboard, Builder, and reference feature modules.