feat: add transactional module migration runner
This commit is contained in:
ekklesia
@@ -0,0 +1,167 @@
|
||||
# 0009: Transactional module migration execution
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-07-21
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0008 defines deterministic, module-owned Drizzle migration artifacts and offline validation.
|
||||
It deliberately stops before connecting to PostgreSQL. Ekklesia now needs one Core-controlled path
|
||||
that compares that immutable inventory with applied database history and either reports a plan or
|
||||
applies every pending migration safely.
|
||||
|
||||
Migration execution is a platform-wide coordination problem. Independently installed modules can
|
||||
share one database, their dependencies determine current execution order, and a concurrent deploy
|
||||
must not race another runner. Destructive declarations also need approval from outside the module
|
||||
artifact so editing SQL invalidates an earlier approval. These invariants cannot be delegated to
|
||||
module entrypoints or Drizzle Kit configuration.
|
||||
|
||||
## Decision
|
||||
|
||||
### Core owns connectivity and execution
|
||||
|
||||
Only Core opens the PostgreSQL connection, reads applied migration history, acquires the migration
|
||||
lock, executes SQL, and records success. Modules contribute only the validated artifacts defined by
|
||||
ADR 0008. Migration execution is not a module lifecycle hook and does not occur automatically when
|
||||
the runtime starts.
|
||||
|
||||
Core uses the pinned `pg` 8.22.0 driver. A run uses one explicitly managed client session because
|
||||
PostgreSQL transactions and session-level advisory locks must remain on the same connection. The
|
||||
client is released in every success and failure path.
|
||||
|
||||
### Fixed applied-history contract
|
||||
|
||||
Core owns the fixed table `public.ekklesia_core_migrations`. Modules may neither migrate nor query
|
||||
it as private module state. Its columns are:
|
||||
|
||||
| Column | PostgreSQL type | Contract |
|
||||
| --- | --- | --- |
|
||||
| `identity` | `text` | Unique exact `<module-id>:<tag>` identity |
|
||||
| `module_id` | `text` | Owning module ID |
|
||||
| `migration_tag` | `text` | Drizzle journal tag |
|
||||
| `migration_index` | `integer` | Module-local contiguous journal index |
|
||||
| `checksum` | `text` | Lowercase SHA-256 of the exact packaged SQL bytes |
|
||||
| `applied_order` | `bigint` identity | Primary key and database-assigned insertion order |
|
||||
| `applied_at` | `timestamp with time zone` | Database application time, defaulting to `clock_timestamp()` |
|
||||
|
||||
The table also has unique constraints for `(module_id, migration_index)` and
|
||||
`(module_id, migration_tag)`. A missing table means no migrations have been applied. Planning does
|
||||
not create it. Applying creates it inside the migration transaction only when at least one
|
||||
migration is pending, so a no-op apply does not mutate the database.
|
||||
|
||||
Packaged SQL remains the source of migration identity and exact contents; the database history is
|
||||
the authority for what was successfully applied. Every history row must identify a migration in
|
||||
the current validated inventory and match its module, tag, index, and checksum. Unknown identities,
|
||||
duplicate or inconsistent metadata, and checksum mismatches stop planning and applying.
|
||||
|
||||
Within each module, applied history must be an exact prefix of its current Drizzle journal. A gap
|
||||
or a later migration without all predecessors is invalid. Historical rows are not required to be
|
||||
one prefix of today's global aggregate: dependencies and installed module sets can change, so
|
||||
valid historical migrations from different modules may interleave. `applied_order` records that
|
||||
historical interleaving for audit rather than redefining module-local order. Its values must be
|
||||
unique and increasing but need not be contiguous because PostgreSQL identity sequences can retain
|
||||
gaps after a rollback. An applied migration
|
||||
which is absent from the current inventory is still rejected; removing an installed history
|
||||
requires a future explicit platform policy.
|
||||
|
||||
Migrations remain forward-only. There are no downgrade or uninstall operations.
|
||||
|
||||
### Read-only planning and locked applying
|
||||
|
||||
`migrations:plan` completes static module and migration discovery, reads history, validates it,
|
||||
and reports each migration as applied, pending, or awaiting destructive approval. It does not
|
||||
acquire the apply lock or write to PostgreSQL. Its result is informational and can become stale as
|
||||
soon as another process applies migrations.
|
||||
|
||||
`migrations:apply` completes the same atomic discovery before opening a database session. It then
|
||||
acquires a session advisory lock and rereads history while holding that lock before it calculates
|
||||
the authoritative pending set. The lock uses PostgreSQL's two-key advisory-lock namespace with the
|
||||
stable signed 32-bit keys `1162562380` (`EKKL`) and `1296648018` (`MIGR`). Core polls
|
||||
`pg_try_advisory_lock` for at most ten seconds rather than waiting indefinitely. A timeout fails
|
||||
without executing SQL. Core explicitly releases the lock in cleanup and closes the session as a
|
||||
second release guarantee, including after transaction, approval, or execution failures.
|
||||
|
||||
### One transaction for the whole pending batch
|
||||
|
||||
After history and approval preflight succeed, Core applies the complete dependency-first pending
|
||||
aggregate in one PostgreSQL transaction. This matches Drizzle ORM 0.45.2's PostgreSQL migration
|
||||
transaction boundary. The history table is created if needed, each migration is executed in
|
||||
aggregate order, and its history row is inserted only after that migration succeeds. A later
|
||||
failure rolls back every schema change and history insert from the current invocation. Migrations
|
||||
committed by earlier invocations remain applied.
|
||||
|
||||
Core matches Drizzle's generated statement-breakpoint behavior: it splits only on the exact
|
||||
literal `--> statement-breakpoint` marker. It never splits SQL on semicolons or tries to parse,
|
||||
classify, reorder, or repair arbitrary SQL. Transaction-incompatible SQL fails the transaction;
|
||||
there is no non-transactional fallback and no regular-expression safety claim. A module author
|
||||
must express a compatible forward migration or wait for a future explicitly designed execution
|
||||
mode.
|
||||
|
||||
### Destructive approval contract
|
||||
|
||||
Before beginning the transaction, Core resolves approvals from an externally supplied approval
|
||||
provider and preflights the entire pending batch. Every migration declared
|
||||
`approval-required` needs an exact approval triple:
|
||||
|
||||
```text
|
||||
module ID + migration identity + exact SHA-256 checksum
|
||||
```
|
||||
|
||||
All three values must match. Wildcards, module-wide grants, a command-line force flag, an
|
||||
approve-all mode, and an `approved` value in the module sidecar are not accepted. The default
|
||||
provider supplies no approvals, so destructive migrations are blocked until a deployment composes
|
||||
a trusted external provider. Because all approvals are checked before the transaction begins, a
|
||||
missing or stale approval causes no migration mutation.
|
||||
|
||||
### Deterministic and sanitized reporting
|
||||
|
||||
The runner consumes only the fully validated, frozen migration aggregate and publishes frozen,
|
||||
deterministically ordered plans or results. Errors use stable categories and safe inventory
|
||||
identifiers. They do not expose SQL contents, connection URLs, credentials, driver exception
|
||||
messages, machine-specific paths, causes, stacks, or partial execution results. Cleanup failures
|
||||
are reported without hiding the primary failure and without copying underlying exception text.
|
||||
|
||||
The developer commands read `DATABASE_URL` from the environment. A local uncommitted `.env` may
|
||||
supply it, but commands never print it. This slice verifies database behavior through a narrow
|
||||
in-memory database port and unit tests. A real PostgreSQL integration suite for lock contention,
|
||||
DDL rollback, history creation, and driver behavior is deliberately deferred rather than claimed
|
||||
by the current verification gate.
|
||||
|
||||
The transaction and single-client choices follow node-postgres's
|
||||
[transaction guidance](https://node-postgres.com/features/transactions). The non-blocking lock and
|
||||
explicit release behavior follow PostgreSQL's
|
||||
[advisory-lock functions](https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADVISORY-LOCKS).
|
||||
The batch transaction and exact breakpoint delimiter match Drizzle ORM 0.45.2's
|
||||
[PostgreSQL dialect](https://github.com/drizzle-team/drizzle-orm/blob/0.45.2/drizzle-orm/src/pg-core/dialect.ts)
|
||||
and [migration reader](https://github.com/drizzle-team/drizzle-orm/blob/0.45.2/drizzle-orm/src/migrator.ts).
|
||||
|
||||
## Alternatives
|
||||
|
||||
- Running migrations during module startup was rejected because database mutation must remain an
|
||||
explicit deployment operation and complete discovery must precede all execution.
|
||||
- A transaction per module or migration was rejected because a failure would publish a partially
|
||||
applied invocation and diverge from the pinned Drizzle PostgreSQL runner.
|
||||
- Transaction-level advisory locks were rejected because the pending set must be protected while
|
||||
history is reread and approvals are checked before the transaction begins.
|
||||
- An indefinitely blocking advisory lock was rejected because deploy automation needs a bounded,
|
||||
deterministic failure mode.
|
||||
- Trusting only a timestamp or the newest history row was rejected because installed modules can
|
||||
interleave and exact identity and checksum comparison is required.
|
||||
- A source-controlled approval flag or force option was rejected because the migration author
|
||||
could grant their own approval and an edited artifact could reuse it.
|
||||
- Parsing SQL with regular expressions was rejected for the reasons in ADR 0008 and because it
|
||||
would imply safety guarantees the runner does not provide.
|
||||
- Automatically falling back to non-transactional execution was rejected because it would weaken
|
||||
rollback guarantees exactly when a migration is least compatible with them.
|
||||
|
||||
## Consequences
|
||||
|
||||
Operators receive an explicit read-only plan and a serialized transactional apply command. A
|
||||
successful invocation records exact immutable history, while a failed invocation leaves none of
|
||||
that invocation's schema or history changes committed. Destructive migrations are unusable until
|
||||
a trusted approval provider is composed, by design.
|
||||
|
||||
The all-or-nothing batch can hold locks for longer than per-migration transactions and excludes
|
||||
operations PostgreSQL forbids in a transaction. The current unit-tested database port gives fast,
|
||||
deterministic coverage but cannot prove real server behavior. The next infrastructure-focused
|
||||
slice must add live PostgreSQL tests before production operation is considered complete.
|
||||
Reference in New Issue
Block a user