feat: add host-provided module object storage
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Failing after 2m20s

`manifest.storage.buckets` has existed since manifest version 1 and meant
nothing — discovery validated it and no code read it. This turns it into a
real capability, which is what stands between Announcements and the module
system: that product currently fetches a hardcoded LAN S3 endpoint, parses
ListObjectsV2 XML by hand, and hands the browser direct object URLs.

- A module declares a logical bucket and receives a scoped facade at
  `context.storage` — list, head, get, put, delete. The contract names no
  provider: no endpoint, region, credential, or physical bucket is reachable
  from it, which is what lets the same module run under a future standalone
  host over its own or a shared S3-compatible service.
- Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the
  deployment target (ADR 0004), never an API name. Probing the pinned image
  found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's
  `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would
  have failed every second boot. A live test pins it.
- Logical → physical mapping is deterministic and collision-safe: an
  installation prefix, the module and bucket ids joined by `--` (which cannot
  occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide),
  and a digest over the full untruncated identity. Two modules declaring
  `assets` always get separate namespaces; neither can see the physical name.
- Declaring a bucket is now a requirement: with no provider configured the
  module is `blocked · storage_unavailable` through ADR 0028's existing
  resolver, propagating to required dependents. An installation with no object
  storage stays a complete deployment.
- Binding availability is not provider health. Eligibility asks whether
  storage is configured, never whether it answered; bucket creation is lazy on
  first use so a storage outage cannot stop the platform booting; a running
  module whose store blinks stays running and retries. Nothing deletes a
  bucket.
- Credentials are deployment-supplied, never persisted, never routed through
  module configuration, and never visible to modules, module UI, diagnostics,
  or API responses. Provider errors translate to a closed code set carrying no
  endpoint, bucket, request id, or XML.
- Retires ADR 0029's transitional `bucket` configuration key. No manifest
  version change. The template declares no bucket, so it still runs without S3.

See ADR 0030.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 08:45:53 -05:00
co-authored by Claude Opus 5
parent 527239f6de
commit b9ca0d560d
55 changed files with 4420 additions and 74 deletions
+65 -3
View File
@@ -389,9 +389,11 @@ account.
3. Construct the module database facade and the frozen `ModuleRuntimeServices` object from that same
pool (see [Module database access](#module-database-access)).
4. Construct the Core identity runtime, authorization store, and evaluator.
5. Read the stored module enablement configuration and then the stored module configuration, and
resolve, over that exact same discovered inventory, which modules are **eligible** to run and
what each eligible module's frozen configuration snapshot is (`resolveModuleEligibility`,
5. Construct the host object-storage provider from validated configuration, if this installation has
one — building the client opens no connection, so a storage outage can never stop the platform
from booting (ADR 0030). Then read the stored module enablement configuration and the stored module
configuration, and resolve, over that exact same discovered inventory, which modules are
**eligible** to run and what each eligible module's frozen configuration snapshot is (`resolveModuleEligibility`,
`@ekklesia/core/modules` — see [Module lifecycle](#module-lifecycle) and
[Module configuration](#module-configuration)). The two reads are sequential rather than raced so
a broken deployment reports the same error code every time.
@@ -664,6 +666,66 @@ different authorities. Mutations additionally require an exact trusted `Origin`
No response carries a filesystem path, package name, manifest path, caught error, stack, or secret
value.
## Module object storage
Ekklesia can provide modules with logical object storage backed by any S3-compatible service
([ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md)). There is **no
HTTP API for it**: storage is a runtime capability modules use, not a resource the platform exposes,
and no route returns a provider endpoint or credential.
### Host configuration
| variable | meaning |
| --- | --- |
| `S3_ENDPOINT` | Absolute `http`/`https` URL. **Absent means this installation has no object storage.** |
| `S3_ACCESS_KEY`, `S3_SECRET_KEY` | Required once an endpoint is set |
| `S3_REGION` | Defaults to `us-east-1` |
| `S3_FORCE_PATH_STYLE` | Defaults to `true`; self-hosted services need it |
| `S3_BUCKET_PREFIX` | Defaults to `ekklesia`; namespaces this installation on a shared service |
Production requires `https`, exactly as the database transport requires verified TLS. An endpoint
embedding credentials is rejected rather than stripped. These values reach Core's provider and
nothing else — never a module, module UI, browser bundle, diagnostic, or API response.
The repository's development Compose stack already runs SeaweedFS for this, and `.env.example`
already carries matching values.
### Logical buckets, and what modules get
A module declares logical buckets in its manifest (`storage.buckets`) and receives a scoped facade at
`context.storage` on its runtime and HTTP handler contexts. It names a logical bucket; the host maps
it to a physical one deterministically and never exposes that name.
An installation with **no** provider is a complete deployment: modules declaring no bucket run
normally. A module that *does* declare one is reported `blocked · storage_unavailable` on
`GET /api/v1/module-lifecycle`, naming the logical bucket:
```json
{ "runtime": { "state": "blocked", "reason": "storage_unavailable",
"storageBucketId": "announcement-assets" } }
```
Its required dependents block behind it through the ordinary dependency mechanism.
### Binding availability is not provider health
Eligibility asks whether storage is **configured**, never whether the service answered. A configured
provider that is temporarily unreachable is an operational error the module catches and retries —
`running` still means the `start` hook completed, and no lifecycle state changes because a storage
call timed out. Buckets are created lazily on first use rather than at startup, so a storage outage
never prevents the platform from booting, and nothing ever deletes a bucket.
### Serving objects to browsers
There are no signed URLs and no public object routing. A module streams objects through its own
authorized route:
```text
browser → GET /api/v1/modules/<id>/assets/<key> → context.storage → object bytes
```
which keeps every S3 concern server-side and gives module UI no credentials.
## Module HTTP routes
An installed module's routes (declared via `http.route(...)` in its `register` hook — see