feat: add host-provided module object storage
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Failing after 2m20s

`manifest.storage.buckets` has existed since manifest version 1 and meant
nothing — discovery validated it and no code read it. This turns it into a
real capability, which is what stands between Announcements and the module
system: that product currently fetches a hardcoded LAN S3 endpoint, parses
ListObjectsV2 XML by hand, and hands the browser direct object URLs.

- A module declares a logical bucket and receives a scoped facade at
  `context.storage` — list, head, get, put, delete. The contract names no
  provider: no endpoint, region, credential, or physical bucket is reachable
  from it, which is what lets the same module run under a future standalone
  host over its own or a shared S3-compatible service.
- Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the
  deployment target (ADR 0004), never an API name. Probing the pinned image
  found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's
  `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would
  have failed every second boot. A live test pins it.
- Logical → physical mapping is deterministic and collision-safe: an
  installation prefix, the module and bucket ids joined by `--` (which cannot
  occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide),
  and a digest over the full untruncated identity. Two modules declaring
  `assets` always get separate namespaces; neither can see the physical name.
- Declaring a bucket is now a requirement: with no provider configured the
  module is `blocked · storage_unavailable` through ADR 0028's existing
  resolver, propagating to required dependents. An installation with no object
  storage stays a complete deployment.
- Binding availability is not provider health. Eligibility asks whether
  storage is configured, never whether it answered; bucket creation is lazy on
  first use so a storage outage cannot stop the platform booting; a running
  module whose store blinks stays running and retries. Nothing deletes a
  bucket.
- Credentials are deployment-supplied, never persisted, never routed through
  module configuration, and never visible to modules, module UI, diagnostics,
  or API responses. Provider errors translate to a closed code set carrying no
  endpoint, bucket, request id, or XML.
- Retires ADR 0029's transitional `bucket` configuration key. No manifest
  version change. The template declares no bucket, so it still runs without S3.

See ADR 0030.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 08:45:53 -05:00
co-authored by Claude Opus 5
parent 527239f6de
commit b9ca0d560d
55 changed files with 4420 additions and 74 deletions
+108
View File
@@ -969,6 +969,12 @@ write one row, and start nothing. The change takes effect at the next `pnpm api:
so a module that breaks startup can be turned off without the API — the HTTP lifecycle API is served
by the process such a module takes down.
`pnpm doctor`'s `object-storage.integration` check reports whether a storage provider is configured
and reachable, and `object-storage.bindings` lists which modules require storage and where each
logical bucket is bound. Neither prints a credential. A module requiring storage on an installation
with none configured makes the bindings check fail with a remediation line; a provider that is merely
unreachable is a warning, because modules already running are unaffected.
`pnpm doctor`'s `modules.configuration` check reports whether each installed module's configuration
would let it run, naming the offending key when it would not. It prints **no configuration values** —
secret or otherwise — only counts, keys, and reasons.
@@ -1096,6 +1102,108 @@ leaves room for.)
Configuration tells you *how to behave*, never *whether to run*. Whether your module runs is its
host's decision, described next.
## Object storage
Declare the logical buckets you need in `ekklesia.module.json`, and read and write them through
`context.storage` ([ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md)).
```json
"storage": {
"buckets": [
{ "id": "announcement-assets", "description": "Slide images an operator uploads." }
]
}
```
```ts
export default defineModuleRuntime({
id: "announcements",
version: "1.0.0",
register({ http }) {
http.route({
method: "GET",
path: "/assets/:key",
permission: "announcements.assets.read",
maxBodyBytes: 0,
async handler(_request, context) {
const object = await context.storage.bucket("announcement-assets").get(context.params.key);
return new Response(object.body, {
headers: { "content-type": object.contentType ?? "application/octet-stream" },
});
},
});
},
});
```
**You name a logical bucket. You never name a service.** There is no endpoint, region, credential, or
physical bucket anywhere in this contract, and no accessor that could return one. That is what lets
the same module run under Ekklesia and under a future standalone host backed by its own SeaweedFS, a
shared one, MinIO, or AWS S3 — without your code changing.
Never add `storageEndpoint`, `storageAccessKey`, or similar to your module configuration. Those are
the host's, and a module that configures its own object store is a module welded to one deployment.
### Declaring a bucket is a real requirement
If you declare a bucket and the installation has no storage provider configured, your module is
**blocked** — reported as `storage_unavailable` naming that bucket, with its required dependents
blocked behind it. Your `start` hook never runs, so it never has to defend against missing storage.
Declare a bucket only if you need one. A module that declares none is completely unaffected by
whether the installation has object storage, which is why the template declares none.
### The operations
```ts
bucket.list({ prefix, limit, cursor }); // → { objects, cursor? }
bucket.head(key); // → metadata only
bucket.get(key); // → metadata + body: ReadableStream
bucket.put(key, body, { contentType, contentLength });
bucket.delete(key);
```
`get` gives you a Web stream, so hand it straight to a `Response` rather than buffering. `put` takes
a `Uint8Array`, string, `Blob`, or a `ReadableStream` — a stream **must** come with `contentLength`,
because every S3-compatible provider needs the length up front and inferring it would mean buffering
the object you were trying to stream.
A listing is paginated. When `cursor` is present, more objects remain; when it is absent, the listing
is complete. Ignoring it will silently truncate a large bucket.
There is deliberately **no** way to create or delete a bucket. Which buckets exist follows from your
manifest and an operator's installation, and nothing your module does can destroy one.
### Object keys
Keys may nest (`slides/2026/welcome.png`). They are not filesystem paths — `..` means nothing to an
object store, and your bucket comes from the host's binding rather than from the key, so a key cannot
escape it. Empty keys, keys over 1024 UTF-8 bytes, control characters, backslashes, and a leading `/`
are rejected as `STORAGE_KEY_INVALID`.
### Storage failures are yours to handle, and never change your lifecycle
```ts
try {
const listing = await context.storage.bucket("announcement-assets").list();
} catch (error) {
// STORAGE_PROVIDER_UNAVAILABLE, STORAGE_OBJECT_NOT_FOUND, STORAGE_OPERATION_DENIED, ...
}
```
`ModuleStorageError` carries a code and nothing else — no endpoint, credential, bucket name, or
provider body — so it is safe to log or return.
**A storage outage is not a lifecycle event.** If the object store is briefly unreachable, your module
stays `running`; retry on your own schedule and report health in your own UI. The platform will never
disable or block a running module because a storage call timed out.
### Serving objects to a browser
Module UI gets no storage access and no credentials. Serve bytes through your own authorized route,
as in the example above: `browser → your module API → your scoped storage`. Signed URLs and public
object delivery do not exist yet.
### Whether your module runs at all is your host's decision
Between discovery and startup, the platform consults its own **enablement configuration** and