feat: deliver module objects directly to browsers through opaque read URLs
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 20s
CI / migration-smoke (pull_request) Successful in 56s

ADR 0030 left exactly one way to get an image to a browser: read the
object into the module's process and write it back out of a module HTTP
route. Every byte of every asset travelled storage -> Ekklesia -> browser.
For the slideshow this platform exists to replace that means Ekklesia in
the path of every image on every display, permanently, for bytes that
never change — and it would make Ekklesia responsible for re-implementing
byte ranges, without which video seeking cannot work at all.

A module can now ask a logical bucket for a temporary browser URL:

    const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");

The contract names intent, not mechanism. There is no signing, expiry
format, endpoint, or bucket in it — a host with no S3 anywhere could
satisfy the same method, which is what keeps a future single-module
standalone host able to implement it. The URL is opaque to module code and
a bearer capability to whoever holds it, so a module authorizes the
request in its own route before asking: storage bounds reach, never
identity. Lifetimes run one minute to six hours, fifteen by default, and
out-of-range values are refused rather than clamped, so "expires forever"
is unreachable. Logical keys stay canonical state; URLs are generated on
demand and never persisted or logged.

The provider endpoint and the browser delivery origin are separated
because measurement showed they cannot be collapsed: an S3-compatible
signature covers the request Host, so a URL signed for one origin and
rewritten to another is rejected. The browser-facing origin is therefore a
signing input (S3_DELIVERY_ENDPOINT), defaulting to S3_ENDPOINT — so the
standard deployment configures nothing and delivery just works. The scheme
is not signed, so a reverse proxy may terminate TLS provided it passes the
original Host through; that is pinned by a test rather than described and
hoped for.

Read URLs are stable within a window of one third of their lifetime.
Signing is a pure function of the clock, so naive re-signing yields a new
URL per second — a guaranteed browser cache miss, and the reference client
keys its <img> by URL, so an unchanged image would be torn down and
re-downloaded continuously. Quantizing the signing timestamp makes
repeated calls byte-identical while leaving at least two-thirds of the
lifetime; Cache-Control is set at signing time to the guaranteed
remainder, because the pinned SeaweedFS image sends none of its own.

Verified against live SeaweedFS: byte ranges answered directly with 206
and a correct Content-Range, so no range proxy is needed in Core; signing
is local (200 URLs in 66ms), so no batch API is needed; unsigned requests
for the same object are refused, so delivery never depends on a public
bucket; tampered signatures, swapped keys, and swapped buckets all
rejected. Signed write and delete URLs are deliberately not built.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 17:12:07 -05:00
co-authored by Claude Opus 5
parent 9a3b8dab8f
commit 7ff063b87a
23 changed files with 1593 additions and 42 deletions
+34 -4
View File
@@ -682,6 +682,7 @@ and no route returns a provider endpoint or credential.
| `S3_REGION` | Defaults to `us-east-1` |
| `S3_FORCE_PATH_STYLE` | Defaults to `true`; self-hosted services need it |
| `S3_BUCKET_PREFIX` | Defaults to `ekklesia`; namespaces this installation on a shared service |
| `S3_DELIVERY_ENDPOINT` | The origin a **browser** reaches storage on. Defaults to `S3_ENDPOINT`; set it only when that is not browser-reachable |
Production requires `https`, exactly as the database transport requires verified TLS. An endpoint
embedding credentials is rejected rather than stripped. These values reach Core's provider and
@@ -690,6 +691,16 @@ nothing else — never a module, module UI, browser bundle, diagnostic, or API r
The repository's development Compose stack already runs SeaweedFS for this, and `.env.example`
already carries matching values.
`S3_DELIVERY_ENDPOINT` exists because an S3-compatible signature covers the request's `Host`, so a
delivery URL cannot be signed for one origin and rewritten to another — the browser-facing origin has
to be known when the URL is signed. Leave it unset whenever `S3_ENDPOINT` is itself reachable from a
browser, which is true of the development stack (Compose publishes storage on loopback) and of any
deployment where Core and browsers use the same name. Set it when Core reaches storage privately —
`http://object-storage:8333` — and browsers reach it publicly. The scheme is *not* signed, so a
reverse proxy may terminate TLS and forward plaintext, **provided it passes the original `Host`
header through unchanged**; a proxy that rewrites `Host` to its backend makes every delivery URL fail
with `403`.
### Logical buckets, and what modules get
A module declares logical buckets in its manifest (`storage.buckets`) and receives a scoped facade at
@@ -717,14 +728,33 @@ never prevents the platform from booting, and nothing ever deletes a bucket.
### Serving objects to browsers
There are no signed URLs and no public object routing. A module streams objects through its own
authorized route:
A module can hand a browser a temporary URL and let it fetch the object from storage directly
([ADR 0031](../architecture/decisions/0031-direct-object-delivery.md)):
```text
browser → GET /api/v1/modules/<id>/assets/<key> → context.storage → object bytes
browser → GET /api/v1/modules/<id>/slides → module route → { key, url, expiresAt }
browser → GET <url> ─────────────────────────────────────→ storage service
```
which keeps every S3 concern server-side and gives module UI no credentials.
The module authorizes the request in its own route, asks its logical bucket for a URL, and returns
it. Ekklesia is not in the data path, so byte ranges, caching, and revalidation are the storage
service's job — a `Range` request against a delivery URL is answered directly with `206`, which is
what makes media seeking work without a range proxy in Core.
```ts
const assets = context.storage.bucket("announcement-assets");
const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");
```
A delivery URL is a **bearer capability**: whoever holds it can read that one object until it
expires. Deciding who may have one is the module's authorization decision, made before it asks;
the storage contract only bounds how far the URL reaches. Treat a complete URL as a secret — never
log one (see [Security](security.md#object-storage-is-scoped-by-binding-and-modules-never-hold-credentials)).
Streaming through the module's own route with `context.storage.get(...)` remains available and is
still the right choice when the response must be computed, transformed, or authorized per byte.
There are no signed **write** URLs: uploads go through a module route, where the module can validate
what it is accepting.
## Module HTTP routes
+33
View File
@@ -283,6 +283,39 @@ correctness and portability boundary like every other module-facing mechanism he
What it does guarantee is that Ekklesia never hands a module another module's data, never gives module
UI or a browser a credential, and never prints one in diagnostics.
### Delivery URLs are bearer capabilities, and are treated as secrets
A module can ask a bucket for a temporary URL that lets a browser fetch one object directly
([ADR 0031](../architecture/decisions/0031-direct-object-delivery.md)). Whoever holds that URL can
read that object until it expires, with no further check of who they are.
- **Authorization happens before issuance, not inside the URL.** A module's route authorizes the
request and only then asks for a URL. The storage contract has no notion of a principal and offers
no hook that would let it make a permission decision; it bounds *reach*, not *who*.
- **Lifetimes are bounded and cannot be waived.** One minute at the shortest, six hours at the
longest, fifteen minutes by default. Out-of-range requests are refused rather than clamped, so
"expires forever" is unreachable rather than merely discouraged.
- **One object, one bucket.** The signature covers the URL's own path, so editing the key or the
bucket in a delivery URL yields a rejection rather than a different object — including another
module's object.
- **Never logged, never printed.** A complete delivery URL is treated exactly like a token: it is not
logged, and Doctor never prints one. Where a URL must be identified in a log, identify it by module
id, logical bucket id, key, and expiry — never by its query parameters.
- **Reads only.** There are no signed write or delete URLs. Uploads go through a module route where
the module can validate what it accepts.
- **Never canonical state.** Modules persist logical object keys and generate URLs on demand. A
stored URL is a stored credential with an expiry date in it.
**What a browser sees.** A delivery URL reveals the storage origin, the physical bucket name, the
object key, and signing parameters. Physical bucket names are implementation metadata, not secrets —
nothing is authorized by knowing one, since the bucket is not publicly readable and a tampered URL is
rejected. The guarantee is that *module code* cannot reach those details, not that a browser cannot
see the URL it is fetching.
**Buckets stay private.** Direct delivery deliberately does **not** work by making buckets anonymous;
an unsigned request for the same object is refused, and an integration test asserts it so that a
future change cannot "fix" delivery by opening a bucket up.
### Client-side permission checks are not a security boundary
A host may tell a module UI what permissions it believes the viewer holds
+33 -2
View File
@@ -209,12 +209,43 @@ no endpoint or XML, two modules declaring the same logical bucket staying genuin
server, the longest bucket name the mapping can produce being accepted, a real connection failure
mapping to `STORAGE_PROVIDER_UNAVAILABLE`, and a bucket surviving the deletion of every object in it.
An eighth suite (`core/tests/module-delivery.integration.ts`, same harness) proves direct browser
delivery against that same live server, because every claim it makes is a property of the agreement
between the signer and the provider rather than of code in this repository
([ADR 0031](../architecture/decisions/0031-direct-object-delivery.md)). It pins: a delivery URL
returning the object with its content type intact to an ordinary credential-free `fetch`; the same
object being refused **without** the URL's authorization, so delivery can never be quietly "fixed" by
making a bucket public; tampered signatures, swapped keys, and swapped buckets all rejected; expiry
actually enforced; a `Range` request answered directly with `206` and a correct `Content-Range`;
repeated calls inside a window returning a byte-identical URL so browser caches and URL-keyed DOM
elements hold; `Cache-Control` never outliving the URL; an optional download filename; and two
modules' identical logical bucket names resolving to different objects.
Its most load-bearing tests concern the **browser delivery origin**. They sign for a public origin and
replay the request against the real backend through `node:http` with an explicit `Host` header —
which `fetch` cannot do, since `Host` is a forbidden header name there and it would silently send the
wrong one and prove nothing. Together they establish that a signature survives a reverse proxy that
preserves `Host` (even across TLS termination, because the scheme is not signed), that it fails when
a proxy rewrites `Host`, and that a URL signed for the internal endpoint cannot be rewritten to the
public one afterwards — which is the entire reason the delivery origin is a signing input rather than
a post-processing step.
An Announcements-shaped fixture in the same file models the real slideshow: it lists its own slides,
filters images, builds a delivery URL for each, and fetches every one directly. It is written to
demonstrate what it *cannot* say — its whole world is a `ModuleStorage` and two names from its own
manifest — and it asserts that the module's durable state holds keys while only the URL carries
host detail. It also proves that forty URLs cost no provider round trips.
Unit suites (`core/tests/module-storage.test.ts`, `core/tests/module-storage-lifecycle.test.ts`) use an
injected in-memory provider and never touch the network. They prove the parts the protocol cannot:
binding determinism and collision-safety, scoping and cross-module isolation, key validation, lazy
provisioning and retry-after-failure, configuration validation and credential redaction, and the
lifecycle integration including dependency propagation. See
[ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md).
lifecycle integration including dependency propagation. They also own the delivery bounds, which need
no server at all: every out-of-range lifetime refused rather than clamped, every download filename
that could inject a response header refused, and the module-facing surface pinned so a new accessor
cannot appear unnoticed. See
[ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md) and
[ADR 0031](../architecture/decisions/0031-direct-object-delivery.md).
### Which kind of timing proof lives where