feat: deliver module objects directly to browsers through opaque read URLs
ADR 0030 left exactly one way to get an image to a browser: read the
object into the module's process and write it back out of a module HTTP
route. Every byte of every asset travelled storage -> Ekklesia -> browser.
For the slideshow this platform exists to replace that means Ekklesia in
the path of every image on every display, permanently, for bytes that
never change — and it would make Ekklesia responsible for re-implementing
byte ranges, without which video seeking cannot work at all.
A module can now ask a logical bucket for a temporary browser URL:
const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");
The contract names intent, not mechanism. There is no signing, expiry
format, endpoint, or bucket in it — a host with no S3 anywhere could
satisfy the same method, which is what keeps a future single-module
standalone host able to implement it. The URL is opaque to module code and
a bearer capability to whoever holds it, so a module authorizes the
request in its own route before asking: storage bounds reach, never
identity. Lifetimes run one minute to six hours, fifteen by default, and
out-of-range values are refused rather than clamped, so "expires forever"
is unreachable. Logical keys stay canonical state; URLs are generated on
demand and never persisted or logged.
The provider endpoint and the browser delivery origin are separated
because measurement showed they cannot be collapsed: an S3-compatible
signature covers the request Host, so a URL signed for one origin and
rewritten to another is rejected. The browser-facing origin is therefore a
signing input (S3_DELIVERY_ENDPOINT), defaulting to S3_ENDPOINT — so the
standard deployment configures nothing and delivery just works. The scheme
is not signed, so a reverse proxy may terminate TLS provided it passes the
original Host through; that is pinned by a test rather than described and
hoped for.
Read URLs are stable within a window of one third of their lifetime.
Signing is a pure function of the clock, so naive re-signing yields a new
URL per second — a guaranteed browser cache miss, and the reference client
keys its <img> by URL, so an unchanged image would be torn down and
re-downloaded continuously. Quantizing the signing timestamp makes
repeated calls byte-identical while leaving at least two-thirds of the
lifetime; Cache-Control is set at signing time to the guaranteed
remainder, because the pinned SeaweedFS image sends none of its own.
Verified against live SeaweedFS: byte ranges answered directly with 206
and a correct Content-Range, so no range proxy is needed in Core; signing
is local (200 URLs in 66ms), so no batch API is needed; unsigned requests
for the same object are refused, so delivery never depends on a public
bucket; tampered signatures, swapped keys, and swapped buckets all
rejected. Signed write and delete URLs are deliberately not built.
Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
9a3b8dab8f
commit
7ff063b87a
@@ -0,0 +1,214 @@
|
||||
# 0031: Browsers fetch objects directly, through opaque temporary read URLs
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-08-23
|
||||
- Extends: [0004](0004-replace-unmaintained-minio.md),
|
||||
[0021](0021-modules-as-hostable-products.md),
|
||||
[0028](0028-module-operational-lifecycle.md),
|
||||
[0030](0030-host-provided-module-object-storage.md)
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0030](0030-host-provided-module-object-storage.md) gave a module a scoped `context.storage` and
|
||||
took every provider detail away from it. That is right for module code, but it left exactly one way
|
||||
to get an image to a browser: read the object into the module's process and write it back out of a
|
||||
module HTTP route. Every byte of every asset travels storage → Ekklesia → browser, and Ekklesia's
|
||||
event loop carries it.
|
||||
|
||||
For the product this platform exists to replace that is not a hypothetical cost. `The_River/Announcements`
|
||||
(commit `9aab486`) is a slideshow: a display renders full-screen images for hours, rotating every
|
||||
seven seconds, and the server rebuilds its slide list every thirty. Proxying it would put Ekklesia in
|
||||
the path of every image on every display in the building, permanently, to deliver bytes that never
|
||||
change. Worse, it would make Ekklesia responsible for re-implementing what an object store already
|
||||
does — byte ranges above all, without which video seeking cannot work at all.
|
||||
|
||||
The existing application solves this by making the bucket world-readable and hardcoding its address:
|
||||
|
||||
```js
|
||||
slides = files.map((name) => `http://10.40.0.2:9000/announcements/${encode(name)}`);
|
||||
```
|
||||
|
||||
Direct delivery, at the cost of a permanently public bucket, an origin compiled into the client, and
|
||||
no per-object control of any kind. We want the delivery path and none of the cost.
|
||||
|
||||
## Decision
|
||||
|
||||
A module can ask a logical bucket for a temporary, browser-usable URL for one object:
|
||||
|
||||
```ts
|
||||
const assets = context.storage.bucket("announcement-assets");
|
||||
const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");
|
||||
```
|
||||
|
||||
The browser then fetches the object from the host's storage service directly. Ekklesia is not in the
|
||||
data path.
|
||||
|
||||
### The contract names intent, not mechanism
|
||||
|
||||
`createReadUrl` is the whole surface. There is no `presign`, no expiry format, no signature, no
|
||||
region, no endpoint, and no bucket — a module names the key it already knows and receives something a
|
||||
browser can use. The first implementation is S3-compatible presigned `GET`, but nothing about that
|
||||
reaches the contract: a host with no S3 anywhere in it could satisfy the same method by minting its
|
||||
own opaque token, and no module would change. This is the same reasoning that keeps `ModuleStorage`
|
||||
free of provider vocabulary in ADR 0030, applied to delivery.
|
||||
|
||||
The return value is `{ url, expiresAt }` and deliberately nothing else. `expiresAt` exists so that a
|
||||
caller can decide *when to ask again* without reading the URL, which is what makes "the URL is
|
||||
opaque" an honest rule rather than an aspiration.
|
||||
|
||||
### A read URL is a bearer capability
|
||||
|
||||
Anyone holding the URL can read that one object until it expires. It is not an authenticated
|
||||
request, and this ADR does not pretend otherwise.
|
||||
|
||||
The division of responsibility is therefore explicit:
|
||||
|
||||
- **Whether this person may have the URL** is the module's own authorization decision, made in its
|
||||
route before it asks for one. Storage does not know about people, permissions, or principals.
|
||||
- **How far the URL reaches once issued** is the storage contract's guarantee: one object, in one
|
||||
module's bucket, for a bounded time.
|
||||
|
||||
Storage authorization does not replace domain authorization, and the contract deliberately offers no
|
||||
hook that would let it try.
|
||||
|
||||
### Expiry is bounded, and the bounds come from the reference consumer
|
||||
|
||||
- **Default 15 minutes.** Announcements rebuilds its slide list every 30 seconds and re-broadcasts
|
||||
state every 7. Fifteen minutes is roughly thirty times that natural refresh, so a display that
|
||||
loses its connection for ten minutes still renders every slide the moment it reconnects — without
|
||||
anyone building a refresh mechanism.
|
||||
- **Floor of 1 minute.** Below that, ordinary clock skew between Ekklesia and the storage service
|
||||
can expire a URL before a browser can spend it.
|
||||
- **Ceiling of 6 hours.** This bounds the damage when a URL leaks. It outlasts any single gathering
|
||||
while staying nowhere near a durable credential, and it binds well before the seven-day maximum
|
||||
S3-compatible signing itself permits — so the contract's limit is the one that applies.
|
||||
|
||||
Out-of-range requests are **refused, not clamped**. Silently shortening a lifetime would surface
|
||||
later as URLs expiring for no visible reason; silently lengthening one would weaken a capability
|
||||
without telling anyone. "Expires forever" is unreachable rather than discouraged.
|
||||
|
||||
### Logical keys are canonical state; URLs are not
|
||||
|
||||
A module persists the object key. It generates a URL when something is about to be delivered and then
|
||||
forgets it. Storing a URL would be storing a credential with an expiry date embedded in it, and it
|
||||
would break the moment the deployment's delivery origin changed.
|
||||
|
||||
### Read URLs are stable within a window
|
||||
|
||||
Signing is a pure function of the request and the clock, so signing the same object twice a second
|
||||
apart produces two different URLs for identical bytes. That is not merely untidy. A browser caches by
|
||||
URL, so a changed query string is a guaranteed cache miss — and UI frameworks routinely key list
|
||||
items by URL. The reference client renders:
|
||||
|
||||
```jsx
|
||||
<img key={`current-${currentImage}`} src={currentImage} />
|
||||
```
|
||||
|
||||
A new URL for an unchanged image therefore tears the element down and re-downloads it. A slideshow
|
||||
rebuilding its state every few seconds would re-fetch every slide continuously, having gained a
|
||||
direct path to storage and lost the caching that made it worth having.
|
||||
|
||||
So the signing timestamp is rounded down to a window of one third of the requested lifetime. Repeated
|
||||
calls within that window return a **byte-identical** URL, and at least two-thirds of the lifetime
|
||||
always remains when one is handed out. Ekklesia also sets `Cache-Control: private, max-age=N` at
|
||||
signing time, where `N` is the guaranteed remainder — the pinned SeaweedFS image returns no
|
||||
`Cache-Control` of its own, which would otherwise leave browsers guessing.
|
||||
|
||||
### The provider endpoint and the browser delivery origin are different things
|
||||
|
||||
They cannot be collapsed, and this was measured rather than assumed. An S3-compatible signature
|
||||
covers the request's **`Host` header**, so a URL signed for one origin and then re-pointed at another
|
||||
fails verification. Rewriting after signing is not available.
|
||||
|
||||
The browser-facing origin is therefore a **signing input**. `S3_DELIVERY_ENDPOINT` sets it, and when
|
||||
unset it is the provider endpoint — which is the ordinary case and the reason a normal deployment
|
||||
configures nothing.
|
||||
|
||||
What the signature does **not** cover is the scheme. Terminating TLS at a reverse proxy and
|
||||
forwarding plaintext to the storage service verifies correctly, **provided the proxy passes the
|
||||
original `Host` through unchanged**. That is the single configuration requirement direct delivery
|
||||
imposes on a deployment, and it is pinned by an integration test rather than described here and
|
||||
hoped for.
|
||||
|
||||
### Zero configuration for an ordinary deployment
|
||||
|
||||
The development stack publishes the storage service on loopback, so `S3_ENDPOINT` is already an
|
||||
origin a browser can reach and serves both roles. An operator running the standard stack configures
|
||||
no endpoint, no keys, no bucket, and no delivery origin; modules declare logical buckets, bindings
|
||||
provision themselves, and delivery URLs work. `S3_DELIVERY_ENDPOINT` exists for the deployment where
|
||||
Core reaches storage by a private service name — and only then.
|
||||
|
||||
### What a browser sees
|
||||
|
||||
A read URL reveals the storage origin, the physical bucket name, the object key, and signing query
|
||||
parameters. This is accepted and stated plainly rather than worked around.
|
||||
|
||||
Physical bucket names are **implementation metadata, not secrets**. Nothing is authorized by knowing
|
||||
one: the bucket is not publicly readable, and a URL's signature covers its own path, so editing the
|
||||
bucket or key in a URL produces a rejection rather than another object. Hiding them would mean
|
||||
reintroducing a proxy layer and recreating the data hop this ADR exists to remove — a real cost for a
|
||||
cosmetic gain.
|
||||
|
||||
The distinction that matters is between **module code**, which must never depend on these details and
|
||||
cannot reach them, and **the browser**, which inevitably sees the URL it is fetching.
|
||||
|
||||
## Consequences
|
||||
|
||||
### What this buys
|
||||
|
||||
Byte ranges work directly against the storage service — verified, `206` with a correct
|
||||
`Content-Range` — so video seeking becomes the provider's job and needs no range proxy in Core. This
|
||||
was the single largest reason to prefer direct delivery over proxying, and it removes a whole
|
||||
subsystem from the platform's future.
|
||||
|
||||
Signing is a local computation: 200 URLs took 66 ms with no network traffic. A slideshow can build
|
||||
forty delivery URLs as cheaply as it builds forty strings, so no batch API is needed and none is
|
||||
added.
|
||||
|
||||
### What it costs
|
||||
|
||||
A leaked URL is usable until it expires. That is the nature of the mechanism, and it is why the
|
||||
ceiling is 6 hours rather than a day and why URLs are never logged.
|
||||
|
||||
`createReadUrl` does not check that the object exists. A URL for a missing key — or one deleted after
|
||||
signing — returns `404` when fetched, which is what a browser already handles. Verifying up front
|
||||
would add a round trip to every URL, and a module listing forty slides would pay forty of them for
|
||||
something it just read.
|
||||
|
||||
### Logging and diagnostics
|
||||
|
||||
A read URL is treated as a bearer token. Complete URLs are never logged and never printed by Doctor.
|
||||
Doctor reports the storage origin, and the delivery origin when it differs, and says explicitly that
|
||||
a separate delivery origin was **not** probed — reporting "reachable" without that caveat would let
|
||||
an operator conclude direct delivery works when only half of it was tested.
|
||||
|
||||
### CORS
|
||||
|
||||
The pinned SeaweedFS image reflects any `Origin` back in `Access-Control-Allow-Origin` and does not
|
||||
set `Access-Control-Allow-Credentials`. Read URLs carry no cookies and no browser credentials, so a
|
||||
reflected origin grants nothing beyond what possession of the URL already grants. Ekklesia therefore
|
||||
configures no CORS policy of its own in this pass and relies on the provider's default, which is
|
||||
sufficient for element loads (`<img>`, `<video>`) and for `fetch` without credentials. A deployment
|
||||
that wants a narrower policy sets it on the storage service or its reverse proxy. This is recorded as
|
||||
an observed default rather than an endorsement.
|
||||
|
||||
### Lifecycle is unchanged
|
||||
|
||||
Storage lifecycle semantics stay exactly as ADR 0030 set them. A signing failure is an operational
|
||||
error a module may retry; it never changes a module's runtime state. Nothing here makes a running
|
||||
module blocked.
|
||||
|
||||
### Standalone-host compatibility
|
||||
|
||||
Everything a module touches — `createReadUrl`, the bounds, the `{ url, expiresAt }` shape — lives
|
||||
above the provider seam. A future single-module standalone host implements `StorageProvider` and
|
||||
gets delivery for free. A module never asks which host is running it, and there is deliberately no
|
||||
value it could branch on.
|
||||
|
||||
### Deferred
|
||||
|
||||
Signed **write** and **delete** URLs are not built. Uploads still go through a module route, where
|
||||
the module can validate what it is accepting; handing a browser a capability to write into a
|
||||
module's bucket needs its own decision about size, type, and quota, and does not belong in the pass
|
||||
that established reads. Realtime URL refresh is not built either — ordinary API reloads are
|
||||
sufficient, and a future realtime pass can refresh state and URLs together when it exists.
|
||||
@@ -39,3 +39,4 @@ ADRs capture durable choices that affect more than one package or phase.
|
||||
- [0028: Module operational lifecycle — enablement, eligibility, and what "running" means](0028-module-operational-lifecycle.md) — Accepted
|
||||
- [0029: Module configuration is manifest-declared, host-resolved, and secret-free at rest](0029-module-configuration-and-secrets.md) — Accepted
|
||||
- [0030: Modules declare logical buckets; hosts provide object storage](0030-host-provided-module-object-storage.md) — Accepted
|
||||
- [0031: Browsers fetch objects directly, through opaque temporary read URLs](0031-direct-object-delivery.md) — Accepted
|
||||
|
||||
Reference in New Issue
Block a user