feat: deliver module objects directly to browsers through opaque read URLs
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 20s
CI / migration-smoke (pull_request) Successful in 56s

ADR 0030 left exactly one way to get an image to a browser: read the
object into the module's process and write it back out of a module HTTP
route. Every byte of every asset travelled storage -> Ekklesia -> browser.
For the slideshow this platform exists to replace that means Ekklesia in
the path of every image on every display, permanently, for bytes that
never change — and it would make Ekklesia responsible for re-implementing
byte ranges, without which video seeking cannot work at all.

A module can now ask a logical bucket for a temporary browser URL:

    const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");

The contract names intent, not mechanism. There is no signing, expiry
format, endpoint, or bucket in it — a host with no S3 anywhere could
satisfy the same method, which is what keeps a future single-module
standalone host able to implement it. The URL is opaque to module code and
a bearer capability to whoever holds it, so a module authorizes the
request in its own route before asking: storage bounds reach, never
identity. Lifetimes run one minute to six hours, fifteen by default, and
out-of-range values are refused rather than clamped, so "expires forever"
is unreachable. Logical keys stay canonical state; URLs are generated on
demand and never persisted or logged.

The provider endpoint and the browser delivery origin are separated
because measurement showed they cannot be collapsed: an S3-compatible
signature covers the request Host, so a URL signed for one origin and
rewritten to another is rejected. The browser-facing origin is therefore a
signing input (S3_DELIVERY_ENDPOINT), defaulting to S3_ENDPOINT — so the
standard deployment configures nothing and delivery just works. The scheme
is not signed, so a reverse proxy may terminate TLS provided it passes the
original Host through; that is pinned by a test rather than described and
hoped for.

Read URLs are stable within a window of one third of their lifetime.
Signing is a pure function of the clock, so naive re-signing yields a new
URL per second — a guaranteed browser cache miss, and the reference client
keys its <img> by URL, so an unchanged image would be torn down and
re-downloaded continuously. Quantizing the signing timestamp makes
repeated calls byte-identical while leaving at least two-thirds of the
lifetime; Cache-Control is set at signing time to the guaranteed
remainder, because the pinned SeaweedFS image sends none of its own.

Verified against live SeaweedFS: byte ranges answered directly with 206
and a correct Content-Range, so no range proxy is needed in Core; signing
is local (200 URLs in 66ms), so no batch API is needed; unsigned requests
for the same object are refused, so delivery never depends on a public
bucket; tampered signatures, swapped keys, and swapped buckets all
rejected. Signed write and delete URLs are deliberately not built.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 17:12:07 -05:00
co-authored by Claude Opus 5
parent 9a3b8dab8f
commit 7ff063b87a
23 changed files with 1593 additions and 42 deletions
+9
View File
@@ -1049,6 +1049,15 @@ export async function diagnoseEkklesia(
await dependencies.probeStorage(options.storage);
storageCheck = freezeCheck("object-storage.integration", "pass", "Object storage reachable", [
describeStorageProvider(options.storage),
// Said plainly rather than implied: this probe used the endpoint Core talks to, and a
// separate browser-facing origin is by definition something a browser reaches and this
// process may not. Reporting "reachable" without that caveat would let an operator conclude
// direct delivery works when only half of it was tested.
...(options.storage.deliveryEndpoint === options.storage.endpoint
? []
: [
"The browser delivery origin is configured separately and was not probed; verify it from a browser. Any reverse proxy in front of it must pass the original Host header through unchanged, or signed read URLs will be rejected.",
]),
]);
} catch {
// A warning, never a failure: the provider is configured, and a module that is already
+83 -17
View File
@@ -7,6 +7,9 @@ import {
export type StorageConfigErrorCode =
| "STORAGE_ACCESS_KEY_MISSING"
| "STORAGE_BUCKET_PREFIX_INVALID"
| "STORAGE_DELIVERY_ENDPOINT_INSECURE_IN_PRODUCTION"
| "STORAGE_DELIVERY_ENDPOINT_INVALID"
| "STORAGE_DELIVERY_ENDPOINT_WITH_CREDENTIALS"
| "STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION"
| "STORAGE_ENDPOINT_INVALID"
| "STORAGE_ENDPOINT_WITH_CREDENTIALS"
@@ -16,6 +19,12 @@ export type StorageConfigErrorCode =
const storageConfigMessages: Readonly<Record<StorageConfigErrorCode, string>> = Object.freeze({
STORAGE_ACCESS_KEY_MISSING: "A storage access key is required when a storage endpoint is set.",
STORAGE_BUCKET_PREFIX_INVALID: "The storage bucket prefix must be lowercase kebab-case.",
STORAGE_DELIVERY_ENDPOINT_INSECURE_IN_PRODUCTION:
"A production browser delivery endpoint must use https, or browsers will refuse to load from it.",
STORAGE_DELIVERY_ENDPOINT_INVALID:
"The browser delivery endpoint must be an absolute http or https URL.",
STORAGE_DELIVERY_ENDPOINT_WITH_CREDENTIALS:
"The browser delivery endpoint must not embed credentials.",
STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION:
"A production storage endpoint must use https, exactly as the database transport must use TLS.",
STORAGE_ENDPOINT_INVALID: "The storage endpoint must be an absolute http or https URL.",
@@ -51,6 +60,8 @@ export interface StorageProviderConfigInput {
readonly secretKey?: string | undefined;
readonly forcePathStyle?: string | boolean | undefined;
readonly bucketPrefix?: string | undefined;
/** The origin a **browser** reaches object storage on, when that differs from `endpoint`. */
readonly deliveryEndpoint?: string | undefined;
}
/**
@@ -70,6 +81,53 @@ export interface StorageProviderConfig {
* deployment does not have. */
readonly forcePathStyle: boolean;
readonly bucketPrefix: string;
/**
* The origin browser-facing read URLs are signed for.
*
* Equal to `endpoint` unless the deployment separates them, which is the common case once Core and
* the storage service share a private network: Core reaches the service at something like
* `http://object-storage:8333`, a name that means nothing in a browser, while the browser reaches
* it through a public name.
*
* The two roles cannot be collapsed by rewriting a URL after the fact. An S3-compatible signature
* covers the request's `Host` header, so a URL signed for one host and then re-pointed at another
* fails verification — which is why this is a *signing* input rather than a post-processing step.
* What the signature does **not** cover is the scheme, so terminating TLS at a reverse proxy and
* forwarding plaintext to the storage service verifies correctly, provided the proxy passes the
* original `Host` through unchanged.
*/
readonly deliveryEndpoint: string;
}
interface EndpointErrorCodes {
readonly invalid: StorageConfigErrorCode;
readonly withCredentials: StorageConfigErrorCode;
readonly insecure: StorageConfigErrorCode;
}
/** Applies one rule set to both endpoints, so the browser-facing origin can never be validated more
* loosely than the one Core itself uses. */
function normalizeEndpoint(
value: string,
environment: StorageEnvironment,
codes: EndpointErrorCodes,
): string {
let parsed: URL;
try {
parsed = new URL(value);
} catch {
throw new StorageConfigError(codes.invalid);
}
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
throw new StorageConfigError(codes.invalid);
}
if (parsed.username !== "" || parsed.password !== "") {
throw new StorageConfigError(codes.withCredentials);
}
if (environment === "production" && parsed.protocol !== "https:") {
throw new StorageConfigError(codes.insecure);
}
return parsed.origin + (parsed.pathname === "/" ? "" : parsed.pathname.replace(/\/+$/u, ""));
}
const DEFAULT_REGION = "us-east-1";
@@ -114,22 +172,25 @@ export function createStorageProviderConfig(
return undefined;
}
let parsed: URL;
try {
parsed = new URL(endpoint);
} catch {
throw new StorageConfigError("STORAGE_ENDPOINT_INVALID");
}
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
throw new StorageConfigError("STORAGE_ENDPOINT_INVALID");
}
if (parsed.username !== "" || parsed.password !== "") {
throw new StorageConfigError("STORAGE_ENDPOINT_WITH_CREDENTIALS");
}
const environment = options.environment ?? "production";
if (environment === "production" && parsed.protocol !== "https:") {
throw new StorageConfigError("STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION");
}
const normalizedEndpoint = normalizeEndpoint(endpoint, environment, {
insecure: "STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION",
invalid: "STORAGE_ENDPOINT_INVALID",
withCredentials: "STORAGE_ENDPOINT_WITH_CREDENTIALS",
});
// Unset means the one endpoint serves both roles, which is correct whenever Core and the browser
// reach storage by the same name — the repository's own development stack publishes the service on
// loopback, so the ordinary developer configures nothing here at all.
const deliveryInput = input?.deliveryEndpoint?.trim();
const deliveryEndpoint =
deliveryInput === undefined || deliveryInput === ""
? normalizedEndpoint
: normalizeEndpoint(deliveryInput, environment, {
insecure: "STORAGE_DELIVERY_ENDPOINT_INSECURE_IN_PRODUCTION",
invalid: "STORAGE_DELIVERY_ENDPOINT_INVALID",
withCredentials: "STORAGE_DELIVERY_ENDPOINT_WITH_CREDENTIALS",
});
const region = input?.region?.trim() === "" ? undefined : input?.region?.trim();
if (region !== undefined && !REGION_PATTERN.test(region)) {
@@ -160,7 +221,8 @@ export function createStorageProviderConfig(
bucketPrefix === undefined || bucketPrefix === ""
? DEFAULT_STORAGE_BUCKET_PREFIX
: bucketPrefix,
endpoint: parsed.origin + (parsed.pathname === "/" ? "" : parsed.pathname.replace(/\/+$/u, "")),
deliveryEndpoint,
endpoint: normalizedEndpoint,
forcePathStyle: parseBoolean(input?.forcePathStyle, true),
region: region ?? DEFAULT_REGION,
secretKey,
@@ -177,9 +239,13 @@ export function createStorageProviderConfig(
*/
export function describeStorageProvider(config: StorageProviderConfig): string {
const origin = new URL(config.endpoint).origin;
const deliveryOrigin = new URL(config.deliveryEndpoint).origin;
// Named only when it differs, so the ordinary single-origin deployment reads exactly as it did
// before this distinction existed. Both are host infrastructure names, never credentials.
const delivery = deliveryOrigin === origin ? "" : ` · browser delivery ${deliveryOrigin}`;
return `S3-compatible endpoint ${origin} · region ${config.region} · ${
config.forcePathStyle ? "path-style" : "virtual-host-style"
} · bucket prefix "${config.bucketPrefix}"`;
} · bucket prefix "${config.bucketPrefix}"${delivery}`;
}
export { STORAGE_BUCKET_PREFIX_MAX_LENGTH };
+52 -1
View File
@@ -1,15 +1,21 @@
import {
EMPTY_MODULE_STORAGE,
isValidModuleStorageKey,
isValidModuleStorageReadUrlFilename,
MODULE_STORAGE_READ_URL_DEFAULT_SECONDS,
MODULE_STORAGE_READ_URL_MAX_SECONDS,
MODULE_STORAGE_READ_URL_MIN_SECONDS,
type ModuleStorage,
type ModuleStorageBody,
type ModuleStorageBucket,
ModuleStorageError,
type ModuleStorageListOptions,
type ModuleStorageListing,
type ModuleStorageListOptions,
type ModuleStorageObject,
type ModuleStorageObjectMetadata,
type ModuleStoragePutOptions,
type ModuleStorageReadUrl,
type ModuleStorageReadUrlOptions,
} from "@ekklesia/contracts";
import { type ModuleStorageBinding, resolveModuleStorageBindings } from "./binding.js";
import type { StorageProvider } from "./provider.js";
@@ -58,6 +64,27 @@ function assertValidKey(key: unknown): asserts key is string {
throw new ModuleStorageError("STORAGE_KEY_INVALID", echo);
}
/**
* Resolves a requested read-URL lifetime against the contract's bounds.
*
* Enforced here rather than in a provider so every provider agrees on what a module may ask for, and
* so an out-of-range request is refused rather than quietly clamped — a module that asks for a
* week's lifetime has a misunderstanding worth surfacing, not a value worth rounding.
*/
function resolveExpirySeconds(requested: number | undefined): number {
if (requested === undefined) {
return MODULE_STORAGE_READ_URL_DEFAULT_SECONDS;
}
if (
!Number.isInteger(requested) ||
requested < MODULE_STORAGE_READ_URL_MIN_SECONDS ||
requested > MODULE_STORAGE_READ_URL_MAX_SECONDS
) {
throw new ModuleStorageError("STORAGE_READ_URL_EXPIRY_INVALID");
}
return requested;
}
function createBucketHandle(
provider: StorageProvider,
binding: ModuleStorageBinding,
@@ -69,6 +96,30 @@ function createBucketHandle(
// where the host keeps its data.
id: binding.logicalBucketId,
async createReadUrl(
key: string,
options: ModuleStorageReadUrlOptions = {},
): Promise<ModuleStorageReadUrl> {
assertValidKey(key);
const expiresInSeconds = resolveExpirySeconds(options.expiresIn);
if (
options.downloadFilename !== undefined &&
!isValidModuleStorageReadUrlFilename(options.downloadFilename)
) {
throw new ModuleStorageError("STORAGE_READ_URL_FILENAME_INVALID", key);
}
// The bucket is ensured for the same reason every other operation ensures it: a URL for a
// bucket that was never created would resolve to a failure the browser reports rather than one
// the module can see. Memoized per binding, so a module signing forty slides pays for it once.
await ensure();
return provider.createReadUrl(binding.physicalBucket, key, {
expiresInSeconds,
...(options.downloadFilename === undefined
? {}
: { downloadFilename: options.downloadFilename }),
});
},
async delete(key: string): Promise<void> {
assertValidKey(key);
await ensure();
+21 -1
View File
@@ -1,10 +1,11 @@
import type {
ModuleStorageBody,
ModuleStorageListOptions,
ModuleStorageListing,
ModuleStorageListOptions,
ModuleStorageObject,
ModuleStorageObjectMetadata,
ModuleStoragePutOptions,
ModuleStorageReadUrl,
} from "@ekklesia/contracts";
/**
@@ -42,6 +43,25 @@ export interface StorageProvider {
options: ModuleStoragePutOptions,
): Promise<ModuleStorageObjectMetadata>;
delete(physicalBucket: string, key: string): Promise<void>;
/**
* A URL a browser can read one object from directly, valid for `expiresIn` seconds.
*
* The seconds and the download filename arrive already validated against the contract's bounds —
* a provider enforces neither, so two providers can never disagree about what a module is allowed
* to ask for.
*
* A provider that cannot issue such a URL has no business implementing this interface: the
* facility is part of the storage contract, not an optional extra a module has to feel around for.
* An implementation with no notion of signing could satisfy it by minting its own opaque token.
*/
createReadUrl(
physicalBucket: string,
key: string,
options: {
readonly expiresInSeconds: number;
readonly downloadFilename?: string | undefined;
},
): Promise<ModuleStorageReadUrl>;
/** A cheap reachability probe for diagnostics only. Its result is **never** persisted and never
* influences a module's lifecycle state — a provider being briefly unreachable is an operational
* condition, not an administrative decision (ADR 0030). */
+96
View File
@@ -9,6 +9,7 @@ import {
PutObjectCommand,
S3Client,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import {
type ModuleStorageBody,
ModuleStorageError,
@@ -17,6 +18,7 @@ import {
type ModuleStorageObject,
type ModuleStorageObjectMetadata,
type ModuleStoragePutOptions,
type ModuleStorageReadUrl,
} from "@ekklesia/contracts";
import type { StorageProviderConfig } from "./config.js";
import type { StorageProvider } from "./provider.js";
@@ -173,6 +175,43 @@ const DEFAULT_TRANSPORT_BOUNDS: S3TransportBounds = Object.freeze({
socketIdleTimeoutMs: 5_000,
});
/**
* How coarsely a read URL's signing timestamp is rounded down, as a fraction of its lifetime.
*
* Signing is a pure function of the request and the clock, so signing the same object twice a second
* apart yields two different URLs for the same bytes. That is worse than untidy. A browser caches by
* URL, so a changed query string is a guaranteed cache miss, and UI frameworks routinely key list
* items by URL — the reference consumer this was measured against renders `<img key={url}>`, so a
* new URL for an unchanged image tears the element down and re-downloads it. A slideshow rebuilding
* its state every few seconds would therefore re-fetch every slide continuously, having gained a
* direct path to storage and lost the caching that made it worth having.
*
* Rounding the signing time down to a window makes repeated calls return a **byte-identical** URL
* within that window, so caches hit and keyed elements stay mounted. Dividing by three, rather than
* fixing a duration, keeps the guarantee proportional: whatever lifetime is asked for, at least
* two-thirds of it always remains when a URL is handed out.
*/
const READ_URL_STABILITY_DIVISOR = 3;
interface ReadUrlWindow {
readonly signedAt: Date;
readonly expiresAt: Date;
readonly guaranteedSeconds: number;
}
function readUrlWindow(nowMs: number, expiresInSeconds: number): ReadUrlWindow {
const windowSeconds = Math.max(1, Math.floor(expiresInSeconds / READ_URL_STABILITY_DIVISOR));
const windowMs = windowSeconds * 1000;
const signedAtMs = Math.floor(nowMs / windowMs) * windowMs;
return {
expiresAt: new Date(signedAtMs + expiresInSeconds * 1000),
// Deliberately derived from the window rather than from the current clock: anything that varies
// second by second would end up in the signed query string and undo the stability above.
guaranteedSeconds: expiresInSeconds - windowSeconds,
signedAt: new Date(signedAtMs),
};
}
export function createS3StorageProvider(
config: StorageProviderConfig,
/** Tightened only by the live provider suite, so it can prove the deadline actually fires without
@@ -197,6 +236,25 @@ export function createS3StorageProvider(
},
});
/**
* The client read URLs are signed with.
*
* An S3-compatible signature covers the request's `Host`, so a URL cannot be signed against the
* endpoint Core talks to and then re-pointed at the origin a browser can reach — verification
* fails. The delivery origin therefore has to be a *signing input*, which is what this second
* client supplies. It never sends a request: presigning is a local computation, so this opens no
* socket and is built only when the two origins actually differ.
*/
const signingClient =
config.deliveryEndpoint === config.endpoint
? client
: new S3Client({
credentials: { accessKeyId: config.accessKey, secretAccessKey: config.secretKey },
endpoint: config.deliveryEndpoint,
forcePathStyle: config.forcePathStyle,
region: config.region,
});
return Object.freeze({
async ensureBucket(physicalBucket: string): Promise<void> {
try {
@@ -326,6 +384,41 @@ export function createS3StorageProvider(
}
},
async createReadUrl(
physicalBucket: string,
key: string,
options: {
readonly expiresInSeconds: number;
readonly downloadFilename?: string | undefined;
},
): Promise<ModuleStorageReadUrl> {
const window = readUrlWindow(Date.now(), options.expiresInSeconds);
try {
const url = await getSignedUrl(
signingClient,
new GetObjectCommand({
Bucket: physicalBucket,
Key: key,
// The pinned SeaweedFS image returns no `Cache-Control` of its own, which leaves
// browsers guessing how long an asset may be reused. Stating it turns the stable URL
// above into an actual cache hit. It never outlives the URL itself.
ResponseCacheControl: `private, max-age=${window.guaranteedSeconds}`,
...(options.downloadFilename === undefined
? {}
: {
ResponseContentDisposition: `attachment; filename="${options.downloadFilename}"`,
}),
}),
{ expiresIn: options.expiresInSeconds, signingDate: window.signedAt },
);
return Object.freeze({ expiresAt: window.expiresAt, url });
} catch (error) {
// Signing is local, so a failure here is a malformed request rather than an outage — but it
// is translated exactly like any other so no SDK exception can reach a module.
throw translateStorageError(error, key);
}
},
async probe(): Promise<void> {
try {
// The cheapest call that proves the endpoint is reachable and the credentials sign
@@ -338,6 +431,9 @@ export function createS3StorageProvider(
async close(): Promise<void> {
client.destroy();
if (signingClient !== client) {
signingClient.destroy();
}
},
});
}