Files
Ekklesia/packages/contracts/src/module-storage.ts
T
ekklesiaandClaude Opus 5 7ff063b87a
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 20s
CI / migration-smoke (pull_request) Successful in 56s
feat: deliver module objects directly to browsers through opaque read URLs
ADR 0030 left exactly one way to get an image to a browser: read the
object into the module's process and write it back out of a module HTTP
route. Every byte of every asset travelled storage -> Ekklesia -> browser.
For the slideshow this platform exists to replace that means Ekklesia in
the path of every image on every display, permanently, for bytes that
never change — and it would make Ekklesia responsible for re-implementing
byte ranges, without which video seeking cannot work at all.

A module can now ask a logical bucket for a temporary browser URL:

    const { url, expiresAt } = await assets.createReadUrl("slides/welcome.png");

The contract names intent, not mechanism. There is no signing, expiry
format, endpoint, or bucket in it — a host with no S3 anywhere could
satisfy the same method, which is what keeps a future single-module
standalone host able to implement it. The URL is opaque to module code and
a bearer capability to whoever holds it, so a module authorizes the
request in its own route before asking: storage bounds reach, never
identity. Lifetimes run one minute to six hours, fifteen by default, and
out-of-range values are refused rather than clamped, so "expires forever"
is unreachable. Logical keys stay canonical state; URLs are generated on
demand and never persisted or logged.

The provider endpoint and the browser delivery origin are separated
because measurement showed they cannot be collapsed: an S3-compatible
signature covers the request Host, so a URL signed for one origin and
rewritten to another is rejected. The browser-facing origin is therefore a
signing input (S3_DELIVERY_ENDPOINT), defaulting to S3_ENDPOINT — so the
standard deployment configures nothing and delivery just works. The scheme
is not signed, so a reverse proxy may terminate TLS provided it passes the
original Host through; that is pinned by a test rather than described and
hoped for.

Read URLs are stable within a window of one third of their lifetime.
Signing is a pure function of the clock, so naive re-signing yields a new
URL per second — a guaranteed browser cache miss, and the reference client
keys its <img> by URL, so an unchanged image would be torn down and
re-downloaded continuously. Quantizing the signing timestamp makes
repeated calls byte-identical while leaving at least two-thirds of the
lifetime; Cache-Control is set at signing time to the guaranteed
remainder, because the pinned SeaweedFS image sends none of its own.

Verified against live SeaweedFS: byte ranges answered directly with 206
and a correct Content-Range, so no range proxy is needed in Core; signing
is local (200 URLs in 66ms), so no batch API is needed; unsigned requests
for the same object are refused, so delivery never depends on a public
bucket; tampered signatures, swapped keys, and swapped buckets all
rejected. Signed write and delete URLs are deliberately not built.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-23 17:12:07 -05:00

341 lines
17 KiB
TypeScript

/**
* The one module-facing object-storage contract.
*
* **Nothing in this file names a provider.** There is no endpoint, no region, no credential, no
* bucket policy, no signing, no `S3`, and no `SeaweedFS` — a module asks for the *logical* bucket it
* declared in its manifest and reads and writes objects in it. Which physical service satisfies that
* requirement is entirely the host's business (ADR 0030).
*
* That is what makes the same module code runnable under Ekklesia and under a future standalone
* host: both construct this interface, one over a shared S3-compatible service and the other over
* whatever it has, and the module cannot tell the difference.
*/
/** Every failure a module can observe from storage, as a closed set of codes rather than a provider
* library's exception hierarchy. */
export const MODULE_STORAGE_ERROR_CODES = [
/** The module asked for a logical bucket its own manifest does not declare. A programming error,
* not a runtime condition — the platform cannot invent a binding a module never asked for. */
"STORAGE_BUCKET_UNDECLARED",
/** The key does not exist in this bucket. */
"STORAGE_OBJECT_NOT_FOUND",
/** The bucket this binding names is not usable — absent, or not creatable with the host's
* credentials. Distinct from a provider-wide outage. */
"STORAGE_BUCKET_UNAVAILABLE",
/** The storage service could not be reached at all. Transient by nature: a module may retry, and
* this never changes the module's lifecycle state (ADR 0030). */
"STORAGE_PROVIDER_UNAVAILABLE",
/** The host's credentials do not permit this operation. */
"STORAGE_OPERATION_DENIED",
/** The object key is not one this contract accepts. See `isValidModuleStorageKey`. */
"STORAGE_KEY_INVALID",
/** A `ReadableStream` body was supplied without `contentLength`. Refused rather than buffered:
* buffering would defeat the only reason to pass a stream, and discovering that through memory
* pressure in production is worse than discovering it at the call site. */
"STORAGE_CONTENT_LENGTH_REQUIRED",
/** A requested read-URL lifetime is outside the range this contract allows. Refused rather than
* clamped: silently shortening a lifetime a module asked for would surface later as URLs expiring
* for no visible reason, and silently lengthening one would weaken a bearer capability. */
"STORAGE_READ_URL_EXPIRY_INVALID",
/** A read-URL download filename contains something that cannot be put in a response header. */
"STORAGE_READ_URL_FILENAME_INVALID",
/** Anything else the provider reported, reduced to a code. */
"STORAGE_OPERATION_FAILED",
] as const;
export type ModuleStorageErrorCode = (typeof MODULE_STORAGE_ERROR_CODES)[number];
const storageErrorMessages: Readonly<Record<ModuleStorageErrorCode, string>> = Object.freeze({
STORAGE_BUCKET_UNAVAILABLE: "The requested storage bucket is not available.",
STORAGE_BUCKET_UNDECLARED: "This module did not declare the requested storage bucket.",
STORAGE_CONTENT_LENGTH_REQUIRED: "A streamed storage body requires an explicit content length.",
STORAGE_KEY_INVALID: "The storage object key is invalid.",
STORAGE_OBJECT_NOT_FOUND: "The storage object was not found.",
STORAGE_OPERATION_DENIED: "The storage operation was denied.",
STORAGE_OPERATION_FAILED: "The storage operation failed.",
STORAGE_PROVIDER_UNAVAILABLE: "The storage service could not be reached.",
STORAGE_READ_URL_EXPIRY_INVALID: "The requested read URL lifetime is out of range.",
STORAGE_READ_URL_FILENAME_INVALID: "The read URL download filename is invalid.",
});
/**
* A sanitized storage failure.
*
* Carries a code and a fixed message, and deliberately **nothing else** — no endpoint, no region, no
* access key, no bucket name (physical or logical), no request id, no provider XML body, and no
* caught cause. A module may catch this, log it, and return it to a browser; every one of those
* paths would otherwise be a way for host infrastructure detail to escape. It follows the same
* discipline as `ModuleLifecycleStoreError` and `ModuleConfigurationStoreError`.
*
* `key` echoes a value the **module itself supplied** — an object key for the object errors, or the
* logical bucket id for `STORAGE_BUCKET_UNDECLARED`. It is safe precisely because it never came from
* the host: it is the module's own input handed back so a caught error is actionable.
*/
export class ModuleStorageError extends Error {
readonly code: ModuleStorageErrorCode;
readonly key?: string;
constructor(code: ModuleStorageErrorCode, key?: string) {
super(storageErrorMessages[code]);
this.name = "ModuleStorageError";
this.code = code;
if (key !== undefined) {
this.key = key;
}
this.stack = `${this.name}: ${this.message}`;
Object.freeze(this);
}
}
/** The S3 object-key ceiling, which every S3-compatible provider shares. Measured in UTF-8 bytes,
* not characters, because that is what the wire format bounds. */
export const MODULE_STORAGE_KEY_MAX_BYTES = 1024;
/**
* Whether a value is an object key this contract accepts.
*
* An object key is **not a filesystem path** — `..` has no traversal meaning to an object store, and
* the physical bucket comes from the host's binding rather than from anything in the key, so a key
* structurally cannot escape its bucket. What is rejected here is what would be *ambiguous* or
* unrepresentable rather than what would be dangerous: an empty key, one that exceeds the wire
* limit, control characters (which break header and XML encodings), lone surrogates (unencodable as
* UTF-8), a backslash (whose meaning differs between providers), and a leading `/` (which some
* providers normalize away and others do not, so one key would name two objects).
*
* `/` inside a key is allowed and ordinary: object stores use it as a display and prefix convention,
* and modules genuinely need `slides/welcome.png`.
*/
export function isValidModuleStorageKey(value: unknown): value is string {
if (typeof value !== "string" || value.length === 0) {
return false;
}
if (value.startsWith("/") || value.includes("\\")) {
return false;
}
for (const character of value) {
const codePoint = character.codePointAt(0);
if (codePoint === undefined) {
return false;
}
// C0 controls, DEL, and C1 controls.
if (codePoint <= 0x1f || (codePoint >= 0x7f && codePoint <= 0x9f)) {
return false;
}
// A lone surrogate survives in a JavaScript string but cannot be encoded as UTF-8.
if (codePoint >= 0xd800 && codePoint <= 0xdfff) {
return false;
}
}
return new TextEncoder().encode(value).length <= MODULE_STORAGE_KEY_MAX_BYTES;
}
/** What a module may hand to `put`. A `ReadableStream` is accepted so a large object never has to be
* buffered in memory, but it must be accompanied by `contentLength` — every S3-compatible provider
* needs the length up front for a single-part upload, and guessing it would mean buffering the very
* thing the stream exists to avoid. */
export type ModuleStorageBody = Uint8Array | string | Blob | ReadableStream<Uint8Array>;
/** What the store knows about an object besides its bytes. Deliberately the four fields every
* S3-compatible provider actually returns, rather than a mirror of S3's full metadata surface. */
export interface ModuleStorageObjectMetadata {
readonly key: string;
/** Preserved exactly as it was written. A module serving images through its own route needs this
* to set a correct `Content-Type` without sniffing bytes. */
readonly contentType?: string;
readonly contentLength?: number;
readonly etag?: string;
readonly lastModified?: Date;
}
/** An object's metadata plus its bytes as a Web stream, so a module can pass it straight into a
* `Response` without ever holding the whole object in memory. */
export interface ModuleStorageObject extends ModuleStorageObjectMetadata {
readonly body: ReadableStream<Uint8Array>;
}
export interface ModuleStoragePutOptions {
readonly contentType?: string;
/** Required when `body` is a `ReadableStream`, ignored otherwise. */
readonly contentLength?: number;
}
export interface ModuleStorageListOptions {
readonly prefix?: string;
/** Provider-bounded; a provider may return fewer. */
readonly limit?: number;
/** The `cursor` from a previous listing. Opaque — a module passes it back and never parses it. */
readonly cursor?: string;
}
export interface ModuleStorageListing {
readonly objects: readonly ModuleStorageObjectMetadata[];
/** Present when more objects remain. Absent means the listing is complete — which is why this
* exists at all: every S3-compatible provider caps a single listing, and a contract without a
* cursor would silently truncate a bucket with more objects than that cap. */
readonly cursor?: string;
}
/**
* How long a read URL may live, in seconds.
*
* The floor exists because a lifetime shorter than a minute is not reliably usable: the host and the
* storage service keep their own clocks, and a few seconds of skew between them would expire a URL
* before the browser could spend it.
*
* The default is deliberately far longer than any caller needs to *use* a URL and far shorter than a
* credential. The reference consumer this was measured against — a church slideshow — hands its
* display a complete, freshly built state every few seconds and rebuilds that state from storage
* every thirty; fifteen minutes is roughly thirty times that natural refresh, so a display that
* loses its connection for ten minutes still renders every slide the moment it reconnects, without
* anyone building a refresh mechanism.
*
* The ceiling is what bounds the damage when a URL leaks. A read URL is a bearer capability: whoever
* holds it can read that one object until it expires. Six hours outlasts any single gathering while
* staying nowhere near durable — and it is well inside the seven-day maximum S3-compatible signing
* itself permits, so the contract's limit binds before any provider's does.
*/
export const MODULE_STORAGE_READ_URL_MIN_SECONDS = 60;
export const MODULE_STORAGE_READ_URL_MAX_SECONDS = 6 * 60 * 60;
export const MODULE_STORAGE_READ_URL_DEFAULT_SECONDS = 15 * 60;
/** The longest download filename this contract accepts. Generous for a real filename, short enough
* that no header built from it approaches a provider's limit. */
export const MODULE_STORAGE_READ_URL_FILENAME_MAX_LENGTH = 128;
/**
* Whether a value is a download filename this contract accepts.
*
* This is stricter than what a filesystem allows, and deliberately so: the value ends up inside a
* `Content-Disposition` **response header**. A carriage return or line feed in a header value is a
* header-injection primitive, and a double quote closes the quoted string early and lets the rest be
* read as further header parameters. Rather than escape those and hope every provider unescapes them
* the same way, the contract accepts only characters that need no escaping at all.
*
* A path separator is refused for the same reason a leading `/` is refused in an object key: the
* name is a label the browser saves under, not a location, and one that looks like a path invites a
* browser or an operating system to interpret it as one.
*/
export function isValidModuleStorageReadUrlFilename(value: unknown): value is string {
if (typeof value !== "string" || value === "") {
return false;
}
if (value.length > MODULE_STORAGE_READ_URL_FILENAME_MAX_LENGTH) {
return false;
}
if (value === "." || value === "..") {
return false;
}
// Letters, digits, and a small set of separators every filesystem and header encoding agrees on.
return /^[A-Za-z0-9][A-Za-z0-9 ._()-]*$/u.test(value);
}
export interface ModuleStorageReadUrlOptions {
/** Seconds the URL stays valid, between `MODULE_STORAGE_READ_URL_MIN_SECONDS` and
* `MODULE_STORAGE_READ_URL_MAX_SECONDS`. Omit to take the host's default. */
readonly expiresIn?: number;
/** Offers the object as a download under this name instead of displaying it inline.
*
* Inline is the default because these are page assets — an image or a video a browser is meant to
* render, not save. This exists for the one case a module cannot otherwise reach: serving
* something like a bulletin PDF as a named download would otherwise force the module to proxy
* every byte through its own route, which is the hop this whole facility removes. */
readonly downloadFilename?: string;
}
/**
* A temporary, browser-usable URL for one object.
*
* **`url` is opaque.** Its shape, hostname, path, and query are the host's business and may change
* with the host's deployment or provider; module code must not parse it, rewrite it, derive the
* storage service's identity from it, or store it. Everything a caller legitimately needs to decide
* *when to ask again* is in `expiresAt`, precisely so that nobody has to read the URL to find out.
*
* **It is a bearer capability, not an authenticated request.** Anyone who obtains it can read that
* one object until it expires, with no further check of who they are. Deciding whether a given
* person may have it is the module's own authorization decision, made before it asks for one — this
* contract only guarantees that the URL reaches no further than the single object it names.
*
* **It is never canonical state.** The logical object key is what a module persists; a URL is
* generated when something is about to be delivered and then forgotten. Storing one in a database
* would be storing a credential with an expiry date in it.
*/
export interface ModuleStorageReadUrl {
readonly url: string;
/** When the URL stops working. A caller compares this against the clock to decide whether to ask
* for a fresh one; it never has to inspect `url` to find out. */
readonly expiresAt: Date;
}
/**
* One logical bucket a module declared, already bound to wherever the host keeps it.
*
* `id` is the **logical** id from the module's own manifest. The physical bucket, its naming scheme,
* and the service holding it are not reachable from here by design — there is no accessor for them,
* so a module cannot come to depend on the host's implementation and cannot be ported away from it
* later.
*/
export interface ModuleStorageBucket {
readonly id: string;
list(options?: ModuleStorageListOptions): Promise<ModuleStorageListing>;
head(key: string): Promise<ModuleStorageObjectMetadata>;
get(key: string): Promise<ModuleStorageObject>;
put(
key: string,
body: ModuleStorageBody,
options?: ModuleStoragePutOptions,
): Promise<ModuleStorageObjectMetadata>;
delete(key: string): Promise<void>;
/**
* A temporary URL a browser can fetch this object from directly.
*
* The point is to take the host out of the data path: without it, delivering an image means the
* bytes travel storage → module route → browser, and the module's process carries every byte of
* every asset. With it the browser talks to the host's storage service itself, and the module
* handles one small response instead.
*
* Nothing about *how* is visible here. There is no signing, no endpoint, no expiry format, and no
* provider — a module names the key it already knows and receives something a browser can use. A
* host with no S3 anywhere in it could satisfy this same method.
*
* The object is not checked for existence first. A URL for a key that is missing — or that is
* deleted after signing — simply fails when it is fetched, which is what a browser already knows
* how to handle. Verifying up front would add a round trip to every URL, and a module listing
* forty slides and asking for forty URLs would pay forty of them for information it just read.
*/
createReadUrl(key: string, options?: ModuleStorageReadUrlOptions): Promise<ModuleStorageReadUrl>;
}
/**
* A module's own scoped storage.
*
* Scoped by construction: `bucket()` resolves only against the logical buckets **this** module
* declared, so knowing another module's bucket name buys nothing — there is no name that maps
* outside this module's own bindings, and no accessor that takes a physical bucket, an endpoint, or
* a credential.
*
* There is deliberately no `createBucket`, `deleteBucket`, or bucket-listing operation. Which
* buckets exist is decided by the manifest an operator installed, not by running module code, and
* nothing a module does can destroy a bucket (ADR 0030).
*/
export interface ModuleStorage {
/** The logical bucket ids this module declared, sorted. */
buckets(): readonly string[];
/** Throws `ModuleStorageError("STORAGE_BUCKET_UNDECLARED")` for anything this module did not
* declare — including another module's bucket. It throws rather than returning `undefined` because
* asking for an undeclared bucket is a manifest/code mismatch, and a silent no-op would hide it
* until data went missing. */
bucket(logicalBucketId: string): ModuleStorageBucket;
}
/**
* The storage a module with no declared buckets receives. `context.storage` is therefore always
* present and never `undefined`, so module code never branches on whether the host supplied one —
* the same reasoning `EMPTY_MODULE_CONFIGURATION` follows.
*/
export const EMPTY_MODULE_STORAGE: ModuleStorage = Object.freeze({
bucket: (logicalBucketId: string): ModuleStorageBucket => {
throw new ModuleStorageError("STORAGE_BUCKET_UNDECLARED", logicalBucketId);
},
buckets: (): readonly string[] => Object.freeze([]),
});