feat: let a module route stream an image-sized upload, explicitly and bounded
ADR 0017 capped a module route at 1 MiB and said large media belonged to a future storage design. That design arrived — ADR 0030 gave modules a bucket, ADR 0031 took Ekklesia out of the read path — and it did not solve the write path: signed write URLs were deliberately declined, so the only way bytes reach a bucket is a module's own route, and a photograph is not 1 MiB. Extend the route contract rather than adding a second HTTP system. A route declares `bodyMode`: `buffered` (the default, and what every existing route already meant) keeps the 1 MiB ceiling and materializes the body before the handler runs; `streamed` raises the ceiling to 32 MiB and counts bytes as the handler consumes them, holding nothing. The mode is explicit rather than inferred from the size, because inferring it would mean an author who raised a limit silently changed how their own handler receives its body and when a 413 can reach their own code. 32 MiB holds one 25 MiB image plus bounded overhead and is also the platform's absolute ceiling: no route in any mode may declare more, and no unbounded option exists in the contract. The unit is one file per request — twenty images are twenty individually bounded requests, and nothing here can express a 500 MiB aggregate whose failure would waste everything already sent. Replace the buffering dependency with the platform's own limiter. A declared Content-Length over the limit is refused synchronously, before authorization and before a byte is read. Buffered mode now counts what actually arrives rather than trusting the header, which closes a gap the previous implementation had. Streamed mode wraps the body in a counting transform; on overflow the stream errors, the source is cancelled, and the response is replaced with the same sanitized 413 even if the handler swallowed the error — a limit a handler can swallow is not a limit. An unread body is cancelled so an abandoned upload releases its socket, and an abort stays an ordinary failed request rather than becoming a 413. Ordering is unchanged and now strictly better: in streamed mode no byte is read until the handler reads it, and the handler runs only after authorization succeeds, so an unauthorized caller cannot push an upload through at all. The live suite uploads a real 25 MiB body over a real socket end to end. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
co-authored by
Claude Opus 5
Claude Opus 5
parent
c444ff570b
commit
e36485bf99
@@ -25,21 +25,72 @@ export function isModuleHttpMethod(value: unknown): value is ModuleHttpMethod {
|
||||
}
|
||||
|
||||
/**
|
||||
* The platform-owned hard ceiling on any single module route's declared request-body allowance, in
|
||||
* bytes. No module route may declare a `maxBodyBytes` above this value — validated at registration
|
||||
* (`safeParseModuleRouteDefinition`, below), not merely documented. This is a security contract,
|
||||
* not a manifest capability: it does not gate on `manifestVersion` and is enforced identically for
|
||||
* every installed module regardless of its declared compatibility range. Large media transfer
|
||||
* belongs to a future dedicated storage design, never to widening this ceiling.
|
||||
* How a route's request body reaches its handler, and therefore which ceiling applies to it
|
||||
* (ADR 0042).
|
||||
*
|
||||
* - `buffered` (the default, and everything that existed before): the platform reads and counts the
|
||||
* whole body **before** the handler runs, so the handler is promised a fully available, bounded
|
||||
* body and an oversized request is rejected before any authorization or handler work happens.
|
||||
* Bounded by {@link MODULE_ROUTE_MAX_BODY_BYTES_CEILING}, deliberately small.
|
||||
* - `streamed`: the platform counts bytes **as the handler consumes them** and never materializes
|
||||
* the body. That is what makes an ordinary image upload possible without the platform holding it
|
||||
* in memory, and it is the whole reason the larger ceiling exists.
|
||||
*
|
||||
* Selecting the mode is an explicit, conscious decision rather than something inferred from the size
|
||||
* a route asked for. Inferring it would mean an author who raised a limit silently changed how their
|
||||
* own handler receives its body — and when a `413` can arrive relative to their own code.
|
||||
*/
|
||||
export const MODULE_HTTP_BODY_MODES = ["buffered", "streamed"] as const;
|
||||
|
||||
export type ModuleRouteBodyMode = (typeof MODULE_HTTP_BODY_MODES)[number];
|
||||
|
||||
const moduleRouteBodyModes: ReadonlySet<string> = new Set(MODULE_HTTP_BODY_MODES);
|
||||
|
||||
export function isModuleRouteBodyMode(value: unknown): value is ModuleRouteBodyMode {
|
||||
return typeof value === "string" && moduleRouteBodyModes.has(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* The platform-owned hard ceiling on a **buffered** module route's declared request-body allowance,
|
||||
* in bytes. No buffered route may declare a `maxBodyBytes` above this value — validated at
|
||||
* registration (`safeParseModuleRouteDefinition`, below), not merely documented. This is a security
|
||||
* contract, not a manifest capability: it does not gate on `manifestVersion` and is enforced
|
||||
* identically for every installed module regardless of its declared compatibility range.
|
||||
*
|
||||
* It stays exactly where it was. A body the platform materializes before a handler runs is a body
|
||||
* the platform is holding in memory, and 1 MiB is the most a JSON request has any business being.
|
||||
*/
|
||||
export const MODULE_ROUTE_MAX_BODY_BYTES_CEILING = 1_048_576;
|
||||
|
||||
/**
|
||||
* The platform-owned hard ceiling on a **streamed** module route's declared allowance, in bytes,
|
||||
* and the platform's absolute request-body ceiling: nothing in Ekklesia may declare or accept a
|
||||
* larger single request body, and there is no unbounded option anywhere in the contract.
|
||||
*
|
||||
* 32 MiB, chosen to hold one 25 MiB image plus bounded request overhead — a multipart envelope, its
|
||||
* boundaries and part headers, and a modest allowance for a client that pads. The unit is
|
||||
* deliberately *one file*: a UI offering twenty images issues twenty individually bounded requests,
|
||||
* each of which the platform can reject cheaply, rather than one 500 MiB request whose failure
|
||||
* wastes everything already sent. Nothing in this contract can express an aggregate upload.
|
||||
*/
|
||||
export const MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING = 33_554_432;
|
||||
|
||||
/** The largest body any module route may declare, in any mode. */
|
||||
export const MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES = MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING;
|
||||
|
||||
/** The ceiling that applies to one delivery mode. */
|
||||
export function moduleRouteBodyCeiling(mode: ModuleRouteBodyMode): number {
|
||||
return mode === "streamed"
|
||||
? MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING
|
||||
: MODULE_ROUTE_MAX_BODY_BYTES_CEILING;
|
||||
}
|
||||
|
||||
const moduleRouteMaxBodyBytesSchema = z
|
||||
.number()
|
||||
.int()
|
||||
.finite()
|
||||
.min(0)
|
||||
.max(MODULE_ROUTE_MAX_BODY_BYTES_CEILING);
|
||||
.max(MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES);
|
||||
|
||||
const MODULE_ROUTE_PATH_MAX_LENGTH = 256;
|
||||
const staticSegmentPattern = /^[a-z0-9]+(?:-[a-z0-9]+)*$/u;
|
||||
@@ -265,6 +316,12 @@ export interface ModuleRouteDefinitionInput {
|
||||
readonly path: string;
|
||||
readonly permission: string;
|
||||
readonly maxBodyBytes: number;
|
||||
/**
|
||||
* How the body reaches the handler, and therefore which ceiling `maxBodyBytes` is checked
|
||||
* against (ADR 0042). Omitted means `buffered`, which is what every route written before this
|
||||
* existed meant, so adding this field changed no existing route's behaviour.
|
||||
*/
|
||||
readonly bodyMode?: ModuleRouteBodyMode;
|
||||
readonly handler: ModuleRouteHandler;
|
||||
}
|
||||
|
||||
@@ -275,6 +332,9 @@ export interface ModuleRouteDefinition {
|
||||
readonly path: ModuleRoutePath;
|
||||
readonly permission: PermissionId;
|
||||
readonly maxBodyBytes: number;
|
||||
/** Always present in the canonical form, defaulted to `buffered`, so no consumer downstream has
|
||||
* to re-apply the default and none of them can disagree about it. */
|
||||
readonly bodyMode: ModuleRouteBodyMode;
|
||||
readonly handler: ModuleRouteHandler;
|
||||
}
|
||||
|
||||
@@ -294,10 +354,20 @@ export interface RegisteredModuleHttpRoute {
|
||||
readonly path: ModuleRoutePath;
|
||||
readonly permission: PermissionId;
|
||||
readonly maxBodyBytes: number;
|
||||
readonly bodyMode: ModuleRouteBodyMode;
|
||||
readonly handler: ModuleRouteHandler;
|
||||
}
|
||||
|
||||
const moduleRouteDefinitionKeys = new Set([
|
||||
"method",
|
||||
"path",
|
||||
"permission",
|
||||
"maxBodyBytes",
|
||||
"bodyMode",
|
||||
"handler",
|
||||
]);
|
||||
|
||||
const requiredModuleRouteDefinitionKeys = new Set([
|
||||
"method",
|
||||
"path",
|
||||
"permission",
|
||||
@@ -314,15 +384,18 @@ function hasModuleRouteDefinitionShape(value: unknown): value is {
|
||||
path: unknown;
|
||||
permission: unknown;
|
||||
maxBodyBytes: unknown;
|
||||
bodyMode: unknown;
|
||||
handler: unknown;
|
||||
} {
|
||||
if (!isRecord(value)) {
|
||||
return false;
|
||||
}
|
||||
const keys = Object.keys(value);
|
||||
// Every required field present, `bodyMode` optional, and nothing else at all: a route definition
|
||||
// still cannot carry a baseline, a mount prefix, middleware, or another module's namespace.
|
||||
return (
|
||||
keys.length === moduleRouteDefinitionKeys.size &&
|
||||
keys.every((key) => moduleRouteDefinitionKeys.has(key)) &&
|
||||
[...requiredModuleRouteDefinitionKeys].every((key) => keys.includes(key)) &&
|
||||
typeof value.handler === "function"
|
||||
);
|
||||
}
|
||||
@@ -332,6 +405,7 @@ export type ModuleRouteDefinitionIssueCode =
|
||||
| "MODULE_ROUTE_METHOD_INVALID"
|
||||
| "MODULE_ROUTE_PATH_INVALID"
|
||||
| "MODULE_ROUTE_PERMISSION_INVALID"
|
||||
| "MODULE_ROUTE_BODY_MODE_INVALID"
|
||||
| "MODULE_ROUTE_MAX_BODY_BYTES_INVALID";
|
||||
|
||||
export type ModuleRouteDefinitionParseResult =
|
||||
@@ -360,13 +434,23 @@ export function safeParseModuleRouteDefinition(value: unknown): ModuleRouteDefin
|
||||
if (!parsedPermission.success) {
|
||||
return { success: false, code: "MODULE_ROUTE_PERMISSION_INVALID" };
|
||||
}
|
||||
if (value.bodyMode !== undefined && !isModuleRouteBodyMode(value.bodyMode)) {
|
||||
return { success: false, code: "MODULE_ROUTE_BODY_MODE_INVALID" };
|
||||
}
|
||||
const bodyMode: ModuleRouteBodyMode = isModuleRouteBodyMode(value.bodyMode)
|
||||
? value.bodyMode
|
||||
: "buffered";
|
||||
const parsedMaxBodyBytes = moduleRouteMaxBodyBytesSchema.safeParse(value.maxBodyBytes);
|
||||
if (!parsedMaxBodyBytes.success) {
|
||||
// The ceiling is chosen by the declared mode, so a route asking for more than a buffered body may
|
||||
// hold has to say `streamed` — and thereby accept that its handler consumes a stream and that a
|
||||
// `413` can arrive after its own code has started running.
|
||||
if (!parsedMaxBodyBytes.success || parsedMaxBodyBytes.data > moduleRouteBodyCeiling(bodyMode)) {
|
||||
return { success: false, code: "MODULE_ROUTE_MAX_BODY_BYTES_INVALID" };
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
data: Object.freeze({
|
||||
bodyMode,
|
||||
method: value.method,
|
||||
path: parsedPath.data,
|
||||
permission: parsedPermission.data,
|
||||
|
||||
@@ -2,7 +2,10 @@ import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
isModuleHttpMethod,
|
||||
MODULE_HTTP_METHODS,
|
||||
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES,
|
||||
MODULE_ROUTE_MAX_BODY_BYTES_CEILING,
|
||||
MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING,
|
||||
moduleRouteBodyCeiling,
|
||||
type ModuleRouteDefinitionInput,
|
||||
moduleRoutePathSchema,
|
||||
moduleRouteShapeKey,
|
||||
@@ -153,6 +156,7 @@ describe("safeParseModuleRouteDefinition", () => {
|
||||
expect(result.success).toBe(true);
|
||||
if (result.success) {
|
||||
expect(result.data).toEqual({
|
||||
bodyMode: "buffered",
|
||||
method: "GET",
|
||||
path: "/events/:eventId",
|
||||
permission: "calendar.events.read",
|
||||
@@ -258,6 +262,81 @@ describe("safeParseModuleRouteDefinition", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("defaults an omitted bodyMode to buffered, so an existing route means what it always meant", () => {
|
||||
const parsed = safeParseModuleRouteDefinition(validDefinition());
|
||||
|
||||
expect(parsed.success && parsed.data.bodyMode).toBe("buffered");
|
||||
});
|
||||
|
||||
it("accepts either declared body mode and carries it through", () => {
|
||||
for (const bodyMode of ["buffered", "streamed"] as const) {
|
||||
const parsed = safeParseModuleRouteDefinition({ ...validDefinition(), bodyMode });
|
||||
|
||||
expect(parsed.success && parsed.data.bodyMode).toBe(bodyMode);
|
||||
}
|
||||
});
|
||||
|
||||
it("rejects a body mode outside the closed set", () => {
|
||||
for (const bodyMode of ["stream", "chunked", "", 1, null]) {
|
||||
expect(safeParseModuleRouteDefinition({ ...validDefinition(), bodyMode })).toEqual({
|
||||
success: false,
|
||||
code: "MODULE_ROUTE_BODY_MODE_INVALID",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("checks maxBodyBytes against the ceiling the declared mode selects", () => {
|
||||
// A route asking for more than a buffered body may hold must say `streamed` — and thereby accept
|
||||
// that its handler consumes a stream (ADR 0042).
|
||||
expect(
|
||||
safeParseModuleRouteDefinition(
|
||||
validDefinition({ maxBodyBytes: MODULE_ROUTE_MAX_BODY_BYTES_CEILING + 1 }),
|
||||
),
|
||||
).toEqual({ success: false, code: "MODULE_ROUTE_MAX_BODY_BYTES_INVALID" });
|
||||
expect(
|
||||
safeParseModuleRouteDefinition({
|
||||
...validDefinition({ maxBodyBytes: MODULE_ROUTE_MAX_BODY_BYTES_CEILING + 1 }),
|
||||
bodyMode: "streamed",
|
||||
}).success,
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("holds one 25 MiB image plus bounded overhead, and refuses one byte more", () => {
|
||||
const image = 25 * 1024 * 1024;
|
||||
|
||||
expect(MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING).toBeGreaterThan(image);
|
||||
expect(
|
||||
safeParseModuleRouteDefinition({
|
||||
...validDefinition({ maxBodyBytes: MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING }),
|
||||
bodyMode: "streamed",
|
||||
}).success,
|
||||
).toBe(true);
|
||||
expect(
|
||||
safeParseModuleRouteDefinition({
|
||||
...validDefinition({ maxBodyBytes: MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING + 1 }),
|
||||
bodyMode: "streamed",
|
||||
}),
|
||||
).toEqual({ success: false, code: "MODULE_ROUTE_MAX_BODY_BYTES_INVALID" });
|
||||
});
|
||||
|
||||
it("has one absolute ceiling and no unbounded option in any mode", () => {
|
||||
expect(MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES).toBe(MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING);
|
||||
expect(moduleRouteBodyCeiling("buffered")).toBe(MODULE_ROUTE_MAX_BODY_BYTES_CEILING);
|
||||
expect(moduleRouteBodyCeiling("streamed")).toBe(MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES);
|
||||
for (const bodyMode of ["buffered", "streamed"] as const) {
|
||||
for (const maxBodyBytes of [
|
||||
Number.POSITIVE_INFINITY,
|
||||
Number.MAX_SAFE_INTEGER,
|
||||
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES + 1,
|
||||
]) {
|
||||
expect(
|
||||
safeParseModuleRouteDefinition({ ...validDefinition({ maxBodyBytes }), bodyMode })
|
||||
.success,
|
||||
).toBe(false);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("does not retain the caller's mutable definition object: mutating it afterward cannot change the parsed result", () => {
|
||||
const source = { ...validDefinition() };
|
||||
const result = safeParseModuleRouteDefinition(source);
|
||||
|
||||
Reference in New Issue
Block a user