Zayin/Ekklesia
3
0
Fork 0
Code Issues 1 Pull Requests 3 Actions Packages Projects Releases 8 Wiki Activity

feat: let a module route stream an image-sized upload, explicitly and bounded

Browse Source 
ADR 0017 capped a module route at 1 MiB and said large media belonged to a
future storage design. That design arrived — ADR 0030 gave modules a bucket,
ADR 0031 took Ekklesia out of the read path — and it did not solve the write
path: signed write URLs were deliberately declined, so the only way bytes
reach a bucket is a module's own route, and a photograph is not 1 MiB.

Extend the route contract rather than adding a second HTTP system. A route
declares `bodyMode`: `buffered` (the default, and what every existing route
already meant) keeps the 1 MiB ceiling and materializes the body before the
handler runs; `streamed` raises the ceiling to 32 MiB and counts bytes as the
handler consumes them, holding nothing. The mode is explicit rather than
inferred from the size, because inferring it would mean an author who raised a
limit silently changed how their own handler receives its body and when a 413
can reach their own code.

32 MiB holds one 25 MiB image plus bounded overhead and is also the platform's
absolute ceiling: no route in any mode may declare more, and no unbounded
option exists in the contract. The unit is one file per request — twenty
images are twenty individually bounded requests, and nothing here can express
a 500 MiB aggregate whose failure would waste everything already sent.

Replace the buffering dependency with the platform's own limiter. A declared
Content-Length over the limit is refused synchronously, before authorization
and before a byte is read. Buffered mode now counts what actually arrives
rather than trusting the header, which closes a gap the previous
implementation had. Streamed mode wraps the body in a counting transform;
on overflow the stream errors, the source is cancelled, and the response is
replaced with the same sanitized 413 even if the handler swallowed the error —
a limit a handler can swallow is not a limit. An unread body is cancelled so
an abandoned upload releases its socket, and an abort stays an ordinary failed
request rather than becoming a 413.

Ordering is unchanged and now strictly better: in streamed mode no byte is
read until the handler reads it, and the handler runs only after authorization
succeeds, so an unauthorized caller cannot push an upload through at all.

The live suite uploads a real 25 MiB body over a real socket end to end.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-27 15:57:44 -05:00
co-authored by Claude Opus 5
parent c444ff570b
commit e36485bf99
12 changed files with 1093 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/api/src/body-limits.ts
+187 -17
View File
@@ -1,6 +1,10 @@
import { MODULE_ROUTE_MAX_BODY_BYTES_CEILING } from "@ekklesia/contracts";
import type { Context } from "hono";
import { bodyLimit } from "hono/body-limit";
import {
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES,
MODULE_ROUTE_MAX_BODY_BYTES_CEILING,
MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING,
type ModuleRouteBodyMode,
} from "@ekklesia/contracts";
import type { Context, MiddlewareHandler } from "hono";
/**
* The platform's one request-body policy, centralized here rather than scattered as magic numbers
@@ -19,15 +23,20 @@ import { bodyLimit } from "hono/body-limit";
* field individually bounded well under 1 KiB combined; 16 KiB leaves headroom without inviting
* abuse.
* - `MODULE_ROUTE_MAX_BODY_BYTES_CEILING` (1 MiB, re-exported from `@ekklesia/contracts` — the same
* constant a module's own route declaration is validated against at registration) is the hard
* ceiling no module route may exceed and doubles as the platform's overall request-body ceiling:
* nothing in this platform ever declares, or enforces, a larger single request body than this.
* Large media transfer is explicitly out of scope for ordinary module HTTP bodies — it belongs to
* a future dedicated storage design, not to widening this ceiling.
* constant a module's own **buffered** route declaration is validated against at registration) is
* the ceiling for every body the platform materializes before a handler runs. It has not moved.
* - `MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING` (32 MiB) is what a route may declare when it
* explicitly opts into `bodyMode: "streamed"` (ADR 0042), and doubles as
* `MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES`: no route, in any mode, may declare or accept a larger
* single request body, and no unbounded option exists anywhere in the contract.
*/
export const AUTH_REQUEST_MAX_BODY_BYTES = 8 * 1024;
export const CORE_MUTATION_MAX_BODY_BYTES = 16 * 1024;
export { MODULE_ROUTE_MAX_BODY_BYTES_CEILING };
export {
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES,
MODULE_ROUTE_MAX_BODY_BYTES_CEILING,
MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING,
};
/** The one stable, sanitized public contract for every request-body-size rejection across the
* platform — auth, Core, and module routes alike. Never leaks the configured limit, the offending
@@ -37,12 +46,173 @@ export function requestBodyTooLargeResponse(c: Context): Response {
return c.json({ code: "REQUEST_BODY_TOO_LARGE" }, 413);
}
/** A single shared `hono/body-limit` factory so every call site uses the identical sanitized 413
* contract rather than redefining `onError` per route. Covers both a declared oversized
* `Content-Length` and an oversized chunked/streamed body with no declared length — see
* `hono/body-limit`'s pinned implementation, which checks `Content-Length` synchronously before
* ever reading the stream, and otherwise reads and counts the stream incrementally, rejecting as
* soon as the running total exceeds `maxSize` rather than buffering the whole body first. */
export function platformBodyLimit(maxSize: number) {
return bodyLimit({ maxSize, onError: requestBodyTooLargeResponse });
/**
* A declared `Content-Length`, or `undefined` when there is not exactly one well-formed one.
*
* A malformed or contradictory value is treated as *absent* rather than as a rejection, deliberately:
* the byte counting below bounds the request either way, so refusing here could only ever turn a
* legitimate request away without making an illegitimate one any less bounded. Node's own HTTP
* parser rejects genuinely malformed framing before this code sees the request at all.
*/
function declaredContentLength(request: Request): number | undefined {
const header = request.headers.get("content-length");
if (header === null || !/^\d{1,15}$/u.test(header.trim())) {
return undefined;
}
return Number.parseInt(header.trim(), 10);
}
/** Releases a body nothing consumed, so an abandoned or oversized upload does not hold a socket
* open waiting to be read. Safe on a stream that is locked, already drained, or already errored. */
function releaseUnreadBody(request: Request): void {
const body = request.body;
if (!body || body.locked) {
return;
}
void body.cancel().catch(() => undefined);
}
/**
* Reads at most `maxSize` bytes, stopping and cancelling the source the moment the running total
* exceeds it. Never allocates beyond what it has already accepted, so an over-limit body costs one
* chunk past the limit rather than the whole upload.
*/
async function readBounded(
body: ReadableStream<Uint8Array>,
maxSize: number,
): Promise<readonly Uint8Array[] | "over-limit"> {
const reader = body.getReader();
const chunks: Uint8Array[] = [];
let size = 0;
try {
for (;;) {
const { done, value } = await reader.read();
if (done) {
return chunks;
}
size += value.byteLength;
if (size > maxSize) {
await reader.cancel().catch(() => undefined);
return "over-limit";
}
chunks.push(value);
}
} finally {
reader.releaseLock();
}
}
function streamOf(chunks: readonly Uint8Array[]): ReadableStream<Uint8Array> {
return new ReadableStream<Uint8Array>({
start(controller) {
for (const chunk of chunks) {
controller.enqueue(chunk);
}
controller.close();
},
});
}
/**
* The platform's one request-body limiter, for every route in the application.
*
* ## Both modes reject a declared oversize before anything else happens
*
* A `Content-Length` above the limit is refused synchronously, before authorization, before the
* handler, and before a single body byte is read. That is the cheap rejection an ordinary
* too-large upload takes.
*
* ## `buffered` — the body is materialized, bounded, before the handler runs
*
* The exact behaviour every route had before streaming existed, now with the counting done here
* rather than by a dependency. Bytes are counted as they arrive and the request is rejected the
* moment the running total passes the limit, so a lying or absent `Content-Length` is bounded by
* what actually arrived rather than by what was claimed. The handler then receives a request whose
* body is fully available and provably within its declared allowance — which is what makes
* `await request.json()` safe in a module handler.
*
* The platform holds at most the route's own declared allowance in memory, and a buffered route may
* declare at most 1 MiB. That bound is the whole reason the buffered ceiling has not moved.
*
* ## `streamed` — the body is counted as the handler consumes it, and never materialized
*
* Nothing is buffered. The body is wrapped in a counting transform and handed to the handler, which
* reads it as a stream; the moment the running total passes the limit the stream errors, the source
* is cancelled, and the response is replaced with the same sanitized `413` — even if the handler
* caught the error and tried to answer anyway. That last part matters: a limit a handler can swallow
* is not a limit.
*
* The consequence a module author must know is that in this mode a `413` can arrive **after** the
* handler has started, so a handler that persists as it reads has to be prepared for a partially
* consumed upload. Writing to a temporary key and committing only after the read completes is the
* pattern; the platform cannot do it for you, because only the module knows what a partial write
* means to its own data.
*
* ## Ordering
*
* This runs before permission evaluation for both modes, so a request that is *declared* too large
* never reaches authorization work. In `streamed` mode no body byte is read until the handler reads
* it, and the handler only runs after authorization succeeds — so an unauthorized caller can never
* push a large upload through the platform at all.
*
* ## Timeouts
*
* Body reading is bounded in time by the server's own `headersTimeout`/`requestTimeout`
* (`apps/api/src/server.ts`), not by anything here. A slow upload is a slow *request*, and the
* platform already has one answer for that rather than a second per-route one.
*/
export function platformBodyLimit(
maxSize: number,
mode: ModuleRouteBodyMode = "buffered",
): MiddlewareHandler {
return async function enforceBodyLimit(c, next) {
const request = c.req.raw;
const declared = declaredContentLength(request);
if (declared !== undefined && declared > maxSize) {
return requestBodyTooLargeResponse(c);
}
if (!request.body) {
return next();
}
if (mode === "buffered") {
const chunks = await readBounded(request.body, maxSize);
if (chunks === "over-limit") {
return requestBodyTooLargeResponse(c);
}
c.req.raw = new Request(request, {
body: streamOf(chunks),
// Required by undici whenever a `Request` is constructed from a stream.
duplex: "half",
} as RequestInit);
return next();
}
let overLimit = false;
let total = 0;
const counted = request.body.pipeThrough(
new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
total += chunk.byteLength;
if (total > maxSize) {
overLimit = true;
// Erroring the transform propagates back to the source and cancels it, so the socket is
// released rather than left draining an upload nobody will read.
controller.error(new Error("REQUEST_BODY_TOO_LARGE"));
return;
}
controller.enqueue(chunk);
},
}),
);
c.req.raw = new Request(request, { body: counted, duplex: "half" } as RequestInit);
await next();
if (overLimit) {
c.res = requestBodyTooLargeResponse(c);
}
releaseUnreadBody(c.req.raw);
return;
};
}