feat: let a module route stream an image-sized upload, explicitly and bounded

ADR 0017 capped a module route at 1 MiB and said large media belonged to a
future storage design. That design arrived — ADR 0030 gave modules a bucket,
ADR 0031 took Ekklesia out of the read path — and it did not solve the write
path: signed write URLs were deliberately declined, so the only way bytes
reach a bucket is a module's own route, and a photograph is not 1 MiB.

Extend the route contract rather than adding a second HTTP system. A route
declares `bodyMode`: `buffered` (the default, and what every existing route
already meant) keeps the 1 MiB ceiling and materializes the body before the
handler runs; `streamed` raises the ceiling to 32 MiB and counts bytes as the
handler consumes them, holding nothing. The mode is explicit rather than
inferred from the size, because inferring it would mean an author who raised a
limit silently changed how their own handler receives its body and when a 413
can reach their own code.

32 MiB holds one 25 MiB image plus bounded overhead and is also the platform's
absolute ceiling: no route in any mode may declare more, and no unbounded
option exists in the contract. The unit is one file per request — twenty
images are twenty individually bounded requests, and nothing here can express
a 500 MiB aggregate whose failure would waste everything already sent.

Replace the buffering dependency with the platform's own limiter. A declared
Content-Length over the limit is refused synchronously, before authorization
and before a byte is read. Buffered mode now counts what actually arrives
rather than trusting the header, which closes a gap the previous
implementation had. Streamed mode wraps the body in a counting transform;
on overflow the stream errors, the source is cancelled, and the response is
replaced with the same sanitized 413 even if the handler swallowed the error —
a limit a handler can swallow is not a limit. An unread body is cancelled so
an abandoned upload releases its socket, and an abort stays an ordinary failed
request rather than becoming a 413.

Ordering is unchanged and now strictly better: in streamed mode no byte is
read until the handler reads it, and the handler runs only after authorization
succeeds, so an unauthorized caller cannot push an upload through at all.

The live suite uploads a real 25 MiB body over a real socket end to end.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-27 15:57:44 -05:00
co-authored by Claude Opus 5
parent c444ff570b
commit e36485bf99
12 changed files with 1093 additions and 50 deletions
+100
View File
@@ -432,6 +432,39 @@ async function createFixtureModule(
async handler(request, context) {
return Response.json({ created: true, principal: context.principal }, { status: 201 });
},
});
// An image-sized streamed upload (ADR 0042). The handler counts bytes as they
// arrive and never materializes the body, which is exactly what a real module
// storing an upload would do.
http.route({
method: "POST",
path: "/uploads",
permission: "calendar.events.read",
maxBodyBytes: 33554432,
bodyMode: "streamed",
async handler(request) {
let bytes = 0;
if (request.body) {
const reader = request.body.getReader();
for (;;) {
const chunk = await reader.read();
if (chunk.done) { break; }
bytes += chunk.value.byteLength;
}
}
return Response.json({ bytes }, { status: 201 });
},
});
http.route({
method: "POST",
path: "/small-uploads",
permission: "calendar.events.read",
maxBodyBytes: 64,
bodyMode: "streamed",
async handler(request) {
await request.arrayBuffer().catch(() => undefined);
return Response.json({ swallowed: true }, { status: 201 });
},
});`
: ""
}
@@ -2685,6 +2718,73 @@ describe.sequential("live API — installed module permission end-to-end", () =>
expect(await trustedResponse.json()).toMatchObject({ created: true });
});
it("streams one image-sized upload end to end over real HTTP", async () => {
// 25 MiB — an ordinary photograph, and the size the streamed ceiling exists to accommodate. It
// travels through a real socket, a real Node server, and the platform's byte counter, and the
// platform holds none of it (ADR 0042).
const image = new Uint8Array(25 * 1024 * 1024).fill(0x41);
const response = await fetch(`${origin}/api/v1/modules/calendar/uploads`, {
body: image,
headers: {
"content-type": "application/octet-stream",
cookie: ownerCookie,
origin: dashboardOrigin,
},
method: "POST",
});
expect(response.status).toBe(201);
expect(await response.json()).toEqual({ bytes: image.byteLength });
}, 60_000);
it("refuses a declared oversize on a streamed route with a sanitized 413", async () => {
const response = await fetch(`${origin}/api/v1/modules/calendar/small-uploads`, {
body: "a".repeat(65),
headers: {
"content-type": "text/plain",
cookie: ownerCookie,
origin: dashboardOrigin,
},
method: "POST",
});
expect(response.status).toBe(413);
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
});
it("terminates a chunked streamed body that exceeds the limit, even when the handler swallowed it", async () => {
// No declared `Content-Length`, so only the running byte count can bound this — and the
// handler deliberately ignores the stream error and answers `201`. The platform's `413` still
// wins: a limit a handler can swallow is not a limit.
const oversize = new ReadableStream<Uint8Array>({
pull(controller) {
controller.enqueue(new Uint8Array(512).fill(0x41));
},
});
const response = await fetch(`${origin}/api/v1/modules/calendar/small-uploads`, {
body: oversize,
duplex: "half",
headers: { cookie: ownerCookie, origin: dashboardOrigin },
method: "POST",
} as RequestInit);
expect(response.status).toBe(413);
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
}, 30_000);
it("keeps the server healthy after an oversized upload", async () => {
// The socket was released rather than left draining, so the very next request is served
// normally.
const response = await fetch(`${origin}/api/v1/modules/calendar/events/evt-43`, {
headers: { cookie: ownerCookie },
});
expect(response.status).toBe(200);
expect(await response.json()).toMatchObject({ eventId: "evt-43" });
});
it("rejects a well-formed permission absent from the composed registry, writing no grant or audit row", async () => {
const admin = await openAdministrativeClient(database.url);
try {