feat: let a module route stream an image-sized upload, explicitly and bounded
ADR 0017 capped a module route at 1 MiB and said large media belonged to a future storage design. That design arrived — ADR 0030 gave modules a bucket, ADR 0031 took Ekklesia out of the read path — and it did not solve the write path: signed write URLs were deliberately declined, so the only way bytes reach a bucket is a module's own route, and a photograph is not 1 MiB. Extend the route contract rather than adding a second HTTP system. A route declares `bodyMode`: `buffered` (the default, and what every existing route already meant) keeps the 1 MiB ceiling and materializes the body before the handler runs; `streamed` raises the ceiling to 32 MiB and counts bytes as the handler consumes them, holding nothing. The mode is explicit rather than inferred from the size, because inferring it would mean an author who raised a limit silently changed how their own handler receives its body and when a 413 can reach their own code. 32 MiB holds one 25 MiB image plus bounded overhead and is also the platform's absolute ceiling: no route in any mode may declare more, and no unbounded option exists in the contract. The unit is one file per request — twenty images are twenty individually bounded requests, and nothing here can express a 500 MiB aggregate whose failure would waste everything already sent. Replace the buffering dependency with the platform's own limiter. A declared Content-Length over the limit is refused synchronously, before authorization and before a byte is read. Buffered mode now counts what actually arrives rather than trusting the header, which closes a gap the previous implementation had. Streamed mode wraps the body in a counting transform; on overflow the stream errors, the source is cancelled, and the response is replaced with the same sanitized 413 even if the handler swallowed the error — a limit a handler can swallow is not a limit. An unread body is cancelled so an abandoned upload releases its socket, and an abort stays an ordinary failed request rather than becoming a 413. Ordering is unchanged and now strictly better: in streamed mode no byte is read until the handler reads it, and the handler runs only after authorization succeeds, so an unauthorized caller cannot push an upload through at all. The live suite uploads a real 25 MiB body over a real socket end to end. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
c444ff570b
commit
e36485bf99
+187
-17
@@ -1,6 +1,10 @@
|
||||
import { MODULE_ROUTE_MAX_BODY_BYTES_CEILING } from "@ekklesia/contracts";
|
||||
import type { Context } from "hono";
|
||||
import { bodyLimit } from "hono/body-limit";
|
||||
import {
|
||||
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES,
|
||||
MODULE_ROUTE_MAX_BODY_BYTES_CEILING,
|
||||
MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING,
|
||||
type ModuleRouteBodyMode,
|
||||
} from "@ekklesia/contracts";
|
||||
import type { Context, MiddlewareHandler } from "hono";
|
||||
|
||||
/**
|
||||
* The platform's one request-body policy, centralized here rather than scattered as magic numbers
|
||||
@@ -19,15 +23,20 @@ import { bodyLimit } from "hono/body-limit";
|
||||
* field individually bounded well under 1 KiB combined; 16 KiB leaves headroom without inviting
|
||||
* abuse.
|
||||
* - `MODULE_ROUTE_MAX_BODY_BYTES_CEILING` (1 MiB, re-exported from `@ekklesia/contracts` — the same
|
||||
* constant a module's own route declaration is validated against at registration) is the hard
|
||||
* ceiling no module route may exceed and doubles as the platform's overall request-body ceiling:
|
||||
* nothing in this platform ever declares, or enforces, a larger single request body than this.
|
||||
* Large media transfer is explicitly out of scope for ordinary module HTTP bodies — it belongs to
|
||||
* a future dedicated storage design, not to widening this ceiling.
|
||||
* constant a module's own **buffered** route declaration is validated against at registration) is
|
||||
* the ceiling for every body the platform materializes before a handler runs. It has not moved.
|
||||
* - `MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING` (32 MiB) is what a route may declare when it
|
||||
* explicitly opts into `bodyMode: "streamed"` (ADR 0042), and doubles as
|
||||
* `MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES`: no route, in any mode, may declare or accept a larger
|
||||
* single request body, and no unbounded option exists anywhere in the contract.
|
||||
*/
|
||||
export const AUTH_REQUEST_MAX_BODY_BYTES = 8 * 1024;
|
||||
export const CORE_MUTATION_MAX_BODY_BYTES = 16 * 1024;
|
||||
export { MODULE_ROUTE_MAX_BODY_BYTES_CEILING };
|
||||
export {
|
||||
MODULE_ROUTE_ABSOLUTE_MAX_BODY_BYTES,
|
||||
MODULE_ROUTE_MAX_BODY_BYTES_CEILING,
|
||||
MODULE_ROUTE_STREAMED_MAX_BODY_BYTES_CEILING,
|
||||
};
|
||||
|
||||
/** The one stable, sanitized public contract for every request-body-size rejection across the
|
||||
* platform — auth, Core, and module routes alike. Never leaks the configured limit, the offending
|
||||
@@ -37,12 +46,173 @@ export function requestBodyTooLargeResponse(c: Context): Response {
|
||||
return c.json({ code: "REQUEST_BODY_TOO_LARGE" }, 413);
|
||||
}
|
||||
|
||||
/** A single shared `hono/body-limit` factory so every call site uses the identical sanitized 413
|
||||
* contract rather than redefining `onError` per route. Covers both a declared oversized
|
||||
* `Content-Length` and an oversized chunked/streamed body with no declared length — see
|
||||
* `hono/body-limit`'s pinned implementation, which checks `Content-Length` synchronously before
|
||||
* ever reading the stream, and otherwise reads and counts the stream incrementally, rejecting as
|
||||
* soon as the running total exceeds `maxSize` rather than buffering the whole body first. */
|
||||
export function platformBodyLimit(maxSize: number) {
|
||||
return bodyLimit({ maxSize, onError: requestBodyTooLargeResponse });
|
||||
/**
|
||||
* A declared `Content-Length`, or `undefined` when there is not exactly one well-formed one.
|
||||
*
|
||||
* A malformed or contradictory value is treated as *absent* rather than as a rejection, deliberately:
|
||||
* the byte counting below bounds the request either way, so refusing here could only ever turn a
|
||||
* legitimate request away without making an illegitimate one any less bounded. Node's own HTTP
|
||||
* parser rejects genuinely malformed framing before this code sees the request at all.
|
||||
*/
|
||||
function declaredContentLength(request: Request): number | undefined {
|
||||
const header = request.headers.get("content-length");
|
||||
if (header === null || !/^\d{1,15}$/u.test(header.trim())) {
|
||||
return undefined;
|
||||
}
|
||||
return Number.parseInt(header.trim(), 10);
|
||||
}
|
||||
|
||||
/** Releases a body nothing consumed, so an abandoned or oversized upload does not hold a socket
|
||||
* open waiting to be read. Safe on a stream that is locked, already drained, or already errored. */
|
||||
function releaseUnreadBody(request: Request): void {
|
||||
const body = request.body;
|
||||
if (!body || body.locked) {
|
||||
return;
|
||||
}
|
||||
void body.cancel().catch(() => undefined);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads at most `maxSize` bytes, stopping and cancelling the source the moment the running total
|
||||
* exceeds it. Never allocates beyond what it has already accepted, so an over-limit body costs one
|
||||
* chunk past the limit rather than the whole upload.
|
||||
*/
|
||||
async function readBounded(
|
||||
body: ReadableStream<Uint8Array>,
|
||||
maxSize: number,
|
||||
): Promise<readonly Uint8Array[] | "over-limit"> {
|
||||
const reader = body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let size = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) {
|
||||
return chunks;
|
||||
}
|
||||
size += value.byteLength;
|
||||
if (size > maxSize) {
|
||||
await reader.cancel().catch(() => undefined);
|
||||
return "over-limit";
|
||||
}
|
||||
chunks.push(value);
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
function streamOf(chunks: readonly Uint8Array[]): ReadableStream<Uint8Array> {
|
||||
return new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
for (const chunk of chunks) {
|
||||
controller.enqueue(chunk);
|
||||
}
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The platform's one request-body limiter, for every route in the application.
|
||||
*
|
||||
* ## Both modes reject a declared oversize before anything else happens
|
||||
*
|
||||
* A `Content-Length` above the limit is refused synchronously, before authorization, before the
|
||||
* handler, and before a single body byte is read. That is the cheap rejection an ordinary
|
||||
* too-large upload takes.
|
||||
*
|
||||
* ## `buffered` — the body is materialized, bounded, before the handler runs
|
||||
*
|
||||
* The exact behaviour every route had before streaming existed, now with the counting done here
|
||||
* rather than by a dependency. Bytes are counted as they arrive and the request is rejected the
|
||||
* moment the running total passes the limit, so a lying or absent `Content-Length` is bounded by
|
||||
* what actually arrived rather than by what was claimed. The handler then receives a request whose
|
||||
* body is fully available and provably within its declared allowance — which is what makes
|
||||
* `await request.json()` safe in a module handler.
|
||||
*
|
||||
* The platform holds at most the route's own declared allowance in memory, and a buffered route may
|
||||
* declare at most 1 MiB. That bound is the whole reason the buffered ceiling has not moved.
|
||||
*
|
||||
* ## `streamed` — the body is counted as the handler consumes it, and never materialized
|
||||
*
|
||||
* Nothing is buffered. The body is wrapped in a counting transform and handed to the handler, which
|
||||
* reads it as a stream; the moment the running total passes the limit the stream errors, the source
|
||||
* is cancelled, and the response is replaced with the same sanitized `413` — even if the handler
|
||||
* caught the error and tried to answer anyway. That last part matters: a limit a handler can swallow
|
||||
* is not a limit.
|
||||
*
|
||||
* The consequence a module author must know is that in this mode a `413` can arrive **after** the
|
||||
* handler has started, so a handler that persists as it reads has to be prepared for a partially
|
||||
* consumed upload. Writing to a temporary key and committing only after the read completes is the
|
||||
* pattern; the platform cannot do it for you, because only the module knows what a partial write
|
||||
* means to its own data.
|
||||
*
|
||||
* ## Ordering
|
||||
*
|
||||
* This runs before permission evaluation for both modes, so a request that is *declared* too large
|
||||
* never reaches authorization work. In `streamed` mode no body byte is read until the handler reads
|
||||
* it, and the handler only runs after authorization succeeds — so an unauthorized caller can never
|
||||
* push a large upload through the platform at all.
|
||||
*
|
||||
* ## Timeouts
|
||||
*
|
||||
* Body reading is bounded in time by the server's own `headersTimeout`/`requestTimeout`
|
||||
* (`apps/api/src/server.ts`), not by anything here. A slow upload is a slow *request*, and the
|
||||
* platform already has one answer for that rather than a second per-route one.
|
||||
*/
|
||||
export function platformBodyLimit(
|
||||
maxSize: number,
|
||||
mode: ModuleRouteBodyMode = "buffered",
|
||||
): MiddlewareHandler {
|
||||
return async function enforceBodyLimit(c, next) {
|
||||
const request = c.req.raw;
|
||||
const declared = declaredContentLength(request);
|
||||
if (declared !== undefined && declared > maxSize) {
|
||||
return requestBodyTooLargeResponse(c);
|
||||
}
|
||||
if (!request.body) {
|
||||
return next();
|
||||
}
|
||||
|
||||
if (mode === "buffered") {
|
||||
const chunks = await readBounded(request.body, maxSize);
|
||||
if (chunks === "over-limit") {
|
||||
return requestBodyTooLargeResponse(c);
|
||||
}
|
||||
c.req.raw = new Request(request, {
|
||||
body: streamOf(chunks),
|
||||
// Required by undici whenever a `Request` is constructed from a stream.
|
||||
duplex: "half",
|
||||
} as RequestInit);
|
||||
return next();
|
||||
}
|
||||
|
||||
let overLimit = false;
|
||||
let total = 0;
|
||||
const counted = request.body.pipeThrough(
|
||||
new TransformStream<Uint8Array, Uint8Array>({
|
||||
transform(chunk, controller) {
|
||||
total += chunk.byteLength;
|
||||
if (total > maxSize) {
|
||||
overLimit = true;
|
||||
// Erroring the transform propagates back to the source and cancels it, so the socket is
|
||||
// released rather than left draining an upload nobody will read.
|
||||
controller.error(new Error("REQUEST_BODY_TOO_LARGE"));
|
||||
return;
|
||||
}
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
}),
|
||||
);
|
||||
c.req.raw = new Request(request, { body: counted, duplex: "half" } as RequestInit);
|
||||
|
||||
await next();
|
||||
|
||||
if (overLimit) {
|
||||
c.res = requestBodyTooLargeResponse(c);
|
||||
}
|
||||
releaseUnreadBody(c.req.raw);
|
||||
return;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -235,9 +235,12 @@ export function mountModuleHttpRoutes(
|
||||
|
||||
const requireRoutePermission = requirePermission(middlewareOptions, route.permission);
|
||||
// The route's own conscious body contract (`route.maxBodyBytes` — 0 means no body accepted at
|
||||
// all) is enforced before permission evaluation ever runs, so an oversized request never
|
||||
// reaches authorization work, let alone the handler or the module database.
|
||||
const routeBodyLimit = platformBodyLimit(route.maxBodyBytes);
|
||||
// all) is enforced before permission evaluation ever runs, so a request *declared* oversized
|
||||
// never reaches authorization work, let alone the handler or the module database. In
|
||||
// `streamed` mode no body byte is read until the handler reads it, and the handler runs only
|
||||
// after authorization succeeds, so an unauthorized caller can never push an upload through
|
||||
// (ADR 0042).
|
||||
const routeBodyLimit = platformBodyLimit(route.maxBodyBytes, route.bodyMode);
|
||||
const path = moduleRoutePath(route.moduleId, route.path);
|
||||
const finalHandler = async (c: Context): Promise<Response> => {
|
||||
const context: ModuleRouteHandlerContext = Object.freeze({
|
||||
|
||||
@@ -432,6 +432,39 @@ async function createFixtureModule(
|
||||
async handler(request, context) {
|
||||
return Response.json({ created: true, principal: context.principal }, { status: 201 });
|
||||
},
|
||||
});
|
||||
// An image-sized streamed upload (ADR 0042). The handler counts bytes as they
|
||||
// arrive and never materializes the body, which is exactly what a real module
|
||||
// storing an upload would do.
|
||||
http.route({
|
||||
method: "POST",
|
||||
path: "/uploads",
|
||||
permission: "calendar.events.read",
|
||||
maxBodyBytes: 33554432,
|
||||
bodyMode: "streamed",
|
||||
async handler(request) {
|
||||
let bytes = 0;
|
||||
if (request.body) {
|
||||
const reader = request.body.getReader();
|
||||
for (;;) {
|
||||
const chunk = await reader.read();
|
||||
if (chunk.done) { break; }
|
||||
bytes += chunk.value.byteLength;
|
||||
}
|
||||
}
|
||||
return Response.json({ bytes }, { status: 201 });
|
||||
},
|
||||
});
|
||||
http.route({
|
||||
method: "POST",
|
||||
path: "/small-uploads",
|
||||
permission: "calendar.events.read",
|
||||
maxBodyBytes: 64,
|
||||
bodyMode: "streamed",
|
||||
async handler(request) {
|
||||
await request.arrayBuffer().catch(() => undefined);
|
||||
return Response.json({ swallowed: true }, { status: 201 });
|
||||
},
|
||||
});`
|
||||
: ""
|
||||
}
|
||||
@@ -2685,6 +2718,73 @@ describe.sequential("live API — installed module permission end-to-end", () =>
|
||||
expect(await trustedResponse.json()).toMatchObject({ created: true });
|
||||
});
|
||||
|
||||
it("streams one image-sized upload end to end over real HTTP", async () => {
|
||||
// 25 MiB — an ordinary photograph, and the size the streamed ceiling exists to accommodate. It
|
||||
// travels through a real socket, a real Node server, and the platform's byte counter, and the
|
||||
// platform holds none of it (ADR 0042).
|
||||
const image = new Uint8Array(25 * 1024 * 1024).fill(0x41);
|
||||
|
||||
const response = await fetch(`${origin}/api/v1/modules/calendar/uploads`, {
|
||||
body: image,
|
||||
headers: {
|
||||
"content-type": "application/octet-stream",
|
||||
cookie: ownerCookie,
|
||||
origin: dashboardOrigin,
|
||||
},
|
||||
method: "POST",
|
||||
});
|
||||
|
||||
expect(response.status).toBe(201);
|
||||
expect(await response.json()).toEqual({ bytes: image.byteLength });
|
||||
}, 60_000);
|
||||
|
||||
it("refuses a declared oversize on a streamed route with a sanitized 413", async () => {
|
||||
const response = await fetch(`${origin}/api/v1/modules/calendar/small-uploads`, {
|
||||
body: "a".repeat(65),
|
||||
headers: {
|
||||
"content-type": "text/plain",
|
||||
cookie: ownerCookie,
|
||||
origin: dashboardOrigin,
|
||||
},
|
||||
method: "POST",
|
||||
});
|
||||
|
||||
expect(response.status).toBe(413);
|
||||
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
|
||||
});
|
||||
|
||||
it("terminates a chunked streamed body that exceeds the limit, even when the handler swallowed it", async () => {
|
||||
// No declared `Content-Length`, so only the running byte count can bound this — and the
|
||||
// handler deliberately ignores the stream error and answers `201`. The platform's `413` still
|
||||
// wins: a limit a handler can swallow is not a limit.
|
||||
const oversize = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
controller.enqueue(new Uint8Array(512).fill(0x41));
|
||||
},
|
||||
});
|
||||
|
||||
const response = await fetch(`${origin}/api/v1/modules/calendar/small-uploads`, {
|
||||
body: oversize,
|
||||
duplex: "half",
|
||||
headers: { cookie: ownerCookie, origin: dashboardOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(413);
|
||||
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
|
||||
}, 30_000);
|
||||
|
||||
it("keeps the server healthy after an oversized upload", async () => {
|
||||
// The socket was released rather than left draining, so the very next request is served
|
||||
// normally.
|
||||
const response = await fetch(`${origin}/api/v1/modules/calendar/events/evt-43`, {
|
||||
headers: { cookie: ownerCookie },
|
||||
});
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(await response.json()).toMatchObject({ eventId: "evt-43" });
|
||||
});
|
||||
|
||||
it("rejects a well-formed permission absent from the composed registry, writing no grant or audit row", async () => {
|
||||
const admin = await openAdministrativeClient(database.url);
|
||||
try {
|
||||
|
||||
@@ -58,14 +58,17 @@ function fakeStore(): AuthorizationStore {
|
||||
};
|
||||
}
|
||||
|
||||
/** Defaults `maxBodyBytes` to a generous value for tests that are not themselves exercising the
|
||||
/** Defaults `maxBodyBytes` and `bodyMode` for tests that are not themselves exercising the
|
||||
* body-size contract, so the many pre-existing routing/authorization/CORS scenarios below do not
|
||||
* each need to restate an unrelated body-size choice; the dedicated "module route body limits"
|
||||
* suite further down overrides it explicitly to prove the contract itself. */
|
||||
* suite further down overrides them explicitly to prove the contract itself. */
|
||||
function moduleRoute(
|
||||
fields: Omit<RegisteredModuleHttpRoute, "maxBodyBytes"> & { maxBodyBytes?: number },
|
||||
fields: Omit<RegisteredModuleHttpRoute, "bodyMode" | "maxBodyBytes"> & {
|
||||
bodyMode?: RegisteredModuleHttpRoute["bodyMode"];
|
||||
maxBodyBytes?: number;
|
||||
},
|
||||
): RegisteredModuleHttpRoute {
|
||||
return Object.freeze({ maxBodyBytes: 1024, ...fields });
|
||||
return Object.freeze({ bodyMode: "buffered", maxBodyBytes: 1024, ...fields });
|
||||
}
|
||||
|
||||
/** A small typed fake `ModuleRuntimeServices` — these tests exercise route mounting,
|
||||
@@ -1634,3 +1637,375 @@ describe("module route overlap can never reach mounting", () => {
|
||||
expect(moduleRoutesOverlap(paramRoute, staticRoute)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Explicit, bounded streaming uploads (ADR 0042).
|
||||
*
|
||||
* The distinction being proven throughout is *when* the platform knows a body is too large, and what
|
||||
* it holds while finding out. In `buffered` mode the answer is "before the handler runs, holding at
|
||||
* most the route's own declared allowance". In `streamed` mode it is "as the handler consumes it,
|
||||
* holding nothing" — and the `413` still wins, even when the handler tried to answer anyway.
|
||||
*/
|
||||
describe("module route streamed body limits", () => {
|
||||
const oneMebibyte = 1024 * 1024;
|
||||
|
||||
/** A body of `size` bytes delivered in realistic chunks, with no declared `Content-Length` — the
|
||||
* chunked-transfer shape a browser `fetch` of a `ReadableStream` actually produces. */
|
||||
function chunkedBody(size: number, chunkSize = 64 * 1024): ReadableStream<Uint8Array> {
|
||||
let sent = 0;
|
||||
return new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (sent >= size) {
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
const next = Math.min(chunkSize, size - sent);
|
||||
sent += next;
|
||||
controller.enqueue(new Uint8Array(next).fill(97));
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function streamingApp(options: {
|
||||
readonly handler: ModuleRouteHandler;
|
||||
readonly maxBodyBytes: number;
|
||||
readonly evaluator?: AuthorizationEvaluator;
|
||||
}) {
|
||||
return buildModuleApp({
|
||||
...(options.evaluator ? { evaluator: options.evaluator } : {}),
|
||||
registry: permissionRegistry([{ id: writePermission, baseline: "public" }]),
|
||||
routes: [
|
||||
moduleRoute({
|
||||
bodyMode: "streamed",
|
||||
handler: options.handler,
|
||||
maxBodyBytes: options.maxBodyBytes,
|
||||
method: "POST",
|
||||
moduleId: "calendar",
|
||||
path: "/uploads",
|
||||
permission: writePermission,
|
||||
}),
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
/** Counts what the handler actually received, which is the only honest way to tell "streamed" from
|
||||
* "buffered then handed over". */
|
||||
async function countingHandler(received: { bytes: number }): Promise<ModuleRouteHandler> {
|
||||
return async (request) => {
|
||||
const body = request.body;
|
||||
if (!body) {
|
||||
return Response.json({ bytes: 0 });
|
||||
}
|
||||
const reader = body.getReader();
|
||||
for (;;) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) {
|
||||
break;
|
||||
}
|
||||
received.bytes += value.byteLength;
|
||||
}
|
||||
return Response.json({ bytes: received.bytes });
|
||||
};
|
||||
}
|
||||
|
||||
it("accepts a body of exactly the declared limit", async () => {
|
||||
const received = { bytes: 0 };
|
||||
const { app } = streamingApp({
|
||||
handler: await countingHandler(received),
|
||||
maxBodyBytes: oneMebibyte,
|
||||
});
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(oneMebibyte),
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(await response.json()).toEqual({ bytes: oneMebibyte });
|
||||
});
|
||||
|
||||
it("rejects a declared Content-Length over the limit before the handler runs", async () => {
|
||||
const received = { bytes: 0 };
|
||||
const counting = await countingHandler(received);
|
||||
const handler = vi.fn<ModuleRouteHandler>(counting);
|
||||
const { app } = streamingApp({ handler, maxBodyBytes: 1024 });
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(2048, 16),
|
||||
duplex: "half",
|
||||
headers: { "content-length": String(1025), origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(413);
|
||||
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
|
||||
// The cheap rejection: the declared size alone decided it, so no handler work happened and no
|
||||
// byte of the upload reached module code.
|
||||
expect(handler).not.toHaveBeenCalled();
|
||||
expect(received.bytes).toBe(0);
|
||||
});
|
||||
|
||||
it("terminates a body whose actual size exceeds a smaller declared Content-Length", async () => {
|
||||
// A misleading `Content-Length` buys nothing: the running byte count is what bounds the request,
|
||||
// so a header claiming 8 bytes cannot smuggle a megabyte past the limit.
|
||||
const received = { bytes: 0 };
|
||||
const { app } = streamingApp({
|
||||
handler: await countingHandler(received),
|
||||
maxBodyBytes: 64,
|
||||
});
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(4096, 32),
|
||||
duplex: "half",
|
||||
headers: { "content-length": "8", origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(413);
|
||||
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
|
||||
// It stopped promptly rather than draining the whole upload first.
|
||||
expect(received.bytes).toBeLessThanOrEqual(96);
|
||||
});
|
||||
|
||||
it("answers 413 even when the handler swallowed the stream error and tried to respond", async () => {
|
||||
// A limit a handler can swallow is not a limit.
|
||||
const handler = vi.fn<ModuleRouteHandler>(async (request) => {
|
||||
try {
|
||||
await request.arrayBuffer();
|
||||
} catch {
|
||||
// Deliberately ignored, exactly as a careless module might.
|
||||
}
|
||||
return Response.json({ ok: "handler won" });
|
||||
});
|
||||
const { app } = streamingApp({ handler, maxBodyBytes: 64 });
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(4096, 32),
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(handler).toHaveBeenCalledTimes(1);
|
||||
expect(response.status).toBe(413);
|
||||
expect(await response.json()).toEqual({ code: "REQUEST_BODY_TOO_LARGE" });
|
||||
});
|
||||
|
||||
it("never buffers the body: the handler sees the first bytes before the last are sent", async () => {
|
||||
let released: (() => void) | undefined;
|
||||
const finished = new Promise<void>((resolve) => {
|
||||
released = resolve;
|
||||
});
|
||||
let firstChunkSeen = false;
|
||||
let secondChunkQueued = false;
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
async pull(controller) {
|
||||
if (!secondChunkQueued) {
|
||||
secondChunkQueued = true;
|
||||
controller.enqueue(new Uint8Array(16).fill(97));
|
||||
return;
|
||||
}
|
||||
// Only produce the rest once the handler has demonstrably consumed the first chunk. A
|
||||
// buffering platform would deadlock here rather than reach the assertion below.
|
||||
await finished;
|
||||
controller.close();
|
||||
},
|
||||
});
|
||||
const handler = vi.fn<ModuleRouteHandler>(async (request) => {
|
||||
const reader = (request.body as ReadableStream<Uint8Array>).getReader();
|
||||
const first = await reader.read();
|
||||
firstChunkSeen = !first.done;
|
||||
released?.();
|
||||
for (;;) {
|
||||
const next = await reader.read();
|
||||
if (next.done) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
return Response.json({ ok: true });
|
||||
});
|
||||
const { app } = streamingApp({ handler, maxBodyBytes: oneMebibyte });
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body,
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(firstChunkSeen).toBe(true);
|
||||
expect(response.status).toBe(200);
|
||||
});
|
||||
|
||||
it("releases an aborted upload without taking the platform down", async () => {
|
||||
const aborted = new ReadableStream<Uint8Array>({
|
||||
start(controller) {
|
||||
controller.enqueue(new Uint8Array(16).fill(97));
|
||||
controller.error(new Error("client went away"));
|
||||
},
|
||||
});
|
||||
const handler = vi.fn<ModuleRouteHandler>(async (request) => {
|
||||
await request.arrayBuffer();
|
||||
return Response.json({ ok: true });
|
||||
});
|
||||
const { app } = streamingApp({ handler, maxBodyBytes: oneMebibyte });
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: aborted,
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
// An abort is a failed request, never an unhandled rejection or a 413: nothing exceeded a limit.
|
||||
expect(response.status).toBe(500);
|
||||
expect(await response.json()).toEqual({ code: "MODULE_HANDLER_FAILED" });
|
||||
});
|
||||
|
||||
it("releases a body the handler never read", async () => {
|
||||
let cancelled = false;
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
cancel() {
|
||||
cancelled = true;
|
||||
},
|
||||
pull(controller) {
|
||||
controller.enqueue(new Uint8Array(16).fill(97));
|
||||
},
|
||||
});
|
||||
const { app } = streamingApp({
|
||||
handler: async () => Response.json({ ok: true }),
|
||||
maxBodyBytes: oneMebibyte,
|
||||
});
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body,
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(200);
|
||||
expect(cancelled).toBe(true);
|
||||
});
|
||||
|
||||
it("refuses a declared oversize before authorization, and never streams for a denied caller", async () => {
|
||||
const evaluator: AuthorizationEvaluator = {
|
||||
authorize: vi.fn().mockResolvedValue({ allowed: false }),
|
||||
};
|
||||
const handler = vi.fn<ModuleRouteHandler>(async () => Response.json({ ok: true }));
|
||||
const { app } = buildModuleApp({
|
||||
evaluator,
|
||||
registry: permissionRegistry([{ id: writePermission, baseline: "restricted" }]),
|
||||
routes: [
|
||||
moduleRoute({
|
||||
bodyMode: "streamed",
|
||||
handler,
|
||||
maxBodyBytes: 64,
|
||||
method: "POST",
|
||||
moduleId: "calendar",
|
||||
path: "/uploads",
|
||||
permission: writePermission,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
const declaredOversize = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: "a".repeat(65),
|
||||
headers: { "content-length": "65", origin: trustedOrigin },
|
||||
method: "POST",
|
||||
});
|
||||
expect(declaredOversize.status).toBe(413);
|
||||
expect(evaluator.authorize).not.toHaveBeenCalled();
|
||||
|
||||
// A within-limit body from a denied caller is refused by authorization, and the handler — the
|
||||
// only thing that ever reads a byte in streamed mode — never runs.
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(32, 8),
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
expect(response.status).toBe(403);
|
||||
expect(handler).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still rejects an untrusted Origin before any body handling", async () => {
|
||||
const handler = vi.fn<ModuleRouteHandler>(async () => Response.json({ ok: true }));
|
||||
const { app } = streamingApp({ handler, maxBodyBytes: oneMebibyte });
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: chunkedBody(4096),
|
||||
duplex: "half",
|
||||
headers: { origin: untrustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
|
||||
expect(response.status).toBe(403);
|
||||
expect(await response.json()).toEqual({ code: "UNTRUSTED_ORIGIN" });
|
||||
expect(handler).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps every rejection sanitized, naming no limit, size, or body content", async () => {
|
||||
const { app } = streamingApp({
|
||||
handler: async () => Response.json({ ok: true }),
|
||||
maxBodyBytes: 64,
|
||||
});
|
||||
|
||||
const response = await app.request("/api/v1/modules/calendar/uploads", {
|
||||
body: "secret-payload-".repeat(64),
|
||||
headers: { "content-length": String(15 * 64), origin: trustedOrigin },
|
||||
method: "POST",
|
||||
});
|
||||
const body = await response.text();
|
||||
|
||||
expect(response.status).toBe(413);
|
||||
expect(body).toBe('{"code":"REQUEST_BODY_TOO_LARGE"}');
|
||||
expect(body).not.toContain("secret-payload");
|
||||
expect(body).not.toContain("64");
|
||||
expect(response.headers.get("cache-control")).toBe("no-store");
|
||||
});
|
||||
|
||||
it("still materializes a buffered body before the handler runs", async () => {
|
||||
// The pre-existing promise, unchanged: a buffered handler may call `request.json()` and know the
|
||||
// body is already there and already within its declared allowance.
|
||||
const handler = vi.fn<ModuleRouteHandler>(async (request) => {
|
||||
const text = await request.text();
|
||||
return Response.json({ length: text.length });
|
||||
});
|
||||
const { app } = buildModuleApp({
|
||||
registry: permissionRegistry([{ id: writePermission, baseline: "public" }]),
|
||||
routes: [
|
||||
moduleRoute({
|
||||
bodyMode: "buffered",
|
||||
handler,
|
||||
maxBodyBytes: 128,
|
||||
method: "POST",
|
||||
moduleId: "calendar",
|
||||
path: "/events",
|
||||
permission: writePermission,
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
const withinLimit = await app.request("/api/v1/modules/calendar/events", {
|
||||
body: chunkedBody(128, 16),
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
expect(withinLimit.status).toBe(200);
|
||||
expect(await withinLimit.json()).toEqual({ length: 128 });
|
||||
|
||||
const overLimit = await app.request("/api/v1/modules/calendar/events", {
|
||||
body: chunkedBody(129, 16),
|
||||
duplex: "half",
|
||||
headers: { origin: trustedOrigin },
|
||||
method: "POST",
|
||||
} as RequestInit);
|
||||
expect(overLimit.status).toBe(413);
|
||||
// A buffered route's handler never sees an over-limit request at all.
|
||||
expect(handler).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user