fix(deploy): reach storage over HTTP/1.1, and own storage CORS in the proxy #58

Merged

Zayin_Admin merged 2 commits from fix/storage-delivery-http1-cors into main 2026-10-06 01:39:39 +00:00

Conversation 1 Commits 2 Files Changed 4

+250 -25

ekklesia-bot commented 2026-10-06 00:07:34 +00:00

Member

Copy Link

Copy Source

fix(deploy): reach storage over HTTP/1.1, and own storage CORS in the proxy

1. Every reload of a cached files. image failed with ERR_HTTP2_PROTOCOL_ERROR

Measured layer by layer on a split reference deployment (platform v0.1.1 + Signage, SeaweedFS 4.28, Caddy 2.11.4):

Layer	First load	Same signed URL again (conditional: If-None-Match / If-Modified-Since)
Chromium (DevTools)	200 over h2	net::ERR_HTTP2_PROTOCOL_ERROR, for fetch and <img> alike
curl --http2 through Caddy	200	stream reset, INTERNAL_ERROR. Caddy logs aborting with incomplete response … reading: unexpected EOF
curl --http1.1 directly to SeaweedFS	200	clean 304
curl --http2 directly to SeaweedFS	200	304 with content-type: application/xml and content-length: 327, and 0 body bytes

Cause:

• Caddy's TLS upstream negotiates HTTP/2 with SeaweedFS.
• Over HTTP/2, SeaweedFS's 304 declares a body it never sends.
• Caddy reads for the promised 327 bytes, gets end-of-stream, and resets the browser's stream.

Fix: versions 1.1 for the storage upstream. Browsers still reach Caddy over HTTP/2 and HTTP/3.

The upstream bug is drafted as a SeaweedFS issue for the maintainer to submit.

2. Storage CORS was the object store's, and reflected everything

SeaweedFS's default answered any Origin, an unrelated one included, with PUT, POST, DELETE and any request header. It also answered preflights itself.

Delivery-layer CORS is now the deployment's, decided in the proxy, so it stays the same with R2 or any other S3 backend:

• CORS only for the origins that read objects with fetch():
  • same-origin topology: the Ekklesia origin;
  • split topology: the API origin (displays) and the Dashboard origin. Signage's preview builds a display controller with the default media cache, so it fetch()es storage URLs from the Dashboard.
• GET and HEAD only, with Vary: Origin;
• preflights are answered by Caddy (204) and never reach storage;
• the object store's own CORS headers are stripped.

<img> needs no CORS and is unaffected. Every object URL is still a signed, expiring capability.

Possible follow-up (Signage): if its preview used the in-memory media cache, the Dashboard origin could be dropped from the allowed list.

Commits

1. Failing smoke checks first (scripts/lib/deploy-smoke-client.mjs, both topologies):

   • the same image twice over a real HTTP/2 connection (node:http2), the second load conditional and expected to be a clean 304, by validator and by date;
   • storage CORS: allowed origins only, nothing for others, Caddy answers preflights, no object-store CORS headers.

   On main, same-origin smoke: both revalidations fail with ERR_HTTP2_STREAM_ERROR, and all four CORS checks fail.

2. The fix: both Caddyfile examples (identical apart from the allowed origins), and a new deploy/README.md section Storage delivery: CORS is the deployment's. The existing-proxy checklist and the testing list are updated too.

Verification

• caddy validate and caddy fmt (pinned caddy:2.11.4-alpine) are clean for both examples. The adapted JSON shows the exact Origin allow-list and versions: ["1.1"].
• pnpm deploy:config, tests/deploy.test.ts and tests/ci-scripts.test.ts pass.
• bash scripts/smoke-deploy.sh: passes in same-origin and in split (every new check ok, the browser walk-through included).

For the instance

deploy/Caddyfile is the split example plus a TLS snippet, and scripts/check-caddyfile.sh fails on drift. After the pin bump, vendor-deploy.sh, and re-syncing the Caddyfile, the instance gets both fixes with no other change. Its storage hostname is files.riverofglorymn.org, and the allowed origins come from EKKLESIA_HOSTNAME and EKKLESIA_DASHBOARD_HOSTNAME, which it already sets.

🤖 Generated with Claude Code

## fix(deploy): reach storage over HTTP/1.1, and own storage CORS in the proxy ### 1. Every reload of a cached `files.` image failed with `ERR_HTTP2_PROTOCOL_ERROR` Measured layer by layer on a split reference deployment (platform v0.1.1 + Signage, SeaweedFS 4.28, Caddy 2.11.4): | Layer | First load | Same signed URL again (conditional: `If-None-Match` / `If-Modified-Since`) | |---|---|---| | Chromium (DevTools) | `200` over h2 | **`net::ERR_HTTP2_PROTOCOL_ERROR`**, for `fetch` and `<img>` alike | | `curl --http2` through Caddy | `200` | **stream reset, `INTERNAL_ERROR`**. Caddy logs `aborting with incomplete response … reading: unexpected EOF` | | `curl --http1.1` directly to SeaweedFS | `200` | clean `304` | | `curl --http2` directly to SeaweedFS | `200` | **`304` with `content-type: application/xml` and `content-length: 327`, and 0 body bytes** | **Cause:** - Caddy's TLS upstream negotiates HTTP/2 with SeaweedFS. - Over HTTP/2, SeaweedFS's 304 declares a body it never sends. - Caddy reads for the promised 327 bytes, gets end-of-stream, and resets the browser's stream. **Fix:** `versions 1.1` for the storage upstream. Browsers still reach Caddy over HTTP/2 and HTTP/3. The upstream bug is drafted as a SeaweedFS issue for the maintainer to submit. ### 2. Storage CORS was the object store's, and reflected everything SeaweedFS's default answered **any** `Origin`, an unrelated one included, with `PUT, POST, DELETE` and any request header. It also answered preflights itself. Delivery-layer CORS is now the deployment's, decided in the proxy, so it stays the same with R2 or any other S3 backend: - CORS only for the origins that read objects with `fetch()`: - same-origin topology: the Ekklesia origin; - split topology: the API origin (displays) and the Dashboard origin. Signage's preview builds a display controller with the default media cache, so it `fetch()`es storage URLs from the Dashboard. - `GET` and `HEAD` only, with `Vary: Origin`; - preflights are answered by Caddy (`204`) and never reach storage; - the object store's own CORS headers are stripped. `<img>` needs no CORS and is unaffected. Every object URL is still a signed, expiring capability. **Possible follow-up (Signage):** if its preview used the in-memory media cache, the Dashboard origin could be dropped from the allowed list. ### Commits 1. **Failing smoke checks first** (`scripts/lib/deploy-smoke-client.mjs`, both topologies): - the same image twice over a real HTTP/2 connection (`node:http2`), the second load conditional and expected to be a clean `304`, by validator and by date; - storage CORS: allowed origins only, nothing for others, Caddy answers preflights, no object-store CORS headers. On `main`, same-origin smoke: both revalidations fail with `ERR_HTTP2_STREAM_ERROR`, and all four CORS checks fail. 2. **The fix:** both Caddyfile examples (identical apart from the allowed origins), and a new `deploy/README.md` section *Storage delivery: CORS is the deployment's*. The existing-proxy checklist and the testing list are updated too. ### Verification - `caddy validate` and `caddy fmt` (pinned `caddy:2.11.4-alpine`) are clean for both examples. The adapted JSON shows the exact Origin allow-list and `versions: ["1.1"]`. - `pnpm deploy:config`, `tests/deploy.test.ts` and `tests/ci-scripts.test.ts` pass. - `bash scripts/smoke-deploy.sh`: passes in **same-origin** and in **split** (every new check `ok`, the browser walk-through included). ### For the instance `deploy/Caddyfile` is the split example plus a TLS snippet, and `scripts/check-caddyfile.sh` fails on drift. After the pin bump, `vendor-deploy.sh`, and re-syncing the Caddyfile, the instance gets both fixes with no other change. Its storage hostname is `files.riverofglorymn.org`, and the allowed origins come from `EKKLESIA_HOSTNAME` and `EKKLESIA_DASHBOARD_HOSTNAME`, which it already sets. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Zayin_Admin added 2 commits 2026-10-06 01:19:18 +00:00

test(deploy): load the same image twice over HTTP/2, and pin storage CORS ... dcbd7613ee

Two checks on storage delivery through the proxy, in both topologies.

The same signed image, twice, over a real HTTP/2 connection (node:http2;
Node's fetch speaks HTTP/1.1, where the failure does not exist). The
second load is conditional, as a browser's revalidation is, and must be
a clean 304. Today Caddy reaches SeaweedFS over HTTP/2, SeaweedFS's 304
declares Content-Length: 327 and sends no body, and Caddy resets the
stream. A browser reports that as ERR_HTTP2_PROTOCOL_ERROR on every
reload of a cached files. image.

CORS on delivery is the deployment's own:
- exactly the API origin (and the Dashboard origin when split) is
  allowed, with Vary: Origin and nothing else;
- any other origin gets no CORS header;
- a preflight is answered by the proxy (204, GET and HEAD only) and
  never reaches the object store.

Today SeaweedFS reflects every origin with PUT, POST and DELETE, and
answers the preflight itself.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(deploy): reach storage over HTTP/1.1, and own storage CORS in the proxy ... e1ac1da416

Every reload of a cached files. image failed in the browser with
ERR_HTTP2_PROTOCOL_ERROR. A browser revalidates a cached signed URL
with If-None-Match. Caddy reached SeaweedFS over HTTP/2, and there
SeaweedFS answers with a 304 declaring content-type: application/xml
and content-length: 327 while sending no body. Caddy then logged
"aborting with incomplete response ... unexpected EOF" and reset the
stream. Over HTTP/1.1 SeaweedFS's 304 is clean. Measured layer by layer:
DevTools, curl through Caddy, and curl direct over HTTP/1.1 and HTTP/2.

The storage upstream is now HTTP/1.1 only (`versions 1.1`). Browsers
still reach the proxy over HTTP/2 and HTTP/3.

Delivery-layer CORS is the deployment's, so it no longer depends on
which S3 backend is behind it. SeaweedFS's default reflected any origin
and advertised PUT, POST and DELETE. Now the proxy:
- grants CORS only to the origins that read objects with fetch(): the
  Ekklesia origin, or, when split, the API origin (displays) and the
  Dashboard origin (a module's preview caches images through fetch());
- allows GET and HEAD only, with Vary: Origin;
- answers preflights itself, so they never reach storage;
- strips the object store's own CORS headers.

The deploy README documents the policy, its reasons, and what a
self-run proxy must do.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Zayin_Admin force-pushed fix/storage-delivery-http1-cors from 2dbae9fca3 to e1ac1da416 2026-10-06 01:19:18 +00:00 Compare

Zayin_Admin approved these changes 2026-10-06 01:39:35 +00:00

Zayin_Admin merged commit 2152eb036a into main 2026-10-06 01:39:39 +00:00

Zayin_Admin deleted branch fix/storage-delivery-http1-cors 2026-10-06 01:39:39 +00:00

Zayin_Admin referenced this issue from a commit 2026-10-06 03:56:37 +00:00

chore(release): prepare 0.1.2

ekklesia-bot referenced this pull request 2026-10-06 03:56:48 +00:00

chore(release): prepare 0.1.2 #62

Sign in to join this conversation.

Reviewers

No Reviewers

Zayin_Admin

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

Joey Zayin_Admin ekklesia-bot

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Zayin/Ekklesia#58