fix(deploy): reach storage over HTTP/1.1, and own storage CORS in the proxy #58

Merged
Zayin_Admin merged 2 commits from fix/storage-delivery-http1-cors into main 2026-10-06 01:39:39 +00:00
3 changed files with 135 additions and 25 deletions
Showing only changes of commit e1ac1da416 - Show all commits
+51 -12
View File
@@ -70,18 +70,57 @@
# Browsers fetch signed object URLs here (S3_DELIVERY_ENDPOINT, ADR 0031). # Browsers fetch signed object URLs here (S3_DELIVERY_ENDPOINT, ADR 0031).
{$EKKLESIA_STORAGE_HOSTNAME} { {$EKKLESIA_STORAGE_HOSTNAME} {
reverse_proxy https://object-storage:8443 { # Delivery-layer CORS belongs to the deployment, not to the object store: this block decides it,
# An S3 signature covers the Host header: pass the original through unchanged, or every signed # so it stays the same with any S3 backend behind it. Only the origins whose pages read objects
# URL fails verification. # with fetch() get CORS: the Ekklesia origin, which serves the Dashboard and the displays. An <img> needs none. Every object
header_up Host {host} # URL is still a signed, expiring capability; CORS only decides which pages may read the bytes.
header_up -CF-Connecting-IP @cors_origin header Origin https://{$EKKLESIA_HOSTNAME}
header_up -X-Real-IP
header_up -Forwarded # A preflight is answered here and never reaches storage. Only GET and HEAD are ever allowed, and
header_up -True-Client-IP # no request header beyond the CORS-safelisted ones.
transport http { @preflight method OPTIONS
# object-storage presents a certificate from the installation-private CA. handle @preflight {
tls_trust_pool file /etc/caddy/ekklesia-ca.crt header @cors_origin {
tls_server_name object-storage Access-Control-Allow-Origin {http.request.header.Origin}
Access-Control-Allow-Methods "GET, HEAD"
Access-Control-Max-Age 600
}
header Vary Origin
respond 204
}
handle {
# Set when the response is written, after the upstream's own CORS headers were removed below.
header @cors_origin >Access-Control-Allow-Origin {http.request.header.Origin}
header >Vary Origin
reverse_proxy https://object-storage:8443 {
# An S3 signature covers the Host header: pass the original through unchanged, or every
# signed URL fails verification.
header_up Host {host}
header_up -CF-Connecting-IP
header_up -X-Real-IP
header_up -Forwarded
header_up -True-Client-IP
# The object store's own CORS answer (SeaweedFS reflects any origin, with every method)
# is not the deployment's policy.
header_down -Access-Control-Allow-Origin
header_down -Access-Control-Allow-Methods
header_down -Access-Control-Allow-Headers
header_down -Access-Control-Expose-Headers
header_down -Access-Control-Allow-Credentials
header_down -Access-Control-Max-Age
header_down -Vary
transport http {
# object-storage presents a certificate from the installation-private CA.
tls_trust_pool file /etc/caddy/ekklesia-ca.crt
tls_server_name object-storage
# HTTP/1.1 only. Over HTTP/2, SeaweedFS answers a conditional request with a 304 that
# declares a Content-Length it never sends; Caddy then aborts the browser's stream,
# which a browser reports as ERR_HTTP2_PROTOCOL_ERROR on every revalidation. Its
# HTTP/1.1 304 is clean. Browsers still reach this proxy over HTTP/2 and HTTP/3.
versions 1.1
}
} }
} }
} }
+55 -12
View File
@@ -92,18 +92,61 @@
# Browsers fetch signed object URLs here (S3_DELIVERY_ENDPOINT, ADR 0031). # Browsers fetch signed object URLs here (S3_DELIVERY_ENDPOINT, ADR 0031).
{$EKKLESIA_STORAGE_HOSTNAME} { {$EKKLESIA_STORAGE_HOSTNAME} {
reverse_proxy https://object-storage:8443 { # Delivery-layer CORS belongs to the deployment, not to the object store: this block decides it,
# An S3 signature covers the Host header: pass the original through unchanged, or every signed # so it stays the same with any S3 backend behind it. Only the origins whose pages read objects
# URL fails verification. # with fetch() get CORS: the API origin, where displays run, and the Dashboard origin,
header_up Host {host} # where a module's preview may cache images too. An <img> needs none. Every object
header_up -CF-Connecting-IP # URL is still a signed, expiring capability; CORS only decides which pages may read the bytes.
header_up -X-Real-IP @cors_origin {
header_up -Forwarded header Origin https://{$EKKLESIA_HOSTNAME}
header_up -True-Client-IP header Origin https://{$EKKLESIA_DASHBOARD_HOSTNAME}
transport http { }
# object-storage presents a certificate from the installation-private CA.
tls_trust_pool file /etc/caddy/ekklesia-ca.crt # A preflight is answered here and never reaches storage. Only GET and HEAD are ever allowed, and
tls_server_name object-storage # no request header beyond the CORS-safelisted ones.
@preflight method OPTIONS
handle @preflight {
header @cors_origin {
Access-Control-Allow-Origin {http.request.header.Origin}
Access-Control-Allow-Methods "GET, HEAD"
Access-Control-Max-Age 600
}
header Vary Origin
respond 204
}
handle {
# Set when the response is written, after the upstream's own CORS headers were removed below.
header @cors_origin >Access-Control-Allow-Origin {http.request.header.Origin}
header >Vary Origin
reverse_proxy https://object-storage:8443 {
# An S3 signature covers the Host header: pass the original through unchanged, or every
# signed URL fails verification.
header_up Host {host}
header_up -CF-Connecting-IP
header_up -X-Real-IP
header_up -Forwarded
header_up -True-Client-IP
# The object store's own CORS answer (SeaweedFS reflects any origin, with every method)
# is not the deployment's policy.
header_down -Access-Control-Allow-Origin
header_down -Access-Control-Allow-Methods
header_down -Access-Control-Allow-Headers
header_down -Access-Control-Expose-Headers
header_down -Access-Control-Allow-Credentials
header_down -Access-Control-Max-Age
header_down -Vary
transport http {
# object-storage presents a certificate from the installation-private CA.
tls_trust_pool file /etc/caddy/ekklesia-ca.crt
tls_server_name object-storage
# HTTP/1.1 only. Over HTTP/2, SeaweedFS answers a conditional request with a 304 that
# declares a Content-Length it never sends; Caddy then aborts the browser's stream,
# which a browser reports as ERR_HTTP2_PROTOCOL_ERROR on every revalidation. Its
# HTTP/1.1 304 is clean. Browsers still reach this proxy over HTTP/2 and HTTP/3.
versions 1.1
}
} }
} }
} }
+29 -1
View File
@@ -178,6 +178,29 @@ the topology and warns when the two hostnames are not siblings.
The Dashboard learns the API's origin at run time: the API writes its identity base URL into the page The Dashboard learns the API's origin at run time: the API writes its identity base URL into the page
it serves (`<meta name="ekklesia-api-origin">`). Nothing about the topology is built into the image. it serves (`<meta name="ekklesia-api-origin">`). Nothing about the topology is built into the image.
### Storage delivery: CORS is the deployment's
Browsers fetch objects directly from the storage hostname through signed, expiring URLs (ADR 0031).
**Which pages may read those bytes with `fetch()` is decided by the delivery layer, the proxy, and
never by the object store.** That policy therefore stays the same with SeaweedFS, R2, or any other
S3 backend behind it, and an installation that changes backends changes nothing about CORS.
The reference proxy's storage block:
- grants CORS only to the origins whose pages read objects with `fetch()`: the Ekklesia origin in the
same-origin topology; the API origin (displays) and the Dashboard origin (a module's preview) in
the split topology. An `<img>` needs no CORS and is unaffected;
- allows `GET` and `HEAD` only, and no request header beyond the CORS-safelisted ones, always with
`Vary: Origin`;
- answers preflight `OPTIONS` requests itself, so they never reach storage;
- removes the object store's own CORS headers. SeaweedFS's default reflects any origin and
advertises `PUT`, `POST`, and `DELETE`.
The proxy reaches storage over **HTTP/1.1 only**. Over HTTP/2, SeaweedFS answers a conditional
request with a `304` that declares a `Content-Length` it never sends. The proxy then resets the
browser's stream, and every reload of a cached image fails with `ERR_HTTP2_PROTOCOL_ERROR`.
Browsers still reach the proxy itself over HTTP/2 and HTTP/3.
### Behind an existing reverse proxy ### Behind an existing reverse proxy
Run the base alone and attach `api` and `object-storage` to your proxy's network with an overlay of Run the base alone and attach `api` and `object-storage` to your proxy's network with an overlay of
@@ -213,6 +236,8 @@ Then make your proxy do what `Caddyfile.example` does:
- forward the storage hostname to `https://object-storage:8443`, verifying it against - forward the storage hostname to `https://object-storage:8443`, verifying it against
`/etc/ekklesia/tls/ca.crt` with server name `object-storage`, and **pass the original `Host` `/etc/ekklesia/tls/ca.crt` with server name `object-storage`, and **pass the original `Host`
through unchanged**, or every signed URL fails (ADR 0031). through unchanged**, or every signed URL fails (ADR 0031).
- reach `object-storage` over **HTTP/1.1** only, and own the storage hostname's CORS as described in
[Storage delivery](#storage-delivery-cors-is-the-deployments).
The proxy's address must be fixed and must never be handed to anything else. With Docker, give its The proxy's address must be fixed and must never be handed to anything else. With Docker, give its
network an explicit subnet, limit `ip_range` to part of it, and fix the proxy's address outside that network an explicit subnet, limit `ip_range` to part of it, and fix the proxy's address outside that
@@ -288,7 +313,10 @@ deployment:
platform-only image and a disposable PostgreSQL and SeaweedFS that exist only for that run. It platform-only image and a disposable PostgreSQL and SeaweedFS that exist only for that run. It
follows the first boot above and checks the following through the proxy: follows the first boot above and checks the following through the proxy:
- readiness and the Dashboard; - readiness and the Dashboard;
- a signed object URL fetched through the storage hostname; - a signed object URL fetched through the storage hostname, and the same image loaded twice over
HTTP/2, the second time as a conditional request that must be a clean `304`;
- storage CORS: exactly the allowed origins, `GET` and `HEAD` only, preflights answered by the
proxy, and none of the object store's own CORS headers;
- the client-address chain, both from an ordinary peer and from the tunnel connector's address; - the client-address chain, both from an ordinary peer and from the tunnel connector's address;
- initial-owner sign-in and sign-out; - initial-owner sign-in and sign-out;
- that the API's database connection is TLS 1.3, and that it is refused by a foreign CA and by a - that the API's database connection is TLS 1.3, and that it is refused by a foreign CA and by a