feat: add host-provided module object storage
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Failing after 2m20s

`manifest.storage.buckets` has existed since manifest version 1 and meant
nothing — discovery validated it and no code read it. This turns it into a
real capability, which is what stands between Announcements and the module
system: that product currently fetches a hardcoded LAN S3 endpoint, parses
ListObjectsV2 XML by hand, and hands the browser direct object URLs.

- A module declares a logical bucket and receives a scoped facade at
  `context.storage` — list, head, get, put, delete. The contract names no
  provider: no endpoint, region, credential, or physical bucket is reachable
  from it, which is what lets the same module run under a future standalone
  host over its own or a shared S3-compatible service.
- Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the
  deployment target (ADR 0004), never an API name. Probing the pinned image
  found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's
  `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would
  have failed every second boot. A live test pins it.
- Logical → physical mapping is deterministic and collision-safe: an
  installation prefix, the module and bucket ids joined by `--` (which cannot
  occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide),
  and a digest over the full untruncated identity. Two modules declaring
  `assets` always get separate namespaces; neither can see the physical name.
- Declaring a bucket is now a requirement: with no provider configured the
  module is `blocked · storage_unavailable` through ADR 0028's existing
  resolver, propagating to required dependents. An installation with no object
  storage stays a complete deployment.
- Binding availability is not provider health. Eligibility asks whether
  storage is configured, never whether it answered; bucket creation is lazy on
  first use so a storage outage cannot stop the platform booting; a running
  module whose store blinks stays running and retries. Nothing deletes a
  bucket.
- Credentials are deployment-supplied, never persisted, never routed through
  module configuration, and never visible to modules, module UI, diagnostics,
  or API responses. Provider errors translate to a closed code set carrying no
  endpoint, bucket, request id, or XML.
- Retires ADR 0029's transitional `bucket` configuration key. No manifest
  version change. The template declares no bucket, so it still runs without S3.

See ADR 0030.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 08:45:53 -05:00
co-authored by Claude Opus 5
parent 527239f6de
commit b9ca0d560d
55 changed files with 4420 additions and 74 deletions
+16
View File
@@ -18,6 +18,7 @@ import type {
ModuleLifecycleSnapshot,
ModuleConfigurationLookup,
ModuleLifecycleStore,
ModuleStorageLookup,
ModuleUiPublicationRegistry,
} from "@ekklesia/core/modules";
import { Hono } from "hono";
@@ -93,6 +94,19 @@ export interface CreateApiAppOptions {
* read the same object rather than two independently resolved values. */
readonly snapshots: ModuleConfigurationLookup;
};
/**
* The platform's module object storage (ADR 0030): the per-module scoped facades the runtime
* started with, and whether a host provider is configured at all.
*
* Required rather than defaulted, like every other module option here — a caller exercising no
* storage says so with an empty map and `providerConfigured: false`, never falls through to an
* implicit one. Nothing here carries an endpoint, a region, or a credential: those never leave
* Core's provider.
*/
readonly moduleStorage: {
readonly snapshots: ModuleStorageLookup;
readonly providerConfigured: boolean;
};
/** Core's persistence for how this Dashboard presents installed modules (ADR 0027). Required
* rather than defaulted, like every other module option here: a caller that exercises no
* Dashboard says so with an explicit store, never falls through to an implicit one. Grouping is
@@ -273,6 +287,7 @@ export function createApiApp(options: CreateApiAppOptions): Hono {
secrets: options.moduleConfiguration.secrets,
store: options.moduleConfiguration.store,
},
storage: { providerConfigured: options.moduleStorage.providerConfigured },
snapshot: options.moduleLifecycle.snapshot,
store: options.moduleLifecycle.store,
trustedOrigins: allowedOrigins,
@@ -297,6 +312,7 @@ export function createApiApp(options: CreateApiAppOptions): Hono {
mountModuleHttpRoutes(app, options.moduleRoutes, {
configurations: options.moduleConfiguration.snapshots,
storages: options.moduleStorage.snapshots,
evaluator: options.authorization.evaluator,
registry: options.authorization.registry,
runtime,
+23
View File
@@ -10,6 +10,11 @@ import {
type IdentityConfig,
type IdentityConfigInput,
} from "@ekklesia/core/identity";
import {
createStorageProviderConfig,
type StorageProviderConfig,
type StorageProviderConfigInput,
} from "@ekklesia/core/storage";
import {
type ClientAddressConfig,
type ClientAddressConfigInput,
@@ -55,6 +60,16 @@ export interface ApiRuntimeConfig {
readonly host: string;
readonly identity: IdentityConfig;
readonly port: number;
/**
* The validated host object-storage provider, or `undefined` when this installation has none
* (ADR 0030).
*
* `undefined` is a complete, valid deployment: modules that declare no logical buckets are
* unaffected, and a module that does declare one is reported blocked rather than crashing
* anything. These values reach Core's provider and nowhere else — no module, module UI, browser
* bundle, diagnostic, or API response ever sees the endpoint or the credentials.
*/
readonly storage: StorageProviderConfig | undefined;
}
export interface ApiRuntimeConfigInput {
@@ -68,6 +83,8 @@ export interface ApiRuntimeConfigInput {
readonly host?: unknown;
readonly identity: IdentityConfigInput;
readonly port?: unknown;
/** Optional. Omitted, or with no endpoint, means this installation has no object storage. */
readonly storage?: StorageProviderConfigInput;
}
export interface CreateApiRuntimeConfigOptions {
@@ -142,6 +159,11 @@ export function createApiRuntimeConfig(
// the URL carries TLS parameters that could override the policy.
assertProductionDatabaseTransport(databaseUrl, databaseTransport, environment);
// Validated in the same pass as every other runtime value, before `startApi` constructs a pool,
// a provider client, or a listener — so a malformed endpoint or a half-supplied credential pair
// fails startup with a sanitized code rather than surfacing later as a confusing storage error.
const storage = createStorageProviderConfig(input.storage, { environment });
return Object.freeze({
clientAddress,
databaseTransport,
@@ -149,5 +171,6 @@ export function createApiRuntimeConfig(
host,
identity,
port,
storage,
});
}
+12 -1
View File
@@ -1,11 +1,12 @@
import {
EMPTY_MODULE_CONFIGURATION,
EMPTY_MODULE_STORAGE,
type ModuleId,
type ModuleRouteHandlerContext,
type ModuleRuntimeServices,
type RegisteredModuleHttpRoute,
} from "@ekklesia/contracts";
import type { ModuleConfigurationLookup } from "@ekklesia/core/modules";
import type { ModuleConfigurationLookup, ModuleStorageLookup } from "@ekklesia/core/modules";
import type { Context, Hono } from "hono";
import {
type AuthorizationMiddlewareOptions,
@@ -43,6 +44,15 @@ export interface ModuleHttpRoutesOptions extends AuthorizationMiddlewareOptions
* ever reach its own module's configuration, because that is the only entry looked up for it.
*/
readonly configurations: ModuleConfigurationLookup;
/**
* Per-module scoped object storage — the exact map the module runtime started from (ADR 0030), so
* a handler and that module's lifecycle hooks share one facade rather than two independently
* resolved ones.
*
* Keyed by module id, like `configurations` and unlike `services`: a handler can only ever reach
* its own module's storage, because that is the only entry looked up for it.
*/
readonly storages: ModuleStorageLookup;
}
/** Never carries the offending permission id, route path, module id, or any other configuration
@@ -226,6 +236,7 @@ export function mountModuleHttpRoutes(
const context: ModuleRouteHandlerContext = Object.freeze({
config: options.configurations.get(route.moduleId) ?? EMPTY_MODULE_CONFIGURATION,
moduleId: route.moduleId,
storage: options.storages.get(route.moduleId) ?? EMPTY_MODULE_STORAGE,
params: Object.freeze({ ...c.req.param() }),
principal: getAuthorizedPrincipal(c),
services: options.services,
+22 -8
View File
@@ -8,6 +8,7 @@ import { CORE_MODULES_LIFECYCLE_MANAGE } from "@ekklesia/core/authorization";
import {
type ModuleConfigurationStore,
type ModuleLifecycleSnapshot,
type ModuleStorageAvailability,
type ModuleLifecycleStore,
ModuleLifecycleStoreError,
type ModuleLifecycleView,
@@ -65,6 +66,9 @@ export interface ModuleLifecycleModuleView {
/** Present for a configuration reason (ADR 0029): the offending key, never its value. A key is
* declared in a public manifest; a value may be a credential or arbitrary operator text. */
readonly configurationKey?: string;
/** Present for a storage reason (ADR 0030): the logical bucket the module declared, never an
* endpoint, a credential, or the physical bucket the host maps it to. */
readonly storageBucketId?: string;
};
readonly restartRequired: boolean;
}
@@ -90,6 +94,9 @@ export interface CreateModuleLifecycleRoutesOptions extends AuthorizationMiddlew
readonly store: ModuleConfigurationStore;
readonly secrets: ModuleSecretSource;
};
/** The other axis that decides eligibility (ADR 0030). A boolean, never a probe: whether storage
* is *configured* decides eligibility, while whether it is *reachable right now* never does. */
readonly storage: ModuleStorageAvailability;
readonly trustedOrigins: ReadonlySet<string>;
}
@@ -156,6 +163,9 @@ function present(view: ModuleLifecycleView): ModuleLifecycleModuleView {
...(view.runtime.configurationKey === undefined
? {}
: { configurationKey: view.runtime.configurationKey }),
...(view.runtime.storageBucketId === undefined
? {}
: { storageBucketId: view.runtime.storageBucketId }),
},
version: view.version,
};
@@ -220,10 +230,12 @@ export function createModuleLifecycleRoutes(options: CreateModuleLifecycleRoutes
options.store.readConfiguration(),
options.configuration.store.readSettings(),
]);
const modules = resolveModuleLifecycleView(options.snapshot, configuration, {
secrets: options.configuration.secrets,
settings,
}).map(present);
const modules = resolveModuleLifecycleView(
options.snapshot,
configuration,
{ secrets: options.configuration.secrets, settings },
options.storage,
).map(present);
return c.json({ modules } satisfies ModuleLifecycleResponse, 200);
} catch (error) {
return storeFailure(c, error, "read");
@@ -254,10 +266,12 @@ export function createModuleLifecycleRoutes(options: CreateModuleLifecycleRoutes
options.store.readConfiguration(),
options.configuration.store.readSettings(),
]);
const view = resolveModuleLifecycleView(options.snapshot, configuration, {
secrets: options.configuration.secrets,
settings,
}).find((entry) => entry.moduleId === moduleId);
const view = resolveModuleLifecycleView(
options.snapshot,
configuration,
{ secrets: options.configuration.secrets, settings },
options.storage,
).find((entry) => entry.moduleId === moduleId);
if (!view) {
return sanitizedFailure(c, 503, "MODULE_LIFECYCLE_READ_FAILED");
}
+65 -4
View File
@@ -12,6 +12,11 @@ import {
} from "@ekklesia/core/authorization";
import { createDashboardStore } from "@ekklesia/core/dashboard";
import { createIdentityRuntime, type IdentityRateLimitOverride } from "@ekklesia/core/identity";
import {
createModuleStorageLookup,
createS3StorageProvider,
type StorageProvider,
} from "@ekklesia/core/storage";
import {
createEnvironmentModuleSecretSource,
createModuleConfigurationStore,
@@ -171,6 +176,19 @@ async function safeCloseDatabase(handle: ApiDatabaseHandle): Promise<void> {
}
}
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, for the storage provider's own
* connection pool. */
async function safeCloseStorage(provider: StorageProvider | undefined): Promise<void> {
if (!provider) {
return;
}
try {
await provider.close();
} catch {
// See safeCloseDatabase: never mask a failure that is already propagating.
}
}
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, used only while unwinding a startup
* failure that already occurred after the module runtime started — never during normal shutdown,
* where a module stop failure remains an observable result instead (see `startApi`'s `shutdown`). */
@@ -422,6 +440,13 @@ export async function startApi(
// start, before a single module entrypoint is imported. A failure here fails startup rather than
// falling back to "start everything": the platform cannot know what it was told, and starting a
// module an administrator disabled is worse than not starting at all.
// The host's object-storage provider, constructed from validated configuration before any module
// runs — and **without touching the network**. Building the client opens no connection, so a
// storage outage can never stop the platform from booting; a bucket is ensured lazily on first
// use instead (ADR 0030). An installation with no storage configured gets `undefined` here, which
// is a complete, valid deployment.
const storageProvider = config.storage ? createS3StorageProvider(config.storage) : undefined;
const moduleLifecycleStore = createModuleLifecycleStore(databaseHandle.moduleLifecycle);
const moduleConfigurationStore = createModuleConfigurationStore(
databaseHandle.moduleConfiguration,
@@ -441,15 +466,29 @@ export async function startApi(
const lifecycleConfiguration = await moduleLifecycleStore.readConfiguration();
const settings = await moduleConfigurationStore.readSettings();
moduleConfigurationSettings = settings;
eligibility = resolveModuleEligibility(inventory.modules, lifecycleConfiguration, {
secrets: moduleSecrets,
settings,
});
eligibility = resolveModuleEligibility(
inventory.modules,
lifecycleConfiguration,
{ secrets: moduleSecrets, settings },
{ providerConfigured: storageProvider !== undefined },
);
} catch (error) {
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
// One scoped facade per eligible module, over the exact buckets that module's own manifest
// declares. Built from the eligible set so a blocked module never receives storage at all.
const moduleStorages = createModuleStorageLookup(
storageProvider,
config.storage?.bucketPrefix ?? "",
eligibility.eligibleModules.map((module) => ({
logicalBucketIds: module.manifest.storage.buckets.map((bucket) => bucket.id),
moduleId: module.manifest.id,
})),
);
// Module load/register/start failures use Core's own existing rollback (already-started
// modules are stopped, in reverse order, before this rejects) — there is no separate runtime
// handle to stop here, only the pool this failure leaves open.
@@ -462,11 +501,13 @@ export async function startApi(
try {
moduleRuntime = await startDiscoveredModuleRuntime(eligibility.eligibleModules, services, {
configurations: eligibility.configurations,
storages: moduleStorages,
...(options.moduleHookTimeoutMsOverride === undefined
? {}
: { hookTimeoutMs: options.moduleHookTimeoutMsOverride }),
});
} catch (error) {
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -507,6 +548,10 @@ export async function startApi(
snapshots: eligibility.configurations,
store: moduleConfigurationStore,
},
moduleStorage: {
providerConfigured: storageProvider !== undefined,
snapshots: moduleStorages,
},
// A typed view over the same single pool as identity, authorization, and the module facade.
dashboard: createDashboardStore(databaseHandle.dashboard),
runtime,
@@ -514,6 +559,7 @@ export async function startApi(
});
} catch (error) {
await safeStopModuleRuntime(moduleRuntime);
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -529,6 +575,7 @@ export async function startApi(
server = await bindServer(app, config.host, config.port, clientAddressResolver, timeouts);
} catch (error) {
await safeStopModuleRuntime(moduleRuntime);
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -558,6 +605,9 @@ export async function startApi(
} catch (error) {
moduleStopError = error;
}
// After the module runtime has stopped, so a `stop` hook may still use its storage
// facade, and before the pool closes, matching the existing ordering discipline exactly.
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
// Every stage above ran regardless of what the earlier ones did, so a drain timeout or a
// module-stop failure can never leave the pool open. Both failures are reported together,
@@ -652,6 +702,17 @@ function readConfigFromEnvironment(): {
trustedOrigins,
},
port: process.env.EKKLESIA_API_PORT,
// The one place this process turns its environment into storage configuration. The names are
// the `S3_*` ones ADR 0004 already established for this repository's Compose stack, and Core's
// validator never reads `process.env` itself.
storage: {
accessKey: process.env.S3_ACCESS_KEY,
bucketPrefix: process.env.S3_BUCKET_PREFIX,
endpoint: process.env.S3_ENDPOINT,
forcePathStyle: process.env.S3_FORCE_PATH_STYLE,
region: process.env.S3_REGION,
secretKey: process.env.S3_SECRET_KEY,
},
},
};
}
+22
View File
@@ -22,6 +22,7 @@ import { createApiApp, type CreateApiAppAuthorizationOptions } from "../src/app.
import { AUTH_REQUEST_MAX_BODY_BYTES } from "../src/body-limits.js";
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
import { emptyModuleStorage } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const untrustedOrigin = "https://evil.example.test";
@@ -92,6 +93,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -113,6 +115,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -135,6 +138,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -159,6 +163,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -185,6 +190,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -211,6 +217,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -234,6 +241,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -259,6 +267,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -281,6 +290,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -305,6 +315,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -336,6 +347,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -368,6 +380,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -405,6 +418,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -424,6 +438,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -445,6 +460,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -465,6 +481,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -483,6 +500,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -508,6 +526,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
@@ -536,6 +555,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({ evaluator: { authorize: evaluatorAuthorize } }),
@@ -567,6 +587,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({
@@ -589,6 +610,7 @@ describe("Ekklesia API Hono application", () => {
moduleUi: emptyModuleUiPublication(),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
moduleServices: fakeModuleServices(),
authorization: fakeAuthorization({}),
+2
View File
@@ -31,6 +31,7 @@ import {
moduleLifecycleSnapshotFor,
} from "./support/in-memory-module-lifecycle.js";
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const untrustedOrigin = "https://evil.example.test";
@@ -147,6 +148,7 @@ async function install(options: InstallOptions = {}): Promise<{ app: Hono }> {
store: createInMemoryModuleLifecycleStore(),
},
moduleConfiguration: moduleConfigurationOptionFor(modules),
moduleStorage: moduleStorageOptionFor(modules),
runtime,
trustedOrigins: [trustedOrigin],
});
@@ -39,6 +39,7 @@ import {
createInMemoryModuleConfigurationStore,
} from "./support/in-memory-module-configuration.js";
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const untrustedOrigin = "https://evil.example.test";
@@ -194,6 +195,7 @@ async function install(options: InstallOptions = {}): Promise<Installation> {
snapshots: eligibility.configurations,
store,
},
moduleStorage: moduleStorageOptionFor(modules),
moduleLifecycle: {
snapshot: {
discoveredModules: modules,
@@ -519,12 +519,18 @@ describe.sequential("live API — module database access through ModuleRuntimeSe
const readHandlerEvent = [...events]
.reverse()
.find((event) => event.type === "handler" && event.route === "GET /records/:id");
// The complete handler-context surface, asserted exactly so a new field is a deliberate change
// here rather than an unnoticed one: `config` (ADR 0029) and `storage` (ADR 0030) are the two
// scoped, per-module additions; everything else a module might want — a Hono Context, the
// authorization store, raw PostgreSQL, the storage provider, an endpoint or credential — remains
// unreachable.
expect(readHandlerEvent?.contextKeys).toEqual([
"config",
"moduleId",
"params",
"principal",
"services",
"storage",
]);
expect(readHandlerEvent?.services).toBeDefined();
expect(readHandlerEvent?.services?.database).toBe(startEvent?.services?.database);
+13 -5
View File
@@ -28,6 +28,7 @@ import {
} from "../src/module-http-routes.js";
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
import { emptyModuleStorage } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const untrustedOrigin = "https://evil.example.test";
@@ -113,6 +114,7 @@ function buildModuleApp(options: {
moduleUi: createModuleUiPublicationRegistry([]),
moduleLifecycle: emptyModuleLifecycle(),
moduleConfiguration: emptyModuleConfiguration(),
moduleStorage: emptyModuleStorage(),
dashboard: createInMemoryDashboardStore(),
runtime,
trustedOrigins: options.trustedOrigins ?? [trustedOrigin],
@@ -339,17 +341,19 @@ describe("module route handler adaptation", () => {
expect(body.url).toContain("/api/v1/modules/calendar/events/evt-1/rsvp?notify=true");
expect(capturedRequest).toBeInstanceOf(Request);
expect(capturedRequest?.headers.get("x-custom")).toBe("abc");
// Exactly config, moduleId, params, principal, services — and nothing else. `config` is this
// module's own frozen configuration snapshot, added deliberately by ADR 0029; a Hono Context,
// the authorization store or evaluator, raw PostgreSQL, platform configuration, and another
// module's configuration all remain unreachable, which is what this exact-shape assertion
// exists to keep true.
// Exactly config, moduleId, params, principal, services, storage — and nothing else. `config`
// is this module's own frozen configuration snapshot (ADR 0029) and `storage` its own scoped
// object storage (ADR 0030); a Hono Context, the authorization store or evaluator, raw
// PostgreSQL, the storage provider, an S3 endpoint or credential, platform configuration, and
// another module's configuration or storage all remain unreachable, which is what this
// exact-shape assertion exists to keep true.
expect(Object.keys(capturedContext ?? {}).sort()).toEqual([
"config",
"moduleId",
"params",
"principal",
"services",
"storage",
]);
expect(capturedContext).toMatchObject({
moduleId: "calendar",
@@ -366,6 +370,10 @@ describe("module route handler adaptation", () => {
// This module declared no configuration, so its snapshot is empty rather than absent — module
// code never has to branch on whether the platform supplied one.
expect(capturedContext?.config.keys()).toEqual([]);
// Likewise storage: this module declared no buckets, so it gets a frozen facade that refuses
// every one rather than `undefined`.
expect(Object.isFrozen(capturedContext?.storage)).toBe(true);
expect(capturedContext?.storage.buckets()).toEqual([]);
});
it("gives static and root routes a frozen, empty params object", async () => {
@@ -34,6 +34,7 @@ import { MODULE_UI_INVENTORY_PATH } from "../src/module-ui-inventory.js";
import { createInMemoryDashboardStore } from "./support/in-memory-dashboard-store.js";
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const untrustedOrigin = "https://evil.example.test";
@@ -170,6 +171,7 @@ async function install(options: InstallOptions = {}): Promise<Installation> {
},
dashboard: createInMemoryDashboardStore(),
moduleConfiguration: moduleConfigurationOptionFor(modules),
moduleStorage: moduleStorageOptionFor(modules),
moduleLifecycle: {
snapshot: {
discoveredModules: modules,
@@ -35,6 +35,7 @@ import {
moduleLifecycleSnapshotFor,
} from "./support/in-memory-module-lifecycle.js";
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
/**
* The whole chain, with nothing between the halves faked:
@@ -223,6 +224,7 @@ async function startHarness(options: HarnessOptions = {}): Promise<Harness> {
store: createInMemoryModuleLifecycleStore(),
},
moduleConfiguration: moduleConfigurationOptionFor(modules),
moduleStorage: moduleStorageOptionFor(modules),
dashboard: createInMemoryDashboardStore(),
runtime,
trustedOrigins: ["https://dashboard.example.test"],
@@ -33,6 +33,7 @@ import {
moduleLifecycleSnapshotFor,
} from "./support/in-memory-module-lifecycle.js";
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
const viewerId = parseEkklesiaUserId("11111111-1111-4111-8111-111111111111");
@@ -179,6 +180,7 @@ async function install(options: InstallOptions): Promise<Installation> {
store: createInMemoryModuleLifecycleStore(),
},
moduleConfiguration: moduleConfigurationOptionFor(modules),
moduleStorage: moduleStorageOptionFor(modules),
dashboard: createInMemoryDashboardStore(),
runtime,
trustedOrigins: [trustedOrigin],
+2
View File
@@ -33,6 +33,7 @@ import {
moduleLifecycleSnapshotFor,
} from "./support/in-memory-module-lifecycle.js";
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
const trustedOrigin = "https://dashboard.example.test";
@@ -179,6 +180,7 @@ async function install(fixtures: readonly ModuleFixture[]): Promise<Installation
store: createInMemoryModuleLifecycleStore(),
},
moduleConfiguration: moduleConfigurationOptionFor(modules),
moduleStorage: moduleStorageOptionFor(modules),
dashboard: createInMemoryDashboardStore(),
runtime,
trustedOrigins: [trustedOrigin],
@@ -0,0 +1,195 @@
import {
type ModuleStorage,
type ModuleStorageBody,
ModuleStorageError,
type ModuleStorageListOptions,
type ModuleStorageListing,
type ModuleStorageObject,
type ModuleStorageObjectMetadata,
type ModuleStoragePutOptions,
} from "@ekklesia/contracts";
import type { DiscoveredModule, ModuleStorageLookup } from "@ekklesia/core/modules";
import { createModuleStorageLookup, type StorageProvider } from "@ekklesia/core/storage";
/** Builds and drains Web streams without depending on DOM-only helper types, which this package's
* TypeScript configuration deliberately does not load. */
function streamOf(bytes: Uint8Array): ReadableStream<Uint8Array> {
return new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(bytes);
controller.close();
},
});
}
async function collect(stream: ReadableStream<Uint8Array>): Promise<Uint8Array> {
const reader = stream.getReader();
const chunks: Uint8Array[] = [];
let total = 0;
for (;;) {
const { done, value } = await reader.read();
if (done) {
break;
}
chunks.push(value);
total += value.byteLength;
}
const merged = new Uint8Array(total);
let offset = 0;
for (const chunk of chunks) {
merged.set(chunk, offset);
offset += chunk.byteLength;
}
return merged;
}
interface StoredObject {
readonly bytes: Uint8Array;
readonly contentType?: string;
readonly lastModified: Date;
}
/**
* An in-memory `StorageProvider` for tests that exercise **wiring, scoping, and routing** rather
* than the S3 protocol.
*
* It implements the same interface the real S3 provider does, so everything above the provider seam
* — binding resolution, the scoped facade, lifecycle eligibility, route wiring — is exercised
* unchanged. What it deliberately does *not* reproduce is the protocol itself: real SeaweedFS
* semantics (non-idempotent `CreateBucket`, `NoSuchKey` vs `NoSuchBucket`, ETag quoting, streaming
* bodies) are proven against a live server in `core/tests/module-storage.integration.ts`, so this
* double can never quietly stand in for a provider guarantee nobody checked.
*
* It keys everything by **physical** bucket, exactly as the real provider does, which is what lets a
* test assert that two modules declaring the same logical name really did land in different
* namespaces.
*/
export function createInMemoryStorageProvider(): StorageProvider & {
readonly buckets: ReadonlyMap<string, Map<string, StoredObject>>;
ensureBucketCalls: number;
} {
const buckets = new Map<string, Map<string, StoredObject>>();
const state = { ensureBucketCalls: 0 };
const objectsIn = (physicalBucket: string): Map<string, StoredObject> => {
const bucket = buckets.get(physicalBucket);
if (!bucket) {
throw new ModuleStorageError("STORAGE_BUCKET_UNAVAILABLE");
}
return bucket;
};
const metadataFor = (key: string, stored: StoredObject): ModuleStorageObjectMetadata =>
Object.freeze({
contentLength: stored.bytes.byteLength,
key,
lastModified: stored.lastModified,
...(stored.contentType === undefined ? {} : { contentType: stored.contentType }),
});
const provider: StorageProvider = {
async close(): Promise<void> {
// Nothing to release.
},
async delete(physicalBucket: string, key: string): Promise<void> {
objectsIn(physicalBucket).delete(key);
},
async ensureBucket(physicalBucket: string): Promise<void> {
state.ensureBucketCalls += 1;
if (!buckets.has(physicalBucket)) {
buckets.set(physicalBucket, new Map());
}
},
async get(physicalBucket: string, key: string): Promise<ModuleStorageObject> {
const stored = objectsIn(physicalBucket).get(key);
if (!stored) {
throw new ModuleStorageError("STORAGE_OBJECT_NOT_FOUND", key);
}
return Object.freeze({
...metadataFor(key, stored),
body: streamOf(stored.bytes),
});
},
async head(physicalBucket: string, key: string): Promise<ModuleStorageObjectMetadata> {
const stored = objectsIn(physicalBucket).get(key);
if (!stored) {
throw new ModuleStorageError("STORAGE_OBJECT_NOT_FOUND", key);
}
return metadataFor(key, stored);
},
async list(
physicalBucket: string,
options: ModuleStorageListOptions,
): Promise<ModuleStorageListing> {
const entries = [...objectsIn(physicalBucket).entries()]
.filter(([key]) => options.prefix === undefined || key.startsWith(options.prefix))
.sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0));
return Object.freeze({
objects: Object.freeze(entries.map(([key, stored]) => metadataFor(key, stored))),
});
},
async probe(): Promise<void> {
// Always reachable.
},
async put(
physicalBucket: string,
key: string,
body: ModuleStorageBody,
options: ModuleStoragePutOptions,
): Promise<ModuleStorageObjectMetadata> {
const bytes =
typeof body === "string"
? new TextEncoder().encode(body)
: body instanceof Uint8Array
? body
: await collect(body as ReadableStream<Uint8Array>);
const stored: StoredObject = {
bytes,
lastModified: new Date(0),
...(options.contentType === undefined ? {} : { contentType: options.contentType }),
};
objectsIn(physicalBucket).set(key, stored);
return metadataFor(key, stored);
},
};
return Object.assign(provider, {
buckets: buckets as ReadonlyMap<string, Map<string, StoredObject>>,
get ensureBucketCalls(): number {
return state.ensureBucketCalls;
},
});
}
export interface ModuleStorageOption {
readonly snapshots: ModuleStorageLookup;
readonly providerConfigured: boolean;
}
/**
* Builds the `createApiApp` storage option the same way `startApi` does — resolve every module's
* declared buckets against one provider and one prefix — rather than hand-writing a map, so a test
* can never assert against a shape the real composition would not produce.
*/
export function moduleStorageOptionFor(
discoveredModules: readonly DiscoveredModule[],
provider?: StorageProvider,
): ModuleStorageOption {
return {
providerConfigured: provider !== undefined,
snapshots: createModuleStorageLookup(
provider,
"ekklesia",
discoveredModules.map((module) => ({
logicalBucketIds: module.manifest.storage.buckets.map((bucket) => bucket.id),
moduleId: module.manifest.id,
})),
),
};
}
/** The storage option for a `createApiApp` caller that exercises no storage at all — the explicit
* "this installation has no object storage" statement the required option asks for. */
export function emptyModuleStorage(): ModuleStorageOption {
return { providerConfigured: false, snapshots: new Map<string, ModuleStorage>() };
}