feat: add host-provided module object storage
CI / verify (pull_request) Successful in 36s
CI / infrastructure-smoke (pull_request) Successful in 13s
CI / migration-smoke (pull_request) Failing after 2m20s

`manifest.storage.buckets` has existed since manifest version 1 and meant
nothing — discovery validated it and no code read it. This turns it into a
real capability, which is what stands between Announcements and the module
system: that product currently fetches a hardcoded LAN S3 endpoint, parses
ListObjectsV2 XML by hand, and hands the browser direct object URLs.

- A module declares a logical bucket and receives a scoped facade at
  `context.storage` — list, head, get, put, delete. The contract names no
  provider: no endpoint, region, credential, or physical bucket is reachable
  from it, which is what lets the same module run under a future standalone
  host over its own or a shared S3-compatible service.
- Core's first provider speaks S3 through the AWS SDK. SeaweedFS is the
  deployment target (ADR 0004), never an API name. Probing the pinned image
  found `CreateBucket` reporting `BucketAlreadyExists` rather than AWS's
  `BucketAlreadyOwnedByYou`; treating only the AWS spelling as success would
  have failed every second boot. A live test pins it.
- Logical → physical mapping is deterministic and collision-safe: an
  installation prefix, the module and bucket ids joined by `--` (which cannot
  occur inside either identifier, so `a-b`+`c` and `a`+`b-c` cannot collide),
  and a digest over the full untruncated identity. Two modules declaring
  `assets` always get separate namespaces; neither can see the physical name.
- Declaring a bucket is now a requirement: with no provider configured the
  module is `blocked · storage_unavailable` through ADR 0028's existing
  resolver, propagating to required dependents. An installation with no object
  storage stays a complete deployment.
- Binding availability is not provider health. Eligibility asks whether
  storage is configured, never whether it answered; bucket creation is lazy on
  first use so a storage outage cannot stop the platform booting; a running
  module whose store blinks stays running and retries. Nothing deletes a
  bucket.
- Credentials are deployment-supplied, never persisted, never routed through
  module configuration, and never visible to modules, module UI, diagnostics,
  or API responses. Provider errors translate to a closed code set carrying no
  endpoint, bucket, request id, or XML.
- Retires ADR 0029's transitional `bucket` configuration key. No manifest
  version change. The template declares no bucket, so it still runs without S3.

See ADR 0030.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
ekklesia
2026-08-23 08:45:53 -05:00
co-authored by Claude Opus 5
parent 527239f6de
commit b9ca0d560d
55 changed files with 4420 additions and 74 deletions
+65 -4
View File
@@ -12,6 +12,11 @@ import {
} from "@ekklesia/core/authorization";
import { createDashboardStore } from "@ekklesia/core/dashboard";
import { createIdentityRuntime, type IdentityRateLimitOverride } from "@ekklesia/core/identity";
import {
createModuleStorageLookup,
createS3StorageProvider,
type StorageProvider,
} from "@ekklesia/core/storage";
import {
createEnvironmentModuleSecretSource,
createModuleConfigurationStore,
@@ -171,6 +176,19 @@ async function safeCloseDatabase(handle: ApiDatabaseHandle): Promise<void> {
}
}
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, for the storage provider's own
* connection pool. */
async function safeCloseStorage(provider: StorageProvider | undefined): Promise<void> {
if (!provider) {
return;
}
try {
await provider.close();
} catch {
// See safeCloseDatabase: never mask a failure that is already propagating.
}
}
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, used only while unwinding a startup
* failure that already occurred after the module runtime started — never during normal shutdown,
* where a module stop failure remains an observable result instead (see `startApi`'s `shutdown`). */
@@ -422,6 +440,13 @@ export async function startApi(
// start, before a single module entrypoint is imported. A failure here fails startup rather than
// falling back to "start everything": the platform cannot know what it was told, and starting a
// module an administrator disabled is worse than not starting at all.
// The host's object-storage provider, constructed from validated configuration before any module
// runs — and **without touching the network**. Building the client opens no connection, so a
// storage outage can never stop the platform from booting; a bucket is ensured lazily on first
// use instead (ADR 0030). An installation with no storage configured gets `undefined` here, which
// is a complete, valid deployment.
const storageProvider = config.storage ? createS3StorageProvider(config.storage) : undefined;
const moduleLifecycleStore = createModuleLifecycleStore(databaseHandle.moduleLifecycle);
const moduleConfigurationStore = createModuleConfigurationStore(
databaseHandle.moduleConfiguration,
@@ -441,15 +466,29 @@ export async function startApi(
const lifecycleConfiguration = await moduleLifecycleStore.readConfiguration();
const settings = await moduleConfigurationStore.readSettings();
moduleConfigurationSettings = settings;
eligibility = resolveModuleEligibility(inventory.modules, lifecycleConfiguration, {
secrets: moduleSecrets,
settings,
});
eligibility = resolveModuleEligibility(
inventory.modules,
lifecycleConfiguration,
{ secrets: moduleSecrets, settings },
{ providerConfigured: storageProvider !== undefined },
);
} catch (error) {
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
// One scoped facade per eligible module, over the exact buckets that module's own manifest
// declares. Built from the eligible set so a blocked module never receives storage at all.
const moduleStorages = createModuleStorageLookup(
storageProvider,
config.storage?.bucketPrefix ?? "",
eligibility.eligibleModules.map((module) => ({
logicalBucketIds: module.manifest.storage.buckets.map((bucket) => bucket.id),
moduleId: module.manifest.id,
})),
);
// Module load/register/start failures use Core's own existing rollback (already-started
// modules are stopped, in reverse order, before this rejects) — there is no separate runtime
// handle to stop here, only the pool this failure leaves open.
@@ -462,11 +501,13 @@ export async function startApi(
try {
moduleRuntime = await startDiscoveredModuleRuntime(eligibility.eligibleModules, services, {
configurations: eligibility.configurations,
storages: moduleStorages,
...(options.moduleHookTimeoutMsOverride === undefined
? {}
: { hookTimeoutMs: options.moduleHookTimeoutMsOverride }),
});
} catch (error) {
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -507,6 +548,10 @@ export async function startApi(
snapshots: eligibility.configurations,
store: moduleConfigurationStore,
},
moduleStorage: {
providerConfigured: storageProvider !== undefined,
snapshots: moduleStorages,
},
// A typed view over the same single pool as identity, authorization, and the module facade.
dashboard: createDashboardStore(databaseHandle.dashboard),
runtime,
@@ -514,6 +559,7 @@ export async function startApi(
});
} catch (error) {
await safeStopModuleRuntime(moduleRuntime);
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -529,6 +575,7 @@ export async function startApi(
server = await bindServer(app, config.host, config.port, clientAddressResolver, timeouts);
} catch (error) {
await safeStopModuleRuntime(moduleRuntime);
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
throw error;
}
@@ -558,6 +605,9 @@ export async function startApi(
} catch (error) {
moduleStopError = error;
}
// After the module runtime has stopped, so a `stop` hook may still use its storage
// facade, and before the pool closes, matching the existing ordering discipline exactly.
await safeCloseStorage(storageProvider);
await safeCloseDatabase(databaseHandle);
// Every stage above ran regardless of what the earlier ones did, so a drain timeout or a
// module-stop failure can never leave the pool open. Both failures are reported together,
@@ -652,6 +702,17 @@ function readConfigFromEnvironment(): {
trustedOrigins,
},
port: process.env.EKKLESIA_API_PORT,
// The one place this process turns its environment into storage configuration. The names are
// the `S3_*` ones ADR 0004 already established for this repository's Compose stack, and Core's
// validator never reads `process.env` itself.
storage: {
accessKey: process.env.S3_ACCESS_KEY,
bucketPrefix: process.env.S3_BUCKET_PREFIX,
endpoint: process.env.S3_ENDPOINT,
forcePathStyle: process.env.S3_FORCE_PATH_STYLE,
region: process.env.S3_REGION,
secretKey: process.env.S3_SECRET_KEY,
},
},
};
}