feat: add host-provided module object storage #29
+13
-1
@@ -12,10 +12,22 @@ POSTGRES_PASSWORD=ekklesia-development-only
|
|||||||
POSTGRES_PORT=5432
|
POSTGRES_PORT=5432
|
||||||
DATABASE_URL=postgresql://ekklesia:[email protected]:5432/ekklesia
|
DATABASE_URL=postgresql://ekklesia:[email protected]:5432/ekklesia
|
||||||
|
|
||||||
|
# Ekklesia's object-storage provider (ADR 0030). Absent S3_ENDPOINT means this installation has no
|
||||||
|
# object storage, which is a complete deployment: only a module that declares a logical bucket in its
|
||||||
|
# manifest is affected, and it is reported blocked rather than breaking anything. Production requires
|
||||||
|
# https, exactly as the database transport requires verified TLS.
|
||||||
|
#
|
||||||
|
# These are host infrastructure. No module ever sees them: a module names a logical bucket and the
|
||||||
|
# host resolves the binding.
|
||||||
S3_ACCESS_KEY=ekklesia-development
|
S3_ACCESS_KEY=ekklesia-development
|
||||||
S3_SECRET_KEY=ekklesia-development-secret
|
S3_SECRET_KEY=ekklesia-development-secret
|
||||||
S3_BUCKET=ekklesia-development
|
|
||||||
S3_ENDPOINT=http://127.0.0.1:8333
|
S3_ENDPOINT=http://127.0.0.1:8333
|
||||||
S3_PORT=8333
|
S3_PORT=8333
|
||||||
S3_REGION=us-east-1
|
S3_REGION=us-east-1
|
||||||
S3_FORCE_PATH_STYLE=true
|
S3_FORCE_PATH_STYLE=true
|
||||||
|
# Namespaces every physical bucket this installation owns, so several Ekklesia-compatible deployments
|
||||||
|
# can share one S3-compatible service. Optional; defaults to "ekklesia".
|
||||||
|
S3_BUCKET_PREFIX=ekklesia
|
||||||
|
# Consumed only by the development Compose service's own bootstrap, never by Ekklesia's provider —
|
||||||
|
# physical buckets are derived from a module's manifest declaration, not configured here.
|
||||||
|
S3_BUCKET=ekklesia-development
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ transport), and Phase 5C (module database access) have landed. **Phase 5D (secur
|
|||||||
against an independent security audit) is complete: all five passes have landed and the final
|
against an independent security audit) is complete: all five passes have landed and the final
|
||||||
independent whole-repository re-audit has closed, under the current trusted-contributor,
|
independent whole-repository re-audit has closed, under the current trusted-contributor,
|
||||||
not-yet-deployed threat model — and Phase 6 is unblocked and underway.** Phase 6 has since
|
not-yet-deployed threat model — and Phase 6 is unblocked and underway.** Phase 6 has since
|
||||||
completed through 6D, and **Phase 7A pass 1 (the minimal operational module lifecycle) and
|
completed through 6D, and **Phase 7A passes 1 (the minimal operational module lifecycle), 2 (the module configuration and
|
||||||
pass 2 (the module configuration and secrets contract) have landed.** The operational
|
secrets contract), and 3 (host-provided module object storage) have landed.** The operational
|
||||||
requirements that remain are prerequisites for two specific *future* activities — executing
|
requirements that remain are prerequisites for two specific *future* activities — executing
|
||||||
untrusted CI contributions, and a public production launch — and neither gates Phase 6; see the
|
untrusted CI contributions, and a public production launch — and neither gates Phase 6; see the
|
||||||
pass 5 closure below and
|
pass 5 closure below and
|
||||||
@@ -506,10 +506,71 @@ and credentials for shared infrastructure stay platform concerns: a module decla
|
|||||||
needs, not an S3 endpoint, leaving room for the object storage service Core still lacks. See
|
needs, not an S3 endpoint, leaving room for the object storage service Core still lacks. See
|
||||||
[ADR 0029](docs/architecture/decisions/0029-module-configuration-and-secrets.md).
|
[ADR 0029](docs/architecture/decisions/0029-module-configuration-and-secrets.md).
|
||||||
|
|
||||||
|
Phase 7A pass 3 turns `manifest.storage.buckets` — declarable since manifest version 1 and inert ever
|
||||||
|
since — into a real **host-provided object-storage capability**. This is what stands between
|
||||||
|
Announcements and the module system: it currently fetches `http://10.40.0.2:9000/announcements`, parses
|
||||||
|
S3 XML by hand, and hands the browser direct S3 URLs.
|
||||||
|
|
||||||
|
A module declares a **logical** bucket and gets a scoped facade:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"storage": { "buckets": [{ "id": "announcement-assets" }] }
|
||||||
|
```
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const assets = context.storage.bucket("announcement-assets");
|
||||||
|
const object = await assets.get("slides/welcome.png");
|
||||||
|
return new Response(object.body, { headers: { "content-type": object.contentType } });
|
||||||
|
```
|
||||||
|
|
||||||
|
**The contract names no provider.** No endpoint, region, credential, or physical bucket is reachable
|
||||||
|
from it — not as a field, not as a type, not as a string. That is what lets the same module run under
|
||||||
|
Ekklesia and under a future standalone host backed by its own SeaweedFS, a shared one, MinIO, or AWS
|
||||||
|
S3. A module that configures its own S3 endpoint is a module welded to one deployment, which is the
|
||||||
|
mistake this pass exists to prevent.
|
||||||
|
|
||||||
|
Core's first provider speaks the S3-compatible protocol through the AWS SDK. SeaweedFS is the intended
|
||||||
|
target ([ADR 0004](docs/architecture/decisions/0004-replace-unmaintained-minio.md)) and already runs in
|
||||||
|
the development Compose stack — but it is a deployment choice, never an API name. Probing it directly
|
||||||
|
found a real divergence worth having caught: `CreateBucket` on an existing bucket reports
|
||||||
|
`BucketAlreadyExists`, **not** AWS's `BucketAlreadyOwnedByYou`, so treating only the AWS spelling as
|
||||||
|
success would have failed on every second boot. A live test against the real server pins it.
|
||||||
|
|
||||||
|
**Logical → physical mapping is deterministic and collision-safe.**
|
||||||
|
`ekklesia-announcements--announcement-assets-3f0c1d9a6b21`: an installation prefix (so several
|
||||||
|
deployments can share one storage service), the module and bucket ids joined by `--` — which cannot
|
||||||
|
occur inside either identifier, so `a-b`+`c` and `a`+`b-c` can never collide — and a digest over the
|
||||||
|
full untruncated identity. Two modules both declaring `assets` always get separate namespaces, and a
|
||||||
|
module can never see or choose the physical name.
|
||||||
|
|
||||||
|
**Declaring a bucket is now a real requirement.** With no provider configured, such a module is
|
||||||
|
reported `blocked · storage_unavailable` naming the logical bucket, and its required dependents block
|
||||||
|
behind it through the existing resolver — no second eligibility system. An installation with *no*
|
||||||
|
object storage remains a complete deployment; modules declaring no bucket are unaffected, which is why
|
||||||
|
the template now declares none.
|
||||||
|
|
||||||
|
**A storage outage is not a lifecycle event.** Eligibility asks whether storage is *configured*, never
|
||||||
|
whether it answered. Bucket creation is lazy on first use rather than at startup, so a storage service
|
||||||
|
being down can never stop the platform from booting — and a running module whose object store blinks
|
||||||
|
stays `running`, catches a structured error, and retries, exactly as Announcements does today. Nothing
|
||||||
|
ever deletes a bucket, on disable or otherwise.
|
||||||
|
|
||||||
|
Credentials are deployment-supplied (`S3_ENDPOINT`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`,
|
||||||
|
`S3_BUCKET_PREFIX`), never persisted, never routed through module configuration, and never visible to
|
||||||
|
modules, module UI, diagnostics, or any API response — errors are translated to a closed code set that
|
||||||
|
carries no endpoint, bucket, request id, or XML. This also retires the transitional `bucket`
|
||||||
|
*configuration* key ADR 0029 flagged: the manifest declaration is the requirement.
|
||||||
|
|
||||||
|
Browsers reach objects through a module's own routes (`browser → module API → scoped storage`), so no
|
||||||
|
S3 knowledge or credential goes near the browser. **Signed URLs, public object delivery, range
|
||||||
|
requests, and multipart upload are deliberately deferred**, and the provider is not yet
|
||||||
|
"production-ready" by ADR 0004's own bar. See
|
||||||
|
[ADR 0030](docs/architecture/decisions/0030-host-provided-module-object-storage.md).
|
||||||
|
|
||||||
**Still not the finished Dashboard.** There is no navigation editor or drag and drop, no nested
|
**Still not the finished Dashboard.** There is no navigation editor or drag and drop, no nested
|
||||||
folders, no icons or colors, no Dashboard Home or widgets, no design system, no client-side routing,
|
folders, no icons or colors, no Dashboard Home or widgets, no design system, no client-side routing,
|
||||||
and no module-management or settings UI — module enablement and configuration are an API and a CLI,
|
and no module-management, settings, or storage UI — module enablement and configuration are an API and
|
||||||
not a screen. Sign-in UI, a production CSP and deployment for the built shell, asset publication beyond
|
a CLI, not a screen, and object storage is deployment configuration. Sign-in UI, a production CSP and deployment for the built shell, asset publication beyond
|
||||||
the UI entrypoint, React runtime sharing, the standalone host, and every real feature module also do
|
the UI entrypoint, React runtime sharing, the standalone host, and every real feature module also do
|
||||||
not exist.
|
not exist.
|
||||||
|
|
||||||
@@ -601,8 +662,10 @@ Identity
|
|||||||
Trusts only the actual immediate socket peer for rate limiting; no forwarded header is trusted.
|
Trusts only the actual immediate socket peer for rate limiting; no forwarded header is trusted.
|
||||||
|
|
||||||
Object storage
|
Object storage
|
||||||
○ Check unavailable
|
○ Object storage not configured
|
||||||
Core storage integration has not been implemented yet.
|
No S3_ENDPOINT is set. This is a complete deployment unless a module requires storage.
|
||||||
|
✓ No module requires object storage
|
||||||
|
0 modules declare a logical storage bucket.
|
||||||
|
|
||||||
Result: READY WITH WARNINGS
|
Result: READY WITH WARNINGS
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ import type {
|
|||||||
ModuleLifecycleSnapshot,
|
ModuleLifecycleSnapshot,
|
||||||
ModuleConfigurationLookup,
|
ModuleConfigurationLookup,
|
||||||
ModuleLifecycleStore,
|
ModuleLifecycleStore,
|
||||||
|
ModuleStorageLookup,
|
||||||
ModuleUiPublicationRegistry,
|
ModuleUiPublicationRegistry,
|
||||||
} from "@ekklesia/core/modules";
|
} from "@ekklesia/core/modules";
|
||||||
import { Hono } from "hono";
|
import { Hono } from "hono";
|
||||||
@@ -93,6 +94,19 @@ export interface CreateApiAppOptions {
|
|||||||
* read the same object rather than two independently resolved values. */
|
* read the same object rather than two independently resolved values. */
|
||||||
readonly snapshots: ModuleConfigurationLookup;
|
readonly snapshots: ModuleConfigurationLookup;
|
||||||
};
|
};
|
||||||
|
/**
|
||||||
|
* The platform's module object storage (ADR 0030): the per-module scoped facades the runtime
|
||||||
|
* started with, and whether a host provider is configured at all.
|
||||||
|
*
|
||||||
|
* Required rather than defaulted, like every other module option here — a caller exercising no
|
||||||
|
* storage says so with an empty map and `providerConfigured: false`, never falls through to an
|
||||||
|
* implicit one. Nothing here carries an endpoint, a region, or a credential: those never leave
|
||||||
|
* Core's provider.
|
||||||
|
*/
|
||||||
|
readonly moduleStorage: {
|
||||||
|
readonly snapshots: ModuleStorageLookup;
|
||||||
|
readonly providerConfigured: boolean;
|
||||||
|
};
|
||||||
/** Core's persistence for how this Dashboard presents installed modules (ADR 0027). Required
|
/** Core's persistence for how this Dashboard presents installed modules (ADR 0027). Required
|
||||||
* rather than defaulted, like every other module option here: a caller that exercises no
|
* rather than defaulted, like every other module option here: a caller that exercises no
|
||||||
* Dashboard says so with an explicit store, never falls through to an implicit one. Grouping is
|
* Dashboard says so with an explicit store, never falls through to an implicit one. Grouping is
|
||||||
@@ -273,6 +287,7 @@ export function createApiApp(options: CreateApiAppOptions): Hono {
|
|||||||
secrets: options.moduleConfiguration.secrets,
|
secrets: options.moduleConfiguration.secrets,
|
||||||
store: options.moduleConfiguration.store,
|
store: options.moduleConfiguration.store,
|
||||||
},
|
},
|
||||||
|
storage: { providerConfigured: options.moduleStorage.providerConfigured },
|
||||||
snapshot: options.moduleLifecycle.snapshot,
|
snapshot: options.moduleLifecycle.snapshot,
|
||||||
store: options.moduleLifecycle.store,
|
store: options.moduleLifecycle.store,
|
||||||
trustedOrigins: allowedOrigins,
|
trustedOrigins: allowedOrigins,
|
||||||
@@ -297,6 +312,7 @@ export function createApiApp(options: CreateApiAppOptions): Hono {
|
|||||||
|
|
||||||
mountModuleHttpRoutes(app, options.moduleRoutes, {
|
mountModuleHttpRoutes(app, options.moduleRoutes, {
|
||||||
configurations: options.moduleConfiguration.snapshots,
|
configurations: options.moduleConfiguration.snapshots,
|
||||||
|
storages: options.moduleStorage.snapshots,
|
||||||
evaluator: options.authorization.evaluator,
|
evaluator: options.authorization.evaluator,
|
||||||
registry: options.authorization.registry,
|
registry: options.authorization.registry,
|
||||||
runtime,
|
runtime,
|
||||||
|
|||||||
@@ -10,6 +10,11 @@ import {
|
|||||||
type IdentityConfig,
|
type IdentityConfig,
|
||||||
type IdentityConfigInput,
|
type IdentityConfigInput,
|
||||||
} from "@ekklesia/core/identity";
|
} from "@ekklesia/core/identity";
|
||||||
|
import {
|
||||||
|
createStorageProviderConfig,
|
||||||
|
type StorageProviderConfig,
|
||||||
|
type StorageProviderConfigInput,
|
||||||
|
} from "@ekklesia/core/storage";
|
||||||
import {
|
import {
|
||||||
type ClientAddressConfig,
|
type ClientAddressConfig,
|
||||||
type ClientAddressConfigInput,
|
type ClientAddressConfigInput,
|
||||||
@@ -55,6 +60,16 @@ export interface ApiRuntimeConfig {
|
|||||||
readonly host: string;
|
readonly host: string;
|
||||||
readonly identity: IdentityConfig;
|
readonly identity: IdentityConfig;
|
||||||
readonly port: number;
|
readonly port: number;
|
||||||
|
/**
|
||||||
|
* The validated host object-storage provider, or `undefined` when this installation has none
|
||||||
|
* (ADR 0030).
|
||||||
|
*
|
||||||
|
* `undefined` is a complete, valid deployment: modules that declare no logical buckets are
|
||||||
|
* unaffected, and a module that does declare one is reported blocked rather than crashing
|
||||||
|
* anything. These values reach Core's provider and nowhere else — no module, module UI, browser
|
||||||
|
* bundle, diagnostic, or API response ever sees the endpoint or the credentials.
|
||||||
|
*/
|
||||||
|
readonly storage: StorageProviderConfig | undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface ApiRuntimeConfigInput {
|
export interface ApiRuntimeConfigInput {
|
||||||
@@ -68,6 +83,8 @@ export interface ApiRuntimeConfigInput {
|
|||||||
readonly host?: unknown;
|
readonly host?: unknown;
|
||||||
readonly identity: IdentityConfigInput;
|
readonly identity: IdentityConfigInput;
|
||||||
readonly port?: unknown;
|
readonly port?: unknown;
|
||||||
|
/** Optional. Omitted, or with no endpoint, means this installation has no object storage. */
|
||||||
|
readonly storage?: StorageProviderConfigInput;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface CreateApiRuntimeConfigOptions {
|
export interface CreateApiRuntimeConfigOptions {
|
||||||
@@ -142,6 +159,11 @@ export function createApiRuntimeConfig(
|
|||||||
// the URL carries TLS parameters that could override the policy.
|
// the URL carries TLS parameters that could override the policy.
|
||||||
assertProductionDatabaseTransport(databaseUrl, databaseTransport, environment);
|
assertProductionDatabaseTransport(databaseUrl, databaseTransport, environment);
|
||||||
|
|
||||||
|
// Validated in the same pass as every other runtime value, before `startApi` constructs a pool,
|
||||||
|
// a provider client, or a listener — so a malformed endpoint or a half-supplied credential pair
|
||||||
|
// fails startup with a sanitized code rather than surfacing later as a confusing storage error.
|
||||||
|
const storage = createStorageProviderConfig(input.storage, { environment });
|
||||||
|
|
||||||
return Object.freeze({
|
return Object.freeze({
|
||||||
clientAddress,
|
clientAddress,
|
||||||
databaseTransport,
|
databaseTransport,
|
||||||
@@ -149,5 +171,6 @@ export function createApiRuntimeConfig(
|
|||||||
host,
|
host,
|
||||||
identity,
|
identity,
|
||||||
port,
|
port,
|
||||||
|
storage,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
import {
|
import {
|
||||||
EMPTY_MODULE_CONFIGURATION,
|
EMPTY_MODULE_CONFIGURATION,
|
||||||
|
EMPTY_MODULE_STORAGE,
|
||||||
type ModuleId,
|
type ModuleId,
|
||||||
type ModuleRouteHandlerContext,
|
type ModuleRouteHandlerContext,
|
||||||
type ModuleRuntimeServices,
|
type ModuleRuntimeServices,
|
||||||
type RegisteredModuleHttpRoute,
|
type RegisteredModuleHttpRoute,
|
||||||
} from "@ekklesia/contracts";
|
} from "@ekklesia/contracts";
|
||||||
import type { ModuleConfigurationLookup } from "@ekklesia/core/modules";
|
import type { ModuleConfigurationLookup, ModuleStorageLookup } from "@ekklesia/core/modules";
|
||||||
import type { Context, Hono } from "hono";
|
import type { Context, Hono } from "hono";
|
||||||
import {
|
import {
|
||||||
type AuthorizationMiddlewareOptions,
|
type AuthorizationMiddlewareOptions,
|
||||||
@@ -43,6 +44,15 @@ export interface ModuleHttpRoutesOptions extends AuthorizationMiddlewareOptions
|
|||||||
* ever reach its own module's configuration, because that is the only entry looked up for it.
|
* ever reach its own module's configuration, because that is the only entry looked up for it.
|
||||||
*/
|
*/
|
||||||
readonly configurations: ModuleConfigurationLookup;
|
readonly configurations: ModuleConfigurationLookup;
|
||||||
|
/**
|
||||||
|
* Per-module scoped object storage — the exact map the module runtime started from (ADR 0030), so
|
||||||
|
* a handler and that module's lifecycle hooks share one facade rather than two independently
|
||||||
|
* resolved ones.
|
||||||
|
*
|
||||||
|
* Keyed by module id, like `configurations` and unlike `services`: a handler can only ever reach
|
||||||
|
* its own module's storage, because that is the only entry looked up for it.
|
||||||
|
*/
|
||||||
|
readonly storages: ModuleStorageLookup;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Never carries the offending permission id, route path, module id, or any other configuration
|
/** Never carries the offending permission id, route path, module id, or any other configuration
|
||||||
@@ -226,6 +236,7 @@ export function mountModuleHttpRoutes(
|
|||||||
const context: ModuleRouteHandlerContext = Object.freeze({
|
const context: ModuleRouteHandlerContext = Object.freeze({
|
||||||
config: options.configurations.get(route.moduleId) ?? EMPTY_MODULE_CONFIGURATION,
|
config: options.configurations.get(route.moduleId) ?? EMPTY_MODULE_CONFIGURATION,
|
||||||
moduleId: route.moduleId,
|
moduleId: route.moduleId,
|
||||||
|
storage: options.storages.get(route.moduleId) ?? EMPTY_MODULE_STORAGE,
|
||||||
params: Object.freeze({ ...c.req.param() }),
|
params: Object.freeze({ ...c.req.param() }),
|
||||||
principal: getAuthorizedPrincipal(c),
|
principal: getAuthorizedPrincipal(c),
|
||||||
services: options.services,
|
services: options.services,
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import { CORE_MODULES_LIFECYCLE_MANAGE } from "@ekklesia/core/authorization";
|
|||||||
import {
|
import {
|
||||||
type ModuleConfigurationStore,
|
type ModuleConfigurationStore,
|
||||||
type ModuleLifecycleSnapshot,
|
type ModuleLifecycleSnapshot,
|
||||||
|
type ModuleStorageAvailability,
|
||||||
type ModuleLifecycleStore,
|
type ModuleLifecycleStore,
|
||||||
ModuleLifecycleStoreError,
|
ModuleLifecycleStoreError,
|
||||||
type ModuleLifecycleView,
|
type ModuleLifecycleView,
|
||||||
@@ -65,6 +66,9 @@ export interface ModuleLifecycleModuleView {
|
|||||||
/** Present for a configuration reason (ADR 0029): the offending key, never its value. A key is
|
/** Present for a configuration reason (ADR 0029): the offending key, never its value. A key is
|
||||||
* declared in a public manifest; a value may be a credential or arbitrary operator text. */
|
* declared in a public manifest; a value may be a credential or arbitrary operator text. */
|
||||||
readonly configurationKey?: string;
|
readonly configurationKey?: string;
|
||||||
|
/** Present for a storage reason (ADR 0030): the logical bucket the module declared, never an
|
||||||
|
* endpoint, a credential, or the physical bucket the host maps it to. */
|
||||||
|
readonly storageBucketId?: string;
|
||||||
};
|
};
|
||||||
readonly restartRequired: boolean;
|
readonly restartRequired: boolean;
|
||||||
}
|
}
|
||||||
@@ -90,6 +94,9 @@ export interface CreateModuleLifecycleRoutesOptions extends AuthorizationMiddlew
|
|||||||
readonly store: ModuleConfigurationStore;
|
readonly store: ModuleConfigurationStore;
|
||||||
readonly secrets: ModuleSecretSource;
|
readonly secrets: ModuleSecretSource;
|
||||||
};
|
};
|
||||||
|
/** The other axis that decides eligibility (ADR 0030). A boolean, never a probe: whether storage
|
||||||
|
* is *configured* decides eligibility, while whether it is *reachable right now* never does. */
|
||||||
|
readonly storage: ModuleStorageAvailability;
|
||||||
readonly trustedOrigins: ReadonlySet<string>;
|
readonly trustedOrigins: ReadonlySet<string>;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -156,6 +163,9 @@ function present(view: ModuleLifecycleView): ModuleLifecycleModuleView {
|
|||||||
...(view.runtime.configurationKey === undefined
|
...(view.runtime.configurationKey === undefined
|
||||||
? {}
|
? {}
|
||||||
: { configurationKey: view.runtime.configurationKey }),
|
: { configurationKey: view.runtime.configurationKey }),
|
||||||
|
...(view.runtime.storageBucketId === undefined
|
||||||
|
? {}
|
||||||
|
: { storageBucketId: view.runtime.storageBucketId }),
|
||||||
},
|
},
|
||||||
version: view.version,
|
version: view.version,
|
||||||
};
|
};
|
||||||
@@ -220,10 +230,12 @@ export function createModuleLifecycleRoutes(options: CreateModuleLifecycleRoutes
|
|||||||
options.store.readConfiguration(),
|
options.store.readConfiguration(),
|
||||||
options.configuration.store.readSettings(),
|
options.configuration.store.readSettings(),
|
||||||
]);
|
]);
|
||||||
const modules = resolveModuleLifecycleView(options.snapshot, configuration, {
|
const modules = resolveModuleLifecycleView(
|
||||||
secrets: options.configuration.secrets,
|
options.snapshot,
|
||||||
settings,
|
configuration,
|
||||||
}).map(present);
|
{ secrets: options.configuration.secrets, settings },
|
||||||
|
options.storage,
|
||||||
|
).map(present);
|
||||||
return c.json({ modules } satisfies ModuleLifecycleResponse, 200);
|
return c.json({ modules } satisfies ModuleLifecycleResponse, 200);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
return storeFailure(c, error, "read");
|
return storeFailure(c, error, "read");
|
||||||
@@ -254,10 +266,12 @@ export function createModuleLifecycleRoutes(options: CreateModuleLifecycleRoutes
|
|||||||
options.store.readConfiguration(),
|
options.store.readConfiguration(),
|
||||||
options.configuration.store.readSettings(),
|
options.configuration.store.readSettings(),
|
||||||
]);
|
]);
|
||||||
const view = resolveModuleLifecycleView(options.snapshot, configuration, {
|
const view = resolveModuleLifecycleView(
|
||||||
secrets: options.configuration.secrets,
|
options.snapshot,
|
||||||
settings,
|
configuration,
|
||||||
}).find((entry) => entry.moduleId === moduleId);
|
{ secrets: options.configuration.secrets, settings },
|
||||||
|
options.storage,
|
||||||
|
).find((entry) => entry.moduleId === moduleId);
|
||||||
if (!view) {
|
if (!view) {
|
||||||
return sanitizedFailure(c, 503, "MODULE_LIFECYCLE_READ_FAILED");
|
return sanitizedFailure(c, 503, "MODULE_LIFECYCLE_READ_FAILED");
|
||||||
}
|
}
|
||||||
|
|||||||
+65
-4
@@ -12,6 +12,11 @@ import {
|
|||||||
} from "@ekklesia/core/authorization";
|
} from "@ekklesia/core/authorization";
|
||||||
import { createDashboardStore } from "@ekklesia/core/dashboard";
|
import { createDashboardStore } from "@ekklesia/core/dashboard";
|
||||||
import { createIdentityRuntime, type IdentityRateLimitOverride } from "@ekklesia/core/identity";
|
import { createIdentityRuntime, type IdentityRateLimitOverride } from "@ekklesia/core/identity";
|
||||||
|
import {
|
||||||
|
createModuleStorageLookup,
|
||||||
|
createS3StorageProvider,
|
||||||
|
type StorageProvider,
|
||||||
|
} from "@ekklesia/core/storage";
|
||||||
import {
|
import {
|
||||||
createEnvironmentModuleSecretSource,
|
createEnvironmentModuleSecretSource,
|
||||||
createModuleConfigurationStore,
|
createModuleConfigurationStore,
|
||||||
@@ -171,6 +176,19 @@ async function safeCloseDatabase(handle: ApiDatabaseHandle): Promise<void> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, for the storage provider's own
|
||||||
|
* connection pool. */
|
||||||
|
async function safeCloseStorage(provider: StorageProvider | undefined): Promise<void> {
|
||||||
|
if (!provider) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await provider.close();
|
||||||
|
} catch {
|
||||||
|
// See safeCloseDatabase: never mask a failure that is already propagating.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, used only while unwinding a startup
|
/** Same swallow-on-cleanup-path rule as `safeCloseDatabase`, used only while unwinding a startup
|
||||||
* failure that already occurred after the module runtime started — never during normal shutdown,
|
* failure that already occurred after the module runtime started — never during normal shutdown,
|
||||||
* where a module stop failure remains an observable result instead (see `startApi`'s `shutdown`). */
|
* where a module stop failure remains an observable result instead (see `startApi`'s `shutdown`). */
|
||||||
@@ -422,6 +440,13 @@ export async function startApi(
|
|||||||
// start, before a single module entrypoint is imported. A failure here fails startup rather than
|
// start, before a single module entrypoint is imported. A failure here fails startup rather than
|
||||||
// falling back to "start everything": the platform cannot know what it was told, and starting a
|
// falling back to "start everything": the platform cannot know what it was told, and starting a
|
||||||
// module an administrator disabled is worse than not starting at all.
|
// module an administrator disabled is worse than not starting at all.
|
||||||
|
// The host's object-storage provider, constructed from validated configuration before any module
|
||||||
|
// runs — and **without touching the network**. Building the client opens no connection, so a
|
||||||
|
// storage outage can never stop the platform from booting; a bucket is ensured lazily on first
|
||||||
|
// use instead (ADR 0030). An installation with no storage configured gets `undefined` here, which
|
||||||
|
// is a complete, valid deployment.
|
||||||
|
const storageProvider = config.storage ? createS3StorageProvider(config.storage) : undefined;
|
||||||
|
|
||||||
const moduleLifecycleStore = createModuleLifecycleStore(databaseHandle.moduleLifecycle);
|
const moduleLifecycleStore = createModuleLifecycleStore(databaseHandle.moduleLifecycle);
|
||||||
const moduleConfigurationStore = createModuleConfigurationStore(
|
const moduleConfigurationStore = createModuleConfigurationStore(
|
||||||
databaseHandle.moduleConfiguration,
|
databaseHandle.moduleConfiguration,
|
||||||
@@ -441,15 +466,29 @@ export async function startApi(
|
|||||||
const lifecycleConfiguration = await moduleLifecycleStore.readConfiguration();
|
const lifecycleConfiguration = await moduleLifecycleStore.readConfiguration();
|
||||||
const settings = await moduleConfigurationStore.readSettings();
|
const settings = await moduleConfigurationStore.readSettings();
|
||||||
moduleConfigurationSettings = settings;
|
moduleConfigurationSettings = settings;
|
||||||
eligibility = resolveModuleEligibility(inventory.modules, lifecycleConfiguration, {
|
eligibility = resolveModuleEligibility(
|
||||||
secrets: moduleSecrets,
|
inventory.modules,
|
||||||
settings,
|
lifecycleConfiguration,
|
||||||
});
|
{ secrets: moduleSecrets, settings },
|
||||||
|
{ providerConfigured: storageProvider !== undefined },
|
||||||
|
);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
await safeCloseStorage(storageProvider);
|
||||||
await safeCloseDatabase(databaseHandle);
|
await safeCloseDatabase(databaseHandle);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// One scoped facade per eligible module, over the exact buckets that module's own manifest
|
||||||
|
// declares. Built from the eligible set so a blocked module never receives storage at all.
|
||||||
|
const moduleStorages = createModuleStorageLookup(
|
||||||
|
storageProvider,
|
||||||
|
config.storage?.bucketPrefix ?? "",
|
||||||
|
eligibility.eligibleModules.map((module) => ({
|
||||||
|
logicalBucketIds: module.manifest.storage.buckets.map((bucket) => bucket.id),
|
||||||
|
moduleId: module.manifest.id,
|
||||||
|
})),
|
||||||
|
);
|
||||||
|
|
||||||
// Module load/register/start failures use Core's own existing rollback (already-started
|
// Module load/register/start failures use Core's own existing rollback (already-started
|
||||||
// modules are stopped, in reverse order, before this rejects) — there is no separate runtime
|
// modules are stopped, in reverse order, before this rejects) — there is no separate runtime
|
||||||
// handle to stop here, only the pool this failure leaves open.
|
// handle to stop here, only the pool this failure leaves open.
|
||||||
@@ -462,11 +501,13 @@ export async function startApi(
|
|||||||
try {
|
try {
|
||||||
moduleRuntime = await startDiscoveredModuleRuntime(eligibility.eligibleModules, services, {
|
moduleRuntime = await startDiscoveredModuleRuntime(eligibility.eligibleModules, services, {
|
||||||
configurations: eligibility.configurations,
|
configurations: eligibility.configurations,
|
||||||
|
storages: moduleStorages,
|
||||||
...(options.moduleHookTimeoutMsOverride === undefined
|
...(options.moduleHookTimeoutMsOverride === undefined
|
||||||
? {}
|
? {}
|
||||||
: { hookTimeoutMs: options.moduleHookTimeoutMsOverride }),
|
: { hookTimeoutMs: options.moduleHookTimeoutMsOverride }),
|
||||||
});
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
await safeCloseStorage(storageProvider);
|
||||||
await safeCloseDatabase(databaseHandle);
|
await safeCloseDatabase(databaseHandle);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
@@ -507,6 +548,10 @@ export async function startApi(
|
|||||||
snapshots: eligibility.configurations,
|
snapshots: eligibility.configurations,
|
||||||
store: moduleConfigurationStore,
|
store: moduleConfigurationStore,
|
||||||
},
|
},
|
||||||
|
moduleStorage: {
|
||||||
|
providerConfigured: storageProvider !== undefined,
|
||||||
|
snapshots: moduleStorages,
|
||||||
|
},
|
||||||
// A typed view over the same single pool as identity, authorization, and the module facade.
|
// A typed view over the same single pool as identity, authorization, and the module facade.
|
||||||
dashboard: createDashboardStore(databaseHandle.dashboard),
|
dashboard: createDashboardStore(databaseHandle.dashboard),
|
||||||
runtime,
|
runtime,
|
||||||
@@ -514,6 +559,7 @@ export async function startApi(
|
|||||||
});
|
});
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await safeStopModuleRuntime(moduleRuntime);
|
await safeStopModuleRuntime(moduleRuntime);
|
||||||
|
await safeCloseStorage(storageProvider);
|
||||||
await safeCloseDatabase(databaseHandle);
|
await safeCloseDatabase(databaseHandle);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
@@ -529,6 +575,7 @@ export async function startApi(
|
|||||||
server = await bindServer(app, config.host, config.port, clientAddressResolver, timeouts);
|
server = await bindServer(app, config.host, config.port, clientAddressResolver, timeouts);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
await safeStopModuleRuntime(moduleRuntime);
|
await safeStopModuleRuntime(moduleRuntime);
|
||||||
|
await safeCloseStorage(storageProvider);
|
||||||
await safeCloseDatabase(databaseHandle);
|
await safeCloseDatabase(databaseHandle);
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
@@ -558,6 +605,9 @@ export async function startApi(
|
|||||||
} catch (error) {
|
} catch (error) {
|
||||||
moduleStopError = error;
|
moduleStopError = error;
|
||||||
}
|
}
|
||||||
|
// After the module runtime has stopped, so a `stop` hook may still use its storage
|
||||||
|
// facade, and before the pool closes, matching the existing ordering discipline exactly.
|
||||||
|
await safeCloseStorage(storageProvider);
|
||||||
await safeCloseDatabase(databaseHandle);
|
await safeCloseDatabase(databaseHandle);
|
||||||
// Every stage above ran regardless of what the earlier ones did, so a drain timeout or a
|
// Every stage above ran regardless of what the earlier ones did, so a drain timeout or a
|
||||||
// module-stop failure can never leave the pool open. Both failures are reported together,
|
// module-stop failure can never leave the pool open. Both failures are reported together,
|
||||||
@@ -652,6 +702,17 @@ function readConfigFromEnvironment(): {
|
|||||||
trustedOrigins,
|
trustedOrigins,
|
||||||
},
|
},
|
||||||
port: process.env.EKKLESIA_API_PORT,
|
port: process.env.EKKLESIA_API_PORT,
|
||||||
|
// The one place this process turns its environment into storage configuration. The names are
|
||||||
|
// the `S3_*` ones ADR 0004 already established for this repository's Compose stack, and Core's
|
||||||
|
// validator never reads `process.env` itself.
|
||||||
|
storage: {
|
||||||
|
accessKey: process.env.S3_ACCESS_KEY,
|
||||||
|
bucketPrefix: process.env.S3_BUCKET_PREFIX,
|
||||||
|
endpoint: process.env.S3_ENDPOINT,
|
||||||
|
forcePathStyle: process.env.S3_FORCE_PATH_STYLE,
|
||||||
|
region: process.env.S3_REGION,
|
||||||
|
secretKey: process.env.S3_SECRET_KEY,
|
||||||
|
},
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ import { createApiApp, type CreateApiAppAuthorizationOptions } from "../src/app.
|
|||||||
import { AUTH_REQUEST_MAX_BODY_BYTES } from "../src/body-limits.js";
|
import { AUTH_REQUEST_MAX_BODY_BYTES } from "../src/body-limits.js";
|
||||||
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
|
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
|
||||||
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
|
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { emptyModuleStorage } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const untrustedOrigin = "https://evil.example.test";
|
const untrustedOrigin = "https://evil.example.test";
|
||||||
@@ -92,6 +93,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -113,6 +115,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -135,6 +138,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -159,6 +163,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -185,6 +190,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -211,6 +217,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -234,6 +241,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -259,6 +267,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -281,6 +290,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -305,6 +315,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -336,6 +347,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -368,6 +380,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -405,6 +418,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -424,6 +438,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -445,6 +460,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -465,6 +481,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -483,6 +500,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -508,6 +526,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
@@ -536,6 +555,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({ evaluator: { authorize: evaluatorAuthorize } }),
|
authorization: fakeAuthorization({ evaluator: { authorize: evaluatorAuthorize } }),
|
||||||
@@ -567,6 +587,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({
|
authorization: fakeAuthorization({
|
||||||
@@ -589,6 +610,7 @@ describe("Ekklesia API Hono application", () => {
|
|||||||
moduleUi: emptyModuleUiPublication(),
|
moduleUi: emptyModuleUiPublication(),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleServices: fakeModuleServices(),
|
moduleServices: fakeModuleServices(),
|
||||||
authorization: fakeAuthorization({}),
|
authorization: fakeAuthorization({}),
|
||||||
|
|||||||
@@ -31,6 +31,7 @@ import {
|
|||||||
moduleLifecycleSnapshotFor,
|
moduleLifecycleSnapshotFor,
|
||||||
} from "./support/in-memory-module-lifecycle.js";
|
} from "./support/in-memory-module-lifecycle.js";
|
||||||
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const untrustedOrigin = "https://evil.example.test";
|
const untrustedOrigin = "https://evil.example.test";
|
||||||
@@ -147,6 +148,7 @@ async function install(options: InstallOptions = {}): Promise<{ app: Hono }> {
|
|||||||
store: createInMemoryModuleLifecycleStore(),
|
store: createInMemoryModuleLifecycleStore(),
|
||||||
},
|
},
|
||||||
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
runtime,
|
runtime,
|
||||||
trustedOrigins: [trustedOrigin],
|
trustedOrigins: [trustedOrigin],
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -39,6 +39,7 @@ import {
|
|||||||
createInMemoryModuleConfigurationStore,
|
createInMemoryModuleConfigurationStore,
|
||||||
} from "./support/in-memory-module-configuration.js";
|
} from "./support/in-memory-module-configuration.js";
|
||||||
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
|
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const untrustedOrigin = "https://evil.example.test";
|
const untrustedOrigin = "https://evil.example.test";
|
||||||
@@ -194,6 +195,7 @@ async function install(options: InstallOptions = {}): Promise<Installation> {
|
|||||||
snapshots: eligibility.configurations,
|
snapshots: eligibility.configurations,
|
||||||
store,
|
store,
|
||||||
},
|
},
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
moduleLifecycle: {
|
moduleLifecycle: {
|
||||||
snapshot: {
|
snapshot: {
|
||||||
discoveredModules: modules,
|
discoveredModules: modules,
|
||||||
|
|||||||
@@ -519,12 +519,18 @@ describe.sequential("live API — module database access through ModuleRuntimeSe
|
|||||||
const readHandlerEvent = [...events]
|
const readHandlerEvent = [...events]
|
||||||
.reverse()
|
.reverse()
|
||||||
.find((event) => event.type === "handler" && event.route === "GET /records/:id");
|
.find((event) => event.type === "handler" && event.route === "GET /records/:id");
|
||||||
|
// The complete handler-context surface, asserted exactly so a new field is a deliberate change
|
||||||
|
// here rather than an unnoticed one: `config` (ADR 0029) and `storage` (ADR 0030) are the two
|
||||||
|
// scoped, per-module additions; everything else a module might want — a Hono Context, the
|
||||||
|
// authorization store, raw PostgreSQL, the storage provider, an endpoint or credential — remains
|
||||||
|
// unreachable.
|
||||||
expect(readHandlerEvent?.contextKeys).toEqual([
|
expect(readHandlerEvent?.contextKeys).toEqual([
|
||||||
"config",
|
"config",
|
||||||
"moduleId",
|
"moduleId",
|
||||||
"params",
|
"params",
|
||||||
"principal",
|
"principal",
|
||||||
"services",
|
"services",
|
||||||
|
"storage",
|
||||||
]);
|
]);
|
||||||
expect(readHandlerEvent?.services).toBeDefined();
|
expect(readHandlerEvent?.services).toBeDefined();
|
||||||
expect(readHandlerEvent?.services?.database).toBe(startEvent?.services?.database);
|
expect(readHandlerEvent?.services?.database).toBe(startEvent?.services?.database);
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ import {
|
|||||||
} from "../src/module-http-routes.js";
|
} from "../src/module-http-routes.js";
|
||||||
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
|
import { emptyModuleLifecycle } from "./support/in-memory-module-lifecycle.js";
|
||||||
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
|
import { emptyModuleConfiguration } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { emptyModuleStorage } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const untrustedOrigin = "https://evil.example.test";
|
const untrustedOrigin = "https://evil.example.test";
|
||||||
@@ -113,6 +114,7 @@ function buildModuleApp(options: {
|
|||||||
moduleUi: createModuleUiPublicationRegistry([]),
|
moduleUi: createModuleUiPublicationRegistry([]),
|
||||||
moduleLifecycle: emptyModuleLifecycle(),
|
moduleLifecycle: emptyModuleLifecycle(),
|
||||||
moduleConfiguration: emptyModuleConfiguration(),
|
moduleConfiguration: emptyModuleConfiguration(),
|
||||||
|
moduleStorage: emptyModuleStorage(),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
runtime,
|
runtime,
|
||||||
trustedOrigins: options.trustedOrigins ?? [trustedOrigin],
|
trustedOrigins: options.trustedOrigins ?? [trustedOrigin],
|
||||||
@@ -339,17 +341,19 @@ describe("module route handler adaptation", () => {
|
|||||||
expect(body.url).toContain("/api/v1/modules/calendar/events/evt-1/rsvp?notify=true");
|
expect(body.url).toContain("/api/v1/modules/calendar/events/evt-1/rsvp?notify=true");
|
||||||
expect(capturedRequest).toBeInstanceOf(Request);
|
expect(capturedRequest).toBeInstanceOf(Request);
|
||||||
expect(capturedRequest?.headers.get("x-custom")).toBe("abc");
|
expect(capturedRequest?.headers.get("x-custom")).toBe("abc");
|
||||||
// Exactly config, moduleId, params, principal, services — and nothing else. `config` is this
|
// Exactly config, moduleId, params, principal, services, storage — and nothing else. `config`
|
||||||
// module's own frozen configuration snapshot, added deliberately by ADR 0029; a Hono Context,
|
// is this module's own frozen configuration snapshot (ADR 0029) and `storage` its own scoped
|
||||||
// the authorization store or evaluator, raw PostgreSQL, platform configuration, and another
|
// object storage (ADR 0030); a Hono Context, the authorization store or evaluator, raw
|
||||||
// module's configuration all remain unreachable, which is what this exact-shape assertion
|
// PostgreSQL, the storage provider, an S3 endpoint or credential, platform configuration, and
|
||||||
// exists to keep true.
|
// another module's configuration or storage all remain unreachable, which is what this
|
||||||
|
// exact-shape assertion exists to keep true.
|
||||||
expect(Object.keys(capturedContext ?? {}).sort()).toEqual([
|
expect(Object.keys(capturedContext ?? {}).sort()).toEqual([
|
||||||
"config",
|
"config",
|
||||||
"moduleId",
|
"moduleId",
|
||||||
"params",
|
"params",
|
||||||
"principal",
|
"principal",
|
||||||
"services",
|
"services",
|
||||||
|
"storage",
|
||||||
]);
|
]);
|
||||||
expect(capturedContext).toMatchObject({
|
expect(capturedContext).toMatchObject({
|
||||||
moduleId: "calendar",
|
moduleId: "calendar",
|
||||||
@@ -366,6 +370,10 @@ describe("module route handler adaptation", () => {
|
|||||||
// This module declared no configuration, so its snapshot is empty rather than absent — module
|
// This module declared no configuration, so its snapshot is empty rather than absent — module
|
||||||
// code never has to branch on whether the platform supplied one.
|
// code never has to branch on whether the platform supplied one.
|
||||||
expect(capturedContext?.config.keys()).toEqual([]);
|
expect(capturedContext?.config.keys()).toEqual([]);
|
||||||
|
// Likewise storage: this module declared no buckets, so it gets a frozen facade that refuses
|
||||||
|
// every one rather than `undefined`.
|
||||||
|
expect(Object.isFrozen(capturedContext?.storage)).toBe(true);
|
||||||
|
expect(capturedContext?.storage.buckets()).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("gives static and root routes a frozen, empty params object", async () => {
|
it("gives static and root routes a frozen, empty params object", async () => {
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ import { MODULE_UI_INVENTORY_PATH } from "../src/module-ui-inventory.js";
|
|||||||
import { createInMemoryDashboardStore } from "./support/in-memory-dashboard-store.js";
|
import { createInMemoryDashboardStore } from "./support/in-memory-dashboard-store.js";
|
||||||
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
|
import { createInMemoryModuleLifecycleStore } from "./support/in-memory-module-lifecycle.js";
|
||||||
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const untrustedOrigin = "https://evil.example.test";
|
const untrustedOrigin = "https://evil.example.test";
|
||||||
@@ -170,6 +171,7 @@ async function install(options: InstallOptions = {}): Promise<Installation> {
|
|||||||
},
|
},
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
moduleLifecycle: {
|
moduleLifecycle: {
|
||||||
snapshot: {
|
snapshot: {
|
||||||
discoveredModules: modules,
|
discoveredModules: modules,
|
||||||
|
|||||||
@@ -35,6 +35,7 @@ import {
|
|||||||
moduleLifecycleSnapshotFor,
|
moduleLifecycleSnapshotFor,
|
||||||
} from "./support/in-memory-module-lifecycle.js";
|
} from "./support/in-memory-module-lifecycle.js";
|
||||||
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The whole chain, with nothing between the halves faked:
|
* The whole chain, with nothing between the halves faked:
|
||||||
@@ -223,6 +224,7 @@ async function startHarness(options: HarnessOptions = {}): Promise<Harness> {
|
|||||||
store: createInMemoryModuleLifecycleStore(),
|
store: createInMemoryModuleLifecycleStore(),
|
||||||
},
|
},
|
||||||
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
runtime,
|
runtime,
|
||||||
trustedOrigins: ["https://dashboard.example.test"],
|
trustedOrigins: ["https://dashboard.example.test"],
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ import {
|
|||||||
moduleLifecycleSnapshotFor,
|
moduleLifecycleSnapshotFor,
|
||||||
} from "./support/in-memory-module-lifecycle.js";
|
} from "./support/in-memory-module-lifecycle.js";
|
||||||
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
const viewerId = parseEkklesiaUserId("11111111-1111-4111-8111-111111111111");
|
const viewerId = parseEkklesiaUserId("11111111-1111-4111-8111-111111111111");
|
||||||
@@ -179,6 +180,7 @@ async function install(options: InstallOptions): Promise<Installation> {
|
|||||||
store: createInMemoryModuleLifecycleStore(),
|
store: createInMemoryModuleLifecycleStore(),
|
||||||
},
|
},
|
||||||
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
runtime,
|
runtime,
|
||||||
trustedOrigins: [trustedOrigin],
|
trustedOrigins: [trustedOrigin],
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ import {
|
|||||||
moduleLifecycleSnapshotFor,
|
moduleLifecycleSnapshotFor,
|
||||||
} from "./support/in-memory-module-lifecycle.js";
|
} from "./support/in-memory-module-lifecycle.js";
|
||||||
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
import { moduleConfigurationOptionFor } from "./support/in-memory-module-configuration.js";
|
||||||
|
import { moduleStorageOptionFor } from "./support/in-memory-module-storage.js";
|
||||||
|
|
||||||
const trustedOrigin = "https://dashboard.example.test";
|
const trustedOrigin = "https://dashboard.example.test";
|
||||||
|
|
||||||
@@ -179,6 +180,7 @@ async function install(fixtures: readonly ModuleFixture[]): Promise<Installation
|
|||||||
store: createInMemoryModuleLifecycleStore(),
|
store: createInMemoryModuleLifecycleStore(),
|
||||||
},
|
},
|
||||||
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
moduleConfiguration: moduleConfigurationOptionFor(modules),
|
||||||
|
moduleStorage: moduleStorageOptionFor(modules),
|
||||||
dashboard: createInMemoryDashboardStore(),
|
dashboard: createInMemoryDashboardStore(),
|
||||||
runtime,
|
runtime,
|
||||||
trustedOrigins: [trustedOrigin],
|
trustedOrigins: [trustedOrigin],
|
||||||
|
|||||||
@@ -0,0 +1,195 @@
|
|||||||
|
import {
|
||||||
|
type ModuleStorage,
|
||||||
|
type ModuleStorageBody,
|
||||||
|
ModuleStorageError,
|
||||||
|
type ModuleStorageListOptions,
|
||||||
|
type ModuleStorageListing,
|
||||||
|
type ModuleStorageObject,
|
||||||
|
type ModuleStorageObjectMetadata,
|
||||||
|
type ModuleStoragePutOptions,
|
||||||
|
} from "@ekklesia/contracts";
|
||||||
|
import type { DiscoveredModule, ModuleStorageLookup } from "@ekklesia/core/modules";
|
||||||
|
import { createModuleStorageLookup, type StorageProvider } from "@ekklesia/core/storage";
|
||||||
|
|
||||||
|
/** Builds and drains Web streams without depending on DOM-only helper types, which this package's
|
||||||
|
* TypeScript configuration deliberately does not load. */
|
||||||
|
function streamOf(bytes: Uint8Array): ReadableStream<Uint8Array> {
|
||||||
|
return new ReadableStream<Uint8Array>({
|
||||||
|
start(controller) {
|
||||||
|
controller.enqueue(bytes);
|
||||||
|
controller.close();
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function collect(stream: ReadableStream<Uint8Array>): Promise<Uint8Array> {
|
||||||
|
const reader = stream.getReader();
|
||||||
|
const chunks: Uint8Array[] = [];
|
||||||
|
let total = 0;
|
||||||
|
for (;;) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
chunks.push(value);
|
||||||
|
total += value.byteLength;
|
||||||
|
}
|
||||||
|
const merged = new Uint8Array(total);
|
||||||
|
let offset = 0;
|
||||||
|
for (const chunk of chunks) {
|
||||||
|
merged.set(chunk, offset);
|
||||||
|
offset += chunk.byteLength;
|
||||||
|
}
|
||||||
|
return merged;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface StoredObject {
|
||||||
|
readonly bytes: Uint8Array;
|
||||||
|
readonly contentType?: string;
|
||||||
|
readonly lastModified: Date;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An in-memory `StorageProvider` for tests that exercise **wiring, scoping, and routing** rather
|
||||||
|
* than the S3 protocol.
|
||||||
|
*
|
||||||
|
* It implements the same interface the real S3 provider does, so everything above the provider seam
|
||||||
|
* — binding resolution, the scoped facade, lifecycle eligibility, route wiring — is exercised
|
||||||
|
* unchanged. What it deliberately does *not* reproduce is the protocol itself: real SeaweedFS
|
||||||
|
* semantics (non-idempotent `CreateBucket`, `NoSuchKey` vs `NoSuchBucket`, ETag quoting, streaming
|
||||||
|
* bodies) are proven against a live server in `core/tests/module-storage.integration.ts`, so this
|
||||||
|
* double can never quietly stand in for a provider guarantee nobody checked.
|
||||||
|
*
|
||||||
|
* It keys everything by **physical** bucket, exactly as the real provider does, which is what lets a
|
||||||
|
* test assert that two modules declaring the same logical name really did land in different
|
||||||
|
* namespaces.
|
||||||
|
*/
|
||||||
|
export function createInMemoryStorageProvider(): StorageProvider & {
|
||||||
|
readonly buckets: ReadonlyMap<string, Map<string, StoredObject>>;
|
||||||
|
ensureBucketCalls: number;
|
||||||
|
} {
|
||||||
|
const buckets = new Map<string, Map<string, StoredObject>>();
|
||||||
|
const state = { ensureBucketCalls: 0 };
|
||||||
|
|
||||||
|
const objectsIn = (physicalBucket: string): Map<string, StoredObject> => {
|
||||||
|
const bucket = buckets.get(physicalBucket);
|
||||||
|
if (!bucket) {
|
||||||
|
throw new ModuleStorageError("STORAGE_BUCKET_UNAVAILABLE");
|
||||||
|
}
|
||||||
|
return bucket;
|
||||||
|
};
|
||||||
|
|
||||||
|
const metadataFor = (key: string, stored: StoredObject): ModuleStorageObjectMetadata =>
|
||||||
|
Object.freeze({
|
||||||
|
contentLength: stored.bytes.byteLength,
|
||||||
|
key,
|
||||||
|
lastModified: stored.lastModified,
|
||||||
|
...(stored.contentType === undefined ? {} : { contentType: stored.contentType }),
|
||||||
|
});
|
||||||
|
|
||||||
|
const provider: StorageProvider = {
|
||||||
|
async close(): Promise<void> {
|
||||||
|
// Nothing to release.
|
||||||
|
},
|
||||||
|
async delete(physicalBucket: string, key: string): Promise<void> {
|
||||||
|
objectsIn(physicalBucket).delete(key);
|
||||||
|
},
|
||||||
|
async ensureBucket(physicalBucket: string): Promise<void> {
|
||||||
|
state.ensureBucketCalls += 1;
|
||||||
|
if (!buckets.has(physicalBucket)) {
|
||||||
|
buckets.set(physicalBucket, new Map());
|
||||||
|
}
|
||||||
|
},
|
||||||
|
async get(physicalBucket: string, key: string): Promise<ModuleStorageObject> {
|
||||||
|
const stored = objectsIn(physicalBucket).get(key);
|
||||||
|
if (!stored) {
|
||||||
|
throw new ModuleStorageError("STORAGE_OBJECT_NOT_FOUND", key);
|
||||||
|
}
|
||||||
|
return Object.freeze({
|
||||||
|
...metadataFor(key, stored),
|
||||||
|
body: streamOf(stored.bytes),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
async head(physicalBucket: string, key: string): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
const stored = objectsIn(physicalBucket).get(key);
|
||||||
|
if (!stored) {
|
||||||
|
throw new ModuleStorageError("STORAGE_OBJECT_NOT_FOUND", key);
|
||||||
|
}
|
||||||
|
return metadataFor(key, stored);
|
||||||
|
},
|
||||||
|
async list(
|
||||||
|
physicalBucket: string,
|
||||||
|
options: ModuleStorageListOptions,
|
||||||
|
): Promise<ModuleStorageListing> {
|
||||||
|
const entries = [...objectsIn(physicalBucket).entries()]
|
||||||
|
.filter(([key]) => options.prefix === undefined || key.startsWith(options.prefix))
|
||||||
|
.sort(([left], [right]) => (left < right ? -1 : left > right ? 1 : 0));
|
||||||
|
return Object.freeze({
|
||||||
|
objects: Object.freeze(entries.map(([key, stored]) => metadataFor(key, stored))),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
async probe(): Promise<void> {
|
||||||
|
// Always reachable.
|
||||||
|
},
|
||||||
|
async put(
|
||||||
|
physicalBucket: string,
|
||||||
|
key: string,
|
||||||
|
body: ModuleStorageBody,
|
||||||
|
options: ModuleStoragePutOptions,
|
||||||
|
): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
const bytes =
|
||||||
|
typeof body === "string"
|
||||||
|
? new TextEncoder().encode(body)
|
||||||
|
: body instanceof Uint8Array
|
||||||
|
? body
|
||||||
|
: await collect(body as ReadableStream<Uint8Array>);
|
||||||
|
const stored: StoredObject = {
|
||||||
|
bytes,
|
||||||
|
lastModified: new Date(0),
|
||||||
|
...(options.contentType === undefined ? {} : { contentType: options.contentType }),
|
||||||
|
};
|
||||||
|
objectsIn(physicalBucket).set(key, stored);
|
||||||
|
return metadataFor(key, stored);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
return Object.assign(provider, {
|
||||||
|
buckets: buckets as ReadonlyMap<string, Map<string, StoredObject>>,
|
||||||
|
get ensureBucketCalls(): number {
|
||||||
|
return state.ensureBucketCalls;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface ModuleStorageOption {
|
||||||
|
readonly snapshots: ModuleStorageLookup;
|
||||||
|
readonly providerConfigured: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds the `createApiApp` storage option the same way `startApi` does — resolve every module's
|
||||||
|
* declared buckets against one provider and one prefix — rather than hand-writing a map, so a test
|
||||||
|
* can never assert against a shape the real composition would not produce.
|
||||||
|
*/
|
||||||
|
export function moduleStorageOptionFor(
|
||||||
|
discoveredModules: readonly DiscoveredModule[],
|
||||||
|
provider?: StorageProvider,
|
||||||
|
): ModuleStorageOption {
|
||||||
|
return {
|
||||||
|
providerConfigured: provider !== undefined,
|
||||||
|
snapshots: createModuleStorageLookup(
|
||||||
|
provider,
|
||||||
|
"ekklesia",
|
||||||
|
discoveredModules.map((module) => ({
|
||||||
|
logicalBucketIds: module.manifest.storage.buckets.map((bucket) => bucket.id),
|
||||||
|
moduleId: module.manifest.id,
|
||||||
|
})),
|
||||||
|
),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The storage option for a `createApiApp` caller that exercises no storage at all — the explicit
|
||||||
|
* "this installation has no object storage" statement the required option asks for. */
|
||||||
|
export function emptyModuleStorage(): ModuleStorageOption {
|
||||||
|
return { providerConfigured: false, snapshots: new Map<string, ModuleStorage>() };
|
||||||
|
}
|
||||||
@@ -15,6 +15,10 @@
|
|||||||
"types": "./dist/modules/index.d.ts",
|
"types": "./dist/modules/index.d.ts",
|
||||||
"import": "./dist/modules/index.js"
|
"import": "./dist/modules/index.js"
|
||||||
},
|
},
|
||||||
|
"./storage": {
|
||||||
|
"types": "./dist/storage/index.d.ts",
|
||||||
|
"import": "./dist/storage/index.js"
|
||||||
|
},
|
||||||
"./dashboard": {
|
"./dashboard": {
|
||||||
"types": "./dist/dashboard/index.d.ts",
|
"types": "./dist/dashboard/index.d.ts",
|
||||||
"import": "./dist/dashboard/index.js"
|
"import": "./dist/dashboard/index.js"
|
||||||
@@ -49,6 +53,7 @@
|
|||||||
"typecheck": "tsc --noEmit --project tsconfig.json"
|
"typecheck": "tsc --noEmit --project tsconfig.json"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@aws-sdk/client-s3": "3.1116.0",
|
||||||
"@ekklesia/contracts": "workspace:*",
|
"@ekklesia/contracts": "workspace:*",
|
||||||
"better-auth": "1.6.23",
|
"better-auth": "1.6.23",
|
||||||
"drizzle-orm": "0.45.2",
|
"drizzle-orm": "0.45.2",
|
||||||
|
|||||||
@@ -5,6 +5,8 @@ import {
|
|||||||
createModuleLifecycleDatabase,
|
createModuleLifecycleDatabase,
|
||||||
createModuleLifecycleStore,
|
createModuleLifecycleStore,
|
||||||
discoverModules,
|
discoverModules,
|
||||||
|
isDependencyBlockedDetail,
|
||||||
|
isStorageBlockedDetail,
|
||||||
ModuleDiscoveryError,
|
ModuleDiscoveryError,
|
||||||
ModuleLifecycleStoreError,
|
ModuleLifecycleStoreError,
|
||||||
resolveModuleEligibility,
|
resolveModuleEligibility,
|
||||||
@@ -132,9 +134,13 @@ async function main(): Promise<void> {
|
|||||||
if (!blocked) {
|
if (!blocked) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
// Names the dependency or the configuration key, never a configuration value.
|
// Names the dependency, the configuration key, or the logical storage bucket — never a
|
||||||
const cause =
|
// configuration value, an endpoint, or a credential.
|
||||||
"dependency" in blocked ? blocked.dependency.moduleId : blocked.configurationKey;
|
const cause = isDependencyBlockedDetail(blocked)
|
||||||
|
? blocked.dependency.moduleId
|
||||||
|
: isStorageBlockedDetail(blocked)
|
||||||
|
? blocked.storageBucketId
|
||||||
|
: blocked.configurationKey;
|
||||||
console.log(`- "${entry.moduleId}" will not start: ${blocked.reason} (${cause}).`);
|
console.log(`- "${entry.moduleId}" will not start: ${blocked.reason} (${cause}).`);
|
||||||
}
|
}
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ const sections: readonly DoctorSection[] = Object.freeze([
|
|||||||
section("Data definitions", ["schemas.discovery", "migrations.discovery"]),
|
section("Data definitions", ["schemas.discovery", "migrations.discovery"]),
|
||||||
section("PostgreSQL", ["database.transport", "postgresql.connection", "postgresql.plan"]),
|
section("PostgreSQL", ["database.transport", "postgresql.connection", "postgresql.plan"]),
|
||||||
section("Identity", ["identity.configuration", "identity.client-address"]),
|
section("Identity", ["identity.configuration", "identity.client-address"]),
|
||||||
section("Object storage", ["object-storage.integration"]),
|
section("Object storage", ["object-storage.integration", "object-storage.bindings"]),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
const statusSymbols: Readonly<Record<DoctorCheck["status"], string>> = Object.freeze({
|
const statusSymbols: Readonly<Record<DoctorCheck["status"], string>> = Object.freeze({
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import {
|
|||||||
capabilityIdSchema,
|
capabilityIdSchema,
|
||||||
EMPTY_MODULE_SECRET_SOURCE,
|
EMPTY_MODULE_SECRET_SOURCE,
|
||||||
isVersionCompatible,
|
isVersionCompatible,
|
||||||
|
logicalBucketIdSchema,
|
||||||
moduleConfigurationKeySchema,
|
moduleConfigurationKeySchema,
|
||||||
moduleIdSchema,
|
moduleIdSchema,
|
||||||
type ModuleSecretSource,
|
type ModuleSecretSource,
|
||||||
@@ -42,8 +43,16 @@ import {
|
|||||||
type PostgresMigrationDatabaseOptions,
|
type PostgresMigrationDatabaseOptions,
|
||||||
planModuleMigrations,
|
planModuleMigrations,
|
||||||
} from "../migrations/index.js";
|
} from "../migrations/index.js";
|
||||||
|
import {
|
||||||
|
createS3StorageProvider,
|
||||||
|
describeStorageProvider,
|
||||||
|
physicalBucketName,
|
||||||
|
type StorageProviderConfig,
|
||||||
|
} from "../storage/index.js";
|
||||||
import {
|
import {
|
||||||
composePlatformPermissionRegistry,
|
composePlatformPermissionRegistry,
|
||||||
|
isConfigurationBlockedDetail,
|
||||||
|
isStorageBlockedDetail,
|
||||||
createModuleConfigurationDatabase,
|
createModuleConfigurationDatabase,
|
||||||
createModuleConfigurationStore,
|
createModuleConfigurationStore,
|
||||||
createModuleLifecycleDatabase,
|
createModuleLifecycleDatabase,
|
||||||
@@ -75,7 +84,8 @@ export type DoctorCheckId =
|
|||||||
| "postgresql.plan"
|
| "postgresql.plan"
|
||||||
| "identity.configuration"
|
| "identity.configuration"
|
||||||
| "identity.client-address"
|
| "identity.client-address"
|
||||||
| "object-storage.integration";
|
| "object-storage.integration"
|
||||||
|
| "object-storage.bindings";
|
||||||
|
|
||||||
export type DoctorCheckStatus = "failure" | "pass" | "skipped" | "warning";
|
export type DoctorCheckStatus = "failure" | "pass" | "skipped" | "warning";
|
||||||
export type DoctorResult = "not-ready" | "ready" | "ready-with-warnings";
|
export type DoctorResult = "not-ready" | "ready" | "ready-with-warnings";
|
||||||
@@ -113,6 +123,9 @@ export interface DoctorOptions {
|
|||||||
* matches what `startApi` would resolve. Omitted supplies nothing — Doctor then reports a required
|
* matches what `startApi` would resolve. Omitted supplies nothing — Doctor then reports a required
|
||||||
* secret as missing, which is the honest answer for a deployment that has not set it. */
|
* secret as missing, which is the honest answer for a deployment that has not set it. */
|
||||||
readonly moduleSecrets?: ModuleSecretSource;
|
readonly moduleSecrets?: ModuleSecretSource;
|
||||||
|
/** The validated host storage provider, or `undefined` when this installation has none. Doctor
|
||||||
|
* never receives raw environment values and never prints a credential. */
|
||||||
|
readonly storage?: StorageProviderConfig | undefined;
|
||||||
readonly runtimeRange: string;
|
readonly runtimeRange: string;
|
||||||
readonly runtimeVersion: string;
|
readonly runtimeVersion: string;
|
||||||
}
|
}
|
||||||
@@ -150,6 +163,9 @@ export interface DoctorDependencies {
|
|||||||
readonly database: MigrationDatabase;
|
readonly database: MigrationDatabase;
|
||||||
}) => Promise<MigrationPlanResult>;
|
}) => Promise<MigrationPlanResult>;
|
||||||
readonly runtimeSatisfies: (version: string, range: string) => boolean;
|
readonly runtimeSatisfies: (version: string, range: string) => boolean;
|
||||||
|
/** Probes the configured storage provider for reachability. Behind the dependency boundary like
|
||||||
|
* every other I/O Doctor performs, so the whole report stays exercisable without a network. */
|
||||||
|
readonly probeStorage: (config: StorageProviderConfig) => Promise<void>;
|
||||||
}
|
}
|
||||||
|
|
||||||
const doctorPostgresTimeoutMs = 10_000;
|
const doctorPostgresTimeoutMs = 10_000;
|
||||||
@@ -194,6 +210,16 @@ async function readModuleConfigurationForRuntime(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Opens a provider client, probes, and always closes it — Doctor never leaves a connection behind. */
|
||||||
|
async function probeStorageForRuntime(config: StorageProviderConfig): Promise<void> {
|
||||||
|
const provider = createS3StorageProvider(config);
|
||||||
|
try {
|
||||||
|
await provider.probe();
|
||||||
|
} finally {
|
||||||
|
await provider.close().catch(() => undefined);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const productionDependencies: DoctorDependencies = Object.freeze({
|
const productionDependencies: DoctorDependencies = Object.freeze({
|
||||||
composePermissions: composePlatformPermissionRegistry,
|
composePermissions: composePlatformPermissionRegistry,
|
||||||
createMigrationDatabase: createPostgresMigrationDatabase,
|
createMigrationDatabase: createPostgresMigrationDatabase,
|
||||||
@@ -201,6 +227,7 @@ const productionDependencies: DoctorDependencies = Object.freeze({
|
|||||||
discoverModules,
|
discoverModules,
|
||||||
discoverSchemas: discoverModuleSchemas,
|
discoverSchemas: discoverModuleSchemas,
|
||||||
planMigrations: planModuleMigrations,
|
planMigrations: planModuleMigrations,
|
||||||
|
probeStorage: probeStorageForRuntime,
|
||||||
readModuleConfiguration: readModuleConfigurationForRuntime,
|
readModuleConfiguration: readModuleConfigurationForRuntime,
|
||||||
readModuleLifecycle: readModuleLifecycleForRuntime,
|
readModuleLifecycle: readModuleLifecycleForRuntime,
|
||||||
runtimeSatisfies: isVersionCompatible,
|
runtimeSatisfies: isVersionCompatible,
|
||||||
@@ -333,7 +360,17 @@ function safeLifecycleDetail(entry: ModuleEligibility): string {
|
|||||||
if (!entry.blocked) {
|
if (!entry.blocked) {
|
||||||
return `${identity} · enabled`;
|
return `${identity} · enabled`;
|
||||||
}
|
}
|
||||||
if (!("dependency" in entry.blocked)) {
|
if (isStorageBlockedDetail(entry.blocked)) {
|
||||||
|
// A storage block names the **logical** bucket the module declared — never an endpoint, a
|
||||||
|
// credential, or the physical bucket the host maps it to. Re-validated against the same grammar
|
||||||
|
// the manifest accepted it under, for the same defense-in-depth reason as every other
|
||||||
|
// interpolation here.
|
||||||
|
if (!logicalBucketIdSchema.safeParse(entry.blocked.storageBucketId).success) {
|
||||||
|
throw new TypeError("Unsafe module lifecycle diagnostic value.");
|
||||||
|
}
|
||||||
|
return `${identity} · blocked · ${entry.blocked.reason} · ${entry.blocked.storageBucketId}`;
|
||||||
|
}
|
||||||
|
if (isConfigurationBlockedDetail(entry.blocked)) {
|
||||||
// A configuration block names the offending **key** and never its value: a key is declared in a
|
// A configuration block names the offending **key** and never its value: a key is declared in a
|
||||||
// public manifest, while a value may be a credential or arbitrary operator-entered text. The key
|
// public manifest, while a value may be a credential or arbitrary operator-entered text. The key
|
||||||
// is re-validated against the same grammar the manifest accepted it under, for the same
|
// is re-validated against the same grammar the manifest accepted it under, for the same
|
||||||
@@ -407,7 +444,7 @@ function safeConfigurationDetail(entry: ModuleEligibility, declaredCount: number
|
|||||||
return `${identity} · no configuration declared`;
|
return `${identity} · no configuration declared`;
|
||||||
}
|
}
|
||||||
const declared = `${declaredCount} ${plural(declaredCount, "key")} declared`;
|
const declared = `${declaredCount} ${plural(declaredCount, "key")} declared`;
|
||||||
if (entry.blocked && !("dependency" in entry.blocked)) {
|
if (entry.blocked && isConfigurationBlockedDetail(entry.blocked)) {
|
||||||
if (!moduleConfigurationKeySchema.safeParse(entry.blocked.configurationKey).success) {
|
if (!moduleConfigurationKeySchema.safeParse(entry.blocked.configurationKey).success) {
|
||||||
throw new TypeError("Unsafe module configuration diagnostic value.");
|
throw new TypeError("Unsafe module configuration diagnostic value.");
|
||||||
}
|
}
|
||||||
@@ -458,6 +495,65 @@ function moduleConfigurationCheck(
|
|||||||
return freezeCheck("modules.configuration", "pass", "Module configuration valid", details);
|
return freezeCheck("modules.configuration", "pass", "Module configuration valid", details);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reports which installed modules require object storage, and where the host keeps each binding.
|
||||||
|
*
|
||||||
|
* Names the module, the **logical** bucket it declared, and the physical bucket the host maps it to
|
||||||
|
* — the last of which is safe and useful here precisely because Doctor is an operator tool: someone
|
||||||
|
* debugging a permissions problem needs to know which bucket to look at. It is still never reachable
|
||||||
|
* from module-facing code (ADR 0030). No endpoint, access key, or secret key appears in any form.
|
||||||
|
*/
|
||||||
|
function storageBindingsCheck(
|
||||||
|
modules: readonly DiscoveredModule[],
|
||||||
|
entries: readonly ModuleEligibility[],
|
||||||
|
config: StorageProviderConfig | undefined,
|
||||||
|
): DoctorCheck {
|
||||||
|
const requiring = modules.filter((module) => module.manifest.storage.buckets.length > 0);
|
||||||
|
if (requiring.length === 0) {
|
||||||
|
// An installation whose modules need no object storage is not unhealthy for lacking a provider.
|
||||||
|
return freezeCheck("object-storage.bindings", "pass", "No module requires object storage", [
|
||||||
|
"0 modules declare a logical storage bucket.",
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
const blocked = entries.filter(
|
||||||
|
(entry) => entry.blocked !== undefined && isStorageBlockedDetail(entry.blocked),
|
||||||
|
);
|
||||||
|
const details = requiring.flatMap((module) => {
|
||||||
|
const moduleId = module.manifest.id;
|
||||||
|
if (!moduleIdSchema.safeParse(moduleId).success) {
|
||||||
|
throw new TypeError("Unsafe module storage diagnostic value.");
|
||||||
|
}
|
||||||
|
return module.manifest.storage.buckets.map((bucket) => {
|
||||||
|
if (!logicalBucketIdSchema.safeParse(bucket.id).success) {
|
||||||
|
throw new TypeError("Unsafe module storage diagnostic value.");
|
||||||
|
}
|
||||||
|
return config === undefined
|
||||||
|
? `${moduleId} · ${bucket.id} · unbound (no storage provider configured)`
|
||||||
|
: `${moduleId} · ${bucket.id} → ${physicalBucketName(config.bucketPrefix, moduleId, bucket.id)}`;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
if (config === undefined) {
|
||||||
|
return freezeCheck(
|
||||||
|
"object-storage.bindings",
|
||||||
|
"failure",
|
||||||
|
blocked.length === 1
|
||||||
|
? "1 module requires object storage that is not configured"
|
||||||
|
: `${blocked.length} modules require object storage that is not configured`,
|
||||||
|
[
|
||||||
|
"Configure an S3-compatible Ekklesia storage provider (S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY).",
|
||||||
|
...details,
|
||||||
|
],
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return freezeCheck(
|
||||||
|
"object-storage.bindings",
|
||||||
|
"pass",
|
||||||
|
`${details.length} ${plural(details.length, "storage binding")} resolved`,
|
||||||
|
details,
|
||||||
|
);
|
||||||
|
}
|
||||||
function moduleDiscoveryFailure(error: unknown): DoctorCheck {
|
function moduleDiscoveryFailure(error: unknown): DoctorCheck {
|
||||||
const details =
|
const details =
|
||||||
error instanceof Error && error.name === "ModuleDiscoveryError"
|
error instanceof Error && error.name === "ModuleDiscoveryError"
|
||||||
@@ -788,6 +884,9 @@ export async function diagnoseEkklesia(
|
|||||||
// failed. It needs only a successful discovery and a configured database.
|
// failed. It needs only a successful discovery and a configured database.
|
||||||
let lifecycleCheck: DoctorCheck;
|
let lifecycleCheck: DoctorCheck;
|
||||||
let configurationCheck: DoctorCheck;
|
let configurationCheck: DoctorCheck;
|
||||||
|
// Hoisted so the storage-bindings check below reports the same eligibility this pass resolved,
|
||||||
|
// rather than resolving a second time and risking a different answer.
|
||||||
|
let eligibilityEntries: readonly ModuleEligibility[] = [];
|
||||||
const skippedConfiguration = (detail: string): DoctorCheck =>
|
const skippedConfiguration = (detail: string): DoctorCheck =>
|
||||||
freezeCheck("modules.configuration", "skipped", "Module configuration skipped", [detail]);
|
freezeCheck("modules.configuration", "skipped", "Module configuration skipped", [detail]);
|
||||||
if (!discoveredModules) {
|
if (!discoveredModules) {
|
||||||
@@ -819,10 +918,13 @@ export async function diagnoseEkklesia(
|
|||||||
poolOptions,
|
poolOptions,
|
||||||
);
|
);
|
||||||
const settings = await dependencies.readModuleConfiguration(options.databaseUrl, poolOptions);
|
const settings = await dependencies.readModuleConfiguration(options.databaseUrl, poolOptions);
|
||||||
const resolution = resolveModuleEligibility(discoveredModules, configuration, {
|
const resolution = resolveModuleEligibility(
|
||||||
secrets: options.moduleSecrets ?? EMPTY_MODULE_SECRET_SOURCE,
|
discoveredModules,
|
||||||
settings,
|
configuration,
|
||||||
});
|
{ secrets: options.moduleSecrets ?? EMPTY_MODULE_SECRET_SOURCE, settings },
|
||||||
|
{ providerConfigured: options.storage !== undefined },
|
||||||
|
);
|
||||||
|
eligibilityEntries = resolution.entries;
|
||||||
lifecycleCheck = moduleLifecycleCheck(resolution.entries);
|
lifecycleCheck = moduleLifecycleCheck(resolution.entries);
|
||||||
configurationCheck = moduleConfigurationCheck(
|
configurationCheck = moduleConfigurationCheck(
|
||||||
resolution.entries,
|
resolution.entries,
|
||||||
@@ -930,8 +1032,44 @@ export async function diagnoseEkklesia(
|
|||||||
options.databaseEnvironment,
|
options.databaseEnvironment,
|
||||||
);
|
);
|
||||||
|
|
||||||
const storageCheck = freezeCheck("object-storage.integration", "skipped", "Check unavailable", [
|
// Two separate questions, deliberately. "Is a provider configured and reachable?" is
|
||||||
"Core storage integration has not been implemented yet.",
|
// infrastructure diagnostics; "can every module's declared bucket be bound?" is what decides
|
||||||
|
// whether modules run. Conflating them would let a momentary network blip read as a module
|
||||||
|
// problem (ADR 0030).
|
||||||
|
let storageCheck: DoctorCheck;
|
||||||
|
if (!options.storage) {
|
||||||
|
storageCheck = freezeCheck(
|
||||||
|
"object-storage.integration",
|
||||||
|
"skipped",
|
||||||
|
"Object storage not configured",
|
||||||
|
["No S3_ENDPOINT is set. This is a complete deployment unless a module requires storage."],
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
try {
|
||||||
|
await dependencies.probeStorage(options.storage);
|
||||||
|
storageCheck = freezeCheck("object-storage.integration", "pass", "Object storage reachable", [
|
||||||
|
describeStorageProvider(options.storage),
|
||||||
|
]);
|
||||||
|
} catch {
|
||||||
|
// A warning, never a failure: the provider is configured, and a module that is already
|
||||||
|
// running keeps running through a transient outage. Reporting this as `not-ready` would make
|
||||||
|
// the platform look broken because a network hiccup happened while Doctor ran.
|
||||||
|
storageCheck = freezeCheck(
|
||||||
|
"object-storage.integration",
|
||||||
|
"warning",
|
||||||
|
"Object storage unreachable",
|
||||||
|
[
|
||||||
|
describeStorageProvider(options.storage),
|
||||||
|
"The configured storage service did not respond. Modules already running are unaffected.",
|
||||||
|
],
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const storageBindings = discoveredModules
|
||||||
|
? storageBindingsCheck(discoveredModules, eligibilityEntries, options.storage)
|
||||||
|
: freezeCheck("object-storage.bindings", "skipped", "Storage bindings skipped", [
|
||||||
|
"Complete module discovery is required.",
|
||||||
]);
|
]);
|
||||||
|
|
||||||
return publishReport([
|
return publishReport([
|
||||||
@@ -949,5 +1087,6 @@ export async function diagnoseEkklesia(
|
|||||||
identityCheck,
|
identityCheck,
|
||||||
clientAddressDoctorCheck,
|
clientAddressDoctorCheck,
|
||||||
storageCheck,
|
storageCheck,
|
||||||
|
storageBindings,
|
||||||
]);
|
]);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -62,7 +62,59 @@ export interface ModuleConfigurationBlockedDetail {
|
|||||||
readonly configurationKey: string;
|
readonly configurationKey: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type ModuleBlockedDetail = ModuleDependencyBlockedDetail | ModuleConfigurationBlockedDetail;
|
/** A block caused by unsatisfiable object-storage requirements (ADR 0030), naming the **logical**
|
||||||
|
* bucket the module declared — never an endpoint, a credential, or a physical bucket name. */
|
||||||
|
export interface ModuleStorageBlockedDetail {
|
||||||
|
readonly reason: "storage_unavailable";
|
||||||
|
readonly storageBucketId: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ModuleBlockedDetail =
|
||||||
|
| ModuleDependencyBlockedDetail
|
||||||
|
| ModuleConfigurationBlockedDetail
|
||||||
|
| ModuleStorageBlockedDetail;
|
||||||
|
|
||||||
|
/** Narrows a block to the dependency variant. Exported so every consumer discriminates the same
|
||||||
|
* way — a bare `"dependency" in detail` repeated at each call site is how one of them eventually
|
||||||
|
* gets it wrong as variants are added. */
|
||||||
|
export function isDependencyBlockedDetail(
|
||||||
|
detail: ModuleBlockedDetail,
|
||||||
|
): detail is ModuleDependencyBlockedDetail {
|
||||||
|
return (
|
||||||
|
detail.reason === "required_dependency_disabled" ||
|
||||||
|
detail.reason === "required_dependency_blocked"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isStorageBlockedDetail(
|
||||||
|
detail: ModuleBlockedDetail,
|
||||||
|
): detail is ModuleStorageBlockedDetail {
|
||||||
|
return detail.reason === "storage_unavailable";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isConfigurationBlockedDetail(
|
||||||
|
detail: ModuleBlockedDetail,
|
||||||
|
): detail is ModuleConfigurationBlockedDetail {
|
||||||
|
return (
|
||||||
|
detail.reason === "configuration_missing" ||
|
||||||
|
detail.reason === "configuration_invalid" ||
|
||||||
|
detail.reason === "secret_missing"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether this host can bind the logical object-storage buckets modules declare (ADR 0030).
|
||||||
|
*
|
||||||
|
* A boolean, deliberately, and resolved without touching the network: eligibility must stay a pure,
|
||||||
|
* synchronous function, and it must never depend on whether the storage service happened to answer a
|
||||||
|
* probe during boot. A configured provider that is temporarily unreachable is an operational
|
||||||
|
* condition a module retries through, not an administrative state change — which is the distinction
|
||||||
|
* the real Announcements product already relies on, since its slideshow survives its object store
|
||||||
|
* going away.
|
||||||
|
*/
|
||||||
|
export interface ModuleStorageAvailability {
|
||||||
|
readonly providerConfigured: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
export interface ModuleEligibilityResolution {
|
export interface ModuleEligibilityResolution {
|
||||||
/**
|
/**
|
||||||
@@ -108,6 +160,11 @@ function frozenEligibility(entry: MutableEligibility): ModuleEligibility {
|
|||||||
dependency: Object.freeze({ ...entry.blocked.dependency }),
|
dependency: Object.freeze({ ...entry.blocked.dependency }),
|
||||||
reason: entry.blocked.reason,
|
reason: entry.blocked.reason,
|
||||||
})
|
})
|
||||||
|
: "storageBucketId" in entry.blocked
|
||||||
|
? Object.freeze({
|
||||||
|
reason: entry.blocked.reason,
|
||||||
|
storageBucketId: entry.blocked.storageBucketId,
|
||||||
|
})
|
||||||
: Object.freeze({
|
: Object.freeze({
|
||||||
configurationKey: entry.blocked.configurationKey,
|
configurationKey: entry.blocked.configurationKey,
|
||||||
reason: entry.blocked.reason,
|
reason: entry.blocked.reason,
|
||||||
@@ -162,6 +219,7 @@ export function resolveModuleEligibility(
|
|||||||
discoveredModules: readonly DiscoveredModule[],
|
discoveredModules: readonly DiscoveredModule[],
|
||||||
configuration: ModuleLifecycleConfiguration,
|
configuration: ModuleLifecycleConfiguration,
|
||||||
moduleConfiguration: ModuleConfigurationInput = { settings: { records: [] } },
|
moduleConfiguration: ModuleConfigurationInput = { settings: { records: [] } },
|
||||||
|
storage: ModuleStorageAvailability = { providerConfigured: false },
|
||||||
): ModuleEligibilityResolution {
|
): ModuleEligibilityResolution {
|
||||||
const configuredByModuleId = new Map(
|
const configuredByModuleId = new Map(
|
||||||
configuration.modules.map((record) => [record.moduleId, record.enabled] as const),
|
configuration.modules.map((record) => [record.moduleId, record.enabled] as const),
|
||||||
@@ -180,6 +238,26 @@ export function resolveModuleEligibility(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A module that declares logical object-storage buckets needs a host provider to bind them to. The
|
||||||
|
// check is deliberately about *binding availability*, not provider health: it asks whether this
|
||||||
|
// deployment has storage configured at all, never whether the service answered just now
|
||||||
|
// (ADR 0030). Runs before the dependency pass so a storage-blocked module propagates to its
|
||||||
|
// required dependents through exactly the existing mechanism rather than a second one.
|
||||||
|
for (const discovered of discoveredModules) {
|
||||||
|
const entry = entries.get(discovered.manifest.id);
|
||||||
|
if (!entry?.eligible) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const declaredBucket = discovered.manifest.storage.buckets[0];
|
||||||
|
if (declaredBucket !== undefined && !storage.providerConfigured) {
|
||||||
|
entry.eligible = false;
|
||||||
|
entry.blocked = {
|
||||||
|
reason: "storage_unavailable",
|
||||||
|
storageBucketId: declaredBucket.id,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Validate each enabled module's own configuration before the dependency pass, so a module
|
// Validate each enabled module's own configuration before the dependency pass, so a module
|
||||||
// blocked by its own configuration propagates to its required dependents through exactly the
|
// blocked by its own configuration propagates to its required dependents through exactly the
|
||||||
// existing mechanism rather than a second one. Runs only for modules that are still eligible: a
|
// existing mechanism rather than a second one. Runs only for modules that are still eligible: a
|
||||||
@@ -306,6 +384,9 @@ export interface ModuleLifecycleView {
|
|||||||
readonly dependency?: ModuleBlockingDependency;
|
readonly dependency?: ModuleBlockingDependency;
|
||||||
/** Present for a configuration reason (ADR 0029) — the offending key, never its value. */
|
/** Present for a configuration reason (ADR 0029) — the offending key, never its value. */
|
||||||
readonly configurationKey?: string;
|
readonly configurationKey?: string;
|
||||||
|
/** Present for a storage reason (ADR 0030) — the logical bucket the module declared, never an
|
||||||
|
* endpoint, a credential, or a physical bucket name. */
|
||||||
|
readonly storageBucketId?: string;
|
||||||
};
|
};
|
||||||
/**
|
/**
|
||||||
* `true` when applying the *current* configuration would produce a different running set than
|
* `true` when applying the *current* configuration would produce a different running set than
|
||||||
@@ -347,6 +428,7 @@ export function resolveModuleLifecycleView(
|
|||||||
snapshot: ModuleLifecycleSnapshot,
|
snapshot: ModuleLifecycleSnapshot,
|
||||||
configuration: ModuleLifecycleConfiguration,
|
configuration: ModuleLifecycleConfiguration,
|
||||||
moduleConfiguration: ModuleConfigurationInput = { settings: { records: [] } },
|
moduleConfiguration: ModuleConfigurationInput = { settings: { records: [] } },
|
||||||
|
storage: ModuleStorageAvailability = { providerConfigured: false },
|
||||||
): readonly ModuleLifecycleView[] {
|
): readonly ModuleLifecycleView[] {
|
||||||
const started = new Set(snapshot.startedModuleIds);
|
const started = new Set(snapshot.startedModuleIds);
|
||||||
const bootEligibility = new Map(
|
const bootEligibility = new Map(
|
||||||
@@ -361,6 +443,7 @@ export function resolveModuleLifecycleView(
|
|||||||
snapshot.discoveredModules,
|
snapshot.discoveredModules,
|
||||||
configuration,
|
configuration,
|
||||||
moduleConfiguration,
|
moduleConfiguration,
|
||||||
|
storage,
|
||||||
);
|
);
|
||||||
const wouldRun = new Set(pending.eligibleModules.map((module) => module.manifest.id));
|
const wouldRun = new Set(pending.eligibleModules.map((module) => module.manifest.id));
|
||||||
|
|
||||||
@@ -379,6 +462,12 @@ export function resolveModuleLifecycleView(
|
|||||||
reason: boot.blocked.reason,
|
reason: boot.blocked.reason,
|
||||||
state: "blocked" as const,
|
state: "blocked" as const,
|
||||||
}
|
}
|
||||||
|
: "storageBucketId" in boot.blocked
|
||||||
|
? {
|
||||||
|
reason: boot.blocked.reason,
|
||||||
|
state: "blocked" as const,
|
||||||
|
storageBucketId: boot.blocked.storageBucketId,
|
||||||
|
}
|
||||||
: {
|
: {
|
||||||
configurationKey: boot.blocked.configurationKey,
|
configurationKey: boot.blocked.configurationKey,
|
||||||
reason: boot.blocked.reason,
|
reason: boot.blocked.reason,
|
||||||
|
|||||||
@@ -2,7 +2,9 @@ import { pathToFileURL } from "node:url";
|
|||||||
import {
|
import {
|
||||||
type CapabilityId,
|
type CapabilityId,
|
||||||
EMPTY_MODULE_CONFIGURATION,
|
EMPTY_MODULE_CONFIGURATION,
|
||||||
|
EMPTY_MODULE_STORAGE,
|
||||||
type ModuleConfiguration,
|
type ModuleConfiguration,
|
||||||
|
type ModuleStorage,
|
||||||
type CapabilityRegistrar,
|
type CapabilityRegistrar,
|
||||||
type CapabilityResolver,
|
type CapabilityResolver,
|
||||||
capabilityIdSchema,
|
capabilityIdSchema,
|
||||||
@@ -96,6 +98,20 @@ function configurationFor(
|
|||||||
return configurations.get(moduleId) ?? EMPTY_MODULE_CONFIGURATION;
|
return configurations.get(moduleId) ?? EMPTY_MODULE_CONFIGURATION;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-module scoped object storage, keyed by module id (ADR 0030).
|
||||||
|
*
|
||||||
|
* A lookup for the same reason configuration is one, and emphatically **not** a field on
|
||||||
|
* `ModuleRuntimeServices`: `services` is one shared object identity every module receives, while a
|
||||||
|
* storage facade is scoped to exactly the buckets one module declared. Sharing it would hand every
|
||||||
|
* module every other module's storage.
|
||||||
|
*/
|
||||||
|
export type ModuleStorageLookup = ReadonlyMap<string, ModuleStorage>;
|
||||||
|
|
||||||
|
function storageFor(storages: ModuleStorageLookup, moduleId: string): ModuleStorage {
|
||||||
|
return storages.get(moduleId) ?? EMPTY_MODULE_STORAGE;
|
||||||
|
}
|
||||||
|
|
||||||
export interface ModuleRuntimeTimeoutOptions {
|
export interface ModuleRuntimeTimeoutOptions {
|
||||||
/** Test-only narrowing of `MODULE_STOP_HOOK_TIMEOUT_MS`, applied to both normal stop and startup
|
/** Test-only narrowing of `MODULE_STOP_HOOK_TIMEOUT_MS`, applied to both normal stop and startup
|
||||||
* rollback, so a deadline can be proven in milliseconds instead of ten seconds. Production
|
* rollback, so a deadline can be proven in milliseconds instead of ten seconds. Production
|
||||||
@@ -115,6 +131,12 @@ export interface ModuleRuntimeCompositionOptions extends ModuleRuntimeTimeoutOpt
|
|||||||
* `EMPTY_MODULE_CONFIGURATION`.
|
* `EMPTY_MODULE_CONFIGURATION`.
|
||||||
*/
|
*/
|
||||||
readonly configurations?: ModuleConfigurationLookup;
|
readonly configurations?: ModuleConfigurationLookup;
|
||||||
|
/**
|
||||||
|
* The per-module scoped storage facades this runtime hands to each module's `start`, `stop`, and
|
||||||
|
* HTTP handler contexts (ADR 0030). Omitted by a caller exercising no storage; every module then
|
||||||
|
* receives `EMPTY_MODULE_STORAGE`, which refuses every bucket.
|
||||||
|
*/
|
||||||
|
readonly storages?: ModuleStorageLookup;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface StartModuleRuntimeOptions
|
export interface StartModuleRuntimeOptions
|
||||||
@@ -257,6 +279,7 @@ class StagedCapabilityRegistry {
|
|||||||
phase: "rollback" | "start" | "stop",
|
phase: "rollback" | "start" | "stop",
|
||||||
services: ModuleRuntimeServices,
|
services: ModuleRuntimeServices,
|
||||||
config: ModuleConfiguration,
|
config: ModuleConfiguration,
|
||||||
|
storage: ModuleStorage,
|
||||||
): ModuleRuntimeContext {
|
): ModuleRuntimeContext {
|
||||||
const capabilities: CapabilityResolver = Object.freeze({
|
const capabilities: CapabilityResolver = Object.freeze({
|
||||||
resolve: (capabilityId: CapabilityId): RegisteredCapability | undefined => {
|
resolve: (capabilityId: CapabilityId): RegisteredCapability | undefined => {
|
||||||
@@ -264,7 +287,13 @@ class StagedCapabilityRegistry {
|
|||||||
return this.#registered.get(capabilityId);
|
return this.#registered.get(capabilityId);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
return Object.freeze({ capabilities, config, moduleId: module.manifest.id, services });
|
return Object.freeze({
|
||||||
|
capabilities,
|
||||||
|
config,
|
||||||
|
moduleId: module.manifest.id,
|
||||||
|
services,
|
||||||
|
storage,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
publish(): CapabilityRegistry {
|
publish(): CapabilityRegistry {
|
||||||
@@ -599,6 +628,7 @@ async function stopModules(
|
|||||||
phase: "rollback" | "stop",
|
phase: "rollback" | "stop",
|
||||||
services: ModuleRuntimeServices,
|
services: ModuleRuntimeServices,
|
||||||
configurations: ModuleConfigurationLookup,
|
configurations: ModuleConfigurationLookup,
|
||||||
|
storages: ModuleStorageLookup,
|
||||||
options: StopModulesOptions = {},
|
options: StopModulesOptions = {},
|
||||||
): Promise<readonly ModuleRuntimeIssue[]> {
|
): Promise<readonly ModuleRuntimeIssue[]> {
|
||||||
const timeoutMs = options.hookTimeoutMs ?? MODULE_STOP_HOOK_TIMEOUT_MS;
|
const timeoutMs = options.hookTimeoutMs ?? MODULE_STOP_HOOK_TIMEOUT_MS;
|
||||||
@@ -619,6 +649,7 @@ async function stopModules(
|
|||||||
phase,
|
phase,
|
||||||
services,
|
services,
|
||||||
configurationFor(configurations, moduleId),
|
configurationFor(configurations, moduleId),
|
||||||
|
storageFor(storages, moduleId),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
@@ -657,6 +688,7 @@ async function startModules(
|
|||||||
registry: StagedCapabilityRegistry,
|
registry: StagedCapabilityRegistry,
|
||||||
services: ModuleRuntimeServices,
|
services: ModuleRuntimeServices,
|
||||||
configurations: ModuleConfigurationLookup,
|
configurations: ModuleConfigurationLookup,
|
||||||
|
storages: ModuleStorageLookup,
|
||||||
options: StopModulesOptions = {},
|
options: StopModulesOptions = {},
|
||||||
): Promise<void> {
|
): Promise<void> {
|
||||||
// A module joins the rollback set the moment its `start` hook is *invoked*, not when it
|
// A module joins the rollback set the moment its `start` hook is *invoked*, not when it
|
||||||
@@ -675,6 +707,7 @@ async function startModules(
|
|||||||
"start",
|
"start",
|
||||||
services,
|
services,
|
||||||
configurationFor(configurations, module.discovered.manifest.id),
|
configurationFor(configurations, module.discovered.manifest.id),
|
||||||
|
storageFor(storages, module.discovered.manifest.id),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
@@ -693,6 +726,7 @@ async function startModules(
|
|||||||
"rollback",
|
"rollback",
|
||||||
services,
|
services,
|
||||||
configurations,
|
configurations,
|
||||||
|
storages,
|
||||||
options,
|
options,
|
||||||
);
|
);
|
||||||
throw new ModuleRuntimeError([startIssue, ...rollbackIssues]);
|
throw new ModuleRuntimeError([startIssue, ...rollbackIssues]);
|
||||||
@@ -717,8 +751,9 @@ export async function startDiscoveredModuleRuntime(
|
|||||||
const capabilityRegistry = new StagedCapabilityRegistry();
|
const capabilityRegistry = new StagedCapabilityRegistry();
|
||||||
const httpRoutes = new StagedModuleHttpRoutes();
|
const httpRoutes = new StagedModuleHttpRoutes();
|
||||||
const configurations = timeouts.configurations ?? new Map<string, ModuleConfiguration>();
|
const configurations = timeouts.configurations ?? new Map<string, ModuleConfiguration>();
|
||||||
|
const storages = timeouts.storages ?? new Map<string, ModuleStorage>();
|
||||||
await registerModules(modules, capabilityRegistry, httpRoutes, configurations);
|
await registerModules(modules, capabilityRegistry, httpRoutes, configurations);
|
||||||
await startModules(modules, capabilityRegistry, services, configurations, timeouts);
|
await startModules(modules, capabilityRegistry, services, configurations, storages, timeouts);
|
||||||
|
|
||||||
const capabilities = capabilityRegistry.publish();
|
const capabilities = capabilityRegistry.publish();
|
||||||
const routes = httpRoutes.publish();
|
const routes = httpRoutes.publish();
|
||||||
@@ -744,6 +779,7 @@ export async function startDiscoveredModuleRuntime(
|
|||||||
"stop",
|
"stop",
|
||||||
services,
|
services,
|
||||||
configurations,
|
configurations,
|
||||||
|
storages,
|
||||||
timeouts,
|
timeouts,
|
||||||
);
|
);
|
||||||
if (issues.length > 0) {
|
if (issues.length > 0) {
|
||||||
@@ -761,6 +797,7 @@ export async function startModuleRuntime(
|
|||||||
const discovered = await discoverModules(options);
|
const discovered = await discoverModules(options);
|
||||||
return startDiscoveredModuleRuntime(discovered, options.services, {
|
return startDiscoveredModuleRuntime(discovered, options.services, {
|
||||||
...(options.configurations === undefined ? {} : { configurations: options.configurations }),
|
...(options.configurations === undefined ? {} : { configurations: options.configurations }),
|
||||||
|
...(options.storages === undefined ? {} : { storages: options.storages }),
|
||||||
...(options.hookTimeoutMs === undefined ? {} : { hookTimeoutMs: options.hookTimeoutMs }),
|
...(options.hookTimeoutMs === undefined ? {} : { hookTimeoutMs: options.hookTimeoutMs }),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,116 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* How a module's logical bucket becomes a physical bucket on the host's storage service.
|
||||||
|
*
|
||||||
|
* This is **host implementation detail**. Nothing module-facing exposes the result: a module names
|
||||||
|
* its logical bucket and the binding is resolved for it, so a module can neither observe nor choose
|
||||||
|
* a physical bucket (ADR 0030).
|
||||||
|
*/
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The prefix every physical bucket this installation owns begins with.
|
||||||
|
*
|
||||||
|
* It exists so several Ekklesia-compatible deployments — or an Ekklesia installation and an
|
||||||
|
* unrelated application — can share one S3-compatible service without colliding, which is exactly
|
||||||
|
* the deployment shape the standalone story needs. Bounded and grammar-checked because it becomes
|
||||||
|
* the leading characters of an S3 bucket name.
|
||||||
|
*/
|
||||||
|
export const DEFAULT_STORAGE_BUCKET_PREFIX = "ekklesia";
|
||||||
|
export const STORAGE_BUCKET_PREFIX_MAX_LENGTH = 16;
|
||||||
|
const BUCKET_PREFIX_PATTERN = /^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$/u;
|
||||||
|
|
||||||
|
export function isValidStorageBucketPrefix(value: unknown): value is string {
|
||||||
|
return (
|
||||||
|
typeof value === "string" &&
|
||||||
|
value.length > 0 &&
|
||||||
|
value.length <= STORAGE_BUCKET_PREFIX_MAX_LENGTH &&
|
||||||
|
BUCKET_PREFIX_PATTERN.test(value)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* S3's own bucket-name ceiling, which every S3-compatible provider shares. Verified against the
|
||||||
|
* pinned SeaweedFS 4.28 development image: a 63-character name is accepted.
|
||||||
|
*/
|
||||||
|
const PHYSICAL_BUCKET_MAX_LENGTH = 63;
|
||||||
|
/** Long enough that a collision is not a practical concern, short enough to leave the readable part
|
||||||
|
* of the name legible. */
|
||||||
|
const DIGEST_LENGTH = 12;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The unambiguous logical identity of one binding: a module and one of its declared buckets.
|
||||||
|
*
|
||||||
|
* `--` is the separator because it **cannot occur inside either identifier**. Both `moduleIdSchema`
|
||||||
|
* and `logicalBucketIdSchema` require at least one `[a-z0-9]` after every hyphen, so a doubled
|
||||||
|
* hyphen is unrepresentable in each half. A single hyphen would have been ambiguous in exactly the
|
||||||
|
* way that matters: module `a-b` + bucket `c` and module `a` + bucket `b-c` would both render as
|
||||||
|
* `a-b-c`, silently handing two unrelated modules the same physical namespace.
|
||||||
|
*/
|
||||||
|
function bindingIdentity(moduleId: string, logicalBucketId: string): string {
|
||||||
|
return `${moduleId}--${logicalBucketId}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Maps a module's logical bucket to the physical bucket name the host uses for it.
|
||||||
|
*
|
||||||
|
* ```text
|
||||||
|
* announcements + announcement-assets
|
||||||
|
* → ekklesia-announcements--announcement-assets-3f0c1d9a6b21
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* Deterministic, so the same declaration always resolves to the same bucket across restarts,
|
||||||
|
* deployments, and hosts — which is what makes the binding stable enough to hold data.
|
||||||
|
*
|
||||||
|
* Collision-safe on two independent grounds. The `--` separator makes the logical identity
|
||||||
|
* unambiguous, and the appended digest is taken over that **full untruncated identity**, so even two
|
||||||
|
* modules whose names truncate to the same readable prefix still receive different buckets. Two
|
||||||
|
* different modules both declaring `assets` therefore always land in different physical namespaces —
|
||||||
|
* which is the isolation property this whole scheme exists to provide.
|
||||||
|
*
|
||||||
|
* The readable middle is truncated only as far as S3's 63-character ceiling requires, and never at a
|
||||||
|
* trailing hyphen, so the result always satisfies S3's naming rules: it begins with a letter, ends
|
||||||
|
* with an alphanumeric, contains only lowercase letters, digits, and single hyphens, and can never
|
||||||
|
* resemble an IP address.
|
||||||
|
*/
|
||||||
|
export function physicalBucketName(
|
||||||
|
prefix: string,
|
||||||
|
moduleId: string,
|
||||||
|
logicalBucketId: string,
|
||||||
|
): string {
|
||||||
|
const identity = bindingIdentity(moduleId, logicalBucketId);
|
||||||
|
const digest = createHash("sha256")
|
||||||
|
.update(identity, "utf8")
|
||||||
|
.digest("hex")
|
||||||
|
.slice(0, DIGEST_LENGTH);
|
||||||
|
// prefix + "-" + readable + "-" + digest
|
||||||
|
const budget = PHYSICAL_BUCKET_MAX_LENGTH - prefix.length - 1 - 1 - DIGEST_LENGTH;
|
||||||
|
const readable = identity.slice(0, Math.max(1, budget)).replace(/-+$/u, "");
|
||||||
|
return `${prefix}-${readable}-${digest}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One resolved binding: what the module calls it, and where the host actually keeps it. */
|
||||||
|
export interface ModuleStorageBinding {
|
||||||
|
readonly moduleId: string;
|
||||||
|
readonly logicalBucketId: string;
|
||||||
|
/** Host detail. Never reachable from `ModuleStorage`; present here because Core's provider needs
|
||||||
|
* it, and because diagnostics may name it for an operator. */
|
||||||
|
readonly physicalBucket: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolves every binding one module's manifest declares, in declaration order. */
|
||||||
|
export function resolveModuleStorageBindings(
|
||||||
|
prefix: string,
|
||||||
|
moduleId: string,
|
||||||
|
logicalBucketIds: readonly string[],
|
||||||
|
): readonly ModuleStorageBinding[] {
|
||||||
|
return Object.freeze(
|
||||||
|
logicalBucketIds.map((logicalBucketId) =>
|
||||||
|
Object.freeze({
|
||||||
|
logicalBucketId,
|
||||||
|
moduleId,
|
||||||
|
physicalBucket: physicalBucketName(prefix, moduleId, logicalBucketId),
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,185 @@
|
|||||||
|
import {
|
||||||
|
DEFAULT_STORAGE_BUCKET_PREFIX,
|
||||||
|
isValidStorageBucketPrefix,
|
||||||
|
STORAGE_BUCKET_PREFIX_MAX_LENGTH,
|
||||||
|
} from "./binding.js";
|
||||||
|
|
||||||
|
export type StorageConfigErrorCode =
|
||||||
|
| "STORAGE_ACCESS_KEY_MISSING"
|
||||||
|
| "STORAGE_BUCKET_PREFIX_INVALID"
|
||||||
|
| "STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION"
|
||||||
|
| "STORAGE_ENDPOINT_INVALID"
|
||||||
|
| "STORAGE_ENDPOINT_WITH_CREDENTIALS"
|
||||||
|
| "STORAGE_REGION_INVALID"
|
||||||
|
| "STORAGE_SECRET_KEY_MISSING";
|
||||||
|
|
||||||
|
const storageConfigMessages: Readonly<Record<StorageConfigErrorCode, string>> = Object.freeze({
|
||||||
|
STORAGE_ACCESS_KEY_MISSING: "A storage access key is required when a storage endpoint is set.",
|
||||||
|
STORAGE_BUCKET_PREFIX_INVALID: "The storage bucket prefix must be lowercase kebab-case.",
|
||||||
|
STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION:
|
||||||
|
"A production storage endpoint must use https, exactly as the database transport must use TLS.",
|
||||||
|
STORAGE_ENDPOINT_INVALID: "The storage endpoint must be an absolute http or https URL.",
|
||||||
|
STORAGE_ENDPOINT_WITH_CREDENTIALS:
|
||||||
|
"The storage endpoint must not embed credentials; supply them separately.",
|
||||||
|
STORAGE_REGION_INVALID: "The storage region must be a short alphanumeric identifier.",
|
||||||
|
STORAGE_SECRET_KEY_MISSING: "A storage secret key is required when a storage endpoint is set.",
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Never carries the endpoint, region, access key, secret key, or any other configured value — only a
|
||||||
|
* fixed code and message. Matching `IdentityConfigError` and `DatabaseTransportError` exactly, and
|
||||||
|
* for a sharper reason here: two of the values this validates are credentials.
|
||||||
|
*/
|
||||||
|
export class StorageConfigError extends Error {
|
||||||
|
readonly code: StorageConfigErrorCode;
|
||||||
|
|
||||||
|
constructor(code: StorageConfigErrorCode) {
|
||||||
|
super(storageConfigMessages[code]);
|
||||||
|
this.name = "StorageConfigError";
|
||||||
|
this.code = code;
|
||||||
|
this.stack = `${this.name}: ${this.message}`;
|
||||||
|
Object.freeze(this);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export type StorageEnvironment = "development" | "production";
|
||||||
|
|
||||||
|
export interface StorageProviderConfigInput {
|
||||||
|
readonly endpoint?: string | undefined;
|
||||||
|
readonly region?: string | undefined;
|
||||||
|
readonly accessKey?: string | undefined;
|
||||||
|
readonly secretKey?: string | undefined;
|
||||||
|
readonly forcePathStyle?: string | boolean | undefined;
|
||||||
|
readonly bucketPrefix?: string | undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The validated host storage configuration.
|
||||||
|
*
|
||||||
|
* **Modules never see this.** It reaches Core's provider and nothing else; the module-facing
|
||||||
|
* `ModuleStorage` contract has no accessor that could return an endpoint, a region, or a credential
|
||||||
|
* (ADR 0030).
|
||||||
|
*/
|
||||||
|
export interface StorageProviderConfig {
|
||||||
|
readonly endpoint: string;
|
||||||
|
readonly region: string;
|
||||||
|
readonly accessKey: string;
|
||||||
|
readonly secretKey: string;
|
||||||
|
/** True for every S3-compatible service that is not AWS itself — SeaweedFS and MinIO both require
|
||||||
|
* it — so it defaults on. Virtual-host addressing needs wildcard DNS the typical self-hosted
|
||||||
|
* deployment does not have. */
|
||||||
|
readonly forcePathStyle: boolean;
|
||||||
|
readonly bucketPrefix: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DEFAULT_REGION = "us-east-1";
|
||||||
|
const REGION_PATTERN = /^[a-z0-9][a-z0-9-]{0,31}$/u;
|
||||||
|
|
||||||
|
function parseBoolean(value: string | boolean | undefined, fallback: boolean): boolean {
|
||||||
|
if (value === undefined) {
|
||||||
|
return fallback;
|
||||||
|
}
|
||||||
|
if (typeof value === "boolean") {
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
return value !== "false";
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Validates the host's storage configuration, or reports that storage is not configured at all.
|
||||||
|
*
|
||||||
|
* **Storage is optional.** An installation with no `endpoint` returns `undefined`, and that is a
|
||||||
|
* complete, valid deployment: modules that declare no buckets run exactly as before. Only a module
|
||||||
|
* that actually requires a bucket is affected, and it is reported blocked rather than crashing
|
||||||
|
* anything (ADR 0030).
|
||||||
|
*
|
||||||
|
* Once an endpoint *is* set, credentials become required — a half-configured provider that silently
|
||||||
|
* attempted anonymous access would be a worse failure than a loud one.
|
||||||
|
*
|
||||||
|
* Production requires `https`, on the same reasoning the database transport requires verified TLS:
|
||||||
|
* object bytes for a church may include member photographs and pastoral material, and shipping them
|
||||||
|
* over plaintext because it was convenient is not a tradeoff this platform makes. Development may
|
||||||
|
* use `http`, which is what the repository's own SeaweedFS Compose service speaks.
|
||||||
|
*
|
||||||
|
* An endpoint carrying userinfo is rejected outright rather than stripped: credentials belong in
|
||||||
|
* their own fields, where they are never logged, and a URL that embeds them tends to end up in
|
||||||
|
* places a URL is printed.
|
||||||
|
*/
|
||||||
|
export function createStorageProviderConfig(
|
||||||
|
input: StorageProviderConfigInput | undefined,
|
||||||
|
options: { readonly environment?: StorageEnvironment } = {},
|
||||||
|
): StorageProviderConfig | undefined {
|
||||||
|
const endpoint = input?.endpoint?.trim();
|
||||||
|
if (endpoint === undefined || endpoint === "") {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
let parsed: URL;
|
||||||
|
try {
|
||||||
|
parsed = new URL(endpoint);
|
||||||
|
} catch {
|
||||||
|
throw new StorageConfigError("STORAGE_ENDPOINT_INVALID");
|
||||||
|
}
|
||||||
|
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
|
||||||
|
throw new StorageConfigError("STORAGE_ENDPOINT_INVALID");
|
||||||
|
}
|
||||||
|
if (parsed.username !== "" || parsed.password !== "") {
|
||||||
|
throw new StorageConfigError("STORAGE_ENDPOINT_WITH_CREDENTIALS");
|
||||||
|
}
|
||||||
|
const environment = options.environment ?? "production";
|
||||||
|
if (environment === "production" && parsed.protocol !== "https:") {
|
||||||
|
throw new StorageConfigError("STORAGE_ENDPOINT_INSECURE_IN_PRODUCTION");
|
||||||
|
}
|
||||||
|
|
||||||
|
const region = input?.region?.trim() === "" ? undefined : input?.region?.trim();
|
||||||
|
if (region !== undefined && !REGION_PATTERN.test(region)) {
|
||||||
|
throw new StorageConfigError("STORAGE_REGION_INVALID");
|
||||||
|
}
|
||||||
|
|
||||||
|
const accessKey = input?.accessKey;
|
||||||
|
if (typeof accessKey !== "string" || accessKey.trim() === "") {
|
||||||
|
throw new StorageConfigError("STORAGE_ACCESS_KEY_MISSING");
|
||||||
|
}
|
||||||
|
const secretKey = input?.secretKey;
|
||||||
|
if (typeof secretKey !== "string" || secretKey.trim() === "") {
|
||||||
|
throw new StorageConfigError("STORAGE_SECRET_KEY_MISSING");
|
||||||
|
}
|
||||||
|
|
||||||
|
const bucketPrefix = input?.bucketPrefix?.trim();
|
||||||
|
if (
|
||||||
|
bucketPrefix !== undefined &&
|
||||||
|
bucketPrefix !== "" &&
|
||||||
|
!isValidStorageBucketPrefix(bucketPrefix)
|
||||||
|
) {
|
||||||
|
throw new StorageConfigError("STORAGE_BUCKET_PREFIX_INVALID");
|
||||||
|
}
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
accessKey,
|
||||||
|
bucketPrefix:
|
||||||
|
bucketPrefix === undefined || bucketPrefix === ""
|
||||||
|
? DEFAULT_STORAGE_BUCKET_PREFIX
|
||||||
|
: bucketPrefix,
|
||||||
|
endpoint: parsed.origin + (parsed.pathname === "/" ? "" : parsed.pathname.replace(/\/+$/u, "")),
|
||||||
|
forcePathStyle: parseBoolean(input?.forcePathStyle, true),
|
||||||
|
region: region ?? DEFAULT_REGION,
|
||||||
|
secretKey,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A one-line operator-facing description of the configured provider that is safe to print anywhere —
|
||||||
|
* Doctor output, a log line, a support transcript.
|
||||||
|
*
|
||||||
|
* It names the endpoint's **origin only** (never a path, query, or userinfo), the region, the
|
||||||
|
* addressing style, and the bucket prefix. It never includes the access key, the secret key, or any
|
||||||
|
* value derived from them. `describeDatabaseTransport` sets the same precedent.
|
||||||
|
*/
|
||||||
|
export function describeStorageProvider(config: StorageProviderConfig): string {
|
||||||
|
const origin = new URL(config.endpoint).origin;
|
||||||
|
return `S3-compatible endpoint ${origin} · region ${config.region} · ${
|
||||||
|
config.forcePathStyle ? "path-style" : "virtual-host-style"
|
||||||
|
} · bucket prefix "${config.bucketPrefix}"`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export { STORAGE_BUCKET_PREFIX_MAX_LENGTH };
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
export * from "./binding.js";
|
||||||
|
export * from "./config.js";
|
||||||
|
export * from "./module-storage.js";
|
||||||
|
export * from "./provider.js";
|
||||||
|
export * from "./s3-provider.js";
|
||||||
@@ -0,0 +1,185 @@
|
|||||||
|
import {
|
||||||
|
EMPTY_MODULE_STORAGE,
|
||||||
|
isValidModuleStorageKey,
|
||||||
|
type ModuleStorage,
|
||||||
|
type ModuleStorageBody,
|
||||||
|
type ModuleStorageBucket,
|
||||||
|
ModuleStorageError,
|
||||||
|
type ModuleStorageListOptions,
|
||||||
|
type ModuleStorageListing,
|
||||||
|
type ModuleStorageObject,
|
||||||
|
type ModuleStorageObjectMetadata,
|
||||||
|
type ModuleStoragePutOptions,
|
||||||
|
} from "@ekklesia/contracts";
|
||||||
|
import { type ModuleStorageBinding, resolveModuleStorageBindings } from "./binding.js";
|
||||||
|
import type { StorageProvider } from "./provider.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds the scoped storage facade one module receives.
|
||||||
|
*
|
||||||
|
* **This is where isolation is enforced, and it is enforced by construction.** The facade closes over
|
||||||
|
* exactly the bindings this module's own manifest declared; there is no lookup table spanning
|
||||||
|
* modules, no name that resolves outside those bindings, and no accessor taking a physical bucket.
|
||||||
|
* Module A cannot reach module B's storage by knowing its bucket name, its module id, or its
|
||||||
|
* physical bucket name, because none of those is an input to anything here (ADR 0030).
|
||||||
|
*
|
||||||
|
* Two modules that both declare `assets` receive two facades over two different physical buckets —
|
||||||
|
* see `physicalBucketName` for why that separation is collision-safe rather than merely likely.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** Ensures a binding's physical bucket exists, at most once per process per binding.
|
||||||
|
*
|
||||||
|
* Lazy rather than eager at startup, deliberately: a storage outage must never stop the platform
|
||||||
|
* from booting. The promise is memoized so concurrent first uses share one round trip, and a
|
||||||
|
* *failed* attempt is discarded so a transient outage does not poison the binding for the life of
|
||||||
|
* the process — the next call retries, which is exactly the resilience a module like a slideshow
|
||||||
|
* depends on (ADR 0030). */
|
||||||
|
function createBucketEnsurer(
|
||||||
|
provider: StorageProvider,
|
||||||
|
physicalBucket: string,
|
||||||
|
): () => Promise<void> {
|
||||||
|
let pending: Promise<void> | undefined;
|
||||||
|
return () => {
|
||||||
|
pending ??= provider.ensureBucket(physicalBucket).catch((error: unknown) => {
|
||||||
|
pending = undefined;
|
||||||
|
throw error;
|
||||||
|
});
|
||||||
|
return pending;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Echoes the offending key only when it is a short string — a module's own input is safe to hand
|
||||||
|
* back, but a non-string or an enormous value is not something to put into an error message. */
|
||||||
|
function assertValidKey(key: unknown): asserts key is string {
|
||||||
|
if (isValidModuleStorageKey(key)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const echo = typeof key === "string" && key.length <= 256 ? key : undefined;
|
||||||
|
throw new ModuleStorageError("STORAGE_KEY_INVALID", echo);
|
||||||
|
}
|
||||||
|
|
||||||
|
function createBucketHandle(
|
||||||
|
provider: StorageProvider,
|
||||||
|
binding: ModuleStorageBinding,
|
||||||
|
): ModuleStorageBucket {
|
||||||
|
const ensure = createBucketEnsurer(provider, binding.physicalBucket);
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
// The **logical** id, never the physical bucket. A module that logs this leaks nothing about
|
||||||
|
// where the host keeps its data.
|
||||||
|
id: binding.logicalBucketId,
|
||||||
|
|
||||||
|
async delete(key: string): Promise<void> {
|
||||||
|
assertValidKey(key);
|
||||||
|
await ensure();
|
||||||
|
await provider.delete(binding.physicalBucket, key);
|
||||||
|
},
|
||||||
|
|
||||||
|
async get(key: string): Promise<ModuleStorageObject> {
|
||||||
|
assertValidKey(key);
|
||||||
|
await ensure();
|
||||||
|
return provider.get(binding.physicalBucket, key);
|
||||||
|
},
|
||||||
|
|
||||||
|
async head(key: string): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
assertValidKey(key);
|
||||||
|
await ensure();
|
||||||
|
return provider.head(binding.physicalBucket, key);
|
||||||
|
},
|
||||||
|
|
||||||
|
async list(options: ModuleStorageListOptions = {}): Promise<ModuleStorageListing> {
|
||||||
|
await ensure();
|
||||||
|
return provider.list(binding.physicalBucket, options);
|
||||||
|
},
|
||||||
|
|
||||||
|
async put(
|
||||||
|
key: string,
|
||||||
|
body: ModuleStorageBody,
|
||||||
|
options: ModuleStoragePutOptions = {},
|
||||||
|
): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
assertValidKey(key);
|
||||||
|
if (body instanceof ReadableStream && options.contentLength === undefined) {
|
||||||
|
throw new ModuleStorageError("STORAGE_CONTENT_LENGTH_REQUIRED", key);
|
||||||
|
}
|
||||||
|
await ensure();
|
||||||
|
return provider.put(binding.physicalBucket, key, body, options);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface CreateModuleStorageOptions {
|
||||||
|
readonly provider: StorageProvider;
|
||||||
|
readonly bucketPrefix: string;
|
||||||
|
readonly moduleId: string;
|
||||||
|
/** The logical bucket ids from this module's own manifest, in declaration order. */
|
||||||
|
readonly logicalBucketIds: readonly string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The scoped `ModuleStorage` for one module.
|
||||||
|
*
|
||||||
|
* A module declaring no buckets gets `EMPTY_MODULE_STORAGE` — present, frozen, and refusing every
|
||||||
|
* bucket — so `context.storage` is never `undefined` and module code never branches on whether the
|
||||||
|
* host supplied storage.
|
||||||
|
*/
|
||||||
|
export function createModuleStorage(options: CreateModuleStorageOptions): ModuleStorage {
|
||||||
|
if (options.logicalBucketIds.length === 0) {
|
||||||
|
return EMPTY_MODULE_STORAGE;
|
||||||
|
}
|
||||||
|
const bindings = resolveModuleStorageBindings(
|
||||||
|
options.bucketPrefix,
|
||||||
|
options.moduleId,
|
||||||
|
options.logicalBucketIds,
|
||||||
|
);
|
||||||
|
// A `Map` rather than a plain object: every lookup key here comes from module code, and indexing a
|
||||||
|
// plain object with such a key is the shape of a prototype-pollution bug. With a `Map` there is no
|
||||||
|
// prototype to reach, so `constructor` or `__proto__` are ordinary absent keys.
|
||||||
|
const handles = new Map<string, ModuleStorageBucket>(
|
||||||
|
bindings.map((binding) => [
|
||||||
|
binding.logicalBucketId,
|
||||||
|
createBucketHandle(options.provider, binding),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
const declared = Object.freeze([...handles.keys()].sort());
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
bucket: (logicalBucketId: string): ModuleStorageBucket => {
|
||||||
|
const handle = typeof logicalBucketId === "string" ? handles.get(logicalBucketId) : undefined;
|
||||||
|
if (!handle) {
|
||||||
|
// Including another module's bucket id, which is simply not a key in this map.
|
||||||
|
throw new ModuleStorageError("STORAGE_BUCKET_UNDECLARED", logicalBucketId);
|
||||||
|
}
|
||||||
|
return handle;
|
||||||
|
},
|
||||||
|
buckets: (): readonly string[] => declared,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Every module's storage, resolved once at startup.
|
||||||
|
*
|
||||||
|
* Returns an empty map when the host has no storage provider. That is not an error: an installation
|
||||||
|
* with no object storage is a complete, valid deployment, and only modules that *declared* a bucket
|
||||||
|
* are affected — those are reported blocked by the lifecycle resolver before any of this is reached
|
||||||
|
* (ADR 0030).
|
||||||
|
*/
|
||||||
|
export function createModuleStorageLookup(
|
||||||
|
provider: StorageProvider | undefined,
|
||||||
|
bucketPrefix: string,
|
||||||
|
modules: readonly { readonly moduleId: string; readonly logicalBucketIds: readonly string[] }[],
|
||||||
|
): ReadonlyMap<string, ModuleStorage> {
|
||||||
|
if (!provider) {
|
||||||
|
return new Map();
|
||||||
|
}
|
||||||
|
return new Map(
|
||||||
|
modules.map((module) => [
|
||||||
|
module.moduleId,
|
||||||
|
createModuleStorage({
|
||||||
|
bucketPrefix,
|
||||||
|
logicalBucketIds: module.logicalBucketIds,
|
||||||
|
moduleId: module.moduleId,
|
||||||
|
provider,
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
import type {
|
||||||
|
ModuleStorageBody,
|
||||||
|
ModuleStorageListOptions,
|
||||||
|
ModuleStorageListing,
|
||||||
|
ModuleStorageObject,
|
||||||
|
ModuleStorageObjectMetadata,
|
||||||
|
ModuleStoragePutOptions,
|
||||||
|
} from "@ekklesia/contracts";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The seam between Core's module-facing storage facade and whatever actually stores bytes.
|
||||||
|
*
|
||||||
|
* This is the **portability boundary made executable**. Everything above it — the scoped facade,
|
||||||
|
* binding resolution, lifecycle eligibility, diagnostics — is provider-agnostic and would be
|
||||||
|
* identical in a standalone host. Everything below it is one implementation
|
||||||
|
* (`createS3StorageProvider`) that could be replaced without any module changing a line.
|
||||||
|
*
|
||||||
|
* It deliberately speaks in **physical buckets**: resolving a module's logical bucket to a physical
|
||||||
|
* one has already happened by the time anything reaches here (`./binding.js`), so a provider never
|
||||||
|
* has to know what a module is. That split is what keeps scoping enforcement in exactly one place
|
||||||
|
* rather than duplicated into every provider.
|
||||||
|
*
|
||||||
|
* Also deliberately absent: any operation that *destroys* a bucket. Core never deletes a bucket, so
|
||||||
|
* there is no method to call by mistake, and disabling or removing a module cannot take data with it
|
||||||
|
* (ADR 0030).
|
||||||
|
*/
|
||||||
|
export interface StorageProvider {
|
||||||
|
/**
|
||||||
|
* Makes sure a physical bucket exists, idempotently.
|
||||||
|
*
|
||||||
|
* Called lazily on first use of a binding rather than at startup, so a storage outage can never
|
||||||
|
* prevent the platform from booting.
|
||||||
|
*/
|
||||||
|
ensureBucket(physicalBucket: string): Promise<void>;
|
||||||
|
list(physicalBucket: string, options: ModuleStorageListOptions): Promise<ModuleStorageListing>;
|
||||||
|
head(physicalBucket: string, key: string): Promise<ModuleStorageObjectMetadata>;
|
||||||
|
get(physicalBucket: string, key: string): Promise<ModuleStorageObject>;
|
||||||
|
put(
|
||||||
|
physicalBucket: string,
|
||||||
|
key: string,
|
||||||
|
body: ModuleStorageBody,
|
||||||
|
options: ModuleStoragePutOptions,
|
||||||
|
): Promise<ModuleStorageObjectMetadata>;
|
||||||
|
delete(physicalBucket: string, key: string): Promise<void>;
|
||||||
|
/** A cheap reachability probe for diagnostics only. Its result is **never** persisted and never
|
||||||
|
* influences a module's lifecycle state — a provider being briefly unreachable is an operational
|
||||||
|
* condition, not an administrative decision (ADR 0030). */
|
||||||
|
probe(): Promise<void>;
|
||||||
|
/** Releases provider connections. Called once, during API shutdown. */
|
||||||
|
close(): Promise<void>;
|
||||||
|
}
|
||||||
@@ -0,0 +1,289 @@
|
|||||||
|
import { Readable } from "node:stream";
|
||||||
|
import {
|
||||||
|
CreateBucketCommand,
|
||||||
|
DeleteObjectCommand,
|
||||||
|
GetObjectCommand,
|
||||||
|
HeadObjectCommand,
|
||||||
|
ListBucketsCommand,
|
||||||
|
ListObjectsV2Command,
|
||||||
|
PutObjectCommand,
|
||||||
|
S3Client,
|
||||||
|
} from "@aws-sdk/client-s3";
|
||||||
|
import {
|
||||||
|
type ModuleStorageBody,
|
||||||
|
ModuleStorageError,
|
||||||
|
type ModuleStorageListOptions,
|
||||||
|
type ModuleStorageListing,
|
||||||
|
type ModuleStorageObject,
|
||||||
|
type ModuleStorageObjectMetadata,
|
||||||
|
type ModuleStoragePutOptions,
|
||||||
|
} from "@ekklesia/contracts";
|
||||||
|
import type { StorageProviderConfig } from "./config.js";
|
||||||
|
import type { StorageProvider } from "./provider.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The platform's first real `StorageProvider`, speaking the S3-compatible protocol.
|
||||||
|
*
|
||||||
|
* **It is not named after a product, and it must not become one.** SeaweedFS is what the
|
||||||
|
* repository's development Compose stack runs and the likely deployment target (ADR 0004), but
|
||||||
|
* nothing here depends on SeaweedFS: the same code addresses MinIO, AWS S3, Garage, or anything else
|
||||||
|
* speaking the protocol, selected purely by host configuration (ADR 0030).
|
||||||
|
*
|
||||||
|
* Server-only, and structurally so: this module imports `node:stream` and the AWS SDK, so a browser
|
||||||
|
* bundle importing it would fail to build. `@ekklesia/module-ui-sdk` and `surfaces/*` never reach it,
|
||||||
|
* because a module never imports Core at all.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** Bounds a single listing page. S3-compatible providers cap this at 1000 regardless. */
|
||||||
|
const MAX_LIST_PAGE_SIZE = 1000;
|
||||||
|
|
||||||
|
interface S3LikeError {
|
||||||
|
readonly name?: unknown;
|
||||||
|
readonly $metadata?: { readonly httpStatusCode?: unknown };
|
||||||
|
}
|
||||||
|
|
||||||
|
function errorName(error: unknown): string {
|
||||||
|
const candidate = error as S3LikeError | null;
|
||||||
|
return typeof candidate?.name === "string" ? candidate.name : "";
|
||||||
|
}
|
||||||
|
|
||||||
|
function statusCode(error: unknown): number | undefined {
|
||||||
|
const candidate = error as S3LikeError | null;
|
||||||
|
const status = candidate?.$metadata?.httpStatusCode;
|
||||||
|
return typeof status === "number" ? status : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reduces any provider failure to one of the contract's closed codes.
|
||||||
|
*
|
||||||
|
* **Nothing from the caught error survives this function** — not its message, not its request id,
|
||||||
|
* not the XML body, not the endpoint it was talking to. Provider errors routinely carry the bucket
|
||||||
|
* name, the host, and sometimes the full signed request, and a module is entitled to log whatever it
|
||||||
|
* catches, so translation here is the only place that boundary can be held.
|
||||||
|
*
|
||||||
|
* The mapping is verified against the pinned SeaweedFS 4.28 image rather than assumed from AWS
|
||||||
|
* documentation: a missing object reports `NoSuchKey`, a missing bucket `NoSuchBucket`, and — the
|
||||||
|
* one that actually differs from AWS — creating an existing bucket reports `BucketAlreadyExists`
|
||||||
|
* where AWS reports `BucketAlreadyOwnedByYou`.
|
||||||
|
*/
|
||||||
|
function translateStorageError(error: unknown, key?: string): ModuleStorageError {
|
||||||
|
const name = errorName(error);
|
||||||
|
const status = statusCode(error);
|
||||||
|
|
||||||
|
if (name === "NoSuchKey" || name === "NotFound" || (status === 404 && key !== undefined)) {
|
||||||
|
return new ModuleStorageError("STORAGE_OBJECT_NOT_FOUND", key);
|
||||||
|
}
|
||||||
|
if (name === "NoSuchBucket" || status === 404) {
|
||||||
|
return new ModuleStorageError("STORAGE_BUCKET_UNAVAILABLE");
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
name === "AccessDenied" ||
|
||||||
|
name === "InvalidAccessKeyId" ||
|
||||||
|
name === "SignatureDoesNotMatch"
|
||||||
|
) {
|
||||||
|
return new ModuleStorageError("STORAGE_OPERATION_DENIED");
|
||||||
|
}
|
||||||
|
if (status === 403) {
|
||||||
|
return new ModuleStorageError("STORAGE_OPERATION_DENIED");
|
||||||
|
}
|
||||||
|
// Connection-level failures surface as Node system errors or the SDK's own timeout types rather
|
||||||
|
// than as an HTTP status, which is exactly the transient case a module is expected to retry.
|
||||||
|
if (
|
||||||
|
status === undefined &&
|
||||||
|
(name.startsWith("Timeout") ||
|
||||||
|
name === "TimeoutError" ||
|
||||||
|
name === "AbortError" ||
|
||||||
|
/ECONN|ENOTFOUND|EAI_AGAIN|EHOSTUNREACH|ENETUNREACH|EPIPE|ETIMEDOUT/u.test(
|
||||||
|
`${name} ${String((error as { code?: unknown } | null)?.code ?? "")}`,
|
||||||
|
))
|
||||||
|
) {
|
||||||
|
return new ModuleStorageError("STORAGE_PROVIDER_UNAVAILABLE");
|
||||||
|
}
|
||||||
|
if (status !== undefined && status >= 500) {
|
||||||
|
return new ModuleStorageError("STORAGE_PROVIDER_UNAVAILABLE");
|
||||||
|
}
|
||||||
|
return new ModuleStorageError("STORAGE_OPERATION_FAILED", key);
|
||||||
|
}
|
||||||
|
|
||||||
|
function metadataFrom(
|
||||||
|
key: string,
|
||||||
|
source: {
|
||||||
|
readonly ContentType?: string | undefined;
|
||||||
|
readonly ContentLength?: number | undefined;
|
||||||
|
readonly ETag?: string | undefined;
|
||||||
|
readonly LastModified?: Date | undefined;
|
||||||
|
},
|
||||||
|
): ModuleStorageObjectMetadata {
|
||||||
|
return Object.freeze({
|
||||||
|
key,
|
||||||
|
// Providers quote the ETag; unquoting keeps the value usable as an HTTP validator without every
|
||||||
|
// module learning to strip quotes.
|
||||||
|
...(source.ETag === undefined ? {} : { etag: source.ETag.replaceAll('"', "") }),
|
||||||
|
...(source.ContentLength === undefined ? {} : { contentLength: source.ContentLength }),
|
||||||
|
...(source.ContentType === undefined ? {} : { contentType: source.ContentType }),
|
||||||
|
...(source.LastModified === undefined ? {} : { lastModified: source.LastModified }),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Converts the contract's accepted body shapes into something the SDK can send without this
|
||||||
|
* process buffering a large object. A Web stream becomes a Node stream rather than an array of
|
||||||
|
* bytes, which is the whole reason the contract accepts one. */
|
||||||
|
function toRequestBody(body: ModuleStorageBody): Uint8Array | string | Readable | Blob {
|
||||||
|
if (body instanceof ReadableStream) {
|
||||||
|
return Readable.fromWeb(body as Parameters<typeof Readable.fromWeb>[0]);
|
||||||
|
}
|
||||||
|
return body;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createS3StorageProvider(config: StorageProviderConfig): StorageProvider {
|
||||||
|
const client = new S3Client({
|
||||||
|
credentials: { accessKeyId: config.accessKey, secretAccessKey: config.secretKey },
|
||||||
|
endpoint: config.endpoint,
|
||||||
|
forcePathStyle: config.forcePathStyle,
|
||||||
|
region: config.region,
|
||||||
|
});
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
async ensureBucket(physicalBucket: string): Promise<void> {
|
||||||
|
try {
|
||||||
|
await client.send(new CreateBucketCommand({ Bucket: physicalBucket }));
|
||||||
|
} catch (error) {
|
||||||
|
const name = errorName(error);
|
||||||
|
// Idempotency, spelled out for both dialects. AWS reports `BucketAlreadyOwnedByYou`; the
|
||||||
|
// pinned SeaweedFS image reports `BucketAlreadyExists` with a 409 — verified directly, not
|
||||||
|
// assumed. Treating only the AWS spelling as success would make every second boot fail
|
||||||
|
// against the repository's own development storage service.
|
||||||
|
if (name === "BucketAlreadyOwnedByYou" || name === "BucketAlreadyExists") {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (name === "AccessDenied" || statusCode(error) === 403) {
|
||||||
|
// The host's credentials cannot create it. Surfaced as an unavailable bucket rather than
|
||||||
|
// silently falling back to "maybe it exists": an operator needs to know their credentials
|
||||||
|
// are insufficient, not discover it through a later confusing read failure.
|
||||||
|
throw new ModuleStorageError("STORAGE_BUCKET_UNAVAILABLE");
|
||||||
|
}
|
||||||
|
throw translateStorageError(error);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async list(
|
||||||
|
physicalBucket: string,
|
||||||
|
options: ModuleStorageListOptions,
|
||||||
|
): Promise<ModuleStorageListing> {
|
||||||
|
try {
|
||||||
|
const response = await client.send(
|
||||||
|
new ListObjectsV2Command({
|
||||||
|
Bucket: physicalBucket,
|
||||||
|
...(options.cursor === undefined ? {} : { ContinuationToken: options.cursor }),
|
||||||
|
...(options.limit === undefined
|
||||||
|
? {}
|
||||||
|
: { MaxKeys: Math.max(1, Math.min(options.limit, MAX_LIST_PAGE_SIZE)) }),
|
||||||
|
...(options.prefix === undefined ? {} : { Prefix: options.prefix }),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const objects = (response.Contents ?? []).flatMap((entry) =>
|
||||||
|
entry.Key === undefined
|
||||||
|
? []
|
||||||
|
: [
|
||||||
|
metadataFrom(entry.Key, {
|
||||||
|
ETag: entry.ETag,
|
||||||
|
ContentLength: entry.Size,
|
||||||
|
LastModified: entry.LastModified,
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
return Object.freeze({
|
||||||
|
objects: Object.freeze(objects),
|
||||||
|
// Only forwarded when the provider says the listing is truncated, so an absent cursor
|
||||||
|
// always means "complete" rather than "the provider happened to omit a token".
|
||||||
|
...(response.IsTruncated === true && response.NextContinuationToken !== undefined
|
||||||
|
? { cursor: response.NextContinuationToken }
|
||||||
|
: {}),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
throw translateStorageError(error);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async head(physicalBucket: string, key: string): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
try {
|
||||||
|
const response = await client.send(
|
||||||
|
new HeadObjectCommand({ Bucket: physicalBucket, Key: key }),
|
||||||
|
);
|
||||||
|
return metadataFrom(key, response);
|
||||||
|
} catch (error) {
|
||||||
|
throw translateStorageError(error, key);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async get(physicalBucket: string, key: string): Promise<ModuleStorageObject> {
|
||||||
|
try {
|
||||||
|
const response = await client.send(
|
||||||
|
new GetObjectCommand({ Bucket: physicalBucket, Key: key }),
|
||||||
|
);
|
||||||
|
if (!response.Body) {
|
||||||
|
throw new ModuleStorageError("STORAGE_OPERATION_FAILED", key);
|
||||||
|
}
|
||||||
|
// A Web stream, never a buffered body: a module hands this straight to a `Response`, so a
|
||||||
|
// large object never passes through this process's memory in one piece.
|
||||||
|
const body = response.Body.transformToWebStream() as ReadableStream<Uint8Array>;
|
||||||
|
return Object.freeze({ ...metadataFrom(key, response), body });
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof ModuleStorageError) {
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
throw translateStorageError(error, key);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async put(
|
||||||
|
physicalBucket: string,
|
||||||
|
key: string,
|
||||||
|
body: ModuleStorageBody,
|
||||||
|
options: ModuleStoragePutOptions,
|
||||||
|
): Promise<ModuleStorageObjectMetadata> {
|
||||||
|
try {
|
||||||
|
const response = await client.send(
|
||||||
|
new PutObjectCommand({
|
||||||
|
Body: toRequestBody(body),
|
||||||
|
Bucket: physicalBucket,
|
||||||
|
Key: key,
|
||||||
|
...(options.contentLength === undefined
|
||||||
|
? {}
|
||||||
|
: { ContentLength: options.contentLength }),
|
||||||
|
...(options.contentType === undefined ? {} : { ContentType: options.contentType }),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return metadataFrom(key, {
|
||||||
|
ETag: response.ETag,
|
||||||
|
...(options.contentLength === undefined ? {} : { ContentLength: options.contentLength }),
|
||||||
|
...(options.contentType === undefined ? {} : { ContentType: options.contentType }),
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
throw translateStorageError(error, key);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async delete(physicalBucket: string, key: string): Promise<void> {
|
||||||
|
try {
|
||||||
|
await client.send(new DeleteObjectCommand({ Bucket: physicalBucket, Key: key }));
|
||||||
|
} catch (error) {
|
||||||
|
throw translateStorageError(error, key);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async probe(): Promise<void> {
|
||||||
|
try {
|
||||||
|
// The cheapest call that proves the endpoint is reachable and the credentials sign
|
||||||
|
// correctly, without creating, reading, or writing anything.
|
||||||
|
await client.send(new ListBucketsCommand({}));
|
||||||
|
} catch (error) {
|
||||||
|
throw translateStorageError(error);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
async close(): Promise<void> {
|
||||||
|
client.destroy();
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -47,6 +47,9 @@ function noConfigurationDependencies(): DoctorDependencies {
|
|||||||
planMigrations: async () => {
|
planMigrations: async () => {
|
||||||
throw new Error("database must not be planned");
|
throw new Error("database must not be planned");
|
||||||
},
|
},
|
||||||
|
probeStorage: async () => {
|
||||||
|
throw new Error("storage must not be probed");
|
||||||
|
},
|
||||||
readModuleConfiguration: async () => {
|
readModuleConfiguration: async () => {
|
||||||
throw new Error("module configuration must not be read");
|
throw new Error("module configuration must not be read");
|
||||||
},
|
},
|
||||||
@@ -112,8 +115,10 @@ Identity
|
|||||||
Trusts only the actual immediate socket peer for rate limiting; no forwarded header is trusted.
|
Trusts only the actual immediate socket peer for rate limiting; no forwarded header is trusted.
|
||||||
|
|
||||||
Object storage
|
Object storage
|
||||||
○ Check unavailable
|
○ Object storage not configured
|
||||||
Core storage integration has not been implemented yet.
|
No S3_ENDPOINT is set. This is a complete deployment unless a module requires storage.
|
||||||
|
✓ No module requires object storage
|
||||||
|
0 modules declare a logical storage bucket.
|
||||||
|
|
||||||
Result: READY WITH WARNINGS`);
|
Result: READY WITH WARNINGS`);
|
||||||
});
|
});
|
||||||
|
|||||||
+171
-4
@@ -138,6 +138,7 @@ function dependencies(overrides: Partial<DoctorDependencies> = {}): DoctorDepend
|
|||||||
discoverModules: vi.fn(async () => Object.freeze([])),
|
discoverModules: vi.fn(async () => Object.freeze([])),
|
||||||
discoverSchemas: vi.fn(async () => schemaAggregate()),
|
discoverSchemas: vi.fn(async () => schemaAggregate()),
|
||||||
planMigrations: vi.fn(async () => plan()),
|
planMigrations: vi.fn(async () => plan()),
|
||||||
|
probeStorage: vi.fn(async () => undefined),
|
||||||
readModuleConfiguration: vi.fn(async () => Object.freeze({ records: Object.freeze([]) })),
|
readModuleConfiguration: vi.fn(async () => Object.freeze({ records: Object.freeze([]) })),
|
||||||
readModuleLifecycle: vi.fn(async () => Object.freeze({ modules: Object.freeze([]) })),
|
readModuleLifecycle: vi.fn(async () => Object.freeze({ modules: Object.freeze([]) })),
|
||||||
runtimeSatisfies: vi.fn(() => true),
|
runtimeSatisfies: vi.fn(() => true),
|
||||||
@@ -217,6 +218,7 @@ describe("Ekklesia Doctor diagnostics", () => {
|
|||||||
"identity.configuration",
|
"identity.configuration",
|
||||||
"identity.client-address",
|
"identity.client-address",
|
||||||
"object-storage.integration",
|
"object-storage.integration",
|
||||||
|
"object-storage.bindings",
|
||||||
]);
|
]);
|
||||||
expect(check(report, "modules.discovery").details).toEqual(["people 0.1.0", "calendar 0.2.0"]);
|
expect(check(report, "modules.discovery").details).toEqual(["people 0.1.0", "calendar 0.2.0"]);
|
||||||
expect(check(report, "modules.dependencies").details).toEqual(["Order: people → calendar"]);
|
expect(check(report, "modules.dependencies").details).toEqual(["Order: people → calendar"]);
|
||||||
@@ -553,15 +555,19 @@ describe("Ekklesia Doctor diagnostics", () => {
|
|||||||
expect(serialized).not.toContain("also-secret.invalid");
|
expect(serialized).not.toContain("also-secret.invalid");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("always reports object storage as unavailable without probing configuration", async () => {
|
it("skips the storage check, without probing, when no provider is configured", async () => {
|
||||||
const report = await diagnoseEkklesia(options(), dependencies());
|
const services = dependencies();
|
||||||
|
const report = await diagnoseEkklesia(options(), services);
|
||||||
|
|
||||||
expect(check(report, "object-storage.integration")).toEqual({
|
expect(check(report, "object-storage.integration")).toEqual({
|
||||||
details: ["Core storage integration has not been implemented yet."],
|
details: [
|
||||||
|
"No S3_ENDPOINT is set. This is a complete deployment unless a module requires storage.",
|
||||||
|
],
|
||||||
id: "object-storage.integration",
|
id: "object-storage.integration",
|
||||||
status: "skipped",
|
status: "skipped",
|
||||||
summary: "Check unavailable",
|
summary: "Object storage not configured",
|
||||||
});
|
});
|
||||||
|
expect(services.probeStorage).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
it("continues static discovery when runtime validation fails", async () => {
|
it("continues static discovery when runtime validation fails", async () => {
|
||||||
@@ -1208,3 +1214,164 @@ describe("Ekklesia Doctor — module configuration", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("Ekklesia Doctor — object storage", () => {
|
||||||
|
const storageConfig = {
|
||||||
|
accessKey: "AKIAEXAMPLEACCESSID",
|
||||||
|
bucketPrefix: "ekklesia",
|
||||||
|
endpoint: "https://storage.example.test",
|
||||||
|
forcePathStyle: true,
|
||||||
|
region: "us-east-1",
|
||||||
|
secretKey: "sk-example-secret-material",
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
function storingModule(id: string, buckets: readonly string[]): DiscoveredModule {
|
||||||
|
const base = discoveredModule(id);
|
||||||
|
return {
|
||||||
|
...base,
|
||||||
|
manifest: {
|
||||||
|
...base.manifest,
|
||||||
|
storage: { buckets: buckets.map((bucketId) => ({ id: bucketId })) },
|
||||||
|
} as unknown as ModuleManifest,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
it("does not probe, and stays healthy, when no provider is configured and nothing needs one", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () => Object.freeze([discoveredModule("calendar")])),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(options(databaseUrl), services);
|
||||||
|
|
||||||
|
expect(services.probeStorage).not.toHaveBeenCalled();
|
||||||
|
expect(check(report, "object-storage.integration").status).toBe("skipped");
|
||||||
|
// The load-bearing assertion: an installation whose modules need no object storage is not
|
||||||
|
// unhealthy for lacking a provider.
|
||||||
|
expect(check(report, "object-storage.bindings").status).toBe("pass");
|
||||||
|
expect(check(report, "object-storage.bindings").details).toEqual([
|
||||||
|
"0 modules declare a logical storage bucket.",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fails the bindings check when a module requires storage and none is configured", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () =>
|
||||||
|
Object.freeze([storingModule("announcements", ["announcement-assets"])]),
|
||||||
|
),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(options(databaseUrl), services);
|
||||||
|
const bindings = check(report, "object-storage.bindings");
|
||||||
|
|
||||||
|
expect(bindings.status).toBe("failure");
|
||||||
|
expect(bindings.details[0]).toContain("Configure an S3-compatible Ekklesia storage provider");
|
||||||
|
expect(bindings.details).toContain(
|
||||||
|
"announcements · announcement-assets · unbound (no storage provider configured)",
|
||||||
|
);
|
||||||
|
// And the lifecycle check agrees, because both come from one resolution.
|
||||||
|
expect(check(report, "modules.lifecycle").details).toContain(
|
||||||
|
"announcements 0.1.0 · blocked · storage_unavailable · announcement-assets",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resolves and lists every binding when a provider is configured", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () =>
|
||||||
|
Object.freeze([storingModule("announcements", ["announcement-assets"])]),
|
||||||
|
),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(
|
||||||
|
{ ...options(databaseUrl), storage: storageConfig },
|
||||||
|
services,
|
||||||
|
);
|
||||||
|
const bindings = check(report, "object-storage.bindings");
|
||||||
|
|
||||||
|
expect(bindings.status).toBe("pass");
|
||||||
|
// The physical bucket is named here deliberately: Doctor is an operator tool, and someone
|
||||||
|
// debugging a permissions problem needs to know which bucket to look at. It is still never
|
||||||
|
// reachable from module-facing code.
|
||||||
|
expect(bindings.details[0]).toMatch(
|
||||||
|
/^announcements · announcement-assets → ekklesia-announcements--announcement-assets-[0-9a-f]{12}$/u,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("probes a configured provider and reports it reachable, without printing a credential", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () => Object.freeze([discoveredModule("calendar")])),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(
|
||||||
|
{ ...options(databaseUrl), storage: storageConfig },
|
||||||
|
services,
|
||||||
|
);
|
||||||
|
const integration = check(report, "object-storage.integration");
|
||||||
|
|
||||||
|
expect(services.probeStorage).toHaveBeenCalledWith(storageConfig);
|
||||||
|
expect(integration.status).toBe("pass");
|
||||||
|
expect(integration.details[0]).toContain("https://storage.example.test");
|
||||||
|
const serialized = JSON.stringify(integration);
|
||||||
|
expect(serialized).not.toContain("AKIAEXAMPLEACCESSID");
|
||||||
|
expect(serialized).not.toContain("sk-example-secret-material");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("warns rather than fails when a configured provider is unreachable", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () => Object.freeze([discoveredModule("calendar")])),
|
||||||
|
probeStorage: vi.fn(async () => {
|
||||||
|
throw new Error("connect ECONNREFUSED 10.40.0.2:8333 accessKey=AKIAEXAMPLEACCESSID");
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(
|
||||||
|
{ ...options(databaseUrl), storage: storageConfig },
|
||||||
|
services,
|
||||||
|
);
|
||||||
|
const integration = check(report, "object-storage.integration");
|
||||||
|
|
||||||
|
// A warning, never a failure. A momentary network blip must not make the whole platform read as
|
||||||
|
// NOT READY, and it must never look like a module problem (ADR 0030).
|
||||||
|
expect(integration.status).toBe("warning");
|
||||||
|
expect(report.result).not.toBe("not-ready");
|
||||||
|
expect(integration.details).toContain(
|
||||||
|
"The configured storage service did not respond. Modules already running are unaffected.",
|
||||||
|
);
|
||||||
|
const serialized = JSON.stringify(integration);
|
||||||
|
expect(serialized).not.toContain("AKIAEXAMPLEACCESSID");
|
||||||
|
expect(serialized).not.toContain("10.40.0.2");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("still resolves bindings when the provider is unreachable, because binding is not health", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () =>
|
||||||
|
Object.freeze([storingModule("announcements", ["announcement-assets"])]),
|
||||||
|
),
|
||||||
|
probeStorage: vi.fn(async () => {
|
||||||
|
throw new Error("unreachable");
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(
|
||||||
|
{ ...options(databaseUrl), storage: storageConfig },
|
||||||
|
services,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(check(report, "object-storage.integration").status).toBe("warning");
|
||||||
|
// The module is still eligible: a configured-but-briefly-unreachable provider is an operational
|
||||||
|
// condition, not a lifecycle state.
|
||||||
|
expect(check(report, "object-storage.bindings").status).toBe("pass");
|
||||||
|
expect(check(report, "modules.lifecycle").details).toContain("announcements 0.1.0 · enabled");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips bindings when discovery itself failed", async () => {
|
||||||
|
const services = dependencies({
|
||||||
|
discoverModules: vi.fn(async () => {
|
||||||
|
throw new Error("broken inventory");
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
const report = await diagnoseEkklesia(options(databaseUrl), services);
|
||||||
|
|
||||||
|
expect(check(report, "object-storage.bindings").status).toBe("skipped");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -0,0 +1,309 @@
|
|||||||
|
import { parseModuleManifest } from "@ekklesia/contracts";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import type { DiscoveredModule } from "../src/modules/discovery.js";
|
||||||
|
import {
|
||||||
|
isStorageBlockedDetail,
|
||||||
|
resolveModuleEligibility,
|
||||||
|
resolveModuleLifecycleView,
|
||||||
|
} from "../src/modules/lifecycle.js";
|
||||||
|
|
||||||
|
/** A minimal in-memory `DiscoveredModule`, built through `parseModuleManifest` so a fixture real
|
||||||
|
* discovery would reject can never stand in for one it would accept. */
|
||||||
|
function fakeModule(
|
||||||
|
id: string,
|
||||||
|
options: {
|
||||||
|
readonly buckets?: readonly string[];
|
||||||
|
readonly dependencies?: Record<string, unknown>;
|
||||||
|
readonly configuration?: readonly Record<string, unknown>[];
|
||||||
|
} = {},
|
||||||
|
): DiscoveredModule {
|
||||||
|
const manifest = parseModuleManifest({
|
||||||
|
manifestVersion: 5,
|
||||||
|
id,
|
||||||
|
name: `${id} module`,
|
||||||
|
version: "1.0.0",
|
||||||
|
compatibility: { platform: "^0.1.0", moduleSdk: "^0.1.0" },
|
||||||
|
...(options.configuration ? { configuration: options.configuration } : {}),
|
||||||
|
...(options.dependencies ? { dependencies: options.dependencies } : {}),
|
||||||
|
...(options.buckets ? { storage: { buckets: options.buckets.map((id) => ({ id })) } } : {}),
|
||||||
|
entrypoints: { server: "./dist/does-not-exist/index.js" },
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
directory: `/modules/${id}`,
|
||||||
|
entrypoints: { server: `/modules/${id}/dist/does-not-exist/index.js` },
|
||||||
|
manifest,
|
||||||
|
manifestPath: `/modules/${id}/ekklesia.module.json`,
|
||||||
|
packageName: `@test/${id}`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function enablement(...records: readonly (readonly [string, boolean])[]) {
|
||||||
|
return { modules: records.map(([moduleId, enabled]) => ({ moduleId, enabled })) };
|
||||||
|
}
|
||||||
|
|
||||||
|
const noConfiguration = { settings: { records: [] } };
|
||||||
|
const withStorage = { providerConfigured: true };
|
||||||
|
const withoutStorage = { providerConfigured: false };
|
||||||
|
|
||||||
|
function ids(modules: readonly DiscoveredModule[]): readonly string[] {
|
||||||
|
return modules.map((module) => module.manifest.id);
|
||||||
|
}
|
||||||
|
|
||||||
|
function entry(
|
||||||
|
resolution: ReturnType<typeof resolveModuleEligibility>,
|
||||||
|
moduleId: string,
|
||||||
|
): ReturnType<typeof resolveModuleEligibility>["entries"][number] {
|
||||||
|
const found = resolution.entries.find((candidate) => candidate.moduleId === moduleId);
|
||||||
|
if (!found) {
|
||||||
|
throw new Error(`No eligibility entry for ${moduleId}.`);
|
||||||
|
}
|
||||||
|
return found;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("declared storage decides eligibility", () => {
|
||||||
|
it("keeps a module requiring storage eligible when a provider is configured", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[fakeModule("announcements", { buckets: ["announcement-assets"] })],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(ids(resolution.eligibleModules)).toEqual(["announcements"]);
|
||||||
|
expect(entry(resolution, "announcements").blocked).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("blocks a module requiring storage when the host has no provider, naming the logical bucket", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[fakeModule("announcements", { buckets: ["announcement-assets"] })],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(resolution.eligibleModules).toEqual([]);
|
||||||
|
expect(entry(resolution, "announcements").blocked).toEqual({
|
||||||
|
reason: "storage_unavailable",
|
||||||
|
storageBucketId: "announcement-assets",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaves a module declaring no buckets entirely unaffected by storage", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[fakeModule("calendar")],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
// An installation with no object storage is a complete, valid deployment.
|
||||||
|
expect(ids(resolution.eligibleModules)).toEqual(["calendar"]);
|
||||||
|
expect(entry(resolution, "calendar").blocked).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not evaluate a disabled module's storage requirement", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[fakeModule("announcements", { buckets: ["announcement-assets"] })],
|
||||||
|
enablement(["announcements", false]),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Disabled is not blocked: reporting a storage problem for something nobody asked to run would
|
||||||
|
// be noise.
|
||||||
|
expect(entry(resolution, "announcements").blocked).toBeUndefined();
|
||||||
|
expect(entry(resolution, "announcements").configured.enabled).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("defaults to no provider when the caller supplies no storage input at all", () => {
|
||||||
|
// Every pre-existing caller keeps its exact behaviour: the parameter is optional, and a module
|
||||||
|
// that declares no bucket is unaffected either way.
|
||||||
|
const withoutArgument = resolveModuleEligibility([fakeModule("calendar")], enablement());
|
||||||
|
expect(ids(withoutArgument.eligibleModules)).toEqual(["calendar"]);
|
||||||
|
|
||||||
|
const requiring = resolveModuleEligibility(
|
||||||
|
[fakeModule("announcements", { buckets: ["assets"] })],
|
||||||
|
enablement(),
|
||||||
|
);
|
||||||
|
expect(requiring.eligibleModules).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("names only the logical bucket — never an endpoint, credential, or physical bucket", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[fakeModule("announcements", { buckets: ["announcement-assets"] })],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
const serialized = JSON.stringify(resolution.entries);
|
||||||
|
|
||||||
|
expect(serialized).toContain("announcement-assets");
|
||||||
|
expect(serialized).not.toContain("ekklesia-announcements--");
|
||||||
|
expect(serialized).not.toContain("http");
|
||||||
|
expect(serialized).not.toMatch(/accessKey|secretKey|endpoint/iu);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("storage blocking propagates through the dependency graph", () => {
|
||||||
|
const graph = [
|
||||||
|
fakeModule("media-storage", { buckets: ["objects"] }),
|
||||||
|
fakeModule("media", { dependencies: { modules: { required: { "media-storage": "^1.0.0" } } } }),
|
||||||
|
fakeModule("announcements", { dependencies: { modules: { required: { media: "^1.0.0" } } } }),
|
||||||
|
];
|
||||||
|
|
||||||
|
it("blocks required dependents of a storage-blocked module, transitively", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
graph,
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(resolution.eligibleModules).toEqual([]);
|
||||||
|
// One mechanism, not two: the chain terminates at the module whose own requirement is
|
||||||
|
// unsatisfiable, exactly as it terminates at a disabled or misconfigured module.
|
||||||
|
expect(entry(resolution, "media-storage").blocked).toEqual({
|
||||||
|
reason: "storage_unavailable",
|
||||||
|
storageBucketId: "objects",
|
||||||
|
});
|
||||||
|
expect(entry(resolution, "media").blocked).toEqual({
|
||||||
|
dependency: { kind: "module", moduleId: "media-storage" },
|
||||||
|
reason: "required_dependency_blocked",
|
||||||
|
});
|
||||||
|
expect(entry(resolution, "announcements").blocked).toEqual({
|
||||||
|
dependency: { kind: "module", moduleId: "media" },
|
||||||
|
reason: "required_dependency_blocked",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("unblocks the whole chain once a provider is configured", () => {
|
||||||
|
const resolution = resolveModuleEligibility(graph, enablement(), noConfiguration, withStorage);
|
||||||
|
|
||||||
|
expect(ids(resolution.eligibleModules)).toEqual(["media-storage", "media", "announcements"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("never blocks an unrelated module because another module needs storage", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[...graph, fakeModule("calendar")],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(ids(resolution.eligibleModules)).toEqual(["calendar"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("storage and configuration blocking coexist without a second mechanism", () => {
|
||||||
|
it("reports the storage requirement first when both are unsatisfiable", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[
|
||||||
|
fakeModule("announcements", {
|
||||||
|
buckets: ["assets"],
|
||||||
|
configuration: [{ key: "bucketLabel", type: "string", required: true }],
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Deterministic ordering — storage is evaluated before configuration — so an operator always
|
||||||
|
// sees the same reason for the same state rather than one that depends on evaluation order.
|
||||||
|
const blocked = entry(resolution, "announcements").blocked;
|
||||||
|
expect(blocked && isStorageBlockedDetail(blocked)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls through to the configuration reason once storage is available", () => {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
[
|
||||||
|
fakeModule("announcements", {
|
||||||
|
buckets: ["assets"],
|
||||||
|
configuration: [{ key: "bucketLabel", type: "string", required: true }],
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(entry(resolution, "announcements").blocked).toEqual({
|
||||||
|
configurationKey: "bucketLabel",
|
||||||
|
reason: "configuration_missing",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("the lifecycle view reports storage state honestly", () => {
|
||||||
|
const discoveredModules = [fakeModule("announcements", { buckets: ["announcement-assets"] })];
|
||||||
|
|
||||||
|
function bootSnapshot(storage: { providerConfigured: boolean }) {
|
||||||
|
const resolution = resolveModuleEligibility(
|
||||||
|
discoveredModules,
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
storage,
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
discoveredModules,
|
||||||
|
eligibility: resolution.entries,
|
||||||
|
startedModuleIds: resolution.eligibleModules.map((module) => module.manifest.id),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
it("reports a storage-blocked module with the logical bucket and no pending restart", () => {
|
||||||
|
const view = resolveModuleLifecycleView(
|
||||||
|
bootSnapshot(withoutStorage),
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(view[0]?.runtime).toEqual({
|
||||||
|
reason: "storage_unavailable",
|
||||||
|
state: "blocked",
|
||||||
|
storageBucketId: "announcement-assets",
|
||||||
|
});
|
||||||
|
expect(view[0]?.restartRequired).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports a pending restart once a provider is configured for a blocked module", () => {
|
||||||
|
const view = resolveModuleLifecycleView(
|
||||||
|
bootSnapshot(withoutStorage),
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
// The next boot really will start it, so the honest answer is that a restart is pending — while
|
||||||
|
// the runtime line still describes what *this* process did.
|
||||||
|
expect(view[0]?.runtime.state).toBe("blocked");
|
||||||
|
expect(view[0]?.restartRequired).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports a running module with storage as ordinary and unremarkable", () => {
|
||||||
|
const view = resolveModuleLifecycleView(
|
||||||
|
bootSnapshot(withStorage),
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withStorage,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(view[0]?.runtime).toEqual({ state: "running" });
|
||||||
|
expect(view[0]?.restartRequired).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("puts no infrastructure detail into the view", () => {
|
||||||
|
const view = resolveModuleLifecycleView(
|
||||||
|
bootSnapshot(withoutStorage),
|
||||||
|
enablement(),
|
||||||
|
noConfiguration,
|
||||||
|
withoutStorage,
|
||||||
|
);
|
||||||
|
const serialized = JSON.stringify(view);
|
||||||
|
|
||||||
|
expect(serialized).toContain("announcement-assets");
|
||||||
|
expect(serialized).not.toContain("ekklesia-announcements--");
|
||||||
|
expect(serialized).not.toMatch(/http|accessKey|secretKey|seaweed/iu);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,357 @@
|
|||||||
|
import { randomBytes } from "node:crypto";
|
||||||
|
import { ModuleStorageError } from "@ekklesia/contracts";
|
||||||
|
import { afterAll, describe, expect, it } from "vitest";
|
||||||
|
import { physicalBucketName } from "../src/storage/binding.js";
|
||||||
|
import { createStorageProviderConfig } from "../src/storage/config.js";
|
||||||
|
import { createModuleStorage } from "../src/storage/module-storage.js";
|
||||||
|
import type { StorageProvider } from "../src/storage/provider.js";
|
||||||
|
import { createS3StorageProvider } from "../src/storage/s3-provider.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The storage contract against a **real S3-compatible server**.
|
||||||
|
*
|
||||||
|
* The unit suite proves scoping, binding, and validation with an in-memory provider; this proves the
|
||||||
|
* things only a real server can — and the reason it must exist is concrete. Probing the pinned
|
||||||
|
* SeaweedFS 4.28 image directly revealed that `CreateBucket` on an existing bucket reports
|
||||||
|
* `BucketAlreadyExists`, **not** AWS's `BucketAlreadyOwnedByYou`. A provider that only treated the
|
||||||
|
* AWS spelling as success would have failed on every second boot against this repository's own
|
||||||
|
* development storage service, and no amount of mocking would have caught it.
|
||||||
|
*
|
||||||
|
* SeaweedFS is used rather than a generic mock because it is the intended deployment target
|
||||||
|
* (ADR 0004) and because the repository's Compose file already pins and provisions it, so this adds
|
||||||
|
* no new CI infrastructure. Nothing here is SeaweedFS-specific in the *contract* — the same suite
|
||||||
|
* would pass against MinIO or AWS S3, which is the portability claim being tested.
|
||||||
|
*/
|
||||||
|
const endpointEnvironment = "EKKLESIA_TEST_STORAGE_ENDPOINT";
|
||||||
|
const accessKeyEnvironment = "EKKLESIA_TEST_STORAGE_ACCESS_KEY";
|
||||||
|
const secretKeyEnvironment = "EKKLESIA_TEST_STORAGE_SECRET_KEY";
|
||||||
|
|
||||||
|
class IntegrationHarnessError extends Error {
|
||||||
|
constructor(message: string) {
|
||||||
|
super(message);
|
||||||
|
this.name = "IntegrationHarnessError";
|
||||||
|
this.stack = `${this.name}: ${this.message}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function providerConfig() {
|
||||||
|
const endpoint = process.env[endpointEnvironment];
|
||||||
|
if (typeof endpoint !== "string" || endpoint.trim() === "") {
|
||||||
|
throw new IntegrationHarnessError("The storage integration endpoint is not configured.");
|
||||||
|
}
|
||||||
|
const config = createStorageProviderConfig(
|
||||||
|
{
|
||||||
|
accessKey: process.env[accessKeyEnvironment],
|
||||||
|
endpoint,
|
||||||
|
secretKey: process.env[secretKeyEnvironment],
|
||||||
|
},
|
||||||
|
// The disposable service speaks plaintext on loopback, exactly like the disposable PostgreSQL
|
||||||
|
// this suite already uses; production configuration would refuse it.
|
||||||
|
{ environment: "development" },
|
||||||
|
);
|
||||||
|
if (!config) {
|
||||||
|
throw new IntegrationHarnessError("The storage integration configuration is invalid.");
|
||||||
|
}
|
||||||
|
return config;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A unique prefix per run, so repeated runs and parallel jobs never collide and no test depends on
|
||||||
|
* another's leftovers. */
|
||||||
|
function disposablePrefix(): string {
|
||||||
|
return `ekktest${randomBytes(3).toString("hex")}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const openProviders: StorageProvider[] = [];
|
||||||
|
|
||||||
|
function provider(): StorageProvider {
|
||||||
|
const created = createS3StorageProvider(providerConfig());
|
||||||
|
openProviders.push(created);
|
||||||
|
return created;
|
||||||
|
}
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
for (const open of openProviders.splice(0)) {
|
||||||
|
await open.close().catch(() => undefined);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
describe.sequential("live S3-compatible module storage", () => {
|
||||||
|
it("creates a module's bucket idempotently, across both provider dialects", async () => {
|
||||||
|
const shared = provider();
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = physicalBucketName(prefix, "announcements", "announcement-assets");
|
||||||
|
|
||||||
|
await shared.ensureBucket(bucket);
|
||||||
|
// The assertion this whole suite exists for: the pinned SeaweedFS image answers a repeat
|
||||||
|
// `CreateBucket` with `BucketAlreadyExists` rather than AWS's `BucketAlreadyOwnedByYou`, and the
|
||||||
|
// provider must treat both as success. Every restart depends on this.
|
||||||
|
await expect(shared.ensureBucket(bucket)).resolves.toBeUndefined();
|
||||||
|
await expect(shared.ensureBucket(bucket)).resolves.toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("round-trips an object through the scoped facade with its content type preserved", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const storage = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["announcement-assets"],
|
||||||
|
moduleId: "announcements",
|
||||||
|
provider: provider(),
|
||||||
|
});
|
||||||
|
const bucket = storage.bucket("announcement-assets");
|
||||||
|
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
||||||
|
|
||||||
|
const written = await bucket.put("slides/welcome.png", png, { contentType: "image/png" });
|
||||||
|
expect(written.key).toBe("slides/welcome.png");
|
||||||
|
|
||||||
|
const object = await bucket.get("slides/welcome.png");
|
||||||
|
expect(object.contentType).toBe("image/png");
|
||||||
|
expect(object.contentLength).toBe(png.byteLength);
|
||||||
|
// Unquoted for the caller — providers return a quoted ETag, and every module learning to strip
|
||||||
|
// quotes would be a contract failure rather than a module failure.
|
||||||
|
expect(object.etag).toMatch(/^[0-9a-f]{32}$/u);
|
||||||
|
expect(new Uint8Array(await new Response(object.body).arrayBuffer())).toEqual(png);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("streams a body without buffering, when the length is supplied", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["assets"],
|
||||||
|
moduleId: "media",
|
||||||
|
provider: provider(),
|
||||||
|
}).bucket("assets");
|
||||||
|
const payload = "x".repeat(64 * 1024);
|
||||||
|
const stream = new Blob([payload]).stream() as ReadableStream<Uint8Array>;
|
||||||
|
|
||||||
|
await bucket.put("large.bin", stream, {
|
||||||
|
contentLength: payload.length,
|
||||||
|
contentType: "application/octet-stream",
|
||||||
|
});
|
||||||
|
|
||||||
|
const object = await bucket.get("large.bin");
|
||||||
|
expect(object.contentLength).toBe(payload.length);
|
||||||
|
expect(await new Response(object.body).text()).toBe(payload);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns a Web stream a module can hand straight to a Response", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["assets"],
|
||||||
|
moduleId: "announcements",
|
||||||
|
provider: provider(),
|
||||||
|
}).bucket("assets");
|
||||||
|
await bucket.put("welcome.png", new Uint8Array([1, 2, 3]), { contentType: "image/png" });
|
||||||
|
|
||||||
|
const object = await bucket.get("welcome.png");
|
||||||
|
expect(object.body).toBeInstanceOf(ReadableStream);
|
||||||
|
// The browser-delivery shape this pass settles on: browser → module route → scoped storage,
|
||||||
|
// with no S3 knowledge or credential anywhere near the browser.
|
||||||
|
const response = new Response(object.body, {
|
||||||
|
headers: { "content-type": object.contentType ?? "application/octet-stream" },
|
||||||
|
});
|
||||||
|
expect(response.headers.get("content-type")).toBe("image/png");
|
||||||
|
expect(new Uint8Array(await response.arrayBuffer())).toEqual(new Uint8Array([1, 2, 3]));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("heads, lists with a prefix, and deletes", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["assets"],
|
||||||
|
moduleId: "announcements",
|
||||||
|
provider: provider(),
|
||||||
|
}).bucket("assets");
|
||||||
|
|
||||||
|
await bucket.put("slides/a.png", "a", { contentType: "image/png" });
|
||||||
|
await bucket.put("slides/b.png", "b", { contentType: "image/png" });
|
||||||
|
await bucket.put("other/c.txt", "c");
|
||||||
|
|
||||||
|
const metadata = await bucket.head("slides/a.png");
|
||||||
|
expect(metadata.contentType).toBe("image/png");
|
||||||
|
expect(metadata.contentLength).toBe(1);
|
||||||
|
expect(metadata.lastModified).toBeInstanceOf(Date);
|
||||||
|
|
||||||
|
const listing = await bucket.list({ prefix: "slides/" });
|
||||||
|
expect(listing.objects.map((object) => object.key)).toEqual(["slides/a.png", "slides/b.png"]);
|
||||||
|
expect(listing.objects[0]?.contentLength).toBe(1);
|
||||||
|
// A complete listing carries no cursor, so an absent cursor always means "done".
|
||||||
|
expect(listing.cursor).toBeUndefined();
|
||||||
|
|
||||||
|
await bucket.delete("slides/a.png");
|
||||||
|
expect((await bucket.list({ prefix: "slides/" })).objects.map((o) => o.key)).toEqual([
|
||||||
|
"slides/b.png",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("paginates a listing rather than silently truncating it", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["assets"],
|
||||||
|
moduleId: "paged",
|
||||||
|
provider: provider(),
|
||||||
|
}).bucket("assets");
|
||||||
|
for (const index of [0, 1, 2, 3, 4]) {
|
||||||
|
await bucket.put(`object-${index}.txt`, String(index));
|
||||||
|
}
|
||||||
|
|
||||||
|
const first = await bucket.list({ limit: 2 });
|
||||||
|
expect(first.objects).toHaveLength(2);
|
||||||
|
expect(first.cursor).toBeDefined();
|
||||||
|
|
||||||
|
const keys = [...first.objects.map((object) => object.key)];
|
||||||
|
let cursor = first.cursor;
|
||||||
|
while (cursor !== undefined) {
|
||||||
|
const page = await bucket.list({ cursor, limit: 2 });
|
||||||
|
keys.push(...page.objects.map((object) => object.key));
|
||||||
|
cursor = page.cursor;
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(keys.sort()).toEqual([
|
||||||
|
"object-0.txt",
|
||||||
|
"object-1.txt",
|
||||||
|
"object-2.txt",
|
||||||
|
"object-3.txt",
|
||||||
|
"object-4.txt",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps a missing object to a structured not-found error, never a provider exception", async () => {
|
||||||
|
const prefix = disposablePrefix();
|
||||||
|
const bucket = createModuleStorage({
|
||||||
|
bucketPrefix: prefix,
|
||||||
|
logicalBucketIds: ["assets"],
|
||||||
|
moduleId: "announcements",
|
||||||
|
provider: provider(),
|
||||||
|
}).bucket("assets");
|
||||||
|
await bucket.put("present.txt", "x");
|
||||||
|
|
||||||
|
let caught: unknown;
|
||||||
|
try {
|
||||||
|
await bucket.get("absent.txt");
|
||||||
|
} catch (error) {
|
||||||
|
caught = error;
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(caught).toBeInstanceOf(ModuleStorageError);
|
||||||
|
expect((caught as ModuleStorageError).code).toBe("STORAGE_OBJECT_NOT_FOUND");
|
||||||
|
expect((caught as ModuleStorageError).key).toBe("absent.txt");
|
||||||
|
// Nothing from the provider survives translation: no endpoint, no bucket, no request id, no XML.
|
||||||
|
const rendered = `${String(caught)} ${JSON.stringify(caught)}`;
|
||||||
|
expect(rendered).not.toContain("http");
|
||||||
|
expect(rendered).not.toContain(prefix);
|
||||||
|
expect(rendered).not.toContain("<?xml");
|
||||||
|
expect(rendered).not.toContain(process.env[accessKeyEnvironment] ?? " | ||||||