feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049)
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m42s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Failing after 54s
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m42s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Failing after 54s
A separately versioned module is now distributed as one deterministic .tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by Core before anything is installed. This slice implements the portable `any` target, with the module template as the end-to-end proof. ADR 0049 is accepted. @ekklesia/contracts: the Module Package v1 contract. The strict ekklesia.package.json schema (packageFormat 1, identity, target, runtime, provenance, incorporated Ekklesia packages, strict transitional facts, an exact byte-wise-sorted file list with size and SHA-256), target names, and the generated runtime package.json. @ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir> --target any --out <dir> --source-revision <commit>`. - Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle included; only Node built-ins remain imports. Any warning, unresolved import, external dependency, or native or platform-specific package fails. - Copies browser artifacts as built, after proving them current, and the migrations; generates package.json and ekklesia.package.json; never ships src, tests, declarations, source maps, or workspace ranges. - THIRD_PARTY_NOTICES: every third-party package traced from the bundles' own inputs, credited once with its declared licence and every licence, copying, copyright, and notice file it ships. A missing licence fails; a missing text fails unless its standard text is complete without copyright lines (Apache-2.0, vendored, needed by drizzle-orm). - Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime 0, 0755/0644, no links. Byte-identical across runs and machines. - Writes only the archive and its .sha256; never deploys, links, or installs, so the workspace's install state is untouched. Core: verifyModulePackage and `verify-module-package.js <archive> [--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe or escaping paths, and a second top-level directory; checks metadata, every file, identity, target, Node.js range, SDK, and the transitional Drizzle agreement; runs the platform's module and migration discovery; loads the backend once in a forked process with an empty environment; only then extracts, with normalized modes, into a new directory. Errors are sanitized. @ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs and whether the built file is current. The nested stylesheet build now works from the module directory, so artifacts no longer depend on the caller's working directory. The split deploy smoke builds the package with the canonical packager (scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm deploy packaging), verifies it with the same generic image, installs the verified tree, removes it, and reinstalls the exact same archive. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
45d6506c10
commit
35ef31df9f
+25
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env node
|
||||
// Committed launcher, deliberately not generated.
|
||||
//
|
||||
// pnpm creates `.bin` links during `pnpm install`, which runs before Turborepo builds anything. A
|
||||
// `bin` pointing straight at `dist/cli.js` therefore fails to link on a fresh clone — the compiled
|
||||
// file does not exist yet — and the packager would not link in CI. This file always exists,
|
||||
// so the link always succeeds; it defers to the compiled CLI at run time, by which point
|
||||
// `dependsOn: ["^build"]` has produced it.
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const compiledCli = new URL("../dist/cli.js", import.meta.url);
|
||||
|
||||
try {
|
||||
await import(compiledCli.href);
|
||||
} catch (error) {
|
||||
if (error?.code === "ERR_MODULE_NOT_FOUND") {
|
||||
process.stderr.write(
|
||||
`@ekklesia/module-pack is not built yet (${fileURLToPath(compiledCli)}).\n` +
|
||||
"Run `pnpm build` from the repository root first.\n",
|
||||
);
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"name": "@ekklesia/module-pack",
|
||||
"version": "0.1.2",
|
||||
"type": "module",
|
||||
"files": [
|
||||
"bin",
|
||||
"dist",
|
||||
"licenses"
|
||||
],
|
||||
"bin": {
|
||||
"ekklesia-pack-module": "./bin/ekklesia-pack-module.mjs"
|
||||
},
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./dist/index.d.ts",
|
||||
"import": "./dist/index.js"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc --project tsconfig.build.json",
|
||||
"lint": "biome lint .",
|
||||
"prepack": "pnpm --workspace-root exec turbo run build --filter=@ekklesia/module-pack",
|
||||
"test": "vitest run",
|
||||
"typecheck": "tsc --noEmit --project tsconfig.json"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ekklesia/contracts": "workspace:*",
|
||||
"@ekklesia/module-ui-build": "workspace:*",
|
||||
"esbuild": "0.28.1",
|
||||
"semver": "7.8.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@biomejs/biome": "2.5.3",
|
||||
"@ekklesia/core": "workspace:*",
|
||||
"@ekklesia/module-sdk": "workspace:*",
|
||||
"@ekklesia/module-template": "workspace:*",
|
||||
"@types/node": "24.13.3",
|
||||
"@types/semver": "7.7.1",
|
||||
"drizzle-orm": "0.45.2",
|
||||
"typescript": "7.0.2",
|
||||
"vitest": "4.1.10"
|
||||
},
|
||||
"ekklesia": {
|
||||
"layer": "tooling"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
#!/usr/bin/env node
|
||||
import path from "node:path";
|
||||
import process from "node:process";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { formatMessages } from "esbuild";
|
||||
import { ModulePackError, type PackModuleOptions, packModule } from "./pack-module.js";
|
||||
|
||||
/**
|
||||
* `ekklesia-pack-module` — the canonical Ekklesia module packager (ADR 0049).
|
||||
*
|
||||
* ekklesia-pack-module --module <built module directory> --target any --out <directory>
|
||||
* --source-revision <full commit id> [--repository <url>]
|
||||
*
|
||||
* Writes `<id>-<version>-any.tar.gz` and its `.sha256` line to the output directory, and nothing
|
||||
* else anywhere. Run it in the module's CI after building the module.
|
||||
*/
|
||||
|
||||
class PackArgumentsError extends Error {}
|
||||
|
||||
const USAGE =
|
||||
"Usage: ekklesia-pack-module --module <directory> --target any --out <directory> --source-revision <commit> [--repository <url>]";
|
||||
|
||||
const OPTIONS: Readonly<Record<string, keyof PackModuleOptions>> = Object.freeze({
|
||||
"--module": "moduleDirectory",
|
||||
"--out": "outDirectory",
|
||||
"--repository": "repository",
|
||||
"--source-revision": "sourceRevision",
|
||||
"--target": "target",
|
||||
});
|
||||
|
||||
export function parsePackModuleArguments(argv: readonly string[]): PackModuleOptions {
|
||||
const values = new Map<keyof PackModuleOptions, string>();
|
||||
for (let index = 0; index < argv.length; index += 2) {
|
||||
const option = OPTIONS[argv[index] ?? ""];
|
||||
const value = argv[index + 1];
|
||||
if (
|
||||
option === undefined ||
|
||||
value === undefined ||
|
||||
value.startsWith("--") ||
|
||||
values.has(option)
|
||||
) {
|
||||
throw new PackArgumentsError(USAGE);
|
||||
}
|
||||
values.set(option, value);
|
||||
}
|
||||
const required = ["moduleDirectory", "outDirectory", "sourceRevision", "target"] as const;
|
||||
if (required.some((option) => !values.has(option))) {
|
||||
throw new PackArgumentsError(USAGE);
|
||||
}
|
||||
const repository = values.get("repository");
|
||||
return {
|
||||
moduleDirectory: values.get("moduleDirectory") ?? "",
|
||||
outDirectory: values.get("outDirectory") ?? "",
|
||||
...(repository === undefined ? {} : { repository }),
|
||||
sourceRevision: values.get("sourceRevision") ?? "",
|
||||
target: values.get("target") ?? "",
|
||||
};
|
||||
}
|
||||
|
||||
async function main(): Promise<number> {
|
||||
try {
|
||||
const result = await packModule(parsePackModuleArguments(process.argv.slice(2)));
|
||||
process.stdout.write(
|
||||
`Packed ${result.moduleId}@${result.version} (target ${result.target}): ${path.basename(result.archivePath)}, ${result.archiveBytes} bytes, ${result.fileCount} files\n`,
|
||||
);
|
||||
process.stdout.write(`sha256 ${result.sha256}\n`);
|
||||
return 0;
|
||||
} catch (error) {
|
||||
if (error instanceof PackArgumentsError) {
|
||||
process.stderr.write(`${error.message}\n`);
|
||||
return 64;
|
||||
}
|
||||
if (error instanceof ModulePackError) {
|
||||
process.stderr.write(`[${error.code}] ${error.message}\n`);
|
||||
if (error.diagnostics.length > 0) {
|
||||
const formatted = await formatMessages([...error.diagnostics], {
|
||||
color: false,
|
||||
kind: "error",
|
||||
});
|
||||
process.stderr.write(formatted.join(""));
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
process.stderr.write("Module packaging failed unexpectedly.\n");
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
function isDirectExecution(): boolean {
|
||||
const entrypoint = process.argv[1];
|
||||
// Run through the committed launcher, which imports this file, or directly.
|
||||
return (
|
||||
entrypoint !== undefined &&
|
||||
(import.meta.url === pathToFileURL(path.resolve(entrypoint)).href ||
|
||||
path.basename(entrypoint) === "ekklesia-pack-module.mjs" ||
|
||||
path.basename(entrypoint) === "ekklesia-pack-module")
|
||||
);
|
||||
}
|
||||
|
||||
if (isDirectExecution()) {
|
||||
process.exitCode = await main();
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
export * from "./pack-module.js";
|
||||
@@ -0,0 +1,176 @@
|
||||
import { readdir, readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import type { BundledPackage } from "./packages.js";
|
||||
|
||||
/**
|
||||
* `THIRD_PARTY_NOTICES` (ADR 0049): the licence and notice text that redistributing a module
|
||||
* package requires, generated from what was actually bundled.
|
||||
*
|
||||
* - **Discovery.** Every source file in every bundle (the backend, and each browser artifact) is
|
||||
* traced to the package it belongs to. Packages installed under `node_modules` are third-party
|
||||
* software and are credited here. The Ekklesia platform's own packages, and the module itself,
|
||||
* are not third-party software: the platform packages appear in `ekklesia.package.json` under
|
||||
* `incorporates`.
|
||||
* - **Content.** For each third-party package: its name, version, declared licence, where in the
|
||||
* package it is used, and the complete text of every licence, copying, copyright, and notice file
|
||||
* it ships at its root, plus the file a `SEE LICENSE IN` declaration names.
|
||||
* - **Duplicates.** A package bundled into several artifacts is credited once, listing every
|
||||
* artifact kind it is used in. Different packages are never merged, even when their texts match,
|
||||
* so each component keeps its own notice.
|
||||
* - **Gaps fail.** A package that declares no licence, or whose licence cannot be established,
|
||||
* stops packaging. When a package ships no licence file, the standard text of its declared licence
|
||||
* is supplied only if that licence's standard text is complete without project-specific lines
|
||||
* (Apache-2.0). An MIT, ISC, or BSD package without its file is refused, because its copyright line
|
||||
* exists only in that file.
|
||||
*
|
||||
* Everything is sorted, and line endings are normalized, so the file is byte-identical for identical
|
||||
* inputs.
|
||||
*/
|
||||
|
||||
export type NoticeErrorCode = "MODULE_PACK_LICENCE_TEXT_MISSING" | "MODULE_PACK_LICENCE_UNKNOWN";
|
||||
|
||||
export class NoticeError extends Error {
|
||||
readonly code: NoticeErrorCode;
|
||||
readonly component: string;
|
||||
|
||||
constructor(code: NoticeErrorCode, component: string) {
|
||||
super(code);
|
||||
this.name = "NoticeError";
|
||||
this.code = code;
|
||||
this.component = component;
|
||||
}
|
||||
}
|
||||
|
||||
export type NoticeUse = "backend" | "browser";
|
||||
|
||||
export interface NoticeComponent {
|
||||
readonly bundled: BundledPackage;
|
||||
readonly uses: ReadonlySet<NoticeUse>;
|
||||
}
|
||||
|
||||
/** Licences whose standard text is complete without any project-specific line, so supplying it
|
||||
* for a package that ships none is faithful. Checked into this package (`licenses/`). */
|
||||
const STANDARD_TEXTS: Readonly<Record<string, string>> = Object.freeze({
|
||||
"Apache-2.0": "Apache-2.0.txt",
|
||||
});
|
||||
|
||||
const LICENCE_FILE =
|
||||
/^(?:licen[cs]e(?:[-._][a-z0-9.-]+)?|copying(?:[-._][a-z0-9.-]+)?|notice(?:[-._][a-z0-9.-]+)?|copyright(?:[-._][a-z0-9.-]+)?)$/iu;
|
||||
|
||||
const RULE = "-".repeat(80);
|
||||
|
||||
function licenceOf(bundled: BundledPackage): string | undefined {
|
||||
const declared = bundled.manifest.license;
|
||||
if (typeof declared === "string" && declared.trim() !== "") {
|
||||
return declared.trim();
|
||||
}
|
||||
if (typeof declared === "object" && declared !== null && "type" in declared) {
|
||||
const type = (declared as { type?: unknown }).type;
|
||||
return typeof type === "string" && type.trim() !== "" ? type.trim() : undefined;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function normalized(text: string): string {
|
||||
return `${text.replaceAll("\r\n", "\n").replaceAll("\r", "\n").replace(/\s+$/u, "")}\n`;
|
||||
}
|
||||
|
||||
function compare(left: string, right: string): number {
|
||||
return left < right ? -1 : left > right ? 1 : 0;
|
||||
}
|
||||
|
||||
async function licenceTexts(
|
||||
bundled: BundledPackage,
|
||||
licence: string,
|
||||
): Promise<readonly { readonly title: string; readonly text: string }[]> {
|
||||
const names = (await readdir(bundled.root, { withFileTypes: true }))
|
||||
.filter((entry) => entry.isFile() && LICENCE_FILE.test(entry.name))
|
||||
.map((entry) => entry.name)
|
||||
.sort(compare);
|
||||
const seeFile = /^SEE LICEN[CS]E IN (.+)$/iu.exec(licence)?.[1]?.trim();
|
||||
if (seeFile && !names.includes(seeFile)) {
|
||||
const resolved = path.resolve(bundled.root, seeFile);
|
||||
if (resolved.startsWith(`${bundled.root}${path.sep}`)) {
|
||||
names.push(seeFile);
|
||||
}
|
||||
}
|
||||
const texts: { title: string; text: string }[] = [];
|
||||
for (const name of names) {
|
||||
texts.push({
|
||||
text: normalized(await readFile(path.join(bundled.root, name), "utf8")),
|
||||
title: name,
|
||||
});
|
||||
}
|
||||
return texts;
|
||||
}
|
||||
|
||||
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||
|
||||
async function standardText(licence: string): Promise<string | undefined> {
|
||||
const file = STANDARD_TEXTS[licence];
|
||||
return file === undefined
|
||||
? undefined
|
||||
: normalized(await readFile(path.join(packageRoot, "licenses", file), "utf8"));
|
||||
}
|
||||
|
||||
export async function thirdPartyNotices(
|
||||
components: ReadonlyMap<string, NoticeComponent>,
|
||||
): Promise<string> {
|
||||
const ordered = [...components.values()].sort(
|
||||
(left, right) =>
|
||||
compare(left.bundled.name, right.bundled.name) ||
|
||||
compare(left.bundled.version, right.bundled.version),
|
||||
);
|
||||
const sections: string[] = [];
|
||||
for (const { bundled, uses } of ordered) {
|
||||
const component = `${bundled.name}@${bundled.version}`;
|
||||
const licence = licenceOf(bundled);
|
||||
if (licence === undefined || /^UNLICEN[CS]ED$/iu.test(licence)) {
|
||||
throw new NoticeError("MODULE_PACK_LICENCE_UNKNOWN", component);
|
||||
}
|
||||
let texts = await licenceTexts(bundled, licence);
|
||||
// A NOTICE file alone is not the licence: the licence text must still come from somewhere.
|
||||
if (!texts.some((text) => !/^notice/iu.test(text.title))) {
|
||||
const standard = await standardText(licence);
|
||||
if (standard === undefined) {
|
||||
throw new NoticeError("MODULE_PACK_LICENCE_TEXT_MISSING", component);
|
||||
}
|
||||
texts = [
|
||||
...texts,
|
||||
{
|
||||
text: `The package ships no licence file. This is the standard text of the licence it declares (${licence}).\n\n${standard}`,
|
||||
title: `Standard ${licence} text`,
|
||||
},
|
||||
];
|
||||
}
|
||||
sections.push(
|
||||
[
|
||||
RULE,
|
||||
`${bundled.name} ${bundled.version}`,
|
||||
`License: ${licence}`,
|
||||
`Used in: ${[...uses].sort(compare).join(", ")}`,
|
||||
RULE,
|
||||
"",
|
||||
...texts.flatMap((text) => [`[${text.title}]`, "", text.text]),
|
||||
].join("\n"),
|
||||
);
|
||||
}
|
||||
|
||||
const header = [
|
||||
"THIRD-PARTY SOFTWARE NOTICES",
|
||||
"",
|
||||
"This module package includes the third-party software listed below. Each entry names the",
|
||||
"component, its version, its declared licence, where in the package it is used, and the licence",
|
||||
"and notice text that came with it.",
|
||||
"",
|
||||
"The Ekklesia platform's own packages are not third-party software. They are listed in",
|
||||
'ekklesia.package.json under "incorporates".',
|
||||
"",
|
||||
ordered.length === 0
|
||||
? "This module package includes no third-party software."
|
||||
: `${ordered.length} ${ordered.length === 1 ? "component" : "components"}.`,
|
||||
"",
|
||||
].join("\n");
|
||||
return `${[header, ...sections].join("\n").replace(/\n+$/u, "")}\n`;
|
||||
}
|
||||
@@ -0,0 +1,588 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { lstat, mkdir, readdir, readFile, realpath, rename, writeFile } from "node:fs/promises";
|
||||
import { isBuiltin } from "node:module";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import {
|
||||
MODULE_PACKAGE_ANY_TARGET,
|
||||
MODULE_PACKAGE_FORMAT,
|
||||
MODULE_PACKAGE_METADATA_FILENAME,
|
||||
MODULE_PACKAGE_NOTICES_FILENAME,
|
||||
type ModuleManifest,
|
||||
modulePackageArchiveFileName,
|
||||
modulePackageRuntimePackage,
|
||||
safeParseModuleManifest,
|
||||
safeParseModulePackageMetadata,
|
||||
} from "@ekklesia/contracts";
|
||||
import { analyzeModuleUi, ModuleUiBuildError } from "@ekklesia/module-ui-build";
|
||||
import { build, type Message, type Plugin } from "esbuild";
|
||||
import { minVersion, satisfies, validRange } from "semver";
|
||||
import { NoticeError, type NoticeComponent, type NoticeUse, thirdPartyNotices } from "./notices.js";
|
||||
import { BundledPackageIndex, isNativeOrPlatformSpecific } from "./packages.js";
|
||||
import { deterministicTarGz, TarPathError } from "./tar.js";
|
||||
|
||||
/**
|
||||
* The canonical Ekklesia module packager (ADR 0049): turns a built module directory into one
|
||||
* deterministic Module Package v1 archive and its `.sha256` line.
|
||||
*
|
||||
* It reads the module directory and the dependencies its workspace installed, and writes only the
|
||||
* two output files. It never installs, links, or deploys anything, so the workspace it runs in is
|
||||
* left exactly as it was. In this release it produces the portable `any` target only.
|
||||
*/
|
||||
|
||||
export type ModulePackErrorCode =
|
||||
| "MODULE_PACK_BUNDLE_FAILED"
|
||||
| "MODULE_PACK_DATABASE_RUNTIME_CONFLICT"
|
||||
| "MODULE_PACK_ENTRYPOINT_MISSING"
|
||||
| "MODULE_PACK_ENTRYPOINT_UNSUPPORTED"
|
||||
| "MODULE_PACK_EXTERNAL_DEPENDENCY"
|
||||
| "MODULE_PACK_FORBIDDEN_FILE"
|
||||
| "MODULE_PACK_IDENTITY_MISMATCH"
|
||||
| "MODULE_PACK_INCORPORATED_CONFLICT"
|
||||
| "MODULE_PACK_LAYOUT_CONFLICT"
|
||||
| "MODULE_PACK_LICENCE_TEXT_MISSING"
|
||||
| "MODULE_PACK_LICENCE_UNKNOWN"
|
||||
| "MODULE_PACK_MANIFEST_INVALID"
|
||||
| "MODULE_PACK_METADATA_INVALID"
|
||||
| "MODULE_PACK_MIGRATIONS_INVALID"
|
||||
| "MODULE_PACK_NATIVE_DEPENDENCY"
|
||||
| "MODULE_PACK_OUTPUT_FAILED"
|
||||
| "MODULE_PACK_PATH_UNSUPPORTED"
|
||||
| "MODULE_PACK_REVISION_INVALID"
|
||||
| "MODULE_PACK_RUNTIME_UNKNOWN"
|
||||
| "MODULE_PACK_SDK_INCOMPATIBLE"
|
||||
| "MODULE_PACK_TARGET_UNSUPPORTED"
|
||||
| "MODULE_PACK_UI_STALE";
|
||||
|
||||
const messages: Readonly<Record<ModulePackErrorCode, string>> = Object.freeze({
|
||||
MODULE_PACK_BUNDLE_FAILED:
|
||||
"The backend could not be bundled into self-contained files. Every import must resolve and bundle cleanly.",
|
||||
MODULE_PACK_DATABASE_RUNTIME_CONFLICT: "The backend bundles more than one Drizzle version.",
|
||||
MODULE_PACK_ENTRYPOINT_MISSING:
|
||||
"A declared backend entrypoint does not exist. Build the module before packaging it.",
|
||||
MODULE_PACK_ENTRYPOINT_UNSUPPORTED:
|
||||
"Backend entrypoints must be compiled .js files to be bundled into a Module Package v1.",
|
||||
MODULE_PACK_EXTERNAL_DEPENDENCY:
|
||||
"The backend bundle would import a dependency at run time; an any-target package bundles everything.",
|
||||
MODULE_PACK_FORBIDDEN_FILE:
|
||||
"The package would contain a source, test, declaration, source-map, or dependency file.",
|
||||
MODULE_PACK_IDENTITY_MISMATCH:
|
||||
"package.json must declare a name and the same version as ekklesia.module.json.",
|
||||
MODULE_PACK_INCORPORATED_CONFLICT:
|
||||
"The package bundles more than one version of an Ekklesia package.",
|
||||
MODULE_PACK_LAYOUT_CONFLICT: "Two package files would have the same path.",
|
||||
MODULE_PACK_LICENCE_TEXT_MISSING:
|
||||
"A bundled package ships no licence text, and its licence has no complete standard text to supply.",
|
||||
MODULE_PACK_LICENCE_UNKNOWN: "A bundled package declares no licence that permits redistribution.",
|
||||
MODULE_PACK_MANIFEST_INVALID: "ekklesia.module.json is missing or invalid.",
|
||||
MODULE_PACK_METADATA_INVALID: "The generated package metadata failed its own contract.",
|
||||
MODULE_PACK_MIGRATIONS_INVALID:
|
||||
"The migrations directory must hold only regular files and directories.",
|
||||
MODULE_PACK_NATIVE_DEPENDENCY:
|
||||
"The backend bundles a native or platform-specific dependency, which an any-target package cannot contain.",
|
||||
MODULE_PACK_OUTPUT_FAILED: "The package could not be written to the output directory.",
|
||||
MODULE_PACK_PATH_UNSUPPORTED:
|
||||
"A package path is not portable printable ASCII that fits the archive format.",
|
||||
MODULE_PACK_REVISION_INVALID: "The source revision must be a full 40- or 64-character commit id.",
|
||||
MODULE_PACK_RUNTIME_UNKNOWN:
|
||||
"The Node.js range could not be determined. Run inside an Ekklesia workspace, or pass one.",
|
||||
MODULE_PACK_SDK_INCOMPATIBLE:
|
||||
"The bundled Module SDK is outside the manifest's own compatibility.moduleSdk range.",
|
||||
MODULE_PACK_TARGET_UNSUPPORTED: "This packager produces the any target only.",
|
||||
MODULE_PACK_UI_STALE:
|
||||
"A declared browser artifact is missing or out of date. Build the module before packaging it.",
|
||||
});
|
||||
|
||||
export class ModulePackError extends Error {
|
||||
readonly code: ModulePackErrorCode;
|
||||
/** The package or path the failure concerns, when there is one. A module author's own build
|
||||
* input, never a secret. */
|
||||
readonly detail?: string;
|
||||
/** esbuild's structured diagnostics, for a command line to print. */
|
||||
readonly diagnostics: readonly Message[];
|
||||
|
||||
constructor(code: ModulePackErrorCode, detail?: string, diagnostics: readonly Message[] = []) {
|
||||
super(detail === undefined ? messages[code] : `${messages[code]} (${detail})`);
|
||||
this.name = "ModulePackError";
|
||||
this.code = code;
|
||||
if (detail !== undefined) {
|
||||
this.detail = detail;
|
||||
}
|
||||
this.diagnostics = diagnostics;
|
||||
}
|
||||
}
|
||||
|
||||
export interface PackModuleOptions {
|
||||
/** A built module directory: its manifest, package.json, and compiled entrypoints. */
|
||||
readonly moduleDirectory: string;
|
||||
/** `any` only, in this release. */
|
||||
readonly target: string;
|
||||
readonly outDirectory: string;
|
||||
/** The full commit id the module was built from. */
|
||||
readonly sourceRevision: string;
|
||||
/** Informational only; recorded as given. */
|
||||
readonly repository?: string;
|
||||
/** The Node.js range the package is built for. Defaults to the enclosing workspace's
|
||||
* `engines.node`, the toolchain the module was built and tested with. */
|
||||
readonly nodeRange?: string;
|
||||
}
|
||||
|
||||
export interface ModulePackResult {
|
||||
readonly archivePath: string;
|
||||
readonly sha256Path: string;
|
||||
readonly sha256: string;
|
||||
readonly archiveBytes: number;
|
||||
readonly moduleId: string;
|
||||
readonly version: string;
|
||||
readonly target: string;
|
||||
/** Payload files, the metadata file not included. */
|
||||
readonly fileCount: number;
|
||||
}
|
||||
|
||||
const NATIVE_GUARD = "ekklesia-native-addon-guard";
|
||||
|
||||
/** Refuses a compiled addon the moment the bundle reaches one. */
|
||||
const nativeAddonGuard: Plugin = {
|
||||
name: NATIVE_GUARD,
|
||||
setup(pluginBuild) {
|
||||
pluginBuild.onLoad({ filter: /\.node$/i }, () => ({
|
||||
errors: [{ text: "A compiled Node.js addon cannot be bundled into an any-target package." }],
|
||||
}));
|
||||
},
|
||||
};
|
||||
|
||||
/** Bundled CommonJS code that requires a Node built-in needs a `require` in module scope. Nothing
|
||||
* else can be required at run time: every other dependency is inside the bundle, and the packager
|
||||
* refuses a bundle that would import anything but a built-in. */
|
||||
const REQUIRE_BANNER =
|
||||
'import { createRequire as __ekklesiaCreateRequire } from "node:module";\nconst require = __ekklesiaCreateRequire(import.meta.url);';
|
||||
|
||||
function compare(left: string, right: string): number {
|
||||
return left < right ? -1 : left > right ? 1 : 0;
|
||||
}
|
||||
|
||||
function sha256(data: Uint8Array): string {
|
||||
return createHash("sha256").update(data).digest("hex");
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
async function readJson(file: string): Promise<unknown> {
|
||||
return JSON.parse(await readFile(file, "utf8")) as unknown;
|
||||
}
|
||||
|
||||
const ownManifestPath = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
"..",
|
||||
"package.json",
|
||||
);
|
||||
|
||||
async function packagerIdentity(): Promise<{ readonly name: string; readonly version: string }> {
|
||||
const manifest = (await readJson(ownManifestPath)) as Record<string, unknown>;
|
||||
return { name: String(manifest.name), version: String(manifest.version) };
|
||||
}
|
||||
|
||||
async function workspaceNodeRange(moduleDirectory: string): Promise<string> {
|
||||
let directory = moduleDirectory;
|
||||
for (;;) {
|
||||
try {
|
||||
await lstat(path.join(directory, "pnpm-workspace.yaml"));
|
||||
const manifest = await readJson(path.join(directory, "package.json"));
|
||||
const engines = isRecord(manifest) ? manifest.engines : undefined;
|
||||
const range = isRecord(engines) ? engines.node : undefined;
|
||||
if (typeof range === "string") {
|
||||
return range;
|
||||
}
|
||||
break;
|
||||
} catch {
|
||||
const parent = path.dirname(directory);
|
||||
if (parent === directory) {
|
||||
break;
|
||||
}
|
||||
directory = parent;
|
||||
}
|
||||
}
|
||||
throw new ModulePackError("MODULE_PACK_RUNTIME_UNKNOWN");
|
||||
}
|
||||
|
||||
/** `./dist/server/index.js` → `dist/server/index.js`, inside the module directory. */
|
||||
function moduleRelative(declared: string): string {
|
||||
return path.posix.normalize(declared.replace(/^\.\//u, ""));
|
||||
}
|
||||
|
||||
function portable(relative: string): boolean {
|
||||
const segments = relative.split("/");
|
||||
return (
|
||||
/^[\x20-\x7e]+$/u.test(relative) &&
|
||||
!relative.includes("\\") &&
|
||||
segments.every((segment) => segment !== "" && segment !== "." && segment !== "..")
|
||||
);
|
||||
}
|
||||
|
||||
function forbidden(relative: string): boolean {
|
||||
return (
|
||||
/(^|\/)(src|tests?|node_modules)\//u.test(relative) ||
|
||||
/\.(map|d\.ts|d\.mts|d\.cts)$/u.test(relative)
|
||||
);
|
||||
}
|
||||
|
||||
async function readTree(root: string, relative: string, files: Map<string, Buffer>): Promise<void> {
|
||||
const directory = path.join(root, relative);
|
||||
for (const entry of (await readdir(directory, { withFileTypes: true })).sort((left, right) =>
|
||||
compare(left.name, right.name),
|
||||
)) {
|
||||
const childRelative = `${relative}/${entry.name}`;
|
||||
const stats = await lstat(path.join(directory, entry.name));
|
||||
if (stats.isDirectory()) {
|
||||
await readTree(root, childRelative, files);
|
||||
} else if (stats.isFile()) {
|
||||
files.set(childRelative, await readFile(path.join(directory, entry.name)));
|
||||
} else {
|
||||
throw new ModulePackError("MODULE_PACK_MIGRATIONS_INVALID", childRelative);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
interface BackendBundle {
|
||||
readonly outputs: ReadonlyMap<string, Buffer>;
|
||||
readonly inputs: readonly string[];
|
||||
}
|
||||
|
||||
async function bundleBackend(
|
||||
moduleDirectory: string,
|
||||
manifest: ModuleManifest,
|
||||
nodeRange: string,
|
||||
): Promise<BackendBundle> {
|
||||
const declared = [
|
||||
manifest.entrypoints.server,
|
||||
manifest.entrypoints.schema,
|
||||
manifest.entrypoints.jobs,
|
||||
].filter((entrypoint): entrypoint is string => entrypoint !== undefined);
|
||||
const entryPoints: { in: string; out: string }[] = [];
|
||||
for (const entrypoint of declared) {
|
||||
const relative = moduleRelative(entrypoint);
|
||||
if (!relative.endsWith(".js")) {
|
||||
throw new ModulePackError("MODULE_PACK_ENTRYPOINT_UNSUPPORTED", relative);
|
||||
}
|
||||
try {
|
||||
if (!(await lstat(path.join(moduleDirectory, relative))).isFile()) {
|
||||
throw new Error("not a file");
|
||||
}
|
||||
} catch {
|
||||
throw new ModulePackError("MODULE_PACK_ENTRYPOINT_MISSING", relative);
|
||||
}
|
||||
entryPoints.push({ in: `./${relative}`, out: relative.slice(0, -".js".length) });
|
||||
}
|
||||
|
||||
const minimum = minVersion(nodeRange);
|
||||
let result: Awaited<ReturnType<typeof build>>;
|
||||
try {
|
||||
result = await build({
|
||||
absWorkingDir: moduleDirectory,
|
||||
banner: { js: REQUIRE_BANNER },
|
||||
bundle: true,
|
||||
chunkNames: "dist/chunks/[name]-[hash]",
|
||||
entryPoints,
|
||||
format: "esm",
|
||||
// Licence comments stay in the code as well; THIRD_PARTY_NOTICES is what satisfies the
|
||||
// licences, never these comments alone.
|
||||
legalComments: "eof",
|
||||
logLevel: "silent",
|
||||
metafile: true,
|
||||
minify: false,
|
||||
outdir: moduleDirectory,
|
||||
platform: "node",
|
||||
plugins: [nativeAddonGuard],
|
||||
sourcemap: false,
|
||||
splitting: true,
|
||||
target: `node${minimum?.version ?? "24"}`,
|
||||
write: false,
|
||||
});
|
||||
} catch (error) {
|
||||
const errors =
|
||||
isRecord(error) && Array.isArray(error.errors) ? (error.errors as Message[]) : [];
|
||||
const native = errors.some((message) => message.pluginName === NATIVE_GUARD);
|
||||
throw new ModulePackError(
|
||||
native ? "MODULE_PACK_NATIVE_DEPENDENCY" : "MODULE_PACK_BUNDLE_FAILED",
|
||||
undefined,
|
||||
errors,
|
||||
);
|
||||
}
|
||||
// A warning is a bundle that might not run: a require it could not follow, an import it could
|
||||
// not see. An installed module has nothing to fall back on, so a warning fails packaging.
|
||||
if (result.warnings.length > 0) {
|
||||
throw new ModulePackError("MODULE_PACK_BUNDLE_FAILED", undefined, result.warnings);
|
||||
}
|
||||
const { metafile, outputFiles } = result;
|
||||
if (metafile === undefined || outputFiles === undefined) {
|
||||
throw new ModulePackError("MODULE_PACK_BUNDLE_FAILED");
|
||||
}
|
||||
for (const output of Object.values(metafile.outputs)) {
|
||||
for (const imported of output.imports) {
|
||||
if (imported.external && !isBuiltin(imported.path)) {
|
||||
throw new ModulePackError("MODULE_PACK_EXTERNAL_DEPENDENCY", imported.path);
|
||||
}
|
||||
}
|
||||
}
|
||||
const outputs = new Map<string, Buffer>();
|
||||
for (const outputFile of outputFiles) {
|
||||
outputs.set(
|
||||
path.relative(moduleDirectory, outputFile.path).split(path.sep).join("/"),
|
||||
Buffer.from(outputFile.contents),
|
||||
);
|
||||
}
|
||||
return {
|
||||
inputs: Object.keys(metafile.inputs).map((input) => path.resolve(moduleDirectory, input)),
|
||||
outputs,
|
||||
};
|
||||
}
|
||||
|
||||
export async function packModule(options: PackModuleOptions): Promise<ModulePackResult> {
|
||||
if (options.target !== MODULE_PACKAGE_ANY_TARGET) {
|
||||
throw new ModulePackError("MODULE_PACK_TARGET_UNSUPPORTED", options.target);
|
||||
}
|
||||
if (!/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/u.test(options.sourceRevision)) {
|
||||
throw new ModulePackError("MODULE_PACK_REVISION_INVALID");
|
||||
}
|
||||
const moduleDirectory = await realpath(path.resolve(options.moduleDirectory)).catch(() => {
|
||||
throw new ModulePackError("MODULE_PACK_MANIFEST_INVALID");
|
||||
});
|
||||
|
||||
const manifestBytes = await readFile(path.join(moduleDirectory, "ekklesia.module.json")).catch(
|
||||
() => {
|
||||
throw new ModulePackError("MODULE_PACK_MANIFEST_INVALID");
|
||||
},
|
||||
);
|
||||
let manifestInput: unknown;
|
||||
try {
|
||||
manifestInput = JSON.parse(manifestBytes.toString("utf8"));
|
||||
} catch {
|
||||
throw new ModulePackError("MODULE_PACK_MANIFEST_INVALID");
|
||||
}
|
||||
const parsedManifest = safeParseModuleManifest(manifestInput);
|
||||
if (!parsedManifest.success) {
|
||||
throw new ModulePackError("MODULE_PACK_MANIFEST_INVALID");
|
||||
}
|
||||
const manifest = parsedManifest.data;
|
||||
const developmentPackage = await readJson(path.join(moduleDirectory, "package.json")).catch(
|
||||
() => undefined,
|
||||
);
|
||||
if (
|
||||
!isRecord(developmentPackage) ||
|
||||
typeof developmentPackage.name !== "string" ||
|
||||
developmentPackage.name === "" ||
|
||||
developmentPackage.version !== manifest.version
|
||||
) {
|
||||
throw new ModulePackError("MODULE_PACK_IDENTITY_MISMATCH");
|
||||
}
|
||||
|
||||
const nodeRange = options.nodeRange ?? (await workspaceNodeRange(moduleDirectory));
|
||||
if (validRange(nodeRange) === null) {
|
||||
throw new ModulePackError("MODULE_PACK_RUNTIME_UNKNOWN");
|
||||
}
|
||||
|
||||
// 1. The backend, bundled with every pure-JavaScript dependency it uses.
|
||||
const backend = await bundleBackend(moduleDirectory, manifest, nodeRange);
|
||||
|
||||
// 2. The browser artifacts, exactly as built, with what went into each.
|
||||
let ui: Awaited<ReturnType<typeof analyzeModuleUi>>;
|
||||
try {
|
||||
ui = await analyzeModuleUi({ moduleDirectory });
|
||||
} catch (error) {
|
||||
throw new ModulePackError(
|
||||
"MODULE_PACK_UI_STALE",
|
||||
error instanceof ModuleUiBuildError ? error.code : undefined,
|
||||
);
|
||||
}
|
||||
|
||||
// 3. Every bundled source file traced to its package.
|
||||
const index = new BundledPackageIndex();
|
||||
const components = new Map<
|
||||
string,
|
||||
{ bundled: Awaited<ReturnType<typeof index.packageOf>>; uses: Set<NoticeUse> }
|
||||
>();
|
||||
const incorporates = new Map<string, string>();
|
||||
const drizzleVersions = new Set<string>();
|
||||
const sources: { readonly file: string; readonly use: NoticeUse }[] = [
|
||||
...backend.inputs.map((file) => ({ file, use: "backend" as const })),
|
||||
...ui.artifacts.flatMap((artifact) =>
|
||||
artifact.inputs.map((file) => ({ file, use: "browser" as const })),
|
||||
),
|
||||
];
|
||||
for (const { file, use } of sources) {
|
||||
const bundled = await index.packageOf(file);
|
||||
if (bundled === undefined || bundled.root === moduleDirectory) {
|
||||
continue;
|
||||
}
|
||||
if (!bundled.thirdParty) {
|
||||
if (bundled.name.startsWith("@ekklesia/")) {
|
||||
const known = incorporates.get(bundled.name);
|
||||
if (known !== undefined && known !== bundled.version) {
|
||||
throw new ModulePackError("MODULE_PACK_INCORPORATED_CONFLICT", bundled.name);
|
||||
}
|
||||
incorporates.set(bundled.name, bundled.version);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (use === "backend" && bundled.name === "drizzle-orm") {
|
||||
drizzleVersions.add(bundled.version);
|
||||
}
|
||||
const key = `${bundled.name}@${bundled.version}`;
|
||||
const existing = components.get(key);
|
||||
if (existing) {
|
||||
existing.uses.add(use);
|
||||
} else {
|
||||
if (use === "backend" && (await isNativeOrPlatformSpecific(bundled))) {
|
||||
throw new ModulePackError("MODULE_PACK_NATIVE_DEPENDENCY", key);
|
||||
}
|
||||
components.set(key, { bundled, uses: new Set([use]) });
|
||||
}
|
||||
}
|
||||
if (drizzleVersions.size > 1) {
|
||||
throw new ModulePackError("MODULE_PACK_DATABASE_RUNTIME_CONFLICT");
|
||||
}
|
||||
const sdk = incorporates.get("@ekklesia/module-sdk");
|
||||
if (sdk !== undefined && !satisfies(sdk, manifest.compatibility.moduleSdk)) {
|
||||
throw new ModulePackError("MODULE_PACK_SDK_INCOMPATIBLE", sdk);
|
||||
}
|
||||
|
||||
// 4. The payload.
|
||||
const files = new Map<string, Buffer>();
|
||||
const add = (relative: string, data: Buffer): void => {
|
||||
if (!portable(relative)) {
|
||||
throw new ModulePackError("MODULE_PACK_PATH_UNSUPPORTED", relative);
|
||||
}
|
||||
if (forbidden(relative)) {
|
||||
throw new ModulePackError("MODULE_PACK_FORBIDDEN_FILE", relative);
|
||||
}
|
||||
if (files.has(relative) || relative === MODULE_PACKAGE_METADATA_FILENAME) {
|
||||
throw new ModulePackError("MODULE_PACK_LAYOUT_CONFLICT", relative);
|
||||
}
|
||||
files.set(relative, data);
|
||||
};
|
||||
for (const [relative, data] of backend.outputs) {
|
||||
add(relative, data);
|
||||
}
|
||||
for (const artifact of ui.artifacts) {
|
||||
if (!artifact.builtArtifactCurrent) {
|
||||
throw new ModulePackError(
|
||||
"MODULE_PACK_UI_STALE",
|
||||
path.relative(moduleDirectory, artifact.outputFile),
|
||||
);
|
||||
}
|
||||
add(
|
||||
path.relative(moduleDirectory, artifact.outputFile).split(path.sep).join("/"),
|
||||
await readFile(artifact.outputFile),
|
||||
);
|
||||
}
|
||||
if (manifest.entrypoints.migrations !== undefined) {
|
||||
const migrations = new Map<string, Buffer>();
|
||||
try {
|
||||
await readTree(moduleDirectory, moduleRelative(manifest.entrypoints.migrations), migrations);
|
||||
} catch (error) {
|
||||
if (error instanceof ModulePackError) {
|
||||
throw error;
|
||||
}
|
||||
throw new ModulePackError("MODULE_PACK_MIGRATIONS_INVALID");
|
||||
}
|
||||
for (const [relative, data] of migrations) {
|
||||
add(relative, data);
|
||||
}
|
||||
}
|
||||
add("ekklesia.module.json", manifestBytes);
|
||||
add(
|
||||
"package.json",
|
||||
Buffer.from(
|
||||
`${JSON.stringify(
|
||||
modulePackageRuntimePackage({ name: developmentPackage.name, version: manifest.version }),
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
),
|
||||
);
|
||||
try {
|
||||
add(
|
||||
MODULE_PACKAGE_NOTICES_FILENAME,
|
||||
Buffer.from(await thirdPartyNotices(components as ReadonlyMap<string, NoticeComponent>)),
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof NoticeError) {
|
||||
throw new ModulePackError(error.code, error.component);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
// 5. The metadata, checked against its own contract before anything is written.
|
||||
const listed = [...files.entries()]
|
||||
.sort(([left], [right]) => compare(left, right))
|
||||
.map(([relative, data]) => ({ path: relative, sha256: sha256(data), size: data.length }));
|
||||
const metadata = {
|
||||
packageFormat: MODULE_PACKAGE_FORMAT,
|
||||
module: { id: manifest.id, version: manifest.version },
|
||||
target: MODULE_PACKAGE_ANY_TARGET,
|
||||
runtime: { node: nodeRange },
|
||||
provenance: {
|
||||
revision: options.sourceRevision,
|
||||
...(options.repository === undefined ? {} : { repository: options.repository }),
|
||||
packager: await packagerIdentity(),
|
||||
},
|
||||
incorporates: Object.fromEntries(
|
||||
[...incorporates].sort(([left], [right]) => compare(left, right)),
|
||||
),
|
||||
...(drizzleVersions.size === 1
|
||||
? { transitional: { databaseRuntime: { "drizzle-orm": [...drizzleVersions][0] } } }
|
||||
: {}),
|
||||
files: listed.map((file) => ({ path: file.path, size: file.size, sha256: file.sha256 })),
|
||||
};
|
||||
if (!safeParseModulePackageMetadata(metadata).success) {
|
||||
throw new ModulePackError("MODULE_PACK_METADATA_INVALID");
|
||||
}
|
||||
const payload = new Map(files);
|
||||
payload.set(
|
||||
MODULE_PACKAGE_METADATA_FILENAME,
|
||||
Buffer.from(`${JSON.stringify(metadata, null, 2)}\n`),
|
||||
);
|
||||
|
||||
// 6. The archive and its digest.
|
||||
let archive: Buffer;
|
||||
try {
|
||||
archive = deterministicTarGz(manifest.id, payload);
|
||||
} catch (error) {
|
||||
if (error instanceof TarPathError) {
|
||||
throw new ModulePackError("MODULE_PACK_PATH_UNSUPPORTED", error.path);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const digest = sha256(archive);
|
||||
const fileName = modulePackageArchiveFileName(
|
||||
manifest.id,
|
||||
manifest.version,
|
||||
MODULE_PACKAGE_ANY_TARGET,
|
||||
);
|
||||
const outDirectory = path.resolve(options.outDirectory);
|
||||
const archivePath = path.join(outDirectory, fileName);
|
||||
const sha256Path = `${archivePath}.sha256`;
|
||||
try {
|
||||
await mkdir(outDirectory, { recursive: true });
|
||||
const staging = `${archivePath}.partial-${process.pid}`;
|
||||
await writeFile(staging, archive);
|
||||
await rename(staging, archivePath);
|
||||
await writeFile(`${sha256Path}.partial-${process.pid}`, `${digest} ${fileName}\n`);
|
||||
await rename(`${sha256Path}.partial-${process.pid}`, sha256Path);
|
||||
} catch {
|
||||
throw new ModulePackError("MODULE_PACK_OUTPUT_FAILED");
|
||||
}
|
||||
|
||||
return Object.freeze({
|
||||
archiveBytes: archive.length,
|
||||
archivePath,
|
||||
fileCount: listed.length,
|
||||
moduleId: manifest.id,
|
||||
sha256: digest,
|
||||
sha256Path,
|
||||
target: MODULE_PACKAGE_ANY_TARGET,
|
||||
version: manifest.version,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
import { access, readdir, readFile, realpath } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
/**
|
||||
* What the packager knows about a package whose files were bundled: where it is, what it is called,
|
||||
* and whether it is third-party software. A package installed under a `node_modules` directory is
|
||||
* third-party; one that is not (a workspace package, or the module itself) is the platform's or the
|
||||
* module's own code.
|
||||
*/
|
||||
export interface BundledPackage {
|
||||
readonly root: string;
|
||||
readonly name: string;
|
||||
readonly version: string;
|
||||
readonly thirdParty: boolean;
|
||||
readonly manifest: Readonly<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
}
|
||||
|
||||
async function readManifest(directory: string): Promise<Record<string, unknown> | undefined> {
|
||||
try {
|
||||
const parsed = JSON.parse(
|
||||
await readFile(path.join(directory, "package.json"), "utf8"),
|
||||
) as unknown;
|
||||
return isRecord(parsed) ? parsed : undefined;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/** Finds the package a bundled file belongs to: the nearest enclosing directory whose package.json
|
||||
* names a package and its version. Nested `package.json` files that only set a module type, as
|
||||
* some libraries ship in subdirectories, are skipped. */
|
||||
export class BundledPackageIndex {
|
||||
readonly #byDirectory = new Map<string, BundledPackage | undefined>();
|
||||
|
||||
async packageOf(file: string): Promise<BundledPackage | undefined> {
|
||||
let directory = path.dirname(await realpath(file).catch(() => file));
|
||||
const visited: string[] = [];
|
||||
let found: BundledPackage | undefined;
|
||||
for (;;) {
|
||||
if (this.#byDirectory.has(directory)) {
|
||||
found = this.#byDirectory.get(directory);
|
||||
break;
|
||||
}
|
||||
visited.push(directory);
|
||||
const manifest = await readManifest(directory);
|
||||
if (manifest && typeof manifest.name === "string" && typeof manifest.version === "string") {
|
||||
found = Object.freeze({
|
||||
manifest: Object.freeze(manifest),
|
||||
name: manifest.name,
|
||||
root: directory,
|
||||
thirdParty: directory.split(path.sep).includes("node_modules"),
|
||||
version: manifest.version,
|
||||
});
|
||||
break;
|
||||
}
|
||||
const parent = path.dirname(directory);
|
||||
if (parent === directory) {
|
||||
break;
|
||||
}
|
||||
directory = parent;
|
||||
}
|
||||
for (const seen of visited) {
|
||||
this.#byDirectory.set(seen, found);
|
||||
}
|
||||
return found;
|
||||
}
|
||||
}
|
||||
|
||||
async function exists(file: string): Promise<boolean> {
|
||||
try {
|
||||
await access(file);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function declaresPlatform(manifest: Readonly<Record<string, unknown>>): boolean {
|
||||
return ["os", "cpu", "libc"].some((field) => manifest[field] !== undefined);
|
||||
}
|
||||
|
||||
async function containsCompiledAddon(directory: string): Promise<boolean> {
|
||||
const entries = await readdir(directory, { withFileTypes: true }).catch(() => []);
|
||||
for (const entry of entries) {
|
||||
if (entry.isDirectory()) {
|
||||
if (
|
||||
entry.name !== "node_modules" &&
|
||||
(await containsCompiledAddon(path.join(directory, entry.name)))
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
} else if (entry.name.toLowerCase().endsWith(".node")) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** The installed package a dependency name resolves to from `root`, by Node's own directory walk. */
|
||||
async function installedDependency(
|
||||
root: string,
|
||||
name: string,
|
||||
): Promise<Record<string, unknown> | undefined> {
|
||||
let directory = root;
|
||||
for (;;) {
|
||||
if (path.basename(directory) !== "node_modules") {
|
||||
const manifest = await readManifest(path.join(directory, "node_modules", ...name.split("/")));
|
||||
if (manifest) {
|
||||
return manifest;
|
||||
}
|
||||
}
|
||||
const parent = path.dirname(directory);
|
||||
if (parent === directory) {
|
||||
return undefined;
|
||||
}
|
||||
directory = parent;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a bundled package is native or platform-specific, which an `any` package may never
|
||||
* contain (ADR 0049): it declares an operating system, CPU, or C library; it is built with node-gyp;
|
||||
* it ships a compiled `.node` addon; or one of its installed dependencies declares a platform, as
|
||||
* packages that load a per-platform binary do.
|
||||
*/
|
||||
export async function isNativeOrPlatformSpecific(bundled: BundledPackage): Promise<boolean> {
|
||||
const manifest = bundled.manifest;
|
||||
if (declaresPlatform(manifest) || manifest.gypfile === true) {
|
||||
return true;
|
||||
}
|
||||
if (await exists(path.join(bundled.root, "binding.gyp"))) {
|
||||
return true;
|
||||
}
|
||||
if (await containsCompiledAddon(bundled.root)) {
|
||||
return true;
|
||||
}
|
||||
for (const field of ["dependencies", "optionalDependencies"]) {
|
||||
const dependencies = manifest[field];
|
||||
if (!isRecord(dependencies)) {
|
||||
continue;
|
||||
}
|
||||
for (const name of Object.keys(dependencies)) {
|
||||
const dependency = await installedDependency(bundled.root, name);
|
||||
if (dependency && declaresPlatform(dependency)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
import { gzipSync } from "node:zlib";
|
||||
|
||||
/**
|
||||
* The deterministic archive encoding of Ekklesia Module Package v1 (ADR 0049).
|
||||
*
|
||||
* POSIX ustar, written by hand rather than by a general tar library, so every byte is decided here:
|
||||
* - one top-level directory, then every directory and regular file in byte-wise path order, which
|
||||
* puts every directory before what it contains;
|
||||
* - uid and gid 0, empty user and group names, modification time 0, device fields 0;
|
||||
* - directories 0755 and files 0644; no links, no special files, no extended or long-name headers;
|
||||
* - exactly two zero blocks at the end;
|
||||
* - gzip at a fixed level, with Node's header: no file name and a modification time of 0.
|
||||
*
|
||||
* The same packager version and the same input bytes therefore always produce the same archive.
|
||||
*/
|
||||
|
||||
export class TarPathError extends Error {
|
||||
readonly path: string;
|
||||
|
||||
constructor(entryPath: string) {
|
||||
super("A path does not fit a portable ustar header.");
|
||||
this.name = "TarPathError";
|
||||
this.path = entryPath;
|
||||
}
|
||||
}
|
||||
|
||||
const BLOCK = 512;
|
||||
const GZIP_LEVEL = 9;
|
||||
|
||||
function octal(value: number, width: number): string {
|
||||
return `${value.toString(8).padStart(width - 1, "0")}\u0000`;
|
||||
}
|
||||
|
||||
/** Splits a path into ustar's 155-byte prefix and 100-byte name, at a `/`. */
|
||||
function splitPath(entryPath: string): { readonly name: string; readonly prefix: string } {
|
||||
if (entryPath.length <= 100) {
|
||||
return { name: entryPath, prefix: "" };
|
||||
}
|
||||
for (let index = entryPath.length - 1; index > 0; index -= 1) {
|
||||
if (entryPath[index] === "/") {
|
||||
const prefix = entryPath.slice(0, index);
|
||||
const name = entryPath.slice(index + 1);
|
||||
if (prefix.length <= 155 && name.length > 0 && name.length <= 100) {
|
||||
return { name, prefix };
|
||||
}
|
||||
}
|
||||
}
|
||||
throw new TarPathError(entryPath);
|
||||
}
|
||||
|
||||
function header(entryPath: string, directory: boolean, size: number): Buffer {
|
||||
const block = Buffer.alloc(BLOCK);
|
||||
const { name, prefix } = splitPath(directory ? `${entryPath}/` : entryPath);
|
||||
block.write(name, 0, 100, "latin1");
|
||||
block.write(octal(directory ? 0o755 : 0o644, 8), 100, "latin1");
|
||||
block.write(octal(0, 8), 108, "latin1");
|
||||
block.write(octal(0, 8), 116, "latin1");
|
||||
block.write(octal(size, 12), 124, "latin1");
|
||||
block.write(octal(0, 12), 136, "latin1");
|
||||
block.write(" ", 148, "latin1");
|
||||
block.write(directory ? "5" : "0", 156, "latin1");
|
||||
block.write("ustar\u000000", 257, "latin1");
|
||||
block.write(octal(0, 8), 329, "latin1");
|
||||
block.write(octal(0, 8), 337, "latin1");
|
||||
block.write(prefix, 345, 155, "latin1");
|
||||
let checksum = 0;
|
||||
for (const byte of block) {
|
||||
checksum += byte;
|
||||
}
|
||||
block.write(`${checksum.toString(8).padStart(6, "0")}\u0000 `, 148, "latin1");
|
||||
return block;
|
||||
}
|
||||
|
||||
function compareBytes(left: string, right: string): number {
|
||||
return left < right ? -1 : left > right ? 1 : 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Archives `files` (paths relative to the module directory, printable ASCII) under `topLevel`.
|
||||
* Directories are implied by the file paths.
|
||||
*/
|
||||
export function deterministicTarGz(topLevel: string, files: ReadonlyMap<string, Buffer>): Buffer {
|
||||
const directories = new Set<string>([topLevel]);
|
||||
for (const filePath of files.keys()) {
|
||||
const segments = filePath.split("/");
|
||||
for (let index = 1; index < segments.length; index += 1) {
|
||||
directories.add(`${topLevel}/${segments.slice(0, index).join("/")}`);
|
||||
}
|
||||
}
|
||||
const entries: { readonly path: string; readonly data?: Buffer }[] = [
|
||||
...[...directories].map((directory) => ({ path: directory })),
|
||||
...[...files].map(([filePath, data]) => ({ data, path: `${topLevel}/${filePath}` })),
|
||||
].sort((left, right) => compareBytes(left.path, right.path));
|
||||
|
||||
const parts: Buffer[] = [];
|
||||
for (const entry of entries) {
|
||||
if (entry.data === undefined) {
|
||||
parts.push(header(entry.path, true, 0));
|
||||
} else {
|
||||
parts.push(header(entry.path, false, entry.data.length), entry.data);
|
||||
const padding = (BLOCK - (entry.data.length % BLOCK)) % BLOCK;
|
||||
if (padding > 0) {
|
||||
parts.push(Buffer.alloc(padding));
|
||||
}
|
||||
}
|
||||
}
|
||||
parts.push(Buffer.alloc(BLOCK * 2));
|
||||
return gzipSync(Buffer.concat(parts), { level: GZIP_LEVEL });
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { parsePackModuleArguments } from "../src/cli.js";
|
||||
|
||||
const revision = "86ae907f59458508e3277666cdd214f5bd7c1078";
|
||||
|
||||
describe("ekklesia-pack-module arguments", () => {
|
||||
it("takes the module, the target, the output directory, and the source revision", () => {
|
||||
expect(
|
||||
parsePackModuleArguments([
|
||||
"--module",
|
||||
"templates/module",
|
||||
"--target",
|
||||
"any",
|
||||
"--out",
|
||||
"dist/packages",
|
||||
"--source-revision",
|
||||
revision,
|
||||
]),
|
||||
).toEqual({
|
||||
moduleDirectory: "templates/module",
|
||||
outDirectory: "dist/packages",
|
||||
sourceRevision: revision,
|
||||
target: "any",
|
||||
});
|
||||
});
|
||||
|
||||
it("records a repository only when one is given", () => {
|
||||
expect(
|
||||
parsePackModuleArguments([
|
||||
"--module",
|
||||
".",
|
||||
"--target",
|
||||
"any",
|
||||
"--out",
|
||||
"out",
|
||||
"--source-revision",
|
||||
revision,
|
||||
"--repository",
|
||||
"https://example.org/acme/example.git",
|
||||
]).repository,
|
||||
).toBe("https://example.org/acme/example.git");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["no arguments", []],
|
||||
["a missing target", ["--module", ".", "--out", "o", "--source-revision", revision]],
|
||||
["a missing revision", ["--module", ".", "--target", "any", "--out", "o"]],
|
||||
[
|
||||
"an unknown option",
|
||||
["--module", ".", "--target", "any", "--out", "o", "--source-revision", revision, "--legacy"],
|
||||
],
|
||||
[
|
||||
"a repeated option",
|
||||
[
|
||||
"--module",
|
||||
".",
|
||||
"--module",
|
||||
"..",
|
||||
"--target",
|
||||
"any",
|
||||
"--out",
|
||||
"o",
|
||||
"--source-revision",
|
||||
revision,
|
||||
],
|
||||
],
|
||||
[
|
||||
"a stray positional argument",
|
||||
["module", "--target", "any", "--out", "o", "--source-revision", revision],
|
||||
],
|
||||
])("refuses %s", (_label, argv) => {
|
||||
expect(() => parsePackModuleArguments(argv)).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,434 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { isBuiltin } from "node:module";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { type ModulePackErrorCode, ModulePackError, packModule } from "../src/index.js";
|
||||
import {
|
||||
cleanupFixtures,
|
||||
type FixtureModuleOptions,
|
||||
fixtureModule,
|
||||
SOURCE_REVISION,
|
||||
scratchDirectory,
|
||||
} from "./support/fixture-module.js";
|
||||
import { fileEntries, readTarGz } from "./support/tar.js";
|
||||
|
||||
afterEach(cleanupFixtures);
|
||||
|
||||
const MIT = "MIT License\n\nCopyright (c) 2026 Example Author\n\nPermission is hereby granted.\n";
|
||||
|
||||
async function pack(options: FixtureModuleOptions = {}, overrides: Record<string, unknown> = {}) {
|
||||
const moduleDirectory = await fixtureModule(options);
|
||||
const outDirectory = await scratchDirectory("ekklesia-pack-out-");
|
||||
const result = await packModule({
|
||||
moduleDirectory,
|
||||
nodeRange: ">=24.11.0 <25",
|
||||
outDirectory,
|
||||
sourceRevision: SOURCE_REVISION,
|
||||
target: "any",
|
||||
...overrides,
|
||||
});
|
||||
return { archive: await readFile(result.archivePath), moduleDirectory, outDirectory, result };
|
||||
}
|
||||
|
||||
async function packRejection(
|
||||
options: FixtureModuleOptions,
|
||||
code: ModulePackErrorCode,
|
||||
overrides: Record<string, unknown> = {},
|
||||
): Promise<ModulePackError> {
|
||||
try {
|
||||
await pack(options, overrides);
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(ModulePackError);
|
||||
expect((error as ModulePackError).code).toBe(code);
|
||||
return error as ModulePackError;
|
||||
}
|
||||
throw new Error(`Expected ${code}.`);
|
||||
}
|
||||
|
||||
describe("packModule: what a package contains (ADR 0049)", () => {
|
||||
it("produces one archive and its sha256sum line, under the recommended name", async () => {
|
||||
const { archive, outDirectory, result } = await pack();
|
||||
const digest = createHash("sha256").update(archive).digest("hex");
|
||||
expect(path.basename(result.archivePath)).toBe("example-0.1.0-any.tar.gz");
|
||||
expect(result.sha256).toBe(digest);
|
||||
expect(await readFile(path.join(outDirectory, "example-0.1.0-any.tar.gz.sha256"), "utf8")).toBe(
|
||||
`${digest} example-0.1.0-any.tar.gz\n`,
|
||||
);
|
||||
});
|
||||
|
||||
it("holds exactly the runtime payload, under one top-level directory named with the module id", async () => {
|
||||
const { archive } = await pack({ withSchema: true });
|
||||
const files = [...fileEntries(archive).keys()].sort();
|
||||
expect(files).toEqual([
|
||||
"example/THIRD_PARTY_NOTICES",
|
||||
expect.stringMatching(/^example\/dist\/chunks\/chunk-[A-Z0-9]+\.js$/u),
|
||||
"example/dist/schema/index.js",
|
||||
"example/dist/server/index.js",
|
||||
"example/drizzle/0000_initial.sql",
|
||||
"example/drizzle/meta/0000_snapshot.json",
|
||||
"example/drizzle/meta/_ekklesia.json",
|
||||
"example/drizzle/meta/_journal.json",
|
||||
"example/ekklesia.module.json",
|
||||
"example/ekklesia.package.json",
|
||||
"example/package.json",
|
||||
]);
|
||||
for (const file of files) {
|
||||
expect(file).not.toMatch(/\/src\/|\/tests\/|\.d\.ts$|\.map$|README|node_modules/u);
|
||||
}
|
||||
expect(new Set(readTarGz(archive).map((entry) => entry.path.split("/")[0]))).toEqual(
|
||||
new Set(["example"]),
|
||||
);
|
||||
});
|
||||
|
||||
it("generates the runtime package.json instead of copying the development one", async () => {
|
||||
const { archive } = await pack();
|
||||
const runtimePackage = fileEntries(archive).get("example/package.json")?.toString("utf8");
|
||||
expect(JSON.parse(runtimePackage ?? "")).toEqual({
|
||||
name: "@test/module-example",
|
||||
version: "0.1.0",
|
||||
private: true,
|
||||
type: "module",
|
||||
ekklesia: { layer: "module" },
|
||||
});
|
||||
for (const [file, data] of fileEntries(archive)) {
|
||||
if (file.endsWith(".json")) {
|
||||
expect(data.toString("utf8"), file).not.toContain("workspace:");
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("copies the manifest byte for byte", async () => {
|
||||
const { archive, moduleDirectory } = await pack();
|
||||
expect(fileEntries(archive).get("example/ekklesia.module.json")).toEqual(
|
||||
await readFile(path.join(moduleDirectory, "ekklesia.module.json")),
|
||||
);
|
||||
});
|
||||
|
||||
it("describes the package completely in ekklesia.package.json, never hashing itself", async () => {
|
||||
const { archive } = await pack({ withSchema: true });
|
||||
const files = fileEntries(archive);
|
||||
const metadata = JSON.parse(files.get("example/ekklesia.package.json")?.toString("utf8") ?? "");
|
||||
expect(metadata).toMatchObject({
|
||||
packageFormat: 1,
|
||||
module: { id: "example", version: "0.1.0" },
|
||||
target: "any",
|
||||
runtime: { node: ">=24.11.0 <25" },
|
||||
provenance: {
|
||||
revision: SOURCE_REVISION,
|
||||
packager: { name: "@ekklesia/module-pack", version: expect.any(String) },
|
||||
},
|
||||
incorporates: {
|
||||
"@ekklesia/contracts": expect.any(String),
|
||||
"@ekklesia/module-sdk": expect.any(String),
|
||||
},
|
||||
transitional: { databaseRuntime: { "drizzle-orm": "0.45.2" } },
|
||||
});
|
||||
const listed = metadata.files.map((file: { path: string }) => `example/${file.path}`);
|
||||
expect(listed).not.toContain("example/ekklesia.package.json");
|
||||
expect(new Set(listed)).toEqual(
|
||||
new Set([...files.keys()].filter((file) => file !== "example/ekklesia.package.json")),
|
||||
);
|
||||
for (const file of metadata.files as { path: string; size: number; sha256: string }[]) {
|
||||
const data = files.get(`example/${file.path}`);
|
||||
expect(data?.length).toBe(file.size);
|
||||
expect(
|
||||
createHash("sha256")
|
||||
.update(data ?? "")
|
||||
.digest("hex"),
|
||||
).toBe(file.sha256);
|
||||
}
|
||||
});
|
||||
|
||||
it("records only the provenance it was given, and a repository only when asked", async () => {
|
||||
const { archive } = await pack({}, { repository: "https://example.org/acme/example.git" });
|
||||
const metadata = JSON.parse(
|
||||
fileEntries(archive).get("example/ekklesia.package.json")?.toString("utf8") ?? "",
|
||||
);
|
||||
expect(metadata.provenance.repository).toBe("https://example.org/acme/example.git");
|
||||
});
|
||||
|
||||
it("omits transitional database facts from a module that bundles no Drizzle", async () => {
|
||||
const { archive } = await pack({
|
||||
serverSource: 'export default { id: "example", version: "0.1.0", register() {} };\n',
|
||||
});
|
||||
const metadata = JSON.parse(
|
||||
fileEntries(archive).get("example/ekklesia.package.json")?.toString("utf8") ?? "",
|
||||
);
|
||||
expect(metadata.transitional).toBeUndefined();
|
||||
expect(metadata.incorporates).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe("packModule: a self-contained backend", () => {
|
||||
it("bundles the SDK, contracts, and Drizzle, leaving only Node built-ins to import", async () => {
|
||||
const { archive } = await pack({ withSchema: true });
|
||||
for (const [file, data] of fileEntries(archive)) {
|
||||
if (!file.endsWith(".js")) {
|
||||
continue;
|
||||
}
|
||||
const code = data.toString("utf8");
|
||||
const specifiers = [
|
||||
...code.matchAll(/(?:^|\n)\s*(?:import|export)\s[^;]*?from\s*"([^"]+)"/gu),
|
||||
...code.matchAll(/(?:^|\n)\s*import\s*"([^"]+)"/gu),
|
||||
...code.matchAll(/\bimport\(\s*"([^"]+)"\s*\)/gu),
|
||||
].map((match) => match[1] ?? "");
|
||||
for (const specifier of specifiers) {
|
||||
expect(specifier.startsWith(".") || isBuiltin(specifier), `${file}: ${specifier}`).toBe(
|
||||
true,
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a dependency the bundle cannot resolve rather than leaving it external", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
serverSource:
|
||||
'import pad from "ekklesia-not-installed";\nexport default { id: "example", version: "0.1.0", register() { pad(); } };\n',
|
||||
},
|
||||
"MODULE_PACK_BUNDLE_FAILED",
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"a package that declares an operating system",
|
||||
{
|
||||
name: "fake-native",
|
||||
packageJson: { os: ["linux"], cpu: ["x64"] },
|
||||
files: { "index.js": "export default 1;\n" },
|
||||
},
|
||||
],
|
||||
[
|
||||
"a package built with node-gyp",
|
||||
{
|
||||
name: "fake-gyp",
|
||||
packageJson: { gypfile: true },
|
||||
files: { "index.js": "export default 1;\n", "binding.gyp": "{}" },
|
||||
},
|
||||
],
|
||||
[
|
||||
"a package that ships a compiled addon",
|
||||
{
|
||||
name: "fake-addon",
|
||||
files: {
|
||||
"index.js": "export default 1;\n",
|
||||
"build/Release/addon.node": new Uint8Array([0x7f, 0x45, 0x4c, 0x46]),
|
||||
},
|
||||
},
|
||||
],
|
||||
])("refuses %s in an any-target package", async (_label, fake) => {
|
||||
await packRejection(
|
||||
{
|
||||
fakePackages: [fake],
|
||||
serverSource: `import value from "${fake.name}";\nexport default { id: "example", version: "0.1.0", register() { return value; } };\n`,
|
||||
},
|
||||
"MODULE_PACK_NATIVE_DEPENDENCY",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses a package whose installed dependency is platform-specific", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
fakePackages: [
|
||||
{
|
||||
name: "fake-wrapper",
|
||||
packageJson: { optionalDependencies: { "fake-wrapper-linux-x64": "1.0.0" } },
|
||||
files: { "index.js": "export default 1;\n", LICENSE: MIT },
|
||||
},
|
||||
{
|
||||
name: "fake-wrapper-linux-x64",
|
||||
packageJson: { os: ["linux"], cpu: ["x64"] },
|
||||
files: { "index.js": "export default 2;\n" },
|
||||
},
|
||||
],
|
||||
serverSource:
|
||||
'import value from "fake-wrapper";\nexport default { id: "example", version: "0.1.0", register() { return value; } };\n',
|
||||
},
|
||||
"MODULE_PACK_NATIVE_DEPENDENCY",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses any target but any in this release", async () => {
|
||||
await packRejection({}, "MODULE_PACK_TARGET_UNSUPPORTED", { target: "linux-x64-glibc" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("packModule: identity and inputs", () => {
|
||||
it("refuses a development package.json whose version differs from the manifest", async () => {
|
||||
await packRejection(
|
||||
{ manifest: (manifest) => ({ ...manifest, version: "0.2.0" }) },
|
||||
"MODULE_PACK_IDENTITY_MISMATCH",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses a module whose backend was never built", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
manifest: (manifest) => ({
|
||||
...manifest,
|
||||
entrypoints: { server: "./dist/missing/index.js" },
|
||||
}),
|
||||
},
|
||||
"MODULE_PACK_ENTRYPOINT_MISSING",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses an abbreviated or missing source revision", async () => {
|
||||
await packRejection({}, "MODULE_PACK_REVISION_INVALID", { sourceRevision: "86ae907" });
|
||||
});
|
||||
|
||||
it("refuses an incorporated SDK outside the manifest's own range", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
manifest: (manifest) => ({
|
||||
...manifest,
|
||||
compatibility: { platform: "^0.1.0", moduleSdk: "^9.0.0" },
|
||||
}),
|
||||
},
|
||||
"MODULE_PACK_SDK_INCOMPATIBLE",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("packModule: THIRD_PARTY_NOTICES", () => {
|
||||
const usesFake = (name: string) =>
|
||||
`import value from "${name}";\nexport default { id: "example", version: "0.1.0", register() { return value; } };\n`;
|
||||
|
||||
it("credits every bundled third-party package once, with its licence and notice text", async () => {
|
||||
const { archive } = await pack({
|
||||
fakePackages: [
|
||||
{
|
||||
name: "fake-mit",
|
||||
version: "2.3.4",
|
||||
files: {
|
||||
"index.js": 'import other from "fake-apache-notice";\nexport default other;\n',
|
||||
LICENSE: MIT,
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "fake-apache-notice",
|
||||
version: "1.0.0",
|
||||
packageJson: { license: "Apache-2.0" },
|
||||
files: {
|
||||
"index.js": "export default 1;\n",
|
||||
"LICENSE.txt": "Apache License text shipped by the package.\n",
|
||||
NOTICE: "Fake Apache Notice\nCopyright 2026 The Fake Foundation\n",
|
||||
},
|
||||
},
|
||||
],
|
||||
serverSource: usesFake("fake-mit"),
|
||||
});
|
||||
const notices = fileEntries(archive).get("example/THIRD_PARTY_NOTICES")?.toString("utf8") ?? "";
|
||||
expect(notices).toContain("fake-mit 2.3.4");
|
||||
expect(notices).toContain("Copyright (c) 2026 Example Author");
|
||||
expect(notices).toContain("fake-apache-notice 1.0.0");
|
||||
expect(notices).toContain("Copyright 2026 The Fake Foundation");
|
||||
expect(notices.match(/^fake-mit 2\.3\.4$/gmu)).toHaveLength(1);
|
||||
// Only what the bundle actually contains is credited: this backend uses no Drizzle code.
|
||||
expect(notices).not.toContain("drizzle-orm");
|
||||
// The Ekklesia platform's own packages are credited in ekklesia.package.json, not here.
|
||||
expect(notices).not.toMatch(/^@ekklesia\//mu);
|
||||
});
|
||||
|
||||
it("supplies the standard Apache-2.0 text only for a package that ships none, and says so", async () => {
|
||||
const { archive } = await pack({ withSchema: true });
|
||||
const notices = fileEntries(archive).get("example/THIRD_PARTY_NOTICES")?.toString("utf8") ?? "";
|
||||
const drizzle = notices.slice(notices.indexOf("drizzle-orm 0.45.2"));
|
||||
expect(drizzle).toContain("License: Apache-2.0");
|
||||
expect(drizzle).toContain("ships no licence file");
|
||||
expect(drizzle).toContain("TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION");
|
||||
});
|
||||
|
||||
it("is byte-identical across packages of the same inputs", async () => {
|
||||
const options = { withSchema: true };
|
||||
const first = await pack(options);
|
||||
const second = await pack(options);
|
||||
expect(fileEntries(first.archive).get("example/THIRD_PARTY_NOTICES")).toEqual(
|
||||
fileEntries(second.archive).get("example/THIRD_PARTY_NOTICES"),
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses a bundled package that declares no licence", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
fakePackages: [
|
||||
{
|
||||
name: "fake-unlicensed",
|
||||
packageJson: { license: undefined },
|
||||
files: { "index.js": "export default 1;\n" },
|
||||
},
|
||||
],
|
||||
serverSource: usesFake("fake-unlicensed"),
|
||||
},
|
||||
"MODULE_PACK_LICENCE_UNKNOWN",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses an MIT package that ships no licence text, because only it holds the copyright line", async () => {
|
||||
await packRejection(
|
||||
{
|
||||
fakePackages: [{ name: "fake-textless", files: { "index.js": "export default 1;\n" } }],
|
||||
serverSource: usesFake("fake-textless"),
|
||||
},
|
||||
"MODULE_PACK_LICENCE_TEXT_MISSING",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("packModule: deterministic archives", () => {
|
||||
it("produces byte-identical archives and digests for identical inputs", async () => {
|
||||
const moduleDirectory = await fixtureModule({ withSchema: true });
|
||||
const runs = [];
|
||||
for (let run = 0; run < 2; run += 1) {
|
||||
const outDirectory = await scratchDirectory("ekklesia-pack-out-");
|
||||
const result = await packModule({
|
||||
moduleDirectory,
|
||||
nodeRange: ">=24.11.0 <25",
|
||||
outDirectory,
|
||||
sourceRevision: SOURCE_REVISION,
|
||||
target: "any",
|
||||
});
|
||||
runs.push({ bytes: await readFile(result.archivePath), sha256: result.sha256 });
|
||||
}
|
||||
expect(runs[0]?.sha256).toBe(runs[1]?.sha256);
|
||||
expect(runs[0]?.bytes.equals(runs[1]?.bytes ?? Buffer.alloc(0))).toBe(true);
|
||||
});
|
||||
|
||||
it("normalizes every entry and the gzip header, and contains no links", async () => {
|
||||
const { archive } = await pack({ withSchema: true });
|
||||
// gzip: no file name flag, no modification time.
|
||||
expect(archive[3]).toBe(0);
|
||||
expect(archive.readUInt32LE(4)).toBe(0);
|
||||
|
||||
const entries = readTarGz(archive);
|
||||
const paths = entries.map((entry) => entry.path);
|
||||
expect(paths).toEqual(
|
||||
[...paths].sort((left, right) => (left < right ? -1 : left > right ? 1 : 0)),
|
||||
);
|
||||
for (const entry of entries) {
|
||||
expect(["0", "5"], entry.path).toContain(entry.typeflag);
|
||||
expect(entry.mode, entry.path).toBe(entry.typeflag === "5" ? 0o755 : 0o644);
|
||||
expect([entry.uid, entry.gid, entry.mtime]).toEqual([0, 0, 0]);
|
||||
expect([entry.uname, entry.gname]).toEqual(["", ""]);
|
||||
expect(entry.magic).toBe("ustar/00");
|
||||
}
|
||||
// Every directory precedes what it contains.
|
||||
for (const entry of entries) {
|
||||
const parent = entry.path.split("/").slice(0, -1).join("/");
|
||||
if (parent !== "") {
|
||||
expect(paths.indexOf(parent), entry.path).toBeLessThan(paths.indexOf(entry.path));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("refers to no absolute path of the build machine", async () => {
|
||||
const { archive, moduleDirectory } = await pack({ withSchema: true });
|
||||
const root = path.dirname(path.dirname(moduleDirectory));
|
||||
for (const [file, data] of fileEntries(archive)) {
|
||||
expect(data.toString("utf8"), file).not.toContain(moduleDirectory);
|
||||
expect(data.toString("utf8"), file).not.toContain(root);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,193 @@
|
||||
import { mkdir, mkdtemp, realpath, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
/**
|
||||
* Generated module fixtures for the packager's tests: a built module directory in a temporary
|
||||
* location, whose `node_modules` links to the real Module SDK and Drizzle this package depends on.
|
||||
* Everything a development module directory holds is there, including what must never ship
|
||||
* (`src/`, `tests/`, declarations, source maps, `workspace:` ranges), so a test can prove the
|
||||
* packager leaves it out.
|
||||
*/
|
||||
|
||||
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
|
||||
|
||||
export interface FakePackage {
|
||||
readonly name: string;
|
||||
readonly version?: string;
|
||||
readonly packageJson?: Record<string, unknown>;
|
||||
/** Files relative to the fake package's root. */
|
||||
readonly files?: Record<string, string | Uint8Array>;
|
||||
}
|
||||
|
||||
export interface FixtureModuleOptions {
|
||||
readonly id?: string;
|
||||
readonly version?: string;
|
||||
readonly serverSource?: string;
|
||||
readonly withSchema?: boolean;
|
||||
readonly fakePackages?: readonly FakePackage[];
|
||||
readonly manifest?: (manifest: Record<string, unknown>) => Record<string, unknown>;
|
||||
}
|
||||
|
||||
const temporaryDirectories: string[] = [];
|
||||
|
||||
export async function cleanupFixtures(): Promise<void> {
|
||||
await Promise.all(
|
||||
temporaryDirectories
|
||||
.splice(0)
|
||||
.map((directory) => rm(directory, { force: true, recursive: true })),
|
||||
);
|
||||
}
|
||||
|
||||
export async function scratchDirectory(prefix = "ekklesia-pack-"): Promise<string> {
|
||||
const directory = await realpath(await mkdtemp(path.join(tmpdir(), prefix)));
|
||||
temporaryDirectories.push(directory);
|
||||
return directory;
|
||||
}
|
||||
|
||||
async function write(file: string, contents: string | Uint8Array): Promise<void> {
|
||||
await mkdir(path.dirname(file), { recursive: true });
|
||||
await writeFile(file, contents);
|
||||
}
|
||||
|
||||
const defaultServer = (
|
||||
id: string,
|
||||
version: string,
|
||||
) => `import { defineModuleRuntime } from "@ekklesia/module-sdk";
|
||||
export default defineModuleRuntime({ id: ${JSON.stringify(id)}, version: ${JSON.stringify(version)}, register() {} });
|
||||
`;
|
||||
|
||||
const schemaSource = (
|
||||
id: string,
|
||||
) => `import { createModuleTable, defineModuleSchema } from "@ekklesia/module-sdk";
|
||||
import { text, uuid } from "drizzle-orm/pg-core";
|
||||
export const records = createModuleTable(${JSON.stringify(id)})("records", {
|
||||
id: uuid("id").primaryKey(),
|
||||
value: text("value").notNull(),
|
||||
});
|
||||
export default defineModuleSchema({ moduleId: ${JSON.stringify(id)}, tables: { records } });
|
||||
`;
|
||||
|
||||
export async function fixtureModule(options: FixtureModuleOptions = {}): Promise<string> {
|
||||
const id = options.id ?? "example";
|
||||
const version = options.version ?? "0.1.0";
|
||||
const root = await scratchDirectory("ekklesia-pack-fixture-");
|
||||
const moduleDirectory = path.join(root, id);
|
||||
const tableName = `${id.replaceAll("-", "_")}_records`;
|
||||
|
||||
const baseManifest: Record<string, unknown> = {
|
||||
manifestVersion: 7,
|
||||
id,
|
||||
name: `${id} module`,
|
||||
version,
|
||||
compatibility: { platform: "^0.1.0", moduleSdk: "^0.1.0" },
|
||||
capabilities: [],
|
||||
permissions: [],
|
||||
entrypoints: {
|
||||
server: "./dist/server/index.js",
|
||||
...(options.withSchema ? { schema: "./dist/schema/index.js" } : {}),
|
||||
migrations: "./drizzle",
|
||||
},
|
||||
};
|
||||
const manifest = options.manifest ? options.manifest(baseManifest) : baseManifest;
|
||||
|
||||
await write(
|
||||
path.join(moduleDirectory, "ekklesia.module.json"),
|
||||
`${JSON.stringify(manifest, null, 2)}\n`,
|
||||
);
|
||||
await write(
|
||||
path.join(moduleDirectory, "package.json"),
|
||||
`${JSON.stringify(
|
||||
{
|
||||
name: `@test/module-${id}`,
|
||||
version,
|
||||
private: true,
|
||||
type: "module",
|
||||
scripts: { build: "tsc" },
|
||||
dependencies: { "@ekklesia/module-sdk": "workspace:*", "drizzle-orm": "0.45.2" },
|
||||
devDependencies: { typescript: "7.0.2" },
|
||||
exports: { ".": "./dist/server/index.js" },
|
||||
ekklesia: { layer: "module" },
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`,
|
||||
);
|
||||
await write(
|
||||
path.join(moduleDirectory, "dist/server/index.js"),
|
||||
options.serverSource ?? defaultServer(id, version),
|
||||
);
|
||||
await write(path.join(moduleDirectory, "dist/server/index.js.map"), '{"version":3}\n');
|
||||
await write(path.join(moduleDirectory, "dist/server/index.d.ts"), "export {};\n");
|
||||
if (options.withSchema) {
|
||||
await write(path.join(moduleDirectory, "dist/schema/index.js"), schemaSource(id));
|
||||
}
|
||||
await write(path.join(moduleDirectory, "src/server/index.ts"), "export {};\n");
|
||||
await write(path.join(moduleDirectory, "tests/server.test.ts"), "export {};\n");
|
||||
await write(path.join(moduleDirectory, "README.md"), "# Development notes\n");
|
||||
|
||||
await write(
|
||||
path.join(moduleDirectory, "drizzle/0000_initial.sql"),
|
||||
`CREATE TABLE "${tableName}" ("id" uuid PRIMARY KEY NOT NULL, "value" text NOT NULL);\n`,
|
||||
);
|
||||
await write(
|
||||
path.join(moduleDirectory, "drizzle/meta/_journal.json"),
|
||||
JSON.stringify({
|
||||
version: "7",
|
||||
dialect: "postgresql",
|
||||
entries: [
|
||||
{ idx: 0, version: "7", when: 1_700_000_000_000, tag: "0000_initial", breakpoints: true },
|
||||
],
|
||||
}),
|
||||
);
|
||||
await write(
|
||||
path.join(moduleDirectory, "drizzle/meta/_ekklesia.json"),
|
||||
JSON.stringify({
|
||||
version: 1,
|
||||
moduleId: id,
|
||||
entries: [{ tag: "0000_initial", destructive: { mode: "none" } }],
|
||||
}),
|
||||
);
|
||||
await write(
|
||||
path.join(moduleDirectory, "drizzle/meta/0000_snapshot.json"),
|
||||
JSON.stringify({
|
||||
version: "7",
|
||||
dialect: "postgresql",
|
||||
id: "10000000-0000-4000-8000-000000000001",
|
||||
prevId: "00000000-0000-0000-0000-000000000000",
|
||||
}),
|
||||
);
|
||||
|
||||
// The module's dependencies, as a workspace links them: the real packages this package depends on.
|
||||
const modules = path.join(moduleDirectory, "node_modules");
|
||||
await mkdir(path.join(modules, "@ekklesia"), { recursive: true });
|
||||
await symlink(
|
||||
await realpath(path.join(packageRoot, "node_modules", "@ekklesia", "module-sdk")),
|
||||
path.join(modules, "@ekklesia", "module-sdk"),
|
||||
);
|
||||
await symlink(
|
||||
await realpath(path.join(packageRoot, "node_modules", "drizzle-orm")),
|
||||
path.join(modules, "drizzle-orm"),
|
||||
);
|
||||
for (const fake of options.fakePackages ?? []) {
|
||||
const fakeRoot = path.join(modules, ...fake.name.split("/"));
|
||||
await write(
|
||||
path.join(fakeRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: fake.name,
|
||||
version: fake.version ?? "1.0.0",
|
||||
type: "module",
|
||||
main: "index.js",
|
||||
license: "MIT",
|
||||
...fake.packageJson,
|
||||
}),
|
||||
);
|
||||
for (const [relative, contents] of Object.entries(fake.files ?? {})) {
|
||||
await write(path.join(fakeRoot, relative), contents);
|
||||
}
|
||||
}
|
||||
return moduleDirectory;
|
||||
}
|
||||
|
||||
export const SOURCE_REVISION = "86ae907f59458508e3277666cdd214f5bd7c1078";
|
||||
@@ -0,0 +1,62 @@
|
||||
import { gunzipSync } from "node:zlib";
|
||||
|
||||
/** A test-side view of every ustar header field the determinism rules constrain (ADR 0049). */
|
||||
export interface TarEntry {
|
||||
readonly path: string;
|
||||
readonly typeflag: string;
|
||||
readonly mode: number;
|
||||
readonly uid: number;
|
||||
readonly gid: number;
|
||||
readonly uname: string;
|
||||
readonly gname: string;
|
||||
readonly mtime: number;
|
||||
readonly magic: string;
|
||||
readonly data: Buffer;
|
||||
}
|
||||
|
||||
function text(block: Buffer, offset: number, length: number): string {
|
||||
const raw = block.subarray(offset, offset + length);
|
||||
const end = raw.indexOf(0);
|
||||
return raw.subarray(0, end === -1 ? length : end).toString("latin1");
|
||||
}
|
||||
|
||||
function octal(block: Buffer, offset: number, length: number): number {
|
||||
return Number.parseInt(text(block, offset, length).trim() || "0", 8);
|
||||
}
|
||||
|
||||
export function readTarGz(archive: Buffer): readonly TarEntry[] {
|
||||
const raw = gunzipSync(archive);
|
||||
const entries: TarEntry[] = [];
|
||||
let offset = 0;
|
||||
while (offset + 512 <= raw.length) {
|
||||
const block = raw.subarray(offset, offset + 512);
|
||||
if (block.every((byte) => byte === 0)) {
|
||||
break;
|
||||
}
|
||||
const name = text(block, 0, 100);
|
||||
const prefix = text(block, 345, 155);
|
||||
const size = octal(block, 124, 12);
|
||||
entries.push({
|
||||
data: Buffer.from(raw.subarray(offset + 512, offset + 512 + size)),
|
||||
gid: octal(block, 116, 8),
|
||||
gname: text(block, 297, 32),
|
||||
magic: `${text(block, 257, 6)}/${text(block, 263, 2)}`,
|
||||
mode: octal(block, 100, 8),
|
||||
mtime: octal(block, 136, 12),
|
||||
path: prefix === "" ? name : `${prefix}/${name}`,
|
||||
typeflag: text(block, 156, 1),
|
||||
uid: octal(block, 108, 8),
|
||||
uname: text(block, 265, 32),
|
||||
});
|
||||
offset += 512 + Math.ceil(size / 512) * 512;
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
export function fileEntries(archive: Buffer): Map<string, Buffer> {
|
||||
return new Map(
|
||||
readTarGz(archive)
|
||||
.filter((entry) => entry.typeflag === "0")
|
||||
.map((entry) => [entry.path, entry.data]),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,195 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { createHash } from "node:crypto";
|
||||
import { lstat, readdir, readFile, realpath } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { verifyModulePackage } from "@ekklesia/core/modules";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { packModule } from "../src/index.js";
|
||||
import { cleanupFixtures, SOURCE_REVISION, scratchDirectory } from "./support/fixture-module.js";
|
||||
import { fileEntries } from "./support/tar.js";
|
||||
|
||||
/**
|
||||
* The canonical template module, packaged end to end (ADR 0049): the real built template, the real
|
||||
* packager, and Core's real verifier. Turbo builds the template and the verifier before this suite
|
||||
* runs (both are development dependencies of this package), and runs the template's own tests, which
|
||||
* rebuild its UI in place, before this one (turbo.json).
|
||||
*/
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const repositoryRoot = path.resolve(packageRoot, "..", "..");
|
||||
|
||||
afterEach(cleanupFixtures);
|
||||
|
||||
async function templateDirectory(): Promise<string> {
|
||||
return realpath(path.join(packageRoot, "node_modules", "@ekklesia", "module-template"));
|
||||
}
|
||||
|
||||
async function readJson(file: string): Promise<Record<string, unknown>> {
|
||||
return JSON.parse(await readFile(file, "utf8")) as Record<string, unknown>;
|
||||
}
|
||||
|
||||
async function platform() {
|
||||
const core = await readJson(path.join(repositoryRoot, "core", "package.json"));
|
||||
return {
|
||||
databaseRuntime: {
|
||||
"drizzle-orm": (core.dependencies as Record<string, string>)["drizzle-orm"] ?? "",
|
||||
},
|
||||
moduleSdkVersion: String(
|
||||
(await readJson(path.join(repositoryRoot, "packages", "module-sdk", "package.json"))).version,
|
||||
),
|
||||
platformVersion: String((await readJson(path.join(repositoryRoot, "package.json"))).version),
|
||||
};
|
||||
}
|
||||
|
||||
async function fingerprint(file: string): Promise<string> {
|
||||
try {
|
||||
return createHash("sha256")
|
||||
.update(await readFile(file))
|
||||
.digest("hex");
|
||||
} catch {
|
||||
return "absent";
|
||||
}
|
||||
}
|
||||
|
||||
async function treeListing(directory: string): Promise<readonly string[]> {
|
||||
const entries = await readdir(directory, { recursive: true, withFileTypes: true });
|
||||
return entries
|
||||
.filter((entry) => !path.join(entry.parentPath, entry.name).includes(`${path.sep}node_modules`))
|
||||
.map((entry) => path.relative(directory, path.join(entry.parentPath, entry.name)))
|
||||
.sort();
|
||||
}
|
||||
|
||||
async function packTemplate() {
|
||||
const outDirectory = await scratchDirectory("ekklesia-template-package-");
|
||||
const result = await packModule({
|
||||
moduleDirectory: await templateDirectory(),
|
||||
outDirectory,
|
||||
sourceRevision: SOURCE_REVISION,
|
||||
target: "any",
|
||||
});
|
||||
return { archive: await readFile(result.archivePath), result };
|
||||
}
|
||||
|
||||
describe("the module template as a Module Package v1", () => {
|
||||
it("packs deterministically, without touching the workspace it was built in", async () => {
|
||||
const template = await templateDirectory();
|
||||
const watched = [
|
||||
path.join(repositoryRoot, "node_modules", ".pnpm-workspace-state-v1.json"),
|
||||
path.join(repositoryRoot, "node_modules", ".modules.yaml"),
|
||||
path.join(repositoryRoot, "pnpm-lock.yaml"),
|
||||
];
|
||||
const before = await Promise.all(watched.map(fingerprint));
|
||||
const templateBefore = await treeListing(template);
|
||||
|
||||
const first = await packTemplate();
|
||||
const second = await packTemplate();
|
||||
|
||||
expect(first.result.sha256).toBe(second.result.sha256);
|
||||
expect(first.archive.equals(second.archive)).toBe(true);
|
||||
expect(path.basename(first.result.archivePath)).toMatch(
|
||||
/^example-\d+\.\d+\.\d+-any\.tar\.gz$/u,
|
||||
);
|
||||
expect(await Promise.all(watched.map(fingerprint))).toEqual(before);
|
||||
expect(await treeListing(template)).toEqual(templateBefore);
|
||||
});
|
||||
|
||||
it("verifies against this platform and extracts to an unrelated directory that holds no link", async () => {
|
||||
const { result } = await packTemplate();
|
||||
const target = path.join(await scratchDirectory("ekklesia-installed-elsewhere-"), "example");
|
||||
const verified = await verifyModulePackage({
|
||||
archivePath: result.archivePath,
|
||||
expectedSha256: result.sha256,
|
||||
extractTo: target,
|
||||
platform: await platform(),
|
||||
});
|
||||
expect(verified).toMatchObject({ moduleId: "example", target: "any" });
|
||||
|
||||
for (const entry of await readdir(target, { recursive: true, withFileTypes: true })) {
|
||||
const stats = await lstat(path.join(entry.parentPath, entry.name));
|
||||
expect(stats.isSymbolicLink(), entry.name).toBe(false);
|
||||
expect(stats.isFile() ? stats.nlink : 1, entry.name).toBe(1);
|
||||
}
|
||||
const listing = await treeListing(target);
|
||||
expect(listing).not.toContain("node_modules");
|
||||
expect(listing.some((file) => /(^|\/)(src|tests)(\/|$)|\.d\.ts$|\.map$/u.test(file))).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("loads from its new location without resolving anything outside itself", async () => {
|
||||
const { result } = await packTemplate();
|
||||
const target = path.join(await scratchDirectory("ekklesia-portable-"), "example");
|
||||
await verifyModulePackage({
|
||||
archivePath: result.archivePath,
|
||||
extractTo: target,
|
||||
platform: await platform(),
|
||||
});
|
||||
|
||||
// Every specifier the backend resolves while loading, recorded by Node itself.
|
||||
const script = `
|
||||
import { registerHooks } from "node:module";
|
||||
import { pathToFileURL } from "node:url";
|
||||
const resolved = [];
|
||||
registerHooks({ resolve(specifier, context, next) { const result = next(specifier, context); resolved.push(result.url); return result; } });
|
||||
for (const file of ["dist/server/index.js", "dist/schema/index.js"]) {
|
||||
await import(pathToFileURL(${JSON.stringify(target)} + "/" + file).href);
|
||||
}
|
||||
process.stdout.write(JSON.stringify(resolved));
|
||||
`;
|
||||
const { stdout } = await execFileAsync(
|
||||
process.execPath,
|
||||
["--input-type=module", "-e", script],
|
||||
{
|
||||
cwd: path.dirname(target),
|
||||
env: {},
|
||||
},
|
||||
);
|
||||
const resolved = JSON.parse(stdout) as string[];
|
||||
expect(resolved.length).toBeGreaterThan(0);
|
||||
for (const url of resolved) {
|
||||
const inside = url.startsWith(`file://${target}/`);
|
||||
expect(inside || url.startsWith("node:"), url).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("credits every third-party package its bundles say they contain", async () => {
|
||||
const { archive } = await packTemplate();
|
||||
const files = fileEntries(archive);
|
||||
const notices = files.get("example/THIRD_PARTY_NOTICES")?.toString("utf8") ?? "";
|
||||
const bundled = new Set<string>();
|
||||
for (const [file, data] of files) {
|
||||
if (file.endsWith(".js")) {
|
||||
// esbuild names every bundled source file in a path comment.
|
||||
for (const match of data
|
||||
.toString("utf8")
|
||||
.matchAll(/node_modules\/((?:@[^/]+\/)?[^/.][^/]*)\//gu)) {
|
||||
if (match[1] && !match[1].startsWith("@ekklesia/")) {
|
||||
bundled.add(match[1]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
expect([...bundled]).toEqual(
|
||||
expect.arrayContaining(["drizzle-orm", "react", "react-dom", "zod"]),
|
||||
);
|
||||
for (const name of bundled) {
|
||||
expect(notices, name).toMatch(
|
||||
new RegExp(`^${name.replace(/[.*+?^${}()|[\]\\/]/gu, "\\$&")} \\S+$`, "mu"),
|
||||
);
|
||||
}
|
||||
const licence = await readFile(path.join(packageRoot, "licenses", "Apache-2.0.txt"));
|
||||
expect(createHash("sha256").update(licence).digest("hex")).toBe(
|
||||
"cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30",
|
||||
);
|
||||
});
|
||||
|
||||
it("refers to no absolute path of the checkout it was built in", async () => {
|
||||
const { archive } = await packTemplate();
|
||||
for (const [file, data] of fileEntries(archive)) {
|
||||
expect(data.toString("utf8"), file).not.toContain(repositoryRoot);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"noEmit": false,
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"sourceMap": true,
|
||||
"types": ["node"]
|
||||
},
|
||||
"include": ["src/**/*.ts"],
|
||||
"exclude": ["src/**/*.test.ts", "tests"]
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"extends": "../typescript-config/node.json",
|
||||
"compilerOptions": {
|
||||
"noEmit": true,
|
||||
"types": ["node", "vitest/globals"]
|
||||
},
|
||||
"include": ["src/**/*.ts", "tests/**/*.ts"]
|
||||
}
|
||||
Reference in New Issue
Block a user