feat: four-part versions, and a published version is never used again

0.2.6.1 is a maintenance release of 0.2.6. The build refused four
parts, update_canary.sh's automatic bump could not add one to "6.1", and
neither publish script would have noticed a version being published
twice — which is what happened to 0.2.6 this morning: it was rebuilt
with the Presentations hardening and uploaded over the first 0.2.6, and
a room that had installed the first can never be offered the second.

One release library now holds the version pattern (three or four parts,
no leading zeros, no .0 maintenance), orders by dpkg --compare-versions,
refuses a version that has ever been published or is not newer than all
of them, uploads with --ignore-existing and checks the server's
checksum, writes a manifest only for a package the server holds, and
reads the channel back over HTTPS afterwards. update_canary.sh takes the
version explicitly; update_stable.sh refuses to go backwards.

The classroom updater refuses a manifest whose version, package name or
checksum is malformed, and refuses a download whose own control file is
not the version the manifest promised, so a room can never loop on
reinstalling and rebooting.

tests/test-release-versions.sh: 87 checks against a stand-in server and
real dpkg. The Trixie gate now also rehearses a stable room: 0.2.2 to an
interim 0.2.6.1 upgraded with settings, channel and Presentations=false
intact, and the installed updater answering correctly.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-09-23 18:19:37 -05:00
co-authored by Claude Opus 5.5
parent 6f2d169355
commit 690ef95aff
9 changed files with 1260 additions and 147 deletions
+86
View File
@@ -110,3 +110,89 @@ curl -fsSL -o c.deb https://updates.riverofglorymn.org/canary/latest.deb
sha256sum c.deb sha256sum c.deb
curl -fsS https://updates.riverofglorymn.org/canary/manifest.json curl -fsS https://updates.riverofglorymn.org/canary/manifest.json
``` ```
## Version numbers and releasing
```
0.2.6 a feature release
0.2.6.1 the first maintenance release of 0.2.6
0.2.6.2 the second
0.2.7 the next feature release
0.3.0 a larger milestone
```
Three parts for a feature release, four for a maintenance release of one.
The leading zero stays until Classroom is feature-complete; 1.0.0 belongs
to the wider Ekklesia platform's milestone. Nothing else is a version: no
leading zeros (dpkg reads `0.2.06` as `0.2.6`), no `0.2.6.0`, no suffixes.
The build, both publish scripts and every classroom's updater refuse
anything else, from one pattern held to one list of cases by
`tests/test-release-versions.sh`.
Which of two versions is newer is always `dpkg --compare-versions`, never
our own arithmetic, because dpkg is what installs them:
`0.2.2 < 0.2.6 < 0.2.6.1 < 0.2.6.2 < 0.2.6.10 < 0.2.7`.
### A published version is never used again
0.2.6 was rebuilt with later fixes and uploaded a second time under the
same number, replacing the first upload. A room that had installed the
first one could never be offered the second: the updater only installs a
version newer than its own. So now:
* `update_canary.sh VERSION` refuses a version that has ever been
published — on the server or in `repo/packages` — and one that is not
newer than every package ever published. It checks before building and
again after the Trixie gate.
* Packages are uploaded with `rsync --ignore-existing`, and the checksum on
the server is compared with the one built before anything points at it.
* A manifest is only written once the server holds exactly the package
and checksum it names.
* After publishing, the channel is read back over HTTPS the way a classroom
reads it: manifest, versioned package, `latest.deb` and the version dpkg
reads out of the file must all agree.
A publish that fails after the local copy into `repo/packages` has spent
that number. Use the next one.
### Canary
```
./update_canary.sh 0.2.6.1 # a maintenance release
./update_canary.sh 0.2.7 # the next feature release
```
The version is always given. With four parts there are two possible next
numbers, and which one a release is depends on what is in it. Run with no
argument it prints the current canary and both candidates, and builds
nothing.
### Stable
```
./update_stable.sh 0.2.6.1
```
Only with the owner's written approval for that release. Stable is never
given a package of its own: it is pointed at one canary already published,
byte for byte, and refuses a version that is not newer than stable's
current one or whose package the server does not hold with the same
checksum.
### Checking a channel without changing it
```
ROOT=$PWD bash -c 'source tools/release-lib.sh; release_verify_channel stable; release_verify_channel canary'
```
### Rehearsing an upgrade
`tools/validate-trixie-release.sh NEW.deb [PREVIOUS.deb]` installs into a
clean Debian 13 container. `VALIDATION_CHANNEL=stable` rehearses a stable
room, for example the desktops' real path:
```
VALIDATION_CHANNEL=stable tools/validate-trixie-release.sh \
dist/ekklesia-classroom_0.2.6.1_amd64.deb \
repo/packages/ekklesia-classroom_0.2.2_amd64.deb
```
+94 -3
View File
@@ -5,6 +5,7 @@ import fcntl
import hashlib import hashlib
import json import json
import os import os
import re
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
@@ -13,6 +14,21 @@ from urllib.parse import urljoin
CONFIG = "/etc/ekklesia-classroom/device.conf" CONFIG = "/etc/ekklesia-classroom/device.conf"
LOCKFILE = "/run/ekklesia-classroom-update.lock" LOCKFILE = "/run/ekklesia-classroom-update.lock"
CACHE_DIR = "/var/cache/ekklesia-classroom"
PACKAGE = "ekklesia-classroom"
# What a published version looks like: 0.2.6 for a feature release,
# 0.2.6.1 for a maintenance release of it. Anything else in a manifest is
# a mistake on the server, and a classroom does nothing with it. The same
# pattern is in tools/release-lib.sh; tests/test-release-versions.sh holds
# the two to one list of good and bad versions.
#
# Only the SHAPE is decided here. Which of two versions is newer is always
# dpkg's answer, because dpkg is what installs them.
VERSION_PATTERN = re.compile(
r"^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\.[1-9][0-9]*)?$"
)
SHA256_PATTERN = re.compile(r"^[0-9a-f]{64}$")
def log(message): def log(message):
@@ -50,13 +66,70 @@ def installed_version():
return result.stdout.strip() return result.stdout.strip()
def valid_version(version):
return isinstance(version, str) and bool(VERSION_PATTERN.match(version))
def version_is_newer(candidate, current): def version_is_newer(candidate, current):
result = subprocess.run( result = subprocess.run(
["dpkg", "--compare-versions", candidate, "gt", current] ["dpkg", "--compare-versions", candidate, "gt", current],
stderr=subprocess.DEVNULL,
) )
return result.returncode == 0 return result.returncode == 0
def manifest_problem(manifest):
"""Why this manifest must not be acted on, or None if it may be.
A manifest names a version, the package file for exactly that
version, and that file's checksum. The package name is checked
against the version, not merely read, so a manifest that says 0.2.6.1
and points at the 0.2.6 file is refused rather than installed and
then offered again at every check.
"""
if not isinstance(manifest, dict):
return "the manifest is not an object"
version = manifest.get("version")
if not valid_version(version):
return f"the manifest names an invalid version: {version!r}"
package = manifest.get("package")
expected = f"/packages/{PACKAGE}_{version}_amd64.deb"
if package != expected:
return f"the manifest names {package!r} for version {version}"
checksum = manifest.get("sha256")
if not isinstance(checksum, str) or not SHA256_PATTERN.match(checksum.lower()):
return f"the manifest's sha256 is not a checksum: {checksum!r}"
return None
def package_identity(path):
"""(Package, Version) as dpkg reads them out of a downloaded file."""
result = subprocess.run(
["dpkg-deb", "--field", path, "Package", "Version"],
capture_output=True,
text=True,
)
if result.returncode != 0:
return None, None
fields = {}
for line in result.stdout.splitlines():
if ":" in line:
key, value = line.split(":", 1)
fields[key.strip()] = value.strip()
return fields.get("Package"), fields.get("Version")
def fetch_json(url): def fetch_json(url):
request = urllib.request.Request( request = urllib.request.Request(
url, url,
@@ -134,6 +207,12 @@ def main():
log(f"Unable to fetch update manifest: {exc}") log(f"Unable to fetch update manifest: {exc}")
return 1 return 1
problem = manifest_problem(manifest)
if problem:
log(f"Ignoring the {channel} manifest: {problem}.")
return 1
candidate = manifest["version"] candidate = manifest["version"]
log(f"Available version: {candidate}") log(f"Available version: {candidate}")
@@ -149,10 +228,10 @@ def main():
package_url = urljoin(base_url, manifest["package"].lstrip("/")) package_url = urljoin(base_url, manifest["package"].lstrip("/"))
expected_hash = manifest["sha256"].lower() expected_hash = manifest["sha256"].lower()
os.makedirs("/var/cache/ekklesia-classroom", exist_ok=True) os.makedirs(CACHE_DIR, exist_ok=True)
package_path = os.path.join( package_path = os.path.join(
"/var/cache/ekklesia-classroom", CACHE_DIR,
os.path.basename(manifest["package"]), os.path.basename(manifest["package"]),
) )
@@ -177,6 +256,18 @@ def main():
log("SHA-256 verification passed.") log("SHA-256 verification passed.")
# The checksum proves the file is the one the manifest describes;
# this proves the manifest described the right file. A package
# that installs as some other version would be offered again at
# every check, and installed again, and the room rebooted, for
# ever.
name, version = package_identity(temporary_path)
if name != PACKAGE or version != candidate:
log(f"The package is {name} {version}, not {PACKAGE} {candidate}.")
os.unlink(temporary_path)
return 1
os.replace(temporary_path, package_path) os.replace(temporary_path, package_path)
log(f"Installing {candidate}") log(f"Installing {candidate}")
+42 -19
View File
@@ -235,16 +235,33 @@ check("container service startup is suppressed", "/usr/sbin/policy-rc.d" in vali
print() print()
print("6. canary release gate") print("6. canary release gate")
validation = canary.index('"$ROOT/tools/validate-trixie-release.sh"') validation = canary.index('"$ROOT/tools/validate-trixie-release.sh"')
package_copy = canary.index('cp "$package_path"') package_copy = canary.index('cp -n "$package_path"')
first_rsync = canary.index("rsync -av") first_upload = canary.index("release_upload_package")
check("Trixie validation runs before the repository copy", validation < package_copy) check("Trixie validation runs before the repository copy", validation < package_copy)
check("Trixie validation runs before any upload", validation < first_rsync) check("Trixie validation runs before any upload", validation < first_upload)
# A version is checked before anything is built for it, and again after
# the long gate, immediately before the first upload.
check("a spent version is refused before the build",
canary.index("release_check_new_version") < canary.index("build-release.sh"))
check("and again after the gate",
validation < canary.rindex("release_check_new_version") < first_upload)
check("the published channel is read back afterwards",
canary.index("release_verify_channel canary") > canary.index("release_publish_manifest canary"))
# The gate checks convergence, not just that the package installs: an # The gate checks convergence, not just that the package installs: an
# upgraded room ends on the v2 channel with every setting this release # upgraded room ends on the v2 channel with every setting this release
# has, exactly once, and the command a technician runs is safe to run on # has, exactly once, and the command a technician runs is safe to run on
# a machine that is already converged. # a machine that is already converged.
check("the gate checks a deliberate channel survives", check("the gate checks a deliberate channel survives",
"'^CHANNEL=canary$'" in validation_script) '"^CHANNEL=${channel}\\$"' in validation_script)
check("and that channel can be stable as well as canary",
"VALIDATION_CHANNEL" in validation_script
and "stable|canary)" in validation_script)
check("the gate checks the installed version is the package's own",
"/usr/local/lib/ekklesia-classroom/VERSION" in validation_script)
check("and asks the installed updater about it",
"ekklesia-classroom-update --check" in validation_script)
check("and that an upgrade leaves Presentations off",
"grep -q '^PRESENTATIONS_ENABLED=false$'" in validation_script)
check("and that the experiment's channel comes home", check("and that the experiment's channel comes home",
"'^CHANNEL=stable$'" in validation_script) "'^CHANNEL=stable$'" in validation_script)
check("and that every new setting arrived exactly once", check("and that every new setting arrived exactly once",
@@ -477,27 +494,33 @@ print("7c-iii. the permanent download addresses")
# .../stable/latest.deb and .../canary/latest.deb. One address per channel # .../stable/latest.deb and .../canary/latest.deb. One address per channel
# that never changes, and still no second record of which version is # that never changes, and still no second record of which version is
# current: the pointer is a symlink to the package the manifest names. # current: the pointer is a symlink to the package the manifest names.
# Both channels publish through one function, so the order below is the
# order for both.
release_lib = (root / "tools/release-lib.sh").read_text()
publish = release_lib[release_lib.index("release_publish_manifest() {"):]
publish = publish[:publish.index("\n}\n\n")]
publish_code = "\n".join(line for line in publish.splitlines()
if not line.lstrip().startswith("#"))
for channel, script in (("canary", "update_canary.sh"), for channel, script in (("canary", "update_canary.sh"),
("stable", "update_stable.sh")): ("stable", "update_stable.sh")):
text = (root / script).read_text() text = (root / script).read_text()
code = "\n".join(line for line in text.splitlines() check(f"{script} publishes {channel} through the shared function",
if not line.lstrip().startswith("#")) f"release_publish_manifest {channel} " in text)
check(f"{script} moves the {channel} pointer", check("the pointer is moved to the versioned package",
f"{channel}/latest.deb" in text or "latest.deb" in code) "ln -sfn ../packages/$name latest.deb.new" in publish_code)
check(f"{script} points it at the versioned package",
"ln -sfn ../packages/$package_name" in code)
# Atomic, so nothing downloads a half-written pointer, and after the # Atomic, so nothing downloads a half-written pointer, and after the
# manifest, so the pointer can lag it but never lead it — it can never # manifest, so the pointer can lag it but never lead it — it can never
# offer a package the channel has not published. # offer a package the channel has not published.
check(f"{script} swaps it by rename rather than in place", check("it is swapped by rename rather than in place",
"mv -Tf latest.deb.new latest.deb" in code) "mv -Tf latest.deb.new latest.deb" in publish_code)
# Against the rsync that actually uploads the manifest, not the echo check("after the manifest goes up",
# at the end that prints its URL. publish_code.index("latest.deb") > publish_code.index("$channel/manifest.json\""))
upload = f"classroom-updates/{channel}/manifest.json" check("and refuses to point at a package that is not there",
check(f"{script} moves it after the manifest goes up", "test -f ../packages/$name" in publish_code)
code.index("latest.deb") > code.index(upload)) check("the manifest is refused unless the server holds exactly that package",
check(f"{script} refuses to point at a package that is not there", publish_code.index("sha256sum") < publish_code.index("cat >"))
"test -f ../packages/$package_name" in code) check("a package upload never replaces a published file",
"--ignore-existing" in release_lib)
check("and neither script writes a version anywhere but the manifest", check("and neither script writes a version anywhere but the manifest",
"latest.json" not in (root / "update_canary.sh").read_text() "latest.json" not in (root / "update_canary.sh").read_text()
+549
View File
@@ -0,0 +1,549 @@
#!/bin/bash
# Three- and four-part versions, through everything that builds, publishes
# and installs a classroom package.
#
# Nothing here reaches the real update server. The server is a directory,
# ssh and rsync are stand-ins that act on it, and it is served over HTTP
# on the loopback interface so that the real updater, the real channel
# check and the real dpkg are the ones being asked.
set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
WORK="$(mktemp -d)"
SERVER_PID=""
cleanup() {
[[ -z "$SERVER_PID" ]] || kill "$SERVER_PID" 2>/dev/null
rm -rf "$WORK"
}
trap cleanup EXIT
passed=0
failed=0
check() {
local label="$1"
shift
if "$@" >/dev/null 2>&1; then
echo " ok $label"
passed=$((passed + 1))
else
echo " FAIL $label"
failed=$((failed + 1))
fi
}
refuses() {
! "$@"
}
# ---------------------------------------------------------------------------
# A package that is nothing but its version, the way dpkg will read it.
fake_deb() {
local version="$1" out="$2" stage="$WORK/stage-$version"
mkdir -p "$stage/DEBIAN"
cat > "$stage/DEBIAN/control" <<EOF
Package: ${3:-ekklesia-classroom}
Version: $version
Architecture: amd64
Maintainer: test
Description: test package
EOF
dpkg-deb --root-owner-group --build "$stage" "$out" >/dev/null
}
# ---------------------------------------------------------------------------
# The update server, as a directory.
SERVER="$WORK/server"
LOCAL="$WORK/local"
mkdir -p "$SERVER/packages" "$SERVER/stable" "$SERVER/canary" \
"$LOCAL/packages" "$LOCAL/canary" "$LOCAL/stable" "$WORK/bin"
for version in 0.1.45~hypr1 0.1.45 0.2.2 0.2.5 0.2.6; do
fake_deb "$version" "$SERVER/packages/ekklesia-classroom_${version}_amd64.deb"
cp "$SERVER/packages/ekklesia-classroom_${version}_amd64.deb" "$LOCAL/packages/"
done
# Not a classroom package, and on the real server all the same.
echo v2 > "$SERVER/packages/ekklesia-classroom-v2.deb"
manifest() {
local version="$1" file="${2:-ekklesia-classroom_$1_amd64.deb}" hash
hash="$(sha256sum "$SERVER/packages/$file" | awk '{print $1}')"
printf '{\n "version": "%s",\n "package": "/packages/%s",\n "sha256": "%s",\n "requires_reboot": true\n}\n' \
"$version" "$file" "$hash"
}
manifest 0.2.2 > "$SERVER/stable/manifest.json"
manifest 0.2.6 > "$SERVER/canary/manifest.json"
ln -s ../packages/ekklesia-classroom_0.2.2_amd64.deb "$SERVER/stable/latest.deb"
ln -s ../packages/ekklesia-classroom_0.2.6_amd64.deb "$SERVER/canary/latest.deb"
cp "$SERVER/stable/manifest.json" "$LOCAL/stable/"
cp "$SERVER/canary/manifest.json" "$LOCAL/canary/"
# ssh HOST COMMAND: run COMMAND here. The release scripts only ever talk
# to one host, and it is this directory.
cat > "$WORK/bin/ssh" <<'EOF'
#!/bin/bash
shift
exec bash -c "$*"
EOF
# rsync [-av] [--ignore-existing] -e ssh SOURCE HOST:DEST
cat > "$WORK/bin/rsync" <<'EOF'
#!/bin/bash
keep=0
args=()
while [[ $# -gt 0 ]]; do
case "$1" in
--ignore-existing) keep=1 ;;
-e) shift ;;
-*) ;;
*) args+=("$1") ;;
esac
shift
done
source="${args[0]}"
dest="${args[1]#*:}"
[[ -d "$dest" ]] && dest="$dest/$(basename "$source")"
if [[ "$keep" == 1 && -e "$dest" ]]; then
exit 0
fi
cp "$source" "$dest"
EOF
chmod +x "$WORK/bin/ssh" "$WORK/bin/rsync"
PORT="$(python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])')"
python3 -m http.server --bind 127.0.0.1 --directory "$SERVER" "$PORT" \
>/dev/null 2>&1 &
SERVER_PID=$!
for _ in $(seq 1 50); do
curl -fsS "http://127.0.0.1:$PORT/stable/manifest.json" >/dev/null 2>&1 && break
sleep 0.1
done
echo "0. the library, sourced the way the release scripts source it"
# With nothing in the environment and set -u in force. Every test below
# exports its own RELEASE_* values, and a default that only worked when
# somebody had already set it once stopped the real build dead.
check "it loads under set -u with nothing set" \
env -i PATH="$PATH" bash -c "set -euo pipefail; ROOT=/checkout; source '$ROOT/tools/release-lib.sh'
[[ \$RELEASE_HOST == classroom-control ]]
[[ \$RELEASE_ROOT == /srv/ekklesia/classroom-updates ]]
[[ \$RELEASE_URL == https://updates.riverofglorymn.org ]]
[[ \$RELEASE_LOCAL == /checkout/repo ]]"
check "and build-release.sh gets as far as checking the version" \
bash -c "'$ROOT/tools/build-release.sh' 0.2.06 2>&1 | grep -q 'Invalid version: 0.2.06'"
echo
export PATH="$WORK/bin:$PATH"
export RELEASE_HOST=test-server RELEASE_ROOT="$SERVER" \
RELEASE_URL="http://127.0.0.1:$PORT" RELEASE_LOCAL="$LOCAL"
# shellcheck source=../tools/release-lib.sh
source "$ROOT/tools/release-lib.sh"
GOOD=(0.2.2 0.2.6 0.2.6.1 0.2.6.2 0.2.7 0.3.0 0.2.10 0.2.6.10 1.0.0 10.20.30.40)
BAD=("" 0.2 0.2.6.1.1 0.2.06 00.2.6 0.2.6.0 0.2.6.01 v0.2.6 0.2.6-1 0.2.6~rc1
"0.2.6 " " 0.2.6" 0.2.6. .0.2.6 0..2 0.2.a "0.2.6;reboot" "0.2.6
0.2.7" 0.2.6+1 -0.2.6)
echo "1. what a version may look like"
for version in "${GOOD[@]}"; do
check "the release scripts accept $version" release_version_valid "$version"
done
for version in "${BAD[@]}"; do
check "the release scripts refuse '${version//$'\n'/\\n}'" refuses release_version_valid "$version"
done
# The updater decides with its own copy of the pattern, because it runs
# on a classroom that has no checkout. The two copies are held to the
# same list.
python3 - "$ROOT/scripts/ekklesia-classroom-update" "${GOOD[@]}" -- "${BAD[@]}" <<'PY'
import importlib.machinery
import importlib.util
import sys
loader = importlib.machinery.SourceFileLoader("updater", sys.argv[1])
spec = importlib.util.spec_from_loader("updater", loader)
updater = importlib.util.module_from_spec(spec)
loader.exec_module(updater)
rest = sys.argv[2:]
split = rest.index("--")
good, bad = rest[:split], rest[split + 1:]
wrong = [v for v in good if not updater.valid_version(v)]
wrong += [v for v in bad if updater.valid_version(v)]
wrong += [v for v in (None, 26, 0.26, ["0.2.6"]) if updater.valid_version(v)]
for version in wrong:
print(f" FAIL the updater disagrees about {version!r}")
sys.exit(1 if wrong else 0)
PY
check "the updater agrees about every one of them, and refuses non-strings" test $? -eq 0
echo
echo "2. which version is newer, as dpkg decides it"
ordered=(0.1.45~hypr1 0.1.45 0.2.2 0.2.5 0.2.6 0.2.6.1 0.2.6.2 0.2.6.10 0.2.7 0.2.10 0.3.0)
for ((i = 1; i < ${#ordered[@]}; i++)); do
check "${ordered[i-1]} < ${ordered[i]}" \
release_version_gt "${ordered[i]}" "${ordered[i-1]}"
check "and not the other way round" \
refuses release_version_gt "${ordered[i-1]}" "${ordered[i]}"
done
check "0.2.6.1 is not newer than itself" refuses release_version_gt 0.2.6.1 0.2.6.1
check "the highest of everything published is 0.2.6" \
test "$(release_published_versions | release_highest)" = 0.2.6
echo
echo "3. a version may be published once, and only if it is newer"
check "0.2.6.1 may be published" release_check_new_version 0.2.6.1
check "0.2.7 may be published" release_check_new_version 0.2.7
check "0.2.6 may not: it is already on the server" refuses release_check_new_version 0.2.6
check "0.2.5.1 may not: it is older than 0.2.6" refuses release_check_new_version 0.2.5.1
check "0.2.2 may not" refuses release_check_new_version 0.2.2
check "0.2.06 may not: dpkg would call it 0.2.6" refuses release_check_new_version 0.2.06
check "0.2.6.0 may not" refuses release_check_new_version 0.2.6.0
# Published only in the local copy — an upload that failed after the copy
# was made. The number is spent all the same.
fake_deb 0.2.8 "$LOCAL/packages/ekklesia-classroom_0.2.8_amd64.deb"
check "a version already copied into repo/ is spent too" refuses release_check_new_version 0.2.8
rm "$LOCAL/packages/ekklesia-classroom_0.2.8_amd64.deb"
echo
echo "4. uploading never replaces a published package"
fake_deb 0.2.6.1 "$WORK/ekklesia-classroom_0.2.6.1_amd64.deb"
check "0.2.6.1 uploads and arrives intact" \
release_upload_package "$WORK/ekklesia-classroom_0.2.6.1_amd64.deb"
check "and the server copy is the built one" \
cmp "$WORK/ekklesia-classroom_0.2.6.1_amd64.deb" \
"$SERVER/packages/ekklesia-classroom_0.2.6.1_amd64.deb"
before="$(sha256sum "$SERVER/packages/ekklesia-classroom_0.2.6_amd64.deb")"
mkdir -p "$WORK/rebuilt"
fake_deb 0.2.6 "$WORK/rebuilt/ekklesia-classroom_0.2.6_amd64.deb" ekklesia-classroom-rebuilt
check "a different 0.2.6 is refused" \
refuses release_upload_package "$WORK/rebuilt/ekklesia-classroom_0.2.6_amd64.deb"
check "and the published 0.2.6 is untouched" \
test "$(sha256sum "$SERVER/packages/ekklesia-classroom_0.2.6_amd64.deb")" = "$before"
echo
echo "5. a channel points at exactly what was uploaded"
hash_0261="$(sha256sum "$WORK/ekklesia-classroom_0.2.6.1_amd64.deb" | awk '{print $1}')"
stable_before="$(cat "$SERVER/stable/manifest.json")"
check "a manifest promising a different checksum is refused" \
refuses release_publish_manifest canary 0.2.6.1 "$(printf '0%.0s' {1..64})"
check "and canary still says 0.2.6" test "$(release_channel_version canary)" = 0.2.6
check "canary is moved to 0.2.6.1" release_publish_manifest canary 0.2.6.1 "$hash_0261"
check "its manifest says so" test "$(release_channel_version canary)" = 0.2.6.1
check "latest.deb follows it" \
test "$(readlink "$SERVER/canary/latest.deb")" = ../packages/ekklesia-classroom_0.2.6.1_amd64.deb
check "what a classroom downloads agrees, over HTTP" release_verify_channel canary 0.2.6.1
check "stable's manifest was not touched" \
test "$(cat "$SERVER/stable/manifest.json")" = "$stable_before"
check "and stable still verifies as 0.2.2" release_verify_channel stable 0.2.2
check "the check notices a channel saying the wrong version" \
refuses release_verify_channel stable 0.2.6.1
check "0.2.6.1 is now spent" refuses release_check_new_version 0.2.6.1
check "0.2.6.2 is next in line" release_check_new_version 0.2.6.2
# A package on the server that says it is a different version from its
# name. The channel check has to read the package, not just its name.
fake_deb 0.2.6 "$SERVER/packages/ekklesia-classroom_0.2.9_amd64.deb"
manifest 0.2.9 > "$SERVER/canary/manifest.json"
ln -sfn ../packages/ekklesia-classroom_0.2.9_amd64.deb "$SERVER/canary/latest.deb"
check "a package that says it is another version fails the channel check" \
refuses release_verify_channel canary 0.2.9
rm "$SERVER/packages/ekklesia-classroom_0.2.9_amd64.deb"
manifest 0.2.6.1 > "$SERVER/canary/manifest.json"
ln -sfn ../packages/ekklesia-classroom_0.2.6.1_amd64.deb "$SERVER/canary/latest.deb"
echo
echo "6. update_stable.sh"
cp "$WORK/ekklesia-classroom_0.2.6.1_amd64.deb" "$LOCAL/packages/"
check "refuses a version that is not newer than stable" \
refuses "$ROOT/update_stable.sh" 0.2.2
check "refuses a malformed version" refuses "$ROOT/update_stable.sh" 0.2.06
fake_deb 0.2.9 "$LOCAL/packages/ekklesia-classroom_0.2.9_amd64.deb"
check "refuses a package that was built but never uploaded" \
refuses "$ROOT/update_stable.sh" 0.2.9
rm "$LOCAL/packages/ekklesia-classroom_0.2.9_amd64.deb"
check "stable is still 0.2.2 after all of that" \
test "$(cat "$SERVER/stable/manifest.json")" = "$stable_before"
echo
echo "7. update_canary.sh"
check "without a version it builds nothing and names both choices" \
bash -c "'$ROOT/update_canary.sh' 2>&1 | grep -q '0.2.6.2 .*maintenance' \
&& '$ROOT/update_canary.sh' 2>&1 | grep -q '0.2.7 .*feature' \
&& ! '$ROOT/update_canary.sh'"
check "a published version is refused before anything is built" \
bash -c "! '$ROOT/update_canary.sh' 0.2.6 2>&1 | grep -q 'Validating sources'"
check "a malformed version is refused" refuses "$ROOT/update_canary.sh" 0.2.6.1.1
# ---------------------------------------------------------------------------
echo
echo "8. a classroom's updater, against that server"
# The package database and apt, as far as the updater can tell. The
# stand-in apt installs by reading the version out of the file it is
# given, which is what the real one does.
cat > "$WORK/bin/dpkg-query" <<EOF
#!/bin/bash
[[ -s "$WORK/installed" ]] || exit 1
cat "$WORK/installed"
EOF
cat > "$WORK/bin/apt-get" <<EOF
#!/bin/bash
echo "\$*" >> "$WORK/apt.log"
dpkg-deb --field "\${@: -1}" Version > "$WORK/installed"
EOF
cat > "$WORK/bin/systemctl" <<EOF
#!/bin/bash
echo "\$*" >> "$WORK/systemctl.log"
EOF
chmod +x "$WORK/bin/dpkg-query" "$WORK/bin/apt-get" "$WORK/bin/systemctl"
UPDATE_URL="http://127.0.0.1:$PORT" WORK="$WORK" SERVER="$SERVER" \
python3 - "$ROOT/scripts/ekklesia-classroom-update" <<'PY'
import contextlib
import hashlib
import importlib.machinery
import importlib.util
import io
import json
import os
import shutil
import subprocess
import sys
from pathlib import Path
work = Path(os.environ["WORK"])
server = Path(os.environ["SERVER"])
loader = importlib.machinery.SourceFileLoader("updater", sys.argv[1])
spec = importlib.util.spec_from_loader("updater", loader)
updater = importlib.util.module_from_spec(spec)
loader.exec_module(updater)
updater.LOCKFILE = str(work / "update.lock")
updater.CACHE_DIR = str(work / "cache")
passed = failed = 0
def check(label, condition, detail=""):
global passed, failed
if condition:
print(f" ok {label}")
passed += 1
else:
print(f" FAIL {label}")
if detail:
print(f" {detail}")
failed += 1
def device(channel, installed):
config = work / f"device-{channel}.conf"
config.write_text(
"# a classroom\n"
f"UPDATE_URL={os.environ['UPDATE_URL']}\n"
f"CHANNEL={channel}\n"
"PRESENTATIONS_ENABLED=false\n"
"UI_SCALE=1.25\n"
)
updater.CONFIG = str(config)
(work / "installed").write_text(installed)
for log in ("apt.log", "systemctl.log"):
(work / log).unlink(missing_ok=True)
return config
def run(*args):
sys.argv = ["ekklesia-classroom-update", *args]
out = io.StringIO()
with contextlib.redirect_stdout(out):
status = updater.main()
return status, out.getvalue()
def installs():
log = work / "apt.log"
return log.read_text().splitlines() if log.exists() else []
def installed():
return (work / "installed").read_text().strip()
def deb(version, name=None, package="ekklesia-classroom"):
stage = work / f"stage-{version}-{package}"
(stage / "DEBIAN").mkdir(parents=True, exist_ok=True)
(stage / "DEBIAN" / "control").write_text(
f"Package: {package}\nVersion: {version}\nArchitecture: amd64\n"
"Maintainer: test\nDescription: test\n"
)
out = server / "packages" / (name or f"ekklesia-classroom_{version}_amd64.deb")
out.unlink(missing_ok=True)
subprocess.run(["dpkg-deb", "--root-owner-group", "--build", stage, out],
check=True, capture_output=True)
return out
def publish(channel, version, file=None, sha=None, raw=None):
path = server / channel / "manifest.json"
if raw is not None:
path.write_text(raw)
return
file = file or f"ekklesia-classroom_{version}_amd64.deb"
digest = sha or hashlib.sha256((server / "packages" / file).read_bytes()).hexdigest()
path.write_text(json.dumps({
"version": version, "package": f"/packages/{file}",
"sha256": digest, "requires_reboot": True,
}))
publish("stable", "0.2.2")
publish("canary", "0.2.6.1")
config = device("canary", "0.2.6")
before = config.read_text()
status, out = run("--check")
check("a canary device on 0.2.6 sees 0.2.6.1 as an update",
status == 0 and "Update available: 0.2.6 -> 0.2.6.1" in out, out)
check("and --check installs nothing", installs() == [])
status, out = run("--no-reboot")
check("it installs 0.2.6.1", status == 0 and installed() == "0.2.6.1", out)
check("once, from the versioned file",
len(installs()) == 1 and installs()[0].endswith("ekklesia-classroom_0.2.6.1_amd64.deb"),
installs())
check("--no-reboot is respected", not (work / "systemctl.log").exists())
check("its device.conf is exactly as it was", config.read_text() == before)
for attempt in range(3):
run("--no-reboot")
check("a device on 0.2.6.1 does not install 0.2.6.1 again",
len(installs()) == 1, installs())
status, out = run()
check("nor reboot for it", "Already up to date" in out
and not (work / "systemctl.log").exists(), out)
deb("0.2.7")
publish("canary", "0.2.7")
status, out = run()
check("a device on 0.2.6.1 recognises 0.2.7 as newer and installs it",
status == 0 and installed() == "0.2.7", out)
check("and reboots, because the manifest asks it to",
(work / "systemctl.log").read_text().strip() == "reboot")
publish("canary", "0.2.6.1")
device("canary", "0.2.7")
status, out = run()
check("a device on 0.2.7 is not moved back to 0.2.6.1",
"Already up to date" in out and installs() == [], out)
# Three parts, exactly as before.
publish("canary", "0.2.6")
device("canary", "0.2.5")
status, out = run("--no-reboot")
check("a device on 0.2.5 still updates to 0.2.6", installed() == "0.2.6", out)
publish("canary", "0.2.6.1")
# The channels.
config = device("stable", "0.2.2")
status, out = run("--no-reboot")
check("a stable device on 0.2.2 reads the stable manifest",
f"{os.environ['UPDATE_URL']}/stable/manifest.json" in out, out)
check("and does not install the canary release",
installs() == [] and installed() == "0.2.2" and "Already up to date" in out, out)
deb("0.2.6.2")
publish("stable", "0.2.6.2")
status, out = run("--no-reboot")
check("a stable device goes from 0.2.2 straight to a four-part release when stable has one",
installed() == "0.2.6.2", out)
publish("stable", "0.2.2")
(server / "packages" / "ekklesia-classroom_0.2.6.2_amd64.deb").unlink()
# A room with no package installed at all reports 0.0.0 and takes anything.
device("canary", "")
status, out = run("--check")
check("a machine with nothing installed is offered 0.2.6.1",
"0.0.0 -> 0.2.6.1" in out, out)
# Malformed manifests: nothing downloaded, nothing installed, and a
# failure the timer's journal will show.
sha = hashlib.sha256((server / "packages" / "ekklesia-classroom_0.2.6.1_amd64.deb").read_bytes()).hexdigest()
bad = {
"a version with a suffix": {"version": "0.2.6.1-1"},
"a version with five parts": {"version": "0.2.6.1.1"},
"a version with a leading zero": {"version": "0.2.06"},
"a version that is a number": {"version": 0.3},
"an empty version": {"version": ""},
"a version with a shell in it": {"version": "0.2.7;reboot"},
"no version at all": {"version": None},
"a package for a different version": {"version": "0.2.7",
"package": "/packages/ekklesia-classroom_0.2.6.1_amd64.deb"},
"a package outside /packages": {"version": "0.2.7",
"package": "/../ekklesia-classroom_0.2.7_amd64.deb"},
"a checksum that is not one": {"version": "0.2.7", "sha256": "abc"},
}
for label, fields in bad.items():
manifest = {"version": "0.2.7",
"package": "/packages/ekklesia-classroom_0.2.7_amd64.deb",
"sha256": sha, "requires_reboot": True}
manifest.update(fields)
if manifest["version"] is None:
del manifest["version"]
publish("canary", None, raw=json.dumps(manifest))
device("canary", "0.2.6")
status, out = run("--no-reboot")
check(f"refused: {label}",
status == 1 and installs() == [] and installed() == "0.2.6", out)
for label, raw in (("a manifest that is not JSON", "<html>502</html>"),
("a manifest that is a list", "[]")):
publish("canary", None, raw=raw)
device("canary", "0.2.6")
status, out = run("--no-reboot")
check(f"refused: {label}", status == 1 and installs() == [], out)
# The file arrives, matches its checksum, and is the wrong package.
shutil.rmtree(updater.CACHE_DIR, ignore_errors=True)
deb("0.2.6", name="ekklesia-classroom_0.2.7_amd64.deb")
publish("canary", "0.2.7")
device("canary", "0.2.6")
status, out = run("--no-reboot")
check("a 0.2.7 file that installs as 0.2.6 is refused",
status == 1 and installs() == [] and "not ekklesia-classroom 0.2.7" in out, out)
check("and not left in the cache",
not list(Path(updater.CACHE_DIR).glob("*0.2.7*")))
deb("0.2.7", package="something-else", name="ekklesia-classroom_0.2.7_amd64.deb")
publish("canary", "0.2.7")
status, out = run("--no-reboot")
check("so is a file that is some other package", status == 1 and installs() == [], out)
deb("0.2.7")
publish("canary", "0.2.7", sha="0" * 64)
status, out = run("--no-reboot")
check("and a file whose checksum is wrong",
status == 1 and installs() == [] and "verification FAILED" in out, out)
publish("canary", "0.2.6.1")
print(f"\n updater: {passed} passed, {failed} failed")
sys.exit(1 if failed else 0)
PY
check "every updater case above passed" test $? -eq 0
echo
echo "$passed passed, $failed failed"
[[ "$failed" -eq 0 ]]
+14 -5
View File
@@ -11,11 +11,10 @@ fi
VERSION="$1" VERSION="$1"
if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then # Three parts for a feature release, four for a maintenance release of
echo "Invalid version: $VERSION" # one: 0.2.6, then 0.2.6.1. See tools/release-lib.sh.
echo "Expected MAJOR.MINOR.PATCH, for example 0.1.6" source "$ROOT/tools/release-lib.sh"
exit 1 release_require_version "$VERSION" || exit 1
fi
STAGE="$ROOT/packaging/root" STAGE="$ROOT/packaging/root"
OUTPUT="$ROOT/dist/ekklesia-classroom_${VERSION}_amd64.deb" OUTPUT="$ROOT/dist/ekklesia-classroom_${VERSION}_amd64.deb"
@@ -568,6 +567,16 @@ dpkg-deb \
--root-owner-group \ --root-owner-group \
--build "$STAGE" "$OUTPUT" --build "$STAGE" "$OUTPUT"
# The version dpkg reads out of the file is the one a classroom will
# report, so it is checked rather than assumed to have survived the sed.
built_version="$(release_package_version "$OUTPUT")"
if [[ "$built_version" != "$VERSION" ]]; then
echo "The package says it is $built_version, not $VERSION" >&2
rm -f "$OUTPUT"
exit 1
fi
echo echo
echo "Built:" echo "Built:"
echo "$OUTPUT" echo "$OUTPUT"
+286
View File
@@ -0,0 +1,286 @@
# Release helpers shared by tools/build-release.sh, update_canary.sh and
# update_stable.sh. Sourced, not run.
#
# VERSIONS.
#
# 0.2.6 a feature release
# 0.2.6.1 the first maintenance release of 0.2.6
# 0.2.6.2 the second
# 0.2.7 the next feature release
#
# Three or four dot-separated numbers, and nothing else. No leading zeros,
# because dpkg compares 0.2.06 and 0.2.6 as the SAME version, and two
# packages that dpkg thinks are equal are one package too many. No fourth
# part of 0: maintenance releases count from 1, and 0.2.6.0 would sort
# after 0.2.6 while claiming to be it.
#
# Ordering is dpkg's, always, through `dpkg --compare-versions`. That is
# what a classroom's updater and apt use to decide what is newer, so
# anything else here could only ever disagree with them.
#
# PUBLISHED IS PERMANENT.
#
# 0.2.6 was rebuilt and uploaded a second time under the same version, and
# the second upload replaced the first on the server. A room that had
# installed the first one could never be offered the second: the updater
# only installs a version newer than its own, and they were equal. So a
# version number, once it has been on the server, is never used again,
# and a package file on the server is never written twice.
RELEASE_HOST="${RELEASE_HOST:-classroom-control}"
RELEASE_ROOT="${RELEASE_ROOT:-/srv/ekklesia/classroom-updates}"
RELEASE_URL="${RELEASE_URL:-https://updates.riverofglorymn.org}"
# The local copy of what was published. ROOT is the checkout, set by
# whichever script sourced this.
RELEASE_LOCAL="${RELEASE_LOCAL:-$ROOT/repo}"
# The same pattern is in scripts/ekklesia-classroom-update, which is what
# a classroom checks a manifest against. tests/test-release-versions.sh
# holds both to the same list of good and bad versions.
RELEASE_VERSION_PATTERN='^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(\.[1-9][0-9]*)?$'
release_version_valid() {
[[ "$1" =~ $RELEASE_VERSION_PATTERN ]]
}
release_require_version() {
if ! release_version_valid "$1"; then
echo "Invalid version: $1" >&2
echo "Expected MAJOR.MINOR.PATCH or MAJOR.MINOR.PATCH.MAINTENANCE," >&2
echo "for example 0.2.7 or 0.2.6.1" >&2
return 1
fi
}
release_version_gt() {
dpkg --compare-versions "$1" gt "$2"
}
release_package_name() {
printf 'ekklesia-classroom_%s_amd64.deb\n' "$1"
}
# The version dpkg itself reads out of a built package, which is the one a
# classroom will report after installing it.
release_package_version() {
dpkg-deb --field "$1" Version
}
release_remote() {
ssh "$RELEASE_HOST" "$@"
}
# Every version that has ever been published, one per line, read from the
# package file names on the server and in the local copy of it. Names that
# are not a classroom package (ekklesia-classroom-v2.deb) are skipped;
# experiment builds such as 0.1.45~hypr1 are kept, because dpkg orders
# them and they were published.
release_published_versions() {
{
release_remote "ls -1 '$RELEASE_ROOT/packages'" || return 1
ls -1 "$RELEASE_LOCAL/packages" 2>/dev/null || true
} | sed -n 's/^ekklesia-classroom_\(.*\)_amd64\.deb$/\1/p' | sort -u
}
release_highest() {
local highest="" version
while read -r version; do
[[ -n "$version" ]] || continue
if [[ -z "$highest" ]] || release_version_gt "$version" "$highest"; then
highest="$version"
fi
done
printf '%s\n' "$highest"
}
# The version a channel's manifest on the server names, or nothing.
release_channel_version() {
release_remote "cat '$RELEASE_ROOT/$1/manifest.json' 2>/dev/null" \
| python3 -c 'import json, sys
try:
print(json.load(sys.stdin)["version"])
except Exception:
pass'
}
# May VERSION be published as a new package at all? It has to be valid,
# its file name must never have been used, and it must be newer than every
# package ever published, so that it is newer than whatever any room is
# running whichever channel that room is on.
release_check_new_version() {
local version="$1" name published highest
release_require_version "$version" || return 1
name="$(release_package_name "$version")"
published="$(release_published_versions)" || {
echo "Could not list the published packages on $RELEASE_HOST." >&2
return 1
}
if grep -qxF -- "$version" <<< "$published"; then
echo "$version has already been published." >&2
echo "A published version is never rebuilt or replaced. Use the next one." >&2
return 1
fi
if release_remote "test -e '$RELEASE_ROOT/packages/$name'"; then
echo "$name already exists on $RELEASE_HOST." >&2
return 1
fi
highest="$(release_highest <<< "$published")"
if [[ -n "$highest" ]] && ! release_version_gt "$version" "$highest"; then
echo "$version is not newer than $highest, the highest version already published." >&2
echo "Every release must sort after every one before it." >&2
return 1
fi
}
# Upload a package that has never been on the server, and prove that what
# arrived is what was built. --ignore-existing is the second lock on the
# rule above: even if the check were wrong, rsync would not replace a
# published file.
release_upload_package() {
local package="$1" name local_hash remote_hash
name="$(basename "$package")"
local_hash="$(sha256sum "$package" | awk '{print $1}')"
rsync -av --ignore-existing -e ssh \
"$package" "$RELEASE_HOST:$RELEASE_ROOT/packages/"
remote_hash="$(release_remote "sha256sum '$RELEASE_ROOT/packages/$name'" | awk '{print $1}')"
if [[ "$remote_hash" != "$local_hash" ]]; then
echo "The package on $RELEASE_HOST does not match the one built here." >&2
echo " built: $local_hash" >&2
echo " on server: ${remote_hash:-missing}" >&2
return 1
fi
}
# Write CHANNEL's manifest for VERSION locally, then upload it, then move
# the channel's latest.deb. The package must already be on the server with
# exactly the checksum the manifest is about to promise.
release_publish_manifest() {
local channel="$1" version="$2" hash="$3" name remote_hash
name="$(release_package_name "$version")"
remote_hash="$(release_remote "sha256sum '$RELEASE_ROOT/packages/$name'" | awk '{print $1}')"
if [[ "$remote_hash" != "$hash" ]]; then
echo "Refusing to point $channel at $name:" >&2
echo " the manifest would say $hash" >&2
echo " the server has ${remote_hash:-no such file}" >&2
return 1
fi
mkdir -p "$RELEASE_LOCAL/$channel"
cat > "$RELEASE_LOCAL/$channel/manifest.json" <<EOF
{
"version": "$version",
"package": "/packages/$name",
"sha256": "$hash",
"requires_reboot": true
}
EOF
# Manifest deliberately goes LAST, after the package it names.
rsync -av -e ssh \
"$RELEASE_LOCAL/$channel/manifest.json" \
"$RELEASE_HOST:$RELEASE_ROOT/$channel/manifest.json"
# THE PERMANENT URL.
#
# .../CHANNEL/latest.deb is a symlink beside the manifest pointing at
# the versioned package the manifest names, so there is one download
# address that never changes and still no second place recording which
# version is current. The manifest remains the only answer to that
# question.
#
# It is swapped AFTER the manifest, and by rename, for two reasons. A
# rename is atomic, so nothing ever downloads a half-written pointer;
# and going last means latest.deb can only ever lag the manifest by a
# moment, never lead it — it can never offer a package the channel has
# not published. The versioned file itself is never touched again once
# uploaded.
release_remote "
set -e
cd '$RELEASE_ROOT/$channel'
test -f ../packages/$name
ln -sfn ../packages/$name latest.deb.new
mv -Tf latest.deb.new latest.deb
"
}
# What a classroom on CHANNEL actually sees, read over HTTPS the way the
# updater reads it: the manifest, the versioned package it names, and the
# permanent latest.deb, all agreeing with each other and with EXPECTED if
# one is given. Read-only; safe to run against stable at any time.
#
# A subshell, so the temporary directory is removed however it returns.
release_verify_channel() (
channel="$1"
expected="${2:-}"
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
manifest="$(curl -fsS -H 'Cache-Control: no-cache' \
"$RELEASE_URL/$channel/manifest.json")" || {
echo " FAIL $channel: manifest could not be fetched" >&2
return 1
}
read -r version package hash < <(python3 -c 'import json, sys
m = json.loads(sys.argv[1])
print(m["version"], m["package"], m["sha256"])' "$manifest")
name="$(release_package_name "$version")"
if [[ -n "$expected" && "$version" != "$expected" ]]; then
echo " FAIL $channel: manifest says $version, expected $expected" >&2
return 1
fi
if [[ "$package" != "/packages/$name" ]]; then
echo " FAIL $channel: manifest names $package for version $version" >&2
return 1
fi
curl -fsS -o "$work/versioned.deb" "$RELEASE_URL$package" || {
echo " FAIL $channel: $package could not be downloaded" >&2
return 1
}
curl -fsSL -H 'Cache-Control: no-cache' -o "$work/latest.deb" \
"$RELEASE_URL/$channel/latest.deb" || {
echo " FAIL $channel: latest.deb could not be downloaded" >&2
return 1
}
got_hash="$(sha256sum "$work/versioned.deb" | awk '{print $1}')"
latest_hash="$(sha256sum "$work/latest.deb" | awk '{print $1}')"
deb_version="$(release_package_version "$work/versioned.deb")"
if [[ "$got_hash" != "$hash" ]]; then
echo " FAIL $channel: $name has sha256 $got_hash, manifest says $hash" >&2
return 1
fi
if [[ "$latest_hash" != "$hash" ]]; then
echo " FAIL $channel: latest.deb has sha256 $latest_hash, manifest says $hash" >&2
return 1
fi
if [[ "$deb_version" != "$version" ]]; then
echo " FAIL $channel: the package itself says $deb_version, manifest says $version" >&2
return 1
fi
echo " ok $channel: $version, $name, sha256 $hash, latest.deb agrees"
)
+102 -5
View File
@@ -18,6 +18,16 @@ if [[ $# -eq 2 ]]; then
[[ -f "$previous" ]] || { echo "Package not found: $previous" >&2; exit 1; } [[ -f "$previous" ]] || { echo "Package not found: $previous" >&2; exit 1; }
fi fi
# The channel the upgraded room is on. canary by default, which is the
# machine a new release goes to first; stable to rehearse the day the
# desktops are moved, from the release they are actually on.
channel="${VALIDATION_CHANNEL:-canary}"
case "$channel" in
stable|canary) ;;
*) echo "VALIDATION_CHANNEL must be stable or canary" >&2; exit 2 ;;
esac
echo "Validating $(basename "$candidate") on Debian Trixie" echo "Validating $(basename "$candidate") on Debian Trixie"
docker_args=( docker_args=(
@@ -41,7 +51,7 @@ fi
# instead teaches everybody to re-run it until it passes. # instead teaches everybody to re-run it until it passes.
timeout --signal=TERM --kill-after=15s 2700s \ timeout --signal=TERM --kill-after=15s 2700s \
docker "${docker_args[@]}" debian:trixie-slim \ docker "${docker_args[@]}" debian:trixie-slim \
bash -s -- "${previous:+/packages/previous.deb}" <<'CONTAINER' bash -s -- "${previous:+/packages/previous.deb}" "$channel" <<'CONTAINER'
set -Eeuo pipefail set -Eeuo pipefail
validation_failure() { validation_failure() {
@@ -87,6 +97,7 @@ run_logged() {
run_logged apt-update 300 apt-get update -qq run_logged apt-update 300 apt-get update -qq
previous="${1:-}" previous="${1:-}"
channel="${2:-canary}"
if [[ -n "$previous" ]]; then if [[ -n "$previous" ]]; then
run_logged previous-install 1200 apt-get install -y -qq "$previous" run_logged previous-install 1200 apt-get install -y -qq "$previous"
@@ -94,7 +105,7 @@ if [[ -n "$previous" ]]; then
cat > /etc/ekklesia-classroom/device.conf <<'CONFIG' cat > /etc/ekklesia-classroom/device.conf <<'CONFIG'
# Configuration carried forward from 0.1.36 # Configuration carried forward from 0.1.36
UPDATE_URL=https://updates.riverofglorymn.org UPDATE_URL=https://updates.riverofglorymn.org
CHANNEL=canary CHANNEL=@CHANNEL@
DVD_ENABLED=false DVD_ENABLED=false
UI_SCALE=1.25 UI_SCALE=1.25
MQTT_ENABLED=true MQTT_ENABLED=true
@@ -104,10 +115,28 @@ MQTT_USERNAME=classroom
MQTT_PASSWORD=preserve-me MQTT_PASSWORD=preserve-me
CUSTOM_SETTING=preserve-me-too CUSTOM_SETTING=preserve-me-too
CONFIG CONFIG
sed -i "s/@CHANNEL@/${channel}/" /etc/ekklesia-classroom/device.conf
previous_version="$(dpkg-query -W -f='${Version}' ekklesia-classroom)"
fi fi
run_logged candidate-install 1200 apt-get install -y -qq /packages/candidate.deb run_logged candidate-install 1200 apt-get install -y -qq /packages/candidate.deb
# THE VERSION, AS THE MACHINE REPORTS IT.
#
# Three places say which release a classroom is running: dpkg's database,
# which the updater compares against the manifest; the VERSION file the
# package carries; and the file name the release was published under.
# They have to be one number, or a room either never updates or updates
# for ever.
candidate_version="$(dpkg-deb --field /packages/candidate.deb Version)"
[[ "$(dpkg-query -W -f='${Version}' ekklesia-classroom)" == "$candidate_version" ]]
[[ "$(cat /usr/local/lib/ekklesia-classroom/VERSION)" == "$candidate_version" ]]
echo "installed ekklesia-classroom $candidate_version${previous_version:+ over $previous_version}"
if [[ -n "${previous_version:-}" ]]; then
dpkg --compare-versions "$candidate_version" gt "$previous_version"
fi
command -v nmtui >/dev/null command -v nmtui >/dev/null
command -v lpadmin >/dev/null command -v lpadmin >/dev/null
command -v lpinfo >/dev/null command -v lpinfo >/dev/null
@@ -376,6 +405,64 @@ timeout --signal=TERM --kill-after=5s 45s classroom-provision \
>/tmp/provision-second.log >/tmp/provision-second.log
grep -q 'Elmer is already correctly configured' /tmp/provision-second.log grep -q 'Elmer is already correctly configured' /tmp/provision-second.log
# THE UPDATER THIS PACKAGE INSTALLED, AGAINST A MANIFEST.
#
# Served from this container, so nothing reaches the real update server.
# The installed version has to be "up to date" against a manifest naming
# itself — or the room reinstalls and reboots at every check — and has to
# see the next feature release as newer.
mkdir -p /tmp/updates/packages "/tmp/updates/$channel"
cp /packages/candidate.deb "/tmp/updates/packages/ekklesia-classroom_${candidate_version}_amd64.deb"
serve_manifest() {
python3 - "$1" "$channel" <<'MANIFEST'
import hashlib, json, sys
version, channel = sys.argv[1], sys.argv[2]
name = f"ekklesia-classroom_{version}_amd64.deb"
try:
digest = hashlib.sha256(open(f"/tmp/updates/packages/{name}", "rb").read()).hexdigest()
except FileNotFoundError:
digest = "0" * 64
json.dump({"version": version, "package": f"/packages/{name}",
"sha256": digest, "requires_reboot": True},
open(f"/tmp/updates/{channel}/manifest.json", "w"))
MANIFEST
}
python3 -m http.server --bind 127.0.0.1 --directory /tmp/updates 8765 \
>/tmp/updates.log 2>&1 &
updates_server=$!
sleep 1
cp /etc/ekklesia-classroom/device.conf /tmp/device.conf.before-updater
sed -i 's|^UPDATE_URL=.*|UPDATE_URL=http://127.0.0.1:8765|' \
/etc/ekklesia-classroom/device.conf
serve_manifest "$candidate_version"
ekklesia-classroom-update --check >/tmp/updater-same.log 2>&1
grep -q "Already up to date" /tmp/updater-same.log
# The next milestone after whatever this is, and a release every
# classroom has long since passed.
IFS=. read -r major minor _ <<< "$candidate_version"
next_version="${major}.$((minor + 1)).0"
serve_manifest "$next_version"
ekklesia-classroom-update --check >/tmp/updater-next.log 2>&1
grep -q "Update available: $candidate_version -> $next_version" /tmp/updater-next.log
serve_manifest 0.2.2
ekklesia-classroom-update --check >/tmp/updater-older.log 2>&1
grep -q "Already up to date" /tmp/updater-older.log
echo '{"version": "0.2.06", "package": "/packages/x.deb", "sha256": "x"}' \
> "/tmp/updates/$channel/manifest.json"
! ekklesia-classroom-update --check >/tmp/updater-bad.log 2>&1
grep -q "invalid version" /tmp/updater-bad.log
kill "$updates_server"
cp /tmp/device.conf.before-updater /etc/ekklesia-classroom/device.conf
if [[ -n "$previous" ]]; then if [[ -n "$previous" ]]; then
# The room's own values, untouched by the upgrade. # The room's own values, untouched by the upgrade.
grep -q '^DVD_ENABLED=false$' /etc/ekklesia-classroom/device.conf grep -q '^DVD_ENABLED=false$' /etc/ekklesia-classroom/device.conf
@@ -386,7 +473,17 @@ if [[ -n "$previous" ]]; then
# The channel a room was put on deliberately survives the upgrade. # The channel a room was put on deliberately survives the upgrade.
# canary is where the next development release goes, and the Plasma # canary is where the next development release goes, and the Plasma
# classroom is the stable line now, so nothing here has to move. # classroom is the stable line now, so nothing here has to move.
grep -q '^CHANNEL=canary$' /etc/ekklesia-classroom/device.conf grep -q "^CHANNEL=${channel}\$" /etc/ekklesia-classroom/device.conf
# An optional feature an existing room never had stays off. A
# Presentations launcher appearing on the desktops' panels by itself
# would be an upgrade changing a classroom, which upgrades do not do.
[[ "$(grep -c '^PRESENTATIONS_ENABLED=' /etc/ekklesia-classroom/device.conf)" -eq 1 ]]
grep -q '^PRESENTATIONS_ENABLED=false$' /etc/ekklesia-classroom/device.conf
classroom-presentations 2>&1 | grep -q 'PRESENTATIONS_ENABLED is false'
classroom-desktop-layout >/dev/null 2>&1 || true
grep -q 'var PRESENTATIONS = false;' \
/usr/share/plasma/look-and-feel/org.ekklesia.classroom.desktop/contents/layouts/org.kde.plasma.desktop-layout.js
# And a room that was carried through the experiment on its own # And a room that was carried through the experiment on its own
# channel comes home, because that channel is the obsolete one now. # channel comes home, because that channel is the obsolete one now.
@@ -395,7 +492,7 @@ if [[ -n "$previous" ]]; then
grep -q '^CHANNEL=stable$' /etc/ekklesia-classroom/device.conf grep -q '^CHANNEL=stable$' /etc/ekklesia-classroom/device.conf
grep -q '^MQTT_PASSWORD=preserve-me$' /etc/ekklesia-classroom/device.conf grep -q '^MQTT_PASSWORD=preserve-me$' /etc/ekklesia-classroom/device.conf
grep -q '^CUSTOM_SETTING=preserve-me-too$' /etc/ekklesia-classroom/device.conf grep -q '^CUSTOM_SETTING=preserve-me-too$' /etc/ekklesia-classroom/device.conf
sed -i 's/^CHANNEL=.*/CHANNEL=canary/' /etc/ekklesia-classroom/device.conf sed -i "s/^CHANNEL=.*/CHANNEL=${channel}/" /etc/ekklesia-classroom/device.conf
# Every setting this release has that the room predates, appended # Every setting this release has that the room predates, appended
# exactly once. classroom-migrate hands device.conf to the shared # exactly once. classroom-migrate hands device.conf to the shared
@@ -415,7 +512,7 @@ if [[ -n "$previous" ]]; then
done done
grep -q '^MQTT_PASSWORD=preserve-me$' /etc/ekklesia-classroom/device.conf grep -q '^MQTT_PASSWORD=preserve-me$' /etc/ekklesia-classroom/device.conf
grep -q '^CUSTOM_SETTING=preserve-me-too$' /etc/ekklesia-classroom/device.conf grep -q '^CUSTOM_SETTING=preserve-me-too$' /etc/ekklesia-classroom/device.conf
grep -q '^CHANNEL=canary$' /etc/ekklesia-classroom/device.conf grep -q "^CHANNEL=${channel}\$" /etc/ekklesia-classroom/device.conf
# And by hand, on a machine that is already converged: the command a # And by hand, on a machine that is already converged: the command a
# technician runs must be safe to run twice. # technician runs must be safe to run twice.
+43 -57
View File
@@ -4,32 +4,50 @@ set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT" cd "$ROOT"
current_version="$(cat VERSION)" source "$ROOT/tools/release-lib.sh"
if [[ $# -eq 1 ]]; then # The version is always said out loud.
version="$1" #
else # With only three parts there was one obvious next number. With four
IFS='.' read -r major minor patch <<< "$current_version" # there are two — 0.2.6 is followed by 0.2.6.1 if it is a fix and by
version="${major}.${minor}.$((patch + 1))" # 0.2.7 if it is not — and which one a release is, is a decision about
# what is in it, not arithmetic.
if [[ $# -ne 1 ]]; then
current="$(release_channel_version canary || true)"
echo "Usage: $0 VERSION" >&2
echo >&2
echo "Canary is on ${current:-an unknown version}. Name the release, for example:" >&2
if [[ -n "$current" ]] && release_version_valid "$current"; then
IFS='.' read -r major minor patch maintenance <<< "$current"
echo " $0 ${major}.${minor}.${patch}.$(( ${maintenance:-0} + 1 )) a maintenance release" >&2
echo " $0 ${major}.${minor}.$(( patch + 1 )) the next feature release" >&2
fi
exit 2
fi fi
package_name="ekklesia-classroom_${version}_amd64.deb" version="$1"
package_name="$(release_package_name "$version")"
package_path="$ROOT/dist/$package_name" package_path="$ROOT/dist/$package_name"
echo "Current version: $current_version" echo "Current canary: $(release_channel_version canary || echo unknown)"
echo "New canary: $version" echo "New canary: $version"
echo echo
# Before anything is built: a version that has been published, or that
# would not be newer than everything that has been, stops here.
release_check_new_version "$version"
"$ROOT/tools/build-release.sh" "$version" "$ROOT/tools/build-release.sh" "$version"
# Validate the exact artifact that will be published. If a previous canary is # Validate the exact artifact that will be published. If a previous canary is
# available locally, exercise the real upgrade path as well as a fresh install. # available locally, exercise the real upgrade path as well as a fresh install.
validation_args=("$package_path") validation_args=("$package_path")
if [[ -f "$ROOT/repo/canary/manifest.json" ]]; then if [[ -f "$RELEASE_LOCAL/canary/manifest.json" ]]; then
previous_package_name="$(python3 -c \ previous_package_name="$(python3 -c \
'import json; print(json.load(open("repo/canary/manifest.json"))["package"].rsplit("/", 1)[-1])')" 'import json, sys; print(json.load(open(sys.argv[1]))["package"].rsplit("/", 1)[-1])' \
previous_package="$ROOT/repo/packages/$previous_package_name" "$RELEASE_LOCAL/canary/manifest.json")"
previous_package="$RELEASE_LOCAL/packages/$previous_package_name"
if [[ -f "$previous_package" && "$previous_package" != "$package_path" ]]; then if [[ -f "$previous_package" && "$previous_package" != "$package_path" ]]; then
validation_args+=("$previous_package") validation_args+=("$previous_package")
@@ -38,62 +56,30 @@ fi
"$ROOT/tools/validate-trixie-release.sh" "${validation_args[@]}" "$ROOT/tools/validate-trixie-release.sh" "${validation_args[@]}"
# Checked again after the gate, which takes long enough for somebody else
# to have published in the meantime.
release_check_new_version "$version"
hash="$(sha256sum "$package_path" | awk '{print $1}')" hash="$(sha256sum "$package_path" | awk '{print $1}')"
mkdir -p "$ROOT/repo/packages" "$ROOT/repo/canary" mkdir -p "$RELEASE_LOCAL/packages"
cp -n "$package_path" "$RELEASE_LOCAL/packages/$package_name"
cp "$package_path" "$ROOT/repo/packages/$package_name"
cat > "$ROOT/repo/canary/manifest.json" <<EOF
{
"version": "$version",
"package": "/packages/$package_name",
"sha256": "$hash",
"requires_reboot": true
}
EOF
echo echo
echo "Publishing package..." echo "Publishing package..."
release_upload_package "$RELEASE_LOCAL/packages/$package_name"
rsync -av \
-e ssh \
"$ROOT/repo/packages/$package_name" \
classroom-control:/srv/ekklesia/classroom-updates/packages/
echo echo
echo "Publishing canary manifest..." echo "Publishing canary manifest..."
release_publish_manifest canary "$version" "$hash"
# Manifest deliberately goes LAST. echo "$RELEASE_URL/canary/latest.deb -> $package_name"
rsync -av \
-e ssh \
"$ROOT/repo/canary/manifest.json" \
classroom-control:/srv/ekklesia/classroom-updates/canary/manifest.json
echo
# THE PERMANENT URL. echo "Checking what a classroom will see..."
# release_verify_channel canary "$version"
# .../canary/latest.deb is a symlink beside the manifest pointing at the echo " stable is still on $(release_channel_version stable || echo unknown)"
# versioned package the manifest names, so there is one download address
# that never changes and still no second place recording which version is
# current. The manifest remains the only answer to that question.
#
# It is swapped AFTER the manifest, and by rename, for two reasons. A
# rename is atomic, so nothing ever downloads a half-written pointer; and
# going last means latest.deb can only ever lag the manifest by a moment,
# never lead it — it can never offer a package the channel has not
# published. The versioned file itself is never touched again once
# uploaded.
ssh classroom-control "
set -e
cd /srv/ekklesia/classroom-updates/canary
test -f ../packages/$package_name
ln -sfn ../packages/$package_name latest.deb.new
mv -Tf latest.deb.new latest.deb
"
echo "https://updates.riverofglorymn.org/canary/latest.deb -> $package_name"
echo echo
echo "Published Ekklesia Classroom $version to CANARY" echo "Published Ekklesia Classroom $version to CANARY"
echo "https://updates.riverofglorymn.org/canary/manifest.json" echo "$RELEASE_URL/canary/manifest.json"
+36 -50
View File
@@ -4,11 +4,28 @@ set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT" cd "$ROOT"
source "$ROOT/tools/release-lib.sh"
# Stable never gets a package of its own. It is pointed at one canary
# already published, so the file every classroom downloads is byte for
# byte the one the canary machine ran.
if [[ $# -eq 1 ]]; then if [[ $# -eq 1 ]]; then
version="$1" version="$1"
else
if [[ ! -f "$RELEASE_LOCAL/canary/manifest.json" ]]; then
echo "No local canary manifest exists."
exit 1
fi
package_name="ekklesia-classroom_${version}_amd64.deb" version="$(python3 -c \
package_path="$ROOT/repo/packages/$package_name" 'import json, sys; print(json.load(open(sys.argv[1]))["version"])' \
"$RELEASE_LOCAL/canary/manifest.json")"
fi
release_require_version "$version"
package_name="$(release_package_name "$version")"
package_path="$RELEASE_LOCAL/packages/$package_name"
if [[ ! -f "$package_path" ]]; then if [[ ! -f "$package_path" ]]; then
echo "Package does not exist locally:" echo "Package does not exist locally:"
@@ -16,64 +33,33 @@ if [[ $# -eq 1 ]]; then
exit 1 exit 1
fi fi
hash="$(sha256sum "$package_path" | awk '{print $1}')" if [[ "$(release_package_version "$package_path")" != "$version" ]]; then
echo "$package_name says it is $(release_package_version "$package_path")." >&2
mkdir -p "$ROOT/repo/stable"
cat > "$ROOT/repo/stable/manifest.json" <<EOF
{
"version": "$version",
"package": "/packages/$package_name",
"sha256": "$hash",
"requires_reboot": true
}
EOF
else
if [[ ! -f "$ROOT/repo/canary/manifest.json" ]]; then
echo "No local canary manifest exists."
exit 1 exit 1
fi fi
mkdir -p "$ROOT/repo/stable" # A classroom only ever installs a version newer than its own, so a stable
cp "$ROOT/repo/canary/manifest.json" "$ROOT/repo/stable/manifest.json" # manifest that went backwards would reach nobody and would only make the
# channel say something untrue.
current="$(release_channel_version stable || true)"
version="$(python3 -c \ if [[ -n "$current" ]] && ! release_version_gt "$version" "$current"; then
'import json; print(json.load(open("repo/stable/manifest.json"))["version"])')" echo "Stable is on $current; $version is not newer." >&2
exit 1
fi fi
echo "Promoting $version to STABLE" hash="$(sha256sum "$package_path" | awk '{print $1}')"
rsync -av \ echo "Promoting $version to STABLE (from ${current:-nothing})"
-e ssh \
"$ROOT/repo/stable/manifest.json" \
classroom-control:/srv/ekklesia/classroom-updates/stable/manifest.json
# Refuses unless the server already holds exactly this package.
release_publish_manifest stable "$version" "$hash"
package_name="ekklesia-classroom_${version}_amd64.deb" echo "$RELEASE_URL/stable/latest.deb -> $package_name"
# THE PERMANENT URL. echo
# release_verify_channel stable "$version"
# .../stable/latest.deb is a symlink beside the manifest pointing at the
# versioned package the manifest names, so there is one download address
# that never changes and still no second place recording which version is
# current. The manifest remains the only answer to that question.
#
# It is swapped AFTER the manifest, and by rename, for two reasons. A
# rename is atomic, so nothing ever downloads a half-written pointer; and
# going last means latest.deb can only ever lag the manifest by a moment,
# never lead it — it can never offer a package the channel has not
# published. The versioned file itself is never touched again once
# uploaded.
ssh classroom-control "
set -e
cd /srv/ekklesia/classroom-updates/stable
test -f ../packages/$package_name
ln -sfn ../packages/$package_name latest.deb.new
mv -Tf latest.deb.new latest.deb
"
echo "https://updates.riverofglorymn.org/stable/latest.deb -> $package_name"
echo echo
echo "Ekklesia Classroom $version is now STABLE" echo "Ekklesia Classroom $version is now STABLE"
echo "https://updates.riverofglorymn.org/stable/manifest.json" echo "$RELEASE_URL/stable/manifest.json"