Author SHA1 Message Date
Media e924b0d099 fix: harden utility export recovery 2026-08-12 18:59:48 -05:00
Media d339e10c47 docs: document utility export workflow 2026-08-12 18:50:48 -05:00
Media 7e6b916963 feat: add isolated utility export modes 2026-08-12 18:48:03 -05:00
Media b934b69e5a feat: persist isolated utility export jobs 2026-08-12 18:21:29 -05:00
Media 8db46df69e fix: enforce Player download height limit 2026-08-12 18:18:38 -05:00
Media c2544c44c7 fix: close Audio Router self-echo PUBACK/subscription-delivery race
publish_router_mode registered its self-echo correlation marker before
publishing but removed it in a finally block the instant the QoS 1 publish()
call observed its PUBACK. A broker may legally deliver a Controller's own
subscription copy of that same publish after the PUBACK, since MQTT does not
order the two; when that happened, the Controller mistook its own late echo
for live operator intent and relinquished automatic restoration ownership.

Keep the marker claimable for a short, bounded mqtt.router_echo_grace_seconds
window (default 2s) after the PUBACK instead of clearing it immediately.
Genuine external/operator commands are unaffected: once the grace window
elapses, a matching payload is still correctly treated as operator intent
and ownership is relinquished as before.

Verified with deterministic fake-clock unit tests plus a real Mosquitto
broker integration test (tests/test_router_echo_mosquitto_integration.py,
skipped automatically when Docker is unavailable).
2026-08-07 17:13:54 -05:00
Media ec826bb8f3 fix: restrict web ingest to official YouTube hosts and bound submission size
The submit form's URL validation accepted any http/https host. Restrict it
to an explicit allowlist of official YouTube hosts using normalized,
exact hostname comparison (no substring/suffix matching), reject embedded
userinfo and malformed URLs, and add server-side length bounds for the URL,
submitter, note, and total request body.
2026-08-07 17:04:05 -05:00
Media 270ede8b77 fix: allow genuinely new playlist ingest without fresh Player evidence
The evidence gate on enqueue_download blocked every download, including a
URL that could only land in a slot store.reserve_download already proves is
empty and unguarded. After a Controller-only restart with the Player still
connected and idle, no fresh detailed status ever arrives, so this could
deadlock all new ingest indefinitely even though nothing unsafe would occur.

Narrow the gate to only require fresh evidence when the URL already occupies
a non-empty slot (refresh/reuse). Deletion, expiry, and cleanup/recovery
gating are unchanged.
2026-08-07 17:01:39 -05:00
Media 5e81403be3 feat: overhaul video player controller 2026-08-06 20:08:12 -05:00
Media 72cc66df92 fix: pass mqtt credentials through compose 2026-08-06 19:30:22 -05:00
Media 63ea314b18 build: harden controller v2 runtime 2026-08-06 19:23:02 -05:00
Media 4e185b2e1c refactor: wire controller v2 into web runtime 2026-08-06 18:58:05 -05:00
Media 3ee7f94d72 feat: coordinate authoritative playback and audio routing 2026-08-06 18:52:01 -05:00
Media 8b3bdae8a0 feat: reduce authoritative player mqtt state 2026-08-06 17:41:07 -05:00
Media 45083207b4 feat: reserve downloads and protect slot cleanup 2026-08-06 17:34:11 -05:00
Media 3ff9ba4637 feat: add atomic state store and legacy migration 2026-08-06 17:31:02 -05:00
Media 9091f27283 test: characterize contracts and make startup explicit 2026-08-06 17:24:20 -05:00
35 changed files with 15423 additions and 905 deletions
+21
View File
@@ -0,0 +1,21 @@
.git
.gitignore
.env
.env.*
!.env.example
__pycache__/
*.py[cod]
.venv/
venv/
.pytest_cache/
.coverage
htmlcov/
*.sqlite3
*.sqlite3-*
playlists.yaml
playlists.yaml.pre-v2.*.bak
controller-data/
data/
cookies*.txt
secrets/
tests/
+16
View File
@@ -0,0 +1,16 @@
# Copy to .env, keep it private, and replace every example path/value.
FLASK_SECRET_KEY=generate-a-long-random-secret
CONTROLLER_DATA_DIR=/srv/video-player-controller/data
YTPLAYER_MEDIA_DIR=/mnt/Media/YTPlayer
CONTROLLER_SECRETS_DIR=/srv/video-player-controller/secrets
MQTT_HOST=10.40.0.2
MQTT_PORT=1883
MQTT_USERNAME=
MQTT_PASSWORD=
MQTT_CLIENT_ID=yt-controller
CONTROLLER_WEB_PORT=7531
# Match the owner of the host media and data directories.
CONTROLLER_UID=1000
CONTROLLER_GID=1000
+20
View File
@@ -0,0 +1,20 @@
__pycache__/
*.py[cod]
.venv/
venv/
.pytest_cache/
.coverage
htmlcov/
.env
.env.*
!.env.example
*.sqlite3
*.sqlite3-*
controller-data/
data/
playlists.yaml.pre-v2.*.bak
cookies*.txt
secrets/
+39 -23
View File
@@ -1,32 +1,48 @@
FROM python:3.12-slim
# syntax=docker/dockerfile:1.7
# System deps: ffmpeg for muxing, nodejs for yt-dlp JS runtime, ca-certs for TLS
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
nodejs \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# yt-dlp requires Node 22+ for current YouTube JavaScript challenges. Copy
# only the runtime binary into the smaller Python/ffmpeg application image.
FROM node:22.23.1-bookworm-slim AS javascript-runtime
FROM python:3.12-slim-bookworm
ARG CONTROLLER_UID=1000
ARG CONTROLLER_GID=1000
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PIP_NO_CACHE_DIR=1 \
HOME=/tmp/controller-home \
XDG_CACHE_HOME=/tmp/controller-cache \
PORT=7531
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg libatomic1 \
&& rm -rf /var/lib/apt/lists/* \
&& groupadd --gid "${CONTROLLER_GID}" controller \
&& useradd --uid "${CONTROLLER_UID}" --gid controller --create-home \
--shell /usr/sbin/nologin controller \
&& install -d -o controller -g controller /app /controller-data /mnt/Media/YTPlayer
COPY --from=javascript-runtime /usr/local/bin/node /usr/local/bin/node
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY requirements.txt ./requirements.txt
RUN python -m pip install --requirement requirements.txt \
&& node --version \
&& python -c "from yt_dlp.utils._jsruntime import NodeJsRuntime; info = NodeJsRuntime().info; assert info and info.supported, info" \
&& ffmpeg -version
# Optional but strongly recommended: keep yt-dlp fresh (only if installed via pip)
RUN python -m pip install --no-cache-dir -U yt-dlp
COPY --chown=controller:controller controller ./controller
COPY --chown=controller:controller app.py controller_core.py wsgi.py gunicorn.conf.py config.yaml ./
# Make yt-dlp defaults persistent (so your subprocess call doesn't need flags)
# 1) tell yt-dlp to use node as JS runtime
# 2) (optional) force android client to dodge SABR weirdness more often
RUN mkdir -p /etc/yt-dlp && printf "%s\n" \
"--js-runtimes node" \
"--extractor-args youtube:player_client=android" \
> /etc/yt-dlp.conf
USER controller:controller
COPY app.py ./app.py
COPY controller_core.py ./controller_core.py
EXPOSE 7531
ENV PYTHONUNBUFFERED=1
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:7531/healthz', timeout=3).read()"]
EXPOSE 8000
CMD ["python", "app.py"]
CMD ["gunicorn", "--config", "gunicorn.conf.py", "wsgi:application"]
+1075
View File
File diff suppressed because it is too large Load Diff
+290 -29
View File
@@ -1,17 +1,24 @@
from flask import Flask, render_template_string, request, redirect, url_for, flash
from __future__ import annotations
import logging
import os
from urllib.parse import urlsplit
from controller_core import enqueue_download
from flask import Flask, flash, jsonify, redirect, render_template_string, request, url_for
app = Flask(__name__)
app.secret_key = os.environ.get("FLASK_SECRET_KEY", "dev-secret")
from controller.config import ControllerConfig, load_config
from controller.models import StoreError
from controller.service import ControllerService, ControllerUnavailableError
LOG = logging.getLogger(__name__)
HTML_FORM = """
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>YouTube Playlist Submit</title>
<title>YouTube Media Submit</title>
<!-- Make it behave on mobile -->
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
@@ -89,6 +96,33 @@ HTML_FORM = """
font-size: 1rem;
}
input[type="radio"] {
width: auto;
margin: 0;
}
fieldset {
border: 0;
padding: 0;
}
legend {
margin-bottom: 0.35rem;
font-size: 0.92rem;
}
.download-options {
display: grid;
gap: 0.4rem;
}
.download-options label {
display: flex;
align-items: center;
gap: 0.55rem;
margin: 0;
}
input::placeholder,
textarea::placeholder {
color: #6b7280;
@@ -132,6 +166,26 @@ HTML_FORM = """
color: #9ca3af;
}
.export-status {
margin-top: 1rem;
padding-top: 0.9rem;
border-top: 1px solid rgba(148, 163, 184, 0.25);
}
.export-job {
margin-top: 0.65rem;
padding: 0.65rem 0.75rem;
border-radius: 0.55rem;
background: rgba(30, 41, 59, 0.7);
font-size: 0.82rem;
}
.export-path,
.export-error {
overflow-wrap: anywhere;
color: #cbd5e1;
}
@media (max-width: 480px) {
.page-wrapper {
padding: 1rem;
@@ -160,9 +214,9 @@ HTML_FORM = """
<body>
<div class="page-wrapper">
<div class="card">
<h2>Submit YouTube Playlist</h2>
<h2>Submit YouTube Media</h2>
<p class="subtitle">
Paste a playlist link below. It’ll be queued for download and assigned to the next free slot.
Paste a YouTube video or playlist link and choose a download type.
</p>
{% with messages = get_flashed_messages() %}
@@ -205,49 +259,256 @@ HTML_FORM = """
></textarea>
</div>
<button type="submit">Submit Playlist</button>
<fieldset class="form-group">
<legend>Download as:</legend>
<div class="download-options">
<label><input type="radio" name="download_mode" value="player_slot" checked> Player Slot</label>
<label><input type="radio" name="download_mode" value="mp4"> MP4 Export</label>
<label><input type="radio" name="download_mode" value="m4a"> M4A Audio</label>
<label><input type="radio" name="download_mode" value="mp3"> MP3 Audio</label>
</div>
</fieldset>
<button type="submit">Submit</button>
</form>
<p class="helper">
Playlists are kept for roughly 48 hours after they’re downloaded.
Utility exports are kept until an operator removes them.
</p>
{% if export_jobs %}
<section class="export-status">
<strong>Recent utility exports</strong>
{% for job in export_jobs %}
<div class="export-job">
<div>{{ download_mode_labels[job.mode] }} — {{ export_status_labels[job.status] }} (job {{ job.job_id[:8] }})</div>
{% for path in job.output_paths %}
<div class="export-path">{{ path }}</div>
{% endfor %}
{% if job.error %}
<div class="export-error">{{ job.error }}</div>
{% endif %}
</div>
{% endfor %}
</section>
{% endif %}
</div>
</div>
</body>
</html>
"""
@app.route("/", methods=["GET"])
def index():
return render_template_string(HTML_FORM)
ALLOWED_YOUTUBE_HOSTS = frozenset(
{
"youtube.com",
"www.youtube.com",
"m.youtube.com",
"music.youtube.com",
"youtu.be",
}
)
MAX_URL_LENGTH = 2048
MAX_SUBMITTER_LENGTH = 200
MAX_NOTE_LENGTH = 1000
MAX_REQUEST_CONTENT_LENGTH = 32 * 1024
DOWNLOAD_MODE_LABELS = {
"player_slot": "Player Slot",
"mp4": "MP4 Export",
"m4a": "M4A Audio",
"mp3": "MP3 Audio",
}
EXPORT_STATUS_LABELS = {
"queued": "queued",
"running": "downloading",
"completed": "completed",
"failed": "failed",
"interrupted": "interrupted",
}
@app.route("/submit", methods=["POST"])
def submit():
def _valid_media_url(value: str) -> bool:
if not value or len(value) > MAX_URL_LENGTH:
return False
try:
parsed = urlsplit(value)
hostname = parsed.hostname
has_userinfo = parsed.username is not None or parsed.password is not None
except ValueError:
return False
if parsed.scheme.lower() not in {"http", "https"}:
return False
if has_userinfo or not hostname:
return False
return hostname.lower() in ALLOWED_YOUTUBE_HOSTS
def _submission_message(result: dict[str, object]) -> str:
status = result.get("status")
if result.get("kind") == "export":
mode = str(result.get("mode") or "")
label = DOWNLOAD_MODE_LABELS.get(mode, "Utility export")
if status == "queue_full":
return "The utility export queue is full; please try again later."
if status == "shutting_down":
return "The Controller is shutting down; please try again shortly."
if status == "worker_error":
return f"The {label} job was reserved, but its worker could not start."
if status != "queued":
return f"The Controller could not queue that {label} job; please try again."
job_id = str(result.get("job_id") or "")[:8]
identity = f" (job {job_id})" if job_id else ""
return f"{label} queued{identity}. Refresh this page to see its result."
if status == "queue_full":
seconds = max(0, int(result.get("oldest_expires_in_seconds") or 0))
hours = max(1, (seconds + 3599) // 3600)
return f"Queue is full. Oldest playlist expires in about {hours} hour(s)."
if status == "slot_busy":
return f"Slot {result.get('slot')} is already being downloaded."
if status == "slot_protected":
return (
f"Slot {result.get('slot')} is protected by active or uncertain playback; "
"the playlist was not replaced."
)
if status == "shutting_down":
return "The Controller is shutting down; please try again shortly."
if status == "worker_error":
return "The playlist was reserved, but its download worker could not start."
if status != "queued":
return "The Controller could not queue that playlist; please try again."
slot = result.get("slot")
message = f"Playlist queued in slot {slot}."
if result.get("reused_slot"):
message += " (Existing slot – only new videos will be added.)"
else:
message += " It will start downloading shortly."
return message
def create_app(
config: ControllerConfig,
service: ControllerService,
*,
testing: bool = False,
) -> Flask:
"""Create the HTTP surface without starting MQTT, threads, or persistence."""
secret_key = os.environ.get("FLASK_SECRET_KEY")
if not secret_key and not testing:
raise RuntimeError("FLASK_SECRET_KEY must be set for the Controller web service")
web_app = Flask(__name__)
web_app.config.update(
TESTING=testing,
SECRET_KEY=secret_key or "controller-test-only-secret",
MAX_CONTENT_LENGTH=MAX_REQUEST_CONTENT_LENGTH,
)
web_app.extensions["controller_service"] = service
web_app.extensions["controller_config"] = config
@web_app.get("/")
def index():
try:
export_jobs = service.list_export_jobs(
limit=min(100, config.downloads.max_queued_exports + 10)
)
except Exception as exc:
LOG.warning("could not load utility export status: %s", exc)
export_jobs = []
return render_template_string(
HTML_FORM,
export_jobs=export_jobs,
download_mode_labels=DOWNLOAD_MODE_LABELS,
export_status_labels=EXPORT_STATUS_LABELS,
)
@web_app.errorhandler(413)
def _submission_too_large(_error):
flash("The submission was too large.")
return redirect(url_for("index"))
@web_app.post("/submit")
def submit():
url = request.form.get("url", "").strip()
submitter = request.form.get("submitter", "").strip() or None
note = request.form.get("note", "").strip() or None
raw_mode = request.form.get("download_mode", "player_slot")
download_mode = raw_mode.strip() if isinstance(raw_mode, str) else ""
if not url.startswith("http"):
flash("Invalid URL.")
if not _valid_media_url(url):
flash(
"Invalid URL. Only YouTube links are accepted "
"(e.g. https://www.youtube.com/... or https://youtu.be/...)."
)
return redirect(url_for("index"))
if submitter is not None and len(submitter) > MAX_SUBMITTER_LENGTH:
flash(f"Your name must be {MAX_SUBMITTER_LENGTH} characters or fewer.")
return redirect(url_for("index"))
if note is not None and len(note) > MAX_NOTE_LENGTH:
flash(f"The note must be {MAX_NOTE_LENGTH} characters or fewer.")
return redirect(url_for("index"))
if download_mode not in DOWNLOAD_MODE_LABELS:
flash("Invalid download type. Choose one of the available options.")
return redirect(url_for("index"))
result = enqueue_download(url, submitter, note)
if result.get("status") == "queue_full":
secs = result.get("oldest_expires_in_seconds", 0)
hours = max(1, secs // 3600)
flash(f"Queue is full. Oldest playlist expires in about {hours} hour(s).")
try:
if download_mode == "player_slot":
result = service.enqueue_download(url, submitter, note)
else:
slot = result.get("slot")
msg = f"Playlist queued in slot {slot}."
if result.get("reused_slot"):
msg += " (Existing slot – only new videos will be added.)"
result = service.enqueue_export(
url,
download_mode,
submitter,
note,
)
except ControllerUnavailableError as exc:
LOG.warning("playlist submission was rejected: %s", exc)
flash("The Controller is unavailable; please try again shortly.")
except StoreError as exc:
LOG.error("playlist submission hit a storage error: %s", exc)
flash("Controller storage is unavailable; the playlist was not queued.")
except ValueError as exc:
LOG.warning("playlist submission was rejected: %s", exc)
flash(f"The playlist request was rejected: {exc}")
except Exception:
LOG.exception("playlist submission failed unexpectedly")
flash("The Controller could not queue that playlist; please try again.")
else:
msg += " It will start downloading shortly."
flash(msg)
flash(_submission_message(result))
return redirect(url_for("index"))
@web_app.get("/healthz")
def healthz():
try:
health = service.health()
except Exception as exc:
LOG.exception("Controller health check failed")
return jsonify(
{
"status": "unavailable",
"started": False,
"coordinator_alive": False,
"error": str(exc),
}
), 503
serving = bool(health.get("started") and health.get("coordinator_alive"))
return jsonify(health), 200 if serving else 503
return web_app
def _run_development_server() -> None:
config = load_config()
service = ControllerService(config)
web_app = create_app(config, service)
service.start()
try:
web_app.run(host=config.web.host, port=config.web.port)
finally:
service.stop()
if __name__ == "__main__":
app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 7531)))
_run_development_server()
+67 -7
View File
@@ -3,13 +3,73 @@ mqtt:
port: 1883
username: null
password: null
client_id_prefix: "ytplayer"
client_id: yt-controller
keepalive_seconds: 30
reconnect_min_seconds: 1
reconnect_max_seconds: 30
command_queue_size: 512
# QoS 1 is accepted only after the broker PUBACKs within this bound.
publish_ack_timeout_seconds: 2
# Audio Router self-echo correlation stays claimable for this long after
# a Controller-originated mode-set publish is PUBACK'd, so a subscription
# echo that is delivered after the PUBACK is still recognized as ours
# instead of being mistaken for operator intent.
router_echo_grace_seconds: 2
topics:
cmd: "youtube/player/cmd"
status: "youtube/player/status"
player_command: youtube/player/cmd
player_status: youtube/player/status
player_state: youtube/player/state
player_telemetry: youtube/player/telemetry
controller_command: youtube/controller/cmd
ingest_progress: youtube/ingest/progress
playback_base: youtube/media/playback
router_mode_set: audio/player/mode/set
router_mode_state: audio/player/mode/state
# Optional additive result/warning topic. Set null to disable it.
# Existing consumers never depend on this topic.
controller_result: youtube/controller/v2/events
player:
display: ":0"
xdg_runtime_dir: "/run/user/1000"
mpv_binary: "/usr/bin/mpv"
storage:
media_base: /mnt/Media/YTPlayer
data_base: /controller-data
database_name: controller.sqlite3
legacy_yaml_name: playlists.yaml
slot_count: 10
retention_seconds: 172800
expiry_interval_seconds: 60
# Marks uncertain playback for operator review; it never removes protection.
uncertain_review_after_seconds: 900
downloads:
yt_dlp_binary: yt-dlp
cookie_path: /run/secrets/yt-dlp-cookies.txt
cookie_required: false
max_workers: 2
process_timeout_seconds: 7200
max_height: 1080
# Utility exports live below media_base and never participate in slots.
export_directory: exports
export_max_height: 1080
# Dedicated capacity prevents utility exports from starving Player work.
export_max_workers: 1
# Bounds queued + running utility work; Player slots retain their own bound.
max_queued_exports: 20
router:
required_mode: video
confirmation_timeout_seconds: 3
# attempt: request video and wait for the finite confirmation timeout
# continue: request video but do not wait when no state has been observed
# reject: reject playback until a Router state has been observed
when_state_unknown: attempt
continue_on_confirmation_failure: true
fallback_restore_mode: null
homeassistant:
enabled: true
discovery_prefix: homeassistant
web:
host: 0.0.0.0
port: 7531
+5
View File
@@ -0,0 +1,5 @@
"""YTPlayer Controller v2 package."""
from .config import ControllerConfig, ConfigError, load_config
__all__ = ["ControllerConfig", "ConfigError", "load_config"]
+356
View File
@@ -0,0 +1,356 @@
"""Validated configuration for Controller v2."""
from __future__ import annotations
import math
import os
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Mapping
import yaml
from .contracts import SLOT_IDS, Topics
BASE_DIR = Path(__file__).resolve().parent.parent
DEFAULT_CONFIG_PATH = BASE_DIR / "config.yaml"
class ConfigError(ValueError):
"""Raised when Controller configuration is invalid."""
@dataclass(frozen=True)
class MqttConfig:
host: str = "10.40.0.2"
port: int = 1883
username: str | None = None
password: str | None = None
client_id: str = "yt-controller"
keepalive_seconds: int = 30
reconnect_min_seconds: int = 1
reconnect_max_seconds: int = 30
command_queue_size: int = 512
publish_ack_timeout_seconds: float = 2.0
router_echo_grace_seconds: float = 2.0
@dataclass(frozen=True)
class StorageConfig:
media_base: Path = Path("/mnt/Media/YTPlayer")
data_base: Path = Path("/controller-data")
database_name: str = "controller.sqlite3"
legacy_yaml_name: str = "playlists.yaml"
retention_seconds: int = 48 * 3600
expiry_interval_seconds: float = 60.0
uncertain_review_after_seconds: float = 15 * 60.0
@property
def database_path(self) -> Path:
return self.data_base / self.database_name
@property
def legacy_yaml_path(self) -> Path:
return self.data_base / self.legacy_yaml_name
@dataclass(frozen=True)
class DownloadConfig:
yt_dlp_binary: str = "yt-dlp"
cookie_path: Path | None = Path("/run/secrets/yt-dlp-cookies.txt")
cookie_required: bool = False
max_workers: int = 2
process_timeout_seconds: float = 2 * 3600.0
max_height: int = 1080
export_directory: Path = Path("exports")
export_max_height: int = 1080
export_max_workers: int = 1
max_queued_exports: int = 20
@dataclass(frozen=True)
class RouterConfig:
required_mode: str = "video"
confirmation_timeout_seconds: float = 3.0
when_state_unknown: str = "attempt"
continue_on_confirmation_failure: bool = True
fallback_restore_mode: str | None = None
@dataclass(frozen=True)
class HomeAssistantConfig:
enabled: bool = True
discovery_prefix: str = "homeassistant"
@dataclass(frozen=True)
class WebConfig:
host: str = "0.0.0.0"
port: int = 7531
@dataclass(frozen=True)
class ControllerConfig:
mqtt: MqttConfig = field(default_factory=MqttConfig)
topics: Topics = field(default_factory=Topics)
storage: StorageConfig = field(default_factory=StorageConfig)
downloads: DownloadConfig = field(default_factory=DownloadConfig)
router: RouterConfig = field(default_factory=RouterConfig)
homeassistant: HomeAssistantConfig = field(default_factory=HomeAssistantConfig)
web: WebConfig = field(default_factory=WebConfig)
@property
def slot_ids(self) -> tuple[int, ...]:
return SLOT_IDS
def _mapping(value: Any, name: str) -> Mapping[str, Any]:
if value is None:
return {}
if not isinstance(value, Mapping):
raise ConfigError(f"{name} must be a mapping")
return value
def _text(value: Any, name: str, *, allow_none: bool = False) -> str | None:
if value is None and allow_none:
return None
if not isinstance(value, str) or not value.strip():
raise ConfigError(f"{name} must be a non-empty string")
return value.strip()
def _optional_text(value: Any, name: str) -> str | None:
"""Normalize unset/blank optional environment values to ``None``."""
if value is None or (isinstance(value, str) and not value.strip()):
return None
return _text(value, name)
def _integer(value: Any, name: str, *, minimum: int = 1, maximum: int | None = None) -> int:
if isinstance(value, bool):
raise ConfigError(f"{name} must be an integer")
try:
result = int(value)
except (TypeError, ValueError, OverflowError) as exc:
raise ConfigError(f"{name} must be an integer") from exc
if result < minimum or (maximum is not None and result > maximum):
upper = f" and <= {maximum}" if maximum is not None else ""
raise ConfigError(f"{name} must be >= {minimum}{upper}")
return result
def _number(value: Any, name: str, *, minimum: float = 0.0) -> float:
if isinstance(value, bool):
raise ConfigError(f"{name} must be a number")
try:
result = float(value)
except (TypeError, ValueError, OverflowError) as exc:
raise ConfigError(f"{name} must be a number") from exc
if not math.isfinite(result):
raise ConfigError(f"{name} must be finite")
if result < minimum:
raise ConfigError(f"{name} must be >= {minimum}")
return result
def _boolean(value: Any, name: str) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
normalized = value.strip().lower()
if normalized in {"1", "true", "yes", "on"}:
return True
if normalized in {"0", "false", "no", "off"}:
return False
raise ConfigError(f"{name} must be a boolean")
def _optional_path(value: Any, name: str) -> Path | None:
if value in (None, ""):
return None
text = _text(value, name)
assert text is not None
return Path(text)
def _relative_directory(value: Any, name: str) -> Path:
text = _text(value, name)
assert text is not None
path = Path(text)
if (
path.is_absolute()
or not path.parts
or any(part in {".", ".."} for part in path.parts)
or "\\" in text
):
raise ConfigError(f"{name} must be a safe relative directory")
reserved_roots = {f"slot{slot}" for slot in SLOT_IDS} | {".controller-trash"}
if path.parts[0].casefold() in reserved_roots:
raise ConfigError(f"{name} must not overlap Controller slot storage")
return path
def _env_value(env: Mapping[str, str], key: str, fallback: Any) -> Any:
return env[key] if key in env else fallback
def _optional_env_value(env: Mapping[str, str], key: str, fallback: Any) -> Any:
value = env.get(key)
if value is None or not value.strip():
return fallback
return value
def load_config(
path: str | Path | None = None,
*,
environ: Mapping[str, str] | None = None,
) -> ControllerConfig:
"""Load YAML relative to the repository and apply documented env overrides."""
config_path = Path(path) if path is not None else DEFAULT_CONFIG_PATH
raw: Mapping[str, Any] = {}
if config_path.exists():
loaded = yaml.safe_load(config_path.read_text(encoding="utf-8"))
raw = _mapping(loaded, "configuration")
env = os.environ if environ is None else environ
mqtt_raw = _mapping(raw.get("mqtt"), "mqtt")
topics_raw = _mapping(raw.get("topics"), "topics")
storage_raw = _mapping(raw.get("storage"), "storage")
downloads_raw = _mapping(raw.get("downloads"), "downloads")
router_raw = _mapping(raw.get("router"), "router")
ha_raw = _mapping(raw.get("homeassistant"), "homeassistant")
web_raw = _mapping(raw.get("web"), "web")
mqtt = MqttConfig(
host=_text(_env_value(env, "MQTT_HOST", mqtt_raw.get("host", "10.40.0.2")), "mqtt.host"),
port=_integer(_env_value(env, "MQTT_PORT", mqtt_raw.get("port", 1883)), "mqtt.port", maximum=65535),
username=_optional_text(_optional_env_value(env, "MQTT_USERNAME", mqtt_raw.get("username")), "mqtt.username"),
password=_optional_text(_optional_env_value(env, "MQTT_PASSWORD", mqtt_raw.get("password")), "mqtt.password"),
client_id=_text(_env_value(env, "MQTT_CLIENT_ID", mqtt_raw.get("client_id", "yt-controller")), "mqtt.client_id"),
keepalive_seconds=_integer(mqtt_raw.get("keepalive_seconds", 30), "mqtt.keepalive_seconds"),
reconnect_min_seconds=_integer(mqtt_raw.get("reconnect_min_seconds", 1), "mqtt.reconnect_min_seconds"),
reconnect_max_seconds=_integer(mqtt_raw.get("reconnect_max_seconds", 30), "mqtt.reconnect_max_seconds"),
command_queue_size=_integer(mqtt_raw.get("command_queue_size", 512), "mqtt.command_queue_size"),
publish_ack_timeout_seconds=_number(
mqtt_raw.get("publish_ack_timeout_seconds", 2.0),
"mqtt.publish_ack_timeout_seconds",
minimum=0.1,
),
router_echo_grace_seconds=_number(
mqtt_raw.get("router_echo_grace_seconds", 2.0),
"mqtt.router_echo_grace_seconds",
minimum=0.0,
),
)
if mqtt.reconnect_max_seconds < mqtt.reconnect_min_seconds:
raise ConfigError("mqtt.reconnect_max_seconds must be >= reconnect_min_seconds")
if (mqtt.username is None) != (mqtt.password is None):
raise ConfigError("mqtt.username and mqtt.password must either both be set or both be null")
topics = Topics(
player_command=_text(topics_raw.get("player_command", "youtube/player/cmd"), "topics.player_command"),
player_status=_text(topics_raw.get("player_status", "youtube/player/status"), "topics.player_status"),
player_state=_text(topics_raw.get("player_state", "youtube/player/state"), "topics.player_state"),
player_telemetry=_text(topics_raw.get("player_telemetry", "youtube/player/telemetry"), "topics.player_telemetry"),
controller_command=_text(topics_raw.get("controller_command", "youtube/controller/cmd"), "topics.controller_command"),
ingest_progress=_text(topics_raw.get("ingest_progress", "youtube/ingest/progress"), "topics.ingest_progress"),
playback_base=_text(topics_raw.get("playback_base", "youtube/media/playback"), "topics.playback_base"),
router_mode_set=_text(topics_raw.get("router_mode_set", "audio/player/mode/set"), "topics.router_mode_set"),
router_mode_state=_text(topics_raw.get("router_mode_state", "audio/player/mode/state"), "topics.router_mode_state"),
controller_result=_text(
topics_raw.get("controller_result", "youtube/controller/v2/events"),
"topics.controller_result",
allow_none=True,
),
)
configured_slots = _integer(storage_raw.get("slot_count", 10), "storage.slot_count")
if configured_slots != len(SLOT_IDS):
raise ConfigError("storage.slot_count is fixed at 10 for stable slot identities")
storage = StorageConfig(
media_base=Path(_text(_env_value(env, "YTPLAYER_MEDIA_BASE", storage_raw.get("media_base", "/mnt/Media/YTPlayer")), "storage.media_base")),
data_base=Path(_text(_env_value(env, "YTPLAYER_DATA_BASE", storage_raw.get("data_base", "/controller-data")), "storage.data_base")),
database_name=_text(storage_raw.get("database_name", "controller.sqlite3"), "storage.database_name"),
legacy_yaml_name=_text(storage_raw.get("legacy_yaml_name", "playlists.yaml"), "storage.legacy_yaml_name"),
retention_seconds=_integer(_env_value(env, "YTPLAYER_RETENTION_SECONDS", storage_raw.get("retention_seconds", 48 * 3600)), "storage.retention_seconds"),
expiry_interval_seconds=_number(storage_raw.get("expiry_interval_seconds", 60), "storage.expiry_interval_seconds", minimum=0.1),
uncertain_review_after_seconds=_number(storage_raw.get("uncertain_review_after_seconds", 15 * 60), "storage.uncertain_review_after_seconds", minimum=0.0),
)
if Path(storage.database_name).name != storage.database_name:
raise ConfigError("storage.database_name must be a filename")
if Path(storage.legacy_yaml_name).name != storage.legacy_yaml_name:
raise ConfigError("storage.legacy_yaml_name must be a filename")
if not storage.media_base.is_absolute():
raise ConfigError("storage.media_base must be an absolute path")
if not storage.data_base.is_absolute():
raise ConfigError("storage.data_base must be an absolute path")
cookie_raw = _env_value(env, "YTPLAYER_COOKIE_PATH", downloads_raw.get("cookie_path", "/run/secrets/yt-dlp-cookies.txt"))
downloads = DownloadConfig(
yt_dlp_binary=_text(downloads_raw.get("yt_dlp_binary", "yt-dlp"), "downloads.yt_dlp_binary"),
cookie_path=_optional_path(cookie_raw, "downloads.cookie_path"),
cookie_required=_boolean(downloads_raw.get("cookie_required", False), "downloads.cookie_required"),
max_workers=_integer(downloads_raw.get("max_workers", 2), "downloads.max_workers", maximum=10),
process_timeout_seconds=_number(downloads_raw.get("process_timeout_seconds", 2 * 3600), "downloads.process_timeout_seconds", minimum=1.0),
max_height=_integer(downloads_raw.get("max_height", 1080), "downloads.max_height"),
export_directory=_relative_directory(
downloads_raw.get("export_directory", "exports"),
"downloads.export_directory",
),
export_max_height=_integer(
downloads_raw.get("export_max_height", 1080),
"downloads.export_max_height",
),
export_max_workers=_integer(
downloads_raw.get("export_max_workers", 1),
"downloads.export_max_workers",
maximum=10,
),
max_queued_exports=_integer(
downloads_raw.get("max_queued_exports", 20),
"downloads.max_queued_exports",
# The web status endpoint deliberately returns at most 100 rows.
# Keep the queue bound within that view so every active job remains
# visible even when the configured queue is full.
maximum=100,
),
)
if downloads.cookie_required and downloads.cookie_path is None:
raise ConfigError("downloads.cookie_path is required when cookie_required is true")
if downloads.cookie_path is not None and not downloads.cookie_path.is_absolute():
raise ConfigError("downloads.cookie_path must be an absolute path")
unknown_behavior = _text(router_raw.get("when_state_unknown", "attempt"), "router.when_state_unknown")
if unknown_behavior not in {"attempt", "continue", "reject"}:
raise ConfigError("router.when_state_unknown must be attempt, continue, or reject")
fallback = _text(router_raw.get("fallback_restore_mode"), "router.fallback_restore_mode", allow_none=True)
router = RouterConfig(
required_mode=_text(router_raw.get("required_mode", "video"), "router.required_mode"),
confirmation_timeout_seconds=_number(router_raw.get("confirmation_timeout_seconds", 3.0), "router.confirmation_timeout_seconds", minimum=0.1),
when_state_unknown=unknown_behavior,
continue_on_confirmation_failure=_boolean(router_raw.get("continue_on_confirmation_failure", True), "router.continue_on_confirmation_failure"),
fallback_restore_mode=fallback,
)
homeassistant = HomeAssistantConfig(
enabled=_boolean(ha_raw.get("enabled", True), "homeassistant.enabled"),
discovery_prefix=_text(ha_raw.get("discovery_prefix", "homeassistant"), "homeassistant.discovery_prefix"),
)
web = WebConfig(
host=_text(web_raw.get("host", "0.0.0.0"), "web.host"),
port=_integer(_env_value(env, "PORT", web_raw.get("port", 7531)), "web.port", maximum=65535),
)
return ControllerConfig(
mqtt=mqtt,
topics=topics,
storage=storage,
downloads=downloads,
router=router,
homeassistant=homeassistant,
web=web,
)
+48
View File
@@ -0,0 +1,48 @@
"""Stable MQTT contracts shared by the Controller adapters."""
from __future__ import annotations
from dataclasses import dataclass
SLOT_IDS = tuple(range(1, 11))
@dataclass(frozen=True)
class Topics:
"""MQTT topic names.
Defaults preserve the established Player, Home Assistant, and Companion
integrations. Player ``state`` is fixed by the Player Node contract.
"""
player_command: str = "youtube/player/cmd"
player_status: str = "youtube/player/status"
player_state: str = "youtube/player/state"
player_telemetry: str = "youtube/player/telemetry"
controller_command: str = "youtube/controller/cmd"
ingest_progress: str = "youtube/ingest/progress"
playback_base: str = "youtube/media/playback"
router_mode_set: str = "audio/player/mode/set"
router_mode_state: str = "audio/player/mode/state"
controller_result: str | None = "youtube/controller/v2/events"
def slot_base(self, slot: int) -> str:
if slot not in SLOT_IDS:
raise ValueError(f"slot must be in 1..10, got {slot!r}")
return f"{self.playback_base}/{slot}"
PLAYER_COMMAND_QOS = 1
PLAYER_COMMAND_RETAIN = False
PLAYER_STATUS_QOS = 1
PLAYER_STATE_QOS = 0
PLAYER_TELEMETRY_QOS = 0
# Existing Controller-owned HA/Companion state was retained at QoS 0.
LEGACY_STATE_QOS = 0
LEGACY_STATE_RETAIN = True
# Router commands are ephemeral commands just like Player commands.
ROUTER_COMMAND_QOS = 1
ROUTER_COMMAND_RETAIN = False
File diff suppressed because it is too large Load Diff
+98
View File
@@ -0,0 +1,98 @@
"""Home Assistant discovery and legacy slot-publication helpers."""
from __future__ import annotations
import json
from dataclasses import dataclass
from typing import Any, Callable, Iterable
from .config import HomeAssistantConfig
from .contracts import LEGACY_STATE_QOS, LEGACY_STATE_RETAIN, SLOT_IDS, Topics
Publish = Callable[..., Any]
HA_DEVICE = {
"identifiers": ["ytplayer-controller"],
"name": "YouTube Controller",
"manufacturer": "River of Glory",
"model": "YT Controller",
}
@dataclass(frozen=True)
class SlotAnnouncement:
slot: int
title: str
state: str
count: int
items: tuple[str, ...] = ()
def discovery_messages(
topics: Topics,
config: HomeAssistantConfig,
) -> Iterable[tuple[str, str, int, bool]]:
"""Yield the existing thirty discovery documents unchanged."""
for slot in SLOT_IDS:
base = topics.slot_base(slot)
object_id = f"yt_slot_{slot}"
entities = (
(
"title",
{
"name": f"YT Slot {slot} Title",
"state_topic": f"{base}/title",
"unique_id": f"{object_id}_title",
"icon": "mdi:playlist-music",
"device": HA_DEVICE,
},
),
(
"state",
{
"name": f"YT Slot {slot} State",
"state_topic": f"{base}/state",
"unique_id": f"{object_id}_state",
"icon": "mdi:circle-small",
"device": HA_DEVICE,
},
),
(
"count",
{
"name": f"YT Slot {slot} Count",
"state_topic": f"{base}/count",
"unique_id": f"{object_id}_count",
"icon": "mdi:numeric",
"device": HA_DEVICE,
},
),
)
for suffix, payload in entities:
yield (
f"{config.discovery_prefix}/sensor/{object_id}_{suffix}/config",
json.dumps(payload),
LEGACY_STATE_QOS,
LEGACY_STATE_RETAIN,
)
def slot_messages(
announcement: SlotAnnouncement,
topics: Topics,
*,
clear_items_through: int = 0,
) -> Iterable[tuple[str, str, int, bool]]:
"""Yield retained legacy slot topics, including stale item cleanup."""
base = topics.slot_base(announcement.slot)
yield f"{base}/title", announcement.title, LEGACY_STATE_QOS, LEGACY_STATE_RETAIN
yield f"{base}/state", announcement.state, LEGACY_STATE_QOS, LEGACY_STATE_RETAIN
yield f"{base}/count", str(announcement.count), LEGACY_STATE_QOS, LEGACY_STATE_RETAIN
for index, title in enumerate(announcement.items, start=1):
yield f"{base}/items/{index}", title, LEGACY_STATE_QOS, LEGACY_STATE_RETAIN
for index in range(len(announcement.items) + 1, clear_items_through + 1):
yield f"{base}/items/{index}", "", LEGACY_STATE_QOS, LEGACY_STATE_RETAIN
+163
View File
@@ -0,0 +1,163 @@
"""Controller domain records shared by persistence and orchestration."""
from __future__ import annotations
from dataclasses import dataclass
MEDIA_STATES = frozenset(
{"empty", "queued", "downloading", "ready", "error", "deleting"}
)
LIVE_JOB_STATES = frozenset({"reserved", "running"})
LIVE_GUARD_STATES = frozenset({"active", "review_required"})
UTILITY_EXPORT_MODES = frozenset({"mp4", "m4a", "mp3"})
@dataclass(frozen=True)
class SlotRecord:
slot: int
generation: int
media_state: str
url: str | None
yt_id: str | None
title: str
item_count: int
published_item_count: int
created_at: float | None
ready_at: float | None
expires_at: float | None
last_error: str | None
updated_at: float
@dataclass(frozen=True)
class DownloadReservation:
status: str
slot: int | None = None
generation: int | None = None
job_id: str | None = None
reused_slot: bool = False
oldest_expires_at: float | None = None
@dataclass(frozen=True)
class DownloadJob:
job_id: str
slot: int
generation: int
url: str
status: str
submitter: str | None
note: str | None
requested_at: float
started_at: float | None
finished_at: float | None
error: str | None
@dataclass(frozen=True)
class ExportReservation:
status: str
job_id: str | None = None
@dataclass(frozen=True)
class ExportJob:
job_id: str
mode: str
url: str
status: str
submitter: str | None
note: str | None
requested_at: float
started_at: float | None
finished_at: float | None
output_paths: tuple[str, ...]
error: str | None
@dataclass(frozen=True)
class DeleteToken:
slot: int
generation: int
reason: str
@dataclass(frozen=True)
class SlotGuard:
guard_id: str
slot: int
kind: str
state: str
created_at: float
review_after: float | None
cleared_at: float | None
cleared_by: str | None
clear_reason: str | None
@dataclass(frozen=True)
class PlayerObservation:
availability: str
phase: str
active_slot: int | None
active_path: str | None
paused: bool | None
generation: int
reconciliation_state: str
reconciliation_due_at: float | None
reconciliation_id: str | None
last_status_at: float | None
last_telemetry_at: float | None
@dataclass(frozen=True)
class RouterObservation:
mode: str | None
freshness: str
observed_at: float | None
last_warning: str | None
@dataclass(frozen=True)
class AudioLeaseRecord:
lease_id: str
operation_id: str | None
playback_generation: int | None
state: str
previous_mode: str | None
target_mode: str
restoration_mode: str | None
restoration_source: str
controller_changed: bool
confirmation_deadline: float | None
created_at: float
acquired_at: float | None
released_at: float | None
warning: str | None
error: str | None
@dataclass(frozen=True)
class MigrationReport:
migrated: bool
source_path: str | None = None
source_sha256: str | None = None
backup_path: str | None = None
warnings: tuple[str, ...] = ()
class StoreError(RuntimeError):
"""Base persistence error."""
class MigrationError(StoreError):
"""Legacy migration was rejected or could not complete safely."""
class SlotProtectedError(StoreError):
"""A slot cannot be modified because live activity protects it."""
class StaleOperationError(StoreError):
"""An operation token no longer identifies the slot occupant."""
+559
View File
@@ -0,0 +1,559 @@
"""Resilient MQTT transport with explicit Player command guarantees."""
from __future__ import annotations
import json
import logging
import queue
import threading
import time
import uuid
from dataclasses import dataclass
from typing import Any, Mapping
import paho.mqtt.client as mqtt
from .config import ControllerConfig
from .contracts import (
PLAYER_COMMAND_QOS,
PLAYER_COMMAND_RETAIN,
PLAYER_STATE_QOS,
PLAYER_STATUS_QOS,
PLAYER_TELEMETRY_QOS,
ROUTER_COMMAND_QOS,
ROUTER_COMMAND_RETAIN,
)
LOG = logging.getLogger(__name__)
@dataclass(frozen=True)
class MqttEvent:
kind: str
topic: str | None = None
payload: bytes = b""
qos: int = 0
retain: bool = False
duplicate: bool = False
message_id: int | None = None
connection_generation: int | None = None
reason: str | None = None
controller_echo: bool = False
@dataclass(frozen=True)
class PublishReceipt:
accepted: bool
message_id: int | None = None
reason: str | None = None
uncertain: bool = False
class MqttAdapter:
"""Own Paho lifecycle while keeping all domain work off its callbacks."""
def __init__(
self,
config: ControllerConfig,
*,
client: Any | None = None,
events: "queue.Queue[MqttEvent] | None" = None,
monotonic=time.monotonic,
):
self.config = config
self.events: "queue.Queue[MqttEvent]" = events or queue.Queue(
maxsize=config.mqtt.command_queue_size
)
self._injected_client = client is not None
self.client = client or self._new_client()
self._generation_lock = threading.Lock()
self._connection_generation = 0
self._connected = threading.Event()
self._stop_event = threading.Event()
self._lifecycle_lock = threading.Lock()
self._started = False
self._loop_started = False
self._bootstrap_thread: threading.Thread | None = None
self._monotonic = monotonic
self._router_echo_lock = threading.Lock()
# token -> (exact payload, grace deadline). The deadline is ``None``
# while the correlated publish() call has not yet observed its
# PUBACK; such an entry never expires on its own, bounded instead by
# the publish ack timeout. Once PUBACK is observed, the entry keeps
# a short grace window so a subscription echo delivered just after
# the PUBACK is still recognized as ours rather than mistaken for
# operator intent.
self._pending_router_echoes: dict[str, tuple[bytes, float | None]] = {}
self.last_error: str | None = None
self._configure_client()
def _new_client(self):
return mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2,
client_id=self.config.mqtt.client_id,
clean_session=True,
)
def _configure_client(self) -> None:
if self.config.mqtt.username is not None:
self.client.username_pw_set(
self.config.mqtt.username,
self.config.mqtt.password,
)
self.client.on_connect = self._on_connect
self.client.on_connect_fail = self._on_connect_fail
self.client.on_disconnect = self._on_disconnect
self.client.on_message = self._on_message
self.client.suppress_exceptions = True
self.client.manual_ack_set(True)
self.client.max_queued_messages_set(self.config.mqtt.command_queue_size)
self.client.reconnect_delay_set(
min_delay=self.config.mqtt.reconnect_min_seconds,
max_delay=self.config.mqtt.reconnect_max_seconds,
)
@property
def connected(self) -> bool:
return self._connected.is_set()
@property
def connection_generation(self) -> int:
with self._generation_lock:
return self._connection_generation
@property
def started(self) -> bool:
with self._lifecycle_lock:
return self._started
def start(self) -> None:
"""Start without requiring the broker or network to be reachable."""
with self._lifecycle_lock:
if self._started:
return
self._started = True
self._stop_event.clear()
try:
self._begin_async_connection()
except (OSError, mqtt.WebsocketConnectionError, RuntimeError) as exc:
self.last_error = f"initial MQTT setup failed: {exc}"
LOG.warning("%s; retrying in background", self.last_error)
self._start_bootstrap_retry()
def _begin_async_connection(self) -> None:
with self._lifecycle_lock:
if not self._started or self._stop_event.is_set():
return
client = self.client
result = client.connect_async(
self.config.mqtt.host,
self.config.mqtt.port,
keepalive=self.config.mqtt.keepalive_seconds,
)
# Paho 2.x returns None after configuring an asynchronous connect;
# older/test clients may return MQTT_ERR_SUCCESS.
if result not in (None, mqtt.MQTT_ERR_SUCCESS):
raise RuntimeError(f"failed to configure MQTT connection: {result}")
with self._lifecycle_lock:
if (
not self._started
or self._stop_event.is_set()
or client is not self.client
):
return
if not self._loop_started:
result = client.loop_start()
if result != mqtt.MQTT_ERR_SUCCESS:
raise RuntimeError(f"failed to start MQTT network loop: {result}")
self._loop_started = True
def _start_bootstrap_retry(self) -> None:
with self._lifecycle_lock:
if self._bootstrap_thread is not None and self._bootstrap_thread.is_alive():
return
self._bootstrap_thread = threading.Thread(
target=self._bootstrap_retry,
name="mqtt-bootstrap-retry",
daemon=True,
)
self._bootstrap_thread.start()
def _bootstrap_retry(self) -> None:
delay = float(self.config.mqtt.reconnect_min_seconds)
maximum = float(self.config.mqtt.reconnect_max_seconds)
while not self._stop_event.wait(delay):
try:
self._begin_async_connection()
self.last_error = None
return
except (OSError, mqtt.WebsocketConnectionError) as exc:
self.last_error = f"MQTT setup retry failed: {exc}"
LOG.warning("%s", self.last_error)
delay = min(maximum, max(delay * 2, 1.0))
except RuntimeError as exc:
self.last_error = str(exc)
LOG.warning("%s", self.last_error)
delay = min(maximum, max(delay * 2, 1.0))
def stop(self) -> None:
# Signal callbacks/retry work before waiting for the lifecycle lock.
# A callback already inside subscribe/connect then observes shutdown
# and cannot publish a post-stop connected state.
self._stop_event.set()
with self._lifecycle_lock:
if not self._started:
self._connected.clear()
return
self._started = False
loop_started = self._loop_started
self._loop_started = False
self._connected.clear()
if loop_started:
try:
self.client.disconnect()
except Exception:
LOG.exception("MQTT disconnect failed")
self.client.loop_stop()
thread = self._bootstrap_thread
if thread is not None and thread is not threading.current_thread():
thread.join(timeout=1.0)
@staticmethod
def _reason_success(reason_code: Any) -> bool:
try:
return int(reason_code) == 0
except (TypeError, ValueError):
return reason_code == 0
def _on_connect(self, client, userdata, flags, reason_code, properties=None) -> None:
try:
with self._lifecycle_lock:
if (
client is not self.client
or not self._started
or self._stop_event.is_set()
):
return
if not self._reason_success(reason_code):
self.last_error = f"MQTT connect failed: {reason_code}"
LOG.warning("%s", self.last_error)
return
if self._stop_event.is_set():
return
with self._generation_lock:
self._connection_generation += 1
generation = self._connection_generation
self._connected.set()
self.last_error = None
subscriptions = (
(self.config.topics.player_status, PLAYER_STATUS_QOS),
(self.config.topics.player_telemetry, PLAYER_TELEMETRY_QOS),
(self.config.topics.player_state, PLAYER_STATE_QOS),
(self.config.topics.controller_command, 1),
(self.config.topics.router_mode_set, ROUTER_COMMAND_QOS),
(self.config.topics.router_mode_state, 1),
)
for topic, qos in subscriptions:
if self._stop_event.is_set():
self._connected.clear()
return
client.subscribe(topic, qos=qos)
if self._stop_event.is_set():
self._connected.clear()
return
if self._stop_event.is_set():
self._connected.clear()
return
self._enqueue(
MqttEvent(
kind="connected",
connection_generation=generation,
)
)
except BaseException:
LOG.exception("unexpected MQTT on_connect failure")
def _on_connect_fail(self, client, userdata) -> None:
self.last_error = (
f"MQTT connection to {self.config.mqtt.host}:{self.config.mqtt.port} failed"
)
LOG.warning("%s; Paho will retry", self.last_error)
def _on_disconnect(
self,
client,
userdata,
disconnect_flags,
reason_code,
properties=None,
) -> None:
if client is not self.client:
return
self._connected.clear()
try:
self._enqueue(
MqttEvent(
kind="disconnected",
connection_generation=self.connection_generation,
reason=str(reason_code),
)
)
except BaseException:
LOG.exception("unexpected MQTT on_disconnect failure")
def _on_message(self, client, userdata, message) -> None:
try:
# Capture client identity and generation atomically with session
# replacement. A callback that was already running when an
# uncertain command discards the transport must be able to leave a
# full queue so loop_stop() cannot deadlock waiting for it.
with self._lifecycle_lock:
if client is not self.client:
return
with self._generation_lock:
generation = self._connection_generation
topic = str(message.topic)
payload = bytes(message.payload)
retained = bool(getattr(message, "retain", False))
self._enqueue(
MqttEvent(
kind="message",
topic=topic,
payload=payload,
qos=int(getattr(message, "qos", 0)),
retain=retained,
duplicate=bool(getattr(message, "dup", False)),
message_id=getattr(message, "mid", None),
connection_generation=generation,
controller_echo=(
topic == self.config.topics.router_mode_set
and not retained
and self._claim_router_echo(payload)
),
)
)
except BaseException:
LOG.exception("unexpected MQTT on_message failure")
def _enqueue(self, event: MqttEvent) -> bool:
while not self._stop_event.is_set():
if (
event.connection_generation is not None
and event.connection_generation != self.connection_generation
):
return False
try:
self.events.put(event, timeout=0.2)
return True
except queue.Full:
continue
return False
def acknowledge(self, event: MqttEvent) -> bool:
if event.message_id is None or event.qos <= 0:
return True
if event.connection_generation != self.connection_generation:
LOG.warning(
"skipping stale MQTT ack mid=%s from generation=%s",
event.message_id,
event.connection_generation,
)
return False
try:
result = self.client.ack(event.message_id, event.qos)
except BaseException:
LOG.exception("MQTT ack failed mid=%s", event.message_id)
return False
return result == mqtt.MQTT_ERR_SUCCESS
def _publish(
self,
topic: str,
payload: str,
*,
qos: int,
retain: bool,
require_connection: bool,
invalidate_transport_on_uncertain: bool = False,
) -> PublishReceipt:
if require_connection and (not self.started or not self.connected):
return PublishReceipt(False, reason="MQTT is disconnected")
try:
info = self.client.publish(topic, payload, qos=qos, retain=retain)
except BaseException as exc:
LOG.exception("MQTT publish failed topic=%s", topic)
return PublishReceipt(False, reason=str(exc))
if info.rc != mqtt.MQTT_ERR_SUCCESS:
return PublishReceipt(False, message_id=getattr(info, "mid", None), reason=f"MQTT rc={info.rc}")
if qos > 0 and hasattr(info, "wait_for_publish"):
try:
info.wait_for_publish(
timeout=self.config.mqtt.publish_ack_timeout_seconds
)
acknowledged = bool(info.is_published())
except BaseException as exc:
acknowledged = False
detail = str(exc) or exc.__class__.__name__
else:
detail = (
f"no broker PUBACK within "
f"{self.config.mqtt.publish_ack_timeout_seconds:g} seconds"
)
if not acknowledged:
reason = f"MQTT QoS 1 publication is uncertain: {detail}"
if invalidate_transport_on_uncertain:
self._discard_unacknowledged_transport(reason)
return PublishReceipt(
False,
message_id=getattr(info, "mid", None),
reason=reason,
uncertain=True,
)
return PublishReceipt(True, message_id=getattr(info, "mid", None))
def _discard_unacknowledged_transport(self, reason: str) -> None:
"""Drop the Paho session so stale QoS 1 commands cannot replay."""
if self._injected_client:
# Test transports do not own a recreatable Paho session. Production
# always follows the replacement path below.
with self._generation_lock:
self._connection_generation += 1
self._connected.clear()
return
with self._lifecycle_lock:
old_client = self.client
old_loop_started = self._loop_started
self._loop_started = False
should_restart = self._started and not self._stop_event.is_set()
self._connected.clear()
with self._generation_lock:
previous_generation = self._connection_generation
self._connection_generation += 1
self.client = self._new_client()
self._configure_client()
try:
old_client.disconnect()
except BaseException:
LOG.exception("failed to disconnect uncertain MQTT transport")
if old_loop_started:
try:
old_client.loop_stop()
except BaseException:
LOG.exception("failed to stop uncertain MQTT transport")
self.last_error = reason
try:
self.events.put_nowait(
MqttEvent(
kind="disconnected",
connection_generation=previous_generation,
reason=reason,
)
)
except queue.Full:
# This method can run on the sole coordinator thread; blocking on
# its own queue would deadlock. The uncertain PublishReceipt still
# drives the operation-specific conservative transition.
LOG.error("MQTT event queue was full while discarding uncertain transport")
if should_restart:
self._start_bootstrap_retry()
@staticmethod
def _json_payload(payload: Mapping[str, Any]) -> str:
return json.dumps(payload, allow_nan=False, separators=(",", ":"))
def publish_player_command(self, payload: Mapping[str, Any]) -> PublishReceipt:
return self._publish(
self.config.topics.player_command,
self._json_payload(payload),
qos=PLAYER_COMMAND_QOS,
retain=PLAYER_COMMAND_RETAIN,
require_connection=True,
invalidate_transport_on_uncertain=True,
)
def publish_router_mode(self, mode: str) -> PublishReceipt:
# A broker normally sends our publication back through the matching
# subscription. Register the exact payload before publishing because
# Paho's network thread may receive that echo before publish()
# returns. A PUBACK only proves broker receipt of our publish, not
# that the broker has already forwarded (or even scheduled) our own
# subscription copy of it; that copy can legally arrive after the
# PUBACK. So the marker is not removed the instant publish()
# completes: it stays claimable for a short bounded grace window
# afterward, and is removed immediately once actually claimed or
# once the grace window elapses. This closes the specific race where
# the Controller's own late echo would otherwise be mistaken for
# live operator intent and cause automatic restoration ownership to
# be relinquished.
echo_token = uuid.uuid4().hex
payload = mode.encode("utf-8")
with self._router_echo_lock:
self._prune_router_echoes_locked()
self._pending_router_echoes[echo_token] = (payload, None)
receipt = self._publish(
self.config.topics.router_mode_set,
mode,
qos=ROUTER_COMMAND_QOS,
retain=ROUTER_COMMAND_RETAIN,
require_connection=True,
invalidate_transport_on_uncertain=True,
)
grace_deadline = (
self._monotonic() + self.config.mqtt.router_echo_grace_seconds
)
with self._router_echo_lock:
if echo_token in self._pending_router_echoes:
self._pending_router_echoes[echo_token] = (payload, grace_deadline)
return receipt
def _claim_router_echo(self, payload: bytes) -> bool:
"""Claim one exact mode-set echo from an in-flight or recent local publish."""
with self._router_echo_lock:
self._prune_router_echoes_locked()
for token, (expected, _deadline) in tuple(
self._pending_router_echoes.items()
):
if expected == payload:
del self._pending_router_echoes[token]
return True
return False
def _prune_router_echoes_locked(self) -> None:
"""Drop expired grace-window entries. Caller holds ``_router_echo_lock``."""
now = self._monotonic()
expired = [
token
for token, (_, deadline) in self._pending_router_echoes.items()
if deadline is not None and deadline <= now
]
for token in expired:
del self._pending_router_echoes[token]
def publish_legacy(self, topic: str, payload: str, *, retain: bool = True) -> PublishReceipt:
return self._publish(
topic,
payload,
qos=0,
retain=retain,
require_connection=True,
)
def publish_json(
self,
topic: str,
payload: Mapping[str, Any],
*,
qos: int = 0,
retain: bool = False,
) -> PublishReceipt:
return self._publish(
topic,
self._json_payload(payload),
qos=qos,
retain=retain,
require_connection=True,
)
+899
View File
@@ -0,0 +1,899 @@
"""Authoritative Player status/telemetry reduction and operation fencing."""
from __future__ import annotations
import math
import os
import time
import uuid
from dataclasses import dataclass, replace
from typing import Any, Mapping
from .config import ControllerConfig
from .contracts import SLOT_IDS
from .models import PlayerObservation
from .store import ControllerStore
TERMINAL_REASONS = frozenset(
{"stop", "finished", "mpv_error", "sigterm", "sigint", "shutdown"}
)
TERMINAL_ERROR_REASONS = frozenset({"startup_error", "worker_start_error"})
CONTROL_ACTIONS = frozenset({"play", "stop", "pause", "resume", "toggle"})
@dataclass(frozen=True)
class PlayerOperation:
operation_id: str
generation: int
action: str
slot: int | None
expected_path: str | None
deadline: float
reconciliation_id: str | None = None
state: str = "awaiting_player"
message: str | None = None
@dataclass(frozen=True)
class PlaybackEffect:
kind: str
slot: int | None = None
operation_id: str | None = None
message: str | None = None
class OperationInProgressError(RuntimeError):
pass
class PlaybackReducer:
"""Apply Player evidence without projecting Controller intent as success."""
def __init__(
self,
config: ControllerConfig,
store: ControllerStore,
*,
monotonic=time.monotonic,
wall_clock=time.time,
operation_timeout_seconds: float = 10.0,
):
self.config = config
self.store = store
self._monotonic = monotonic
self._wall_clock = wall_clock
self.operation_timeout_seconds = operation_timeout_seconds
self._operation_generation = 0
self.pending: PlayerOperation | None = None
self.last_operation: PlayerOperation | None = None
def begin_operation(
self,
action: str,
*,
slot: int | None = None,
expected_path: str | None = None,
operation_id: str | None = None,
) -> PlayerOperation:
if action not in CONTROL_ACTIONS:
raise ValueError(f"unsupported Player action {action!r}")
if isinstance(slot, bool) or (slot is not None and slot not in SLOT_IDS):
raise ValueError("slot must be in 1..10")
if self.pending is not None:
raise OperationInProgressError(
f"Player operation {self.pending.operation_id} is still pending"
)
self._operation_generation += 1
operation = PlayerOperation(
operation_id=operation_id or uuid.uuid4().hex,
generation=self._operation_generation,
action=action,
slot=slot,
expected_path=expected_path,
deadline=self._monotonic() + self.operation_timeout_seconds,
)
if action == "play" and slot is not None:
self.store.protect_slot(
slot,
"pending_play",
guard_id=f"operation-{operation.operation_id}",
details={"operation_id": operation.operation_id},
)
elif action == "play" and expected_path is not None:
due = None
if self.config.storage.uncertain_review_after_seconds > 0:
due = (
self._wall_clock()
+ self.config.storage.uncertain_review_after_seconds
)
observation = self.store.mark_external_uncertain(
expected_path,
review_after=due,
details={
"operation_id": operation.operation_id,
"reason": "play command pending Player evidence",
},
)
operation = replace(
operation,
reconciliation_id=observation.reconciliation_id,
)
self.pending = operation
return operation
def publish_failed(self, reason: str) -> PlaybackEffect | None:
if self.pending is None:
return None
if (
self.pending.action == "play"
and self.pending.slot is None
and self.pending.reconciliation_id is not None
):
self.store.cancel_external_intent(
self.pending.reconciliation_id,
reason=reason,
)
return self._finish_operation("failed", reason)
def _finish_operation(self, state: str, message: str | None = None) -> PlaybackEffect:
assert self.pending is not None
operation = replace(self.pending, state=state, message=message)
self.pending = None
self.last_operation = operation
if operation.action == "play" and operation.slot is not None and state != "confirmed":
self.store.clear_playback_guards(
slot=operation.slot,
kind="pending_play",
reason=f"play operation {state}: {message or ''}".strip(),
)
return PlaybackEffect(
kind=f"operation_{state}",
slot=operation.slot,
operation_id=operation.operation_id,
message=message,
)
@staticmethod
def _slot(value: Any) -> int | None:
if value is None or isinstance(value, bool):
return None
try:
converted = int(value)
except (TypeError, ValueError, OverflowError):
return None
if isinstance(value, float) and (not math.isfinite(value) or not value.is_integer()):
return None
return converted if converted in SLOT_IDS else None
@staticmethod
def _path(value: Any) -> str | None:
return value if isinstance(value, str) and value else None
def _matches_pending(self, slot: int | None, path: str | None) -> bool:
operation = self.pending
if operation is None:
return False
positive_match = False
if operation.slot is not None and slot is not None:
if operation.slot != slot:
return False
positive_match = True
if operation.expected_path is not None and path is not None:
if os.path.abspath(operation.expected_path) != os.path.abspath(path):
return False
positive_match = True
return positive_match
def _canonical_slot_for_path(self, path: str | None) -> int | None:
if path is None:
return None
candidate = os.path.abspath(path)
for slot in SLOT_IDS:
slot_directory = os.path.abspath(
self.config.storage.media_base / f"slot{slot}"
)
try:
if os.path.commonpath((candidate, slot_directory)) == slot_directory:
return slot
except ValueError:
continue
return None
def _normalized_identity(
self,
payload: Mapping[str, Any],
) -> tuple[int | None, str | None, str | None]:
slot = self._slot(payload.get("slot"))
path = self._path(payload.get("path"))
derived = self._canonical_slot_for_path(path)
if slot is not None and derived is not None and slot != derived:
return (
slot,
path,
f"Player identity conflicts: slot {slot} does not contain path {path!r}",
)
return (derived if slot is None else slot), path, None
@staticmethod
def _same_playback_identity(
previous: PlayerObservation,
slot: int | None,
path: str | None,
) -> bool:
if previous.active_slot != slot:
return False
if slot is None:
return (
previous.active_path is not None
and path is not None
and os.path.abspath(previous.active_path) == os.path.abspath(path)
)
if previous.active_path is None or path is None:
return True
return os.path.abspath(previous.active_path) == os.path.abspath(path)
def _active_evidence(
self,
*,
slot: int | None,
path: str | None,
paused: bool | None,
status: Mapping[str, Any] | None = None,
telemetry: Mapping[str, Any] | None = None,
) -> PlayerObservation:
previous = self.store.player_observation()
if slot is None and (path is None or path == previous.active_path):
slot = previous.active_slot
if path is None and (slot is None or slot == previous.active_slot):
path = previous.active_path
if paused is None and self._same_playback_identity(previous, slot, path):
paused = previous.paused
phase = "paused" if paused is True else "playing"
if slot is not None:
self.store.reconcile_active_guard(
slot,
kind="player_active",
details={"path": path},
)
elif path is not None:
self.store.reconcile_external_active()
return self.store.set_player_observation(
availability="online",
phase=phase,
active_slot=slot,
active_path=path,
paused=paused,
reconciliation_state="confirmed",
reconciliation_due_at=None,
reconciliation_id=None,
status_payload=status,
telemetry_payload=telemetry,
)
def _idle_evidence(
self,
*,
status: Mapping[str, Any] | None = None,
telemetry: Mapping[str, Any] | None = None,
) -> PlayerObservation:
# Persist idle before removing protection. A crash between the two
# steps can leave harmless over-protection, never unsafe under-protection.
observation = self.store.set_player_observation(
availability="online",
phase="idle",
active_slot=None,
active_path=None,
paused=False,
reconciliation_state="clear",
reconciliation_due_at=None,
reconciliation_id=None,
status_payload=status,
telemetry_payload=telemetry,
)
self.store.clear_playback_guards(reason="fresh Player idle evidence")
return observation
def handle_status(self, payload: Any) -> list[PlaybackEffect]:
if not isinstance(payload, Mapping):
return [PlaybackEffect("warning", message="Player status must be an object")]
state = payload.get("state")
if not isinstance(state, str) or not state:
return [PlaybackEffect("warning", message="Player status is missing state")]
state = state.strip()
slot, path, identity_warning = self._normalized_identity(payload)
if identity_warning is not None:
return [PlaybackEffect("warning", slot=slot, message=identity_warning)]
effects: list[PlaybackEffect] = []
if state == "offline":
previous = self.store.player_observation()
due = None
active_known = (
previous.active_slot is not None or previous.active_path is not None
)
if previous.active_slot is not None:
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
guard = self.store.mark_slot_uncertain(
previous.active_slot,
review_after=due,
details={"reason": "Player offline"},
)
due = guard.review_after
self.store.set_player_observation(
availability="offline",
phase="unknown",
active_slot=previous.active_slot,
active_path=previous.active_path,
paused=previous.paused,
reconciliation_state=(
"operator_review"
if guard.state == "review_required"
else "awaiting_evidence"
),
reconciliation_due_at=due,
reconciliation_id=guard.guard_id,
status_payload=payload,
)
elif previous.active_path is not None:
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
self.store.mark_external_uncertain(
previous.active_path,
review_after=due,
availability="offline",
details={"reason": "Player offline"},
status_payload=payload,
)
else:
self.store.set_player_observation(
availability="offline",
phase=previous.phase,
active_slot=None,
active_path=None,
paused=previous.paused,
reconciliation_state="unknown",
reconciliation_due_at=None,
reconciliation_id=None,
status_payload=payload,
)
effects.append(
PlaybackEffect(
"player_offline",
slot=previous.active_slot,
message="Player availability is offline; playback is unknown",
)
)
return effects
if state == "idle" and payload.get("message") == "player online":
if path is not None:
self._active_evidence(slot=slot, path=path, paused=None, status=payload)
effects.append(
PlaybackEffect(
"player_reconnected_active",
slot=slot,
message="Reconnect idle announcement carried an active path",
)
)
if self.pending is not None and self.pending.action == "play":
if self._matches_pending(slot, path):
effects.append(
self._finish_operation(
"confirmed", "matching reconnect evidence"
)
)
else:
effects.append(
self._finish_operation(
"failed",
"Player reconnected with different active playback",
)
)
else:
self._idle_evidence(status=payload)
effects.append(PlaybackEffect("player_idle"))
if self.pending is not None:
if self.pending.action == "stop":
effects.append(self._finish_operation("confirmed"))
elif self.pending.action == "play":
effects.append(
self._finish_operation(
"failed", "Player reconnected without active playback"
)
)
return effects
if state == "idle":
reason = payload.get("reason")
if reason not in TERMINAL_REASONS:
previous = self.store.player_observation()
self.store.set_player_observation(
availability="online",
phase=previous.phase,
active_slot=previous.active_slot,
active_path=previous.active_path,
paused=previous.paused,
reconciliation_state=previous.reconciliation_state,
reconciliation_due_at=previous.reconciliation_due_at,
reconciliation_id=previous.reconciliation_id,
status_payload=payload,
)
return [
PlaybackEffect(
"warning",
slot=previous.active_slot,
message="ambiguous Player idle status was not treated as terminal",
)
]
self._idle_evidence(status=payload)
effects.append(PlaybackEffect("player_idle", slot=slot, message=str(reason or "idle")))
if self.pending is not None:
if self.pending.action == "stop":
effects.append(self._finish_operation("confirmed", str(reason or "idle")))
elif self.pending.action == "play":
effects.append(self._finish_operation("failed", str(reason or "Player remained idle")))
return effects
if state == "playing":
self._active_evidence(
slot=slot,
path=path,
paused=None,
status=payload,
)
effects.append(PlaybackEffect("player_active", slot=slot))
if (
self.pending is not None
and self.pending.action == "play"
):
if self._matches_pending(slot, path):
effects.append(self._finish_operation("confirmed"))
else:
effects.append(
self._finish_operation(
"failed",
"Player became active with a different slot or path",
)
)
return effects
if state in {"paused", "toggled"}:
self._active_evidence(
slot=slot,
path=path,
paused=None,
status=payload,
)
effects.append(
PlaybackEffect(
"control_issued",
slot=slot,
message=f"Player issued {state}; awaiting telemetry",
)
)
if self.pending is not None and self.pending.action == "play":
if self._matches_pending(slot, path):
effects.append(self._finish_operation("confirmed", state))
else:
effects.append(
self._finish_operation(
"failed",
"Player control status identified different playback",
)
)
return effects
if state == "busy":
previous = self.store.player_observation()
if (
slot is not None
or path is not None
or previous.active_slot is not None
or previous.active_path is not None
):
self._active_evidence(
slot=slot,
path=path,
paused=None,
status=payload,
)
else:
self.store.set_player_observation(
availability="online",
phase="unknown",
active_slot=None,
active_path=None,
paused=None,
reconciliation_state="unknown",
reconciliation_due_at=None,
reconciliation_id=None,
status_payload=payload,
)
effects.append(PlaybackEffect("player_busy", slot=slot, message=payload.get("message")))
if self.pending is not None and self.pending.action == "play":
effects.append(self._finish_operation("failed", "Player is busy"))
return effects
if state == "error":
previous = self.store.player_observation()
message = str(payload.get("message") or "Player error")
reason = payload.get("reason")
if reason in TERMINAL_ERROR_REASONS:
terminal_slot = slot if slot is not None else previous.active_slot
self._idle_evidence(status=payload)
effects.extend(
(
PlaybackEffect(
"player_error",
slot=terminal_slot,
message=message,
),
PlaybackEffect(
"player_idle",
slot=terminal_slot,
message=str(reason),
),
)
)
if self.pending is not None:
effects.append(self._finish_operation("failed", message))
return effects
active_slot = slot if slot is not None else previous.active_slot
active_path = path if path is not None else previous.active_path
due = None
if self.config.storage.uncertain_review_after_seconds > 0:
due = (
self._wall_clock()
+ self.config.storage.uncertain_review_after_seconds
)
if active_slot is not None:
guard = self.store.mark_slot_uncertain(
active_slot,
review_after=due,
details={"reason": "Player error", "message": message},
)
due = guard.review_after
self.store.set_player_observation(
availability="online",
phase="error",
active_slot=active_slot,
active_path=active_path,
paused=(
previous.paused
if self._same_playback_identity(
previous,
active_slot,
active_path,
)
else None
),
reconciliation_state=(
"operator_review"
if guard.state == "review_required"
else "awaiting_evidence"
),
reconciliation_due_at=due,
reconciliation_id=guard.guard_id,
status_payload=payload,
)
elif active_path is not None:
self.store.mark_external_uncertain(
active_path,
review_after=due,
availability="online",
details={"reason": "Player error", "message": message},
status_payload=payload,
)
else:
self.store.set_player_observation(
availability="online",
phase="unknown",
active_slot=None,
active_path=None,
paused=None,
reconciliation_state="unknown",
reconciliation_due_at=None,
reconciliation_id=None,
status_payload=payload,
)
effects.append(PlaybackEffect("player_error", slot=active_slot, message=message))
if self.pending is not None:
effects.append(self._finish_operation("failed", message))
return effects
effects.append(PlaybackEffect("warning", slot=slot, message=f"unknown Player state {state!r}"))
return effects
def handle_telemetry(self, payload: Any) -> list[PlaybackEffect]:
if not isinstance(payload, Mapping) or payload.get("state") != "telemetry":
return [PlaybackEffect("warning", message="invalid Player telemetry envelope")]
slot, path, identity_warning = self._normalized_identity(payload)
if identity_warning is not None:
return [PlaybackEffect("warning", slot=slot, message=identity_warning)]
effects: list[PlaybackEffect] = []
if path is None:
self._idle_evidence(telemetry=payload)
effects.append(PlaybackEffect("player_idle", slot=slot, message="cleared telemetry"))
if self.pending is not None:
if self.pending.action == "stop":
effects.append(self._finish_operation("confirmed", "cleared telemetry"))
elif self.pending.action == "play":
effects.append(self._finish_operation("failed", "playback cleared"))
return effects
paused_value = payload.get("paused")
paused = paused_value if isinstance(paused_value, bool) else None
observation = self._active_evidence(
slot=slot,
path=path,
paused=paused,
telemetry=payload,
)
effects.append(PlaybackEffect("player_telemetry", slot=observation.active_slot))
operation = self.pending
if operation is None:
return effects
if not self._matches_pending(slot, path):
effects.append(
self._finish_operation(
"failed",
"Player telemetry identified a different slot or path",
)
)
return effects
if operation.action == "play":
effects.append(self._finish_operation("confirmed", "matching telemetry"))
elif operation.action == "pause" and paused is True:
effects.append(self._finish_operation("confirmed", "telemetry paused=true"))
elif operation.action == "resume" and paused is False:
effects.append(self._finish_operation("confirmed", "telemetry paused=false"))
elif operation.action == "toggle" and paused is not None:
effects.append(self._finish_operation("confirmed", f"telemetry paused={paused}"))
return effects
def check_timeout(self) -> PlaybackEffect | None:
operation = self.pending
if operation is None or self._monotonic() < operation.deadline:
return None
if operation.action == "play":
previous = self.store.player_observation()
has_confirmed_active = (
previous.reconciliation_state == "confirmed"
and (previous.active_slot is not None or previous.active_path is not None)
)
if has_confirmed_active and self._matches_pending(
previous.active_slot,
previous.active_path,
):
return self._finish_operation(
"confirmed", "matching authoritative Player evidence"
)
if not has_confirmed_active:
self._mark_play_uncertain(operation, "play timeout")
elif operation.action == "stop":
previous = self.store.player_observation()
if previous.active_slot is not None:
due = None
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
guard = self.store.mark_slot_uncertain(
previous.active_slot,
review_after=due,
details={"operation_id": operation.operation_id, "reason": "stop timeout"},
)
due = guard.review_after
self.store.set_player_observation(
availability="unknown",
phase="unknown",
active_slot=previous.active_slot,
active_path=previous.active_path,
paused=previous.paused,
reconciliation_state=(
"operator_review"
if guard.state == "review_required"
else "awaiting_evidence"
),
reconciliation_due_at=due,
reconciliation_id=guard.guard_id,
)
elif previous.active_path is not None:
due = None
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
self.store.mark_external_uncertain(
previous.active_path,
review_after=due,
details={
"operation_id": operation.operation_id,
"reason": "stop timeout",
},
)
elif operation.action in {"pause", "resume", "toggle"}:
previous = self.store.player_observation()
if previous.active_slot is not None or previous.active_path is not None:
self.store.set_player_observation(
availability=previous.availability,
phase="unknown",
active_slot=previous.active_slot,
active_path=previous.active_path,
paused=None,
reconciliation_state=previous.reconciliation_state,
reconciliation_due_at=previous.reconciliation_due_at,
reconciliation_id=previous.reconciliation_id,
)
return self._finish_operation("timed_out", "no authoritative Player confirmation")
def _mark_play_uncertain(self, operation: PlayerOperation, reason: str) -> None:
due = None
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
slot = operation.slot or self._canonical_slot_for_path(operation.expected_path)
if slot is not None:
guard = self.store.mark_slot_uncertain(
slot,
review_after=due,
details={"operation_id": operation.operation_id, "reason": reason},
)
due = guard.review_after
self.store.set_player_observation(
availability="unknown",
phase="unknown",
active_slot=slot,
active_path=operation.expected_path,
paused=None,
reconciliation_state=(
"operator_review"
if guard.state == "review_required"
else "awaiting_evidence"
),
reconciliation_due_at=due,
reconciliation_id=guard.guard_id,
)
elif operation.expected_path is not None:
self.store.mark_external_uncertain(
operation.expected_path,
review_after=due,
details={"operation_id": operation.operation_id, "reason": reason},
)
def publication_uncertain(self, reason: str) -> PlaybackEffect | None:
if self.pending is None:
return None
operation = self.pending
if operation.action == "play":
self._mark_play_uncertain(operation, reason)
return self._finish_operation("uncertain", reason)
def mark_transport_uncertain(self, reason: str) -> PlaybackEffect:
previous = self.store.player_observation()
pending_play = (
self.pending
if self.pending is not None and self.pending.action == "play"
else None
)
active_slot = previous.active_slot
active_path = previous.active_path
if active_slot is None and active_path is None and pending_play is not None:
active_slot = pending_play.slot or self._canonical_slot_for_path(
pending_play.expected_path
)
active_path = pending_play.expected_path
due = None
if active_slot is not None:
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
guard = self.store.mark_slot_uncertain(
active_slot,
review_after=due,
details={"reason": reason},
)
due = guard.review_after
self.store.set_player_observation(
availability="unknown",
phase="unknown",
active_slot=active_slot,
active_path=active_path,
paused=previous.paused,
reconciliation_state=(
"operator_review"
if guard.state == "review_required"
else "awaiting_evidence"
),
reconciliation_due_at=due,
reconciliation_id=guard.guard_id,
)
elif active_path is not None:
if self.config.storage.uncertain_review_after_seconds > 0:
due = self._wall_clock() + self.config.storage.uncertain_review_after_seconds
self.store.mark_external_uncertain(
active_path,
review_after=due,
details={"reason": reason},
)
else:
self.store.set_player_observation(
availability="unknown",
phase=previous.phase,
active_slot=None,
active_path=None,
paused=previous.paused,
reconciliation_state="unknown",
reconciliation_due_at=None,
reconciliation_id=None,
)
return PlaybackEffect(
"transport_uncertain",
slot=active_slot,
message=reason,
)
def projected_slot_state(self, slot: int) -> str:
record = self.store.get_slot(slot)
observation = self.store.player_observation()
if observation.active_slot == slot:
if observation.reconciliation_state == "operator_review":
return "operator_review"
if observation.phase == "paused" and observation.paused is True:
return "paused"
if observation.phase == "playing":
return "playing"
if observation.phase in {"unknown", "error"}:
return "unknown"
guard = next(
(guard for guard in self.store.list_live_guards() if guard.slot == slot),
None,
)
if guard is not None:
if guard.kind == "pending_play":
return record.media_state
return "operator_review" if guard.state == "review_required" else "unknown"
return record.media_state
def mark_due_reconciliation(self) -> list[PlaybackEffect]:
now = self._wall_clock()
due = self.store.mark_due_guards_for_review(now)
external_due = self.store.mark_external_reconciliation_due(now)
observation = self.store.player_observation()
if observation.active_slot is not None and any(
guard.slot == observation.active_slot for guard in due
):
self.store.set_player_observation(
availability=observation.availability,
phase="unknown",
active_slot=observation.active_slot,
active_path=observation.active_path,
paused=observation.paused,
reconciliation_state="operator_review",
reconciliation_due_at=observation.reconciliation_due_at,
reconciliation_id=observation.reconciliation_id,
)
effects = [
PlaybackEffect(
"operator_review",
slot=guard.slot,
operation_id=guard.guard_id,
message=(
"Playback remains unknown and protected; fresh Player evidence "
"or an administrative clear is required "
f"(review_id={guard.guard_id})"
),
)
for guard in due
]
if external_due is not None:
review_id, path = external_due
effects.append(
PlaybackEffect(
"operator_review",
operation_id=review_id,
message=(
f"Playback at {path!r} remains unknown and protected; "
"fresh Player evidence or an administrative clear is required "
f"(review_id={review_id})"
),
)
)
return effects
+422
View File
@@ -0,0 +1,422 @@
"""Timeout-bounded Audio Router acquisition and restoration leases."""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass
from typing import Callable
from .config import ControllerConfig
from .models import AudioLeaseRecord
from .mqtt_adapter import PublishReceipt
from .store import ControllerStore
ModePublisher = Callable[[str], PublishReceipt]
@dataclass(frozen=True)
class RouterDecision:
action: str
lease_id: str | None = None
warning: str | None = None
continued: bool = False
class AudioRouterCoordinator:
"""Own one conservative Router lease for one playback session."""
def __init__(
self,
config: ControllerConfig,
store: ControllerStore,
publish_mode: ModePublisher,
*,
clock=time.monotonic,
):
self.config = config
self.store = store
self.publish_mode = publish_mode
self._clock = clock
def observe_mode(self, mode: str) -> RouterDecision:
mode = mode.strip()
if not mode:
warning = "Audio Router published an empty mode"
self._record_warning(warning)
return RouterDecision("warning", warning=warning)
self.store.set_router_observation(mode=mode, freshness="current")
lease = self.store.live_audio_lease()
if lease is None:
return RouterDecision("observed")
if lease.state == "acquiring":
if mode == lease.target_mode:
self.store.update_audio_lease(
lease.lease_id,
state="held",
confirmation_deadline=None,
acquired=True,
)
return RouterDecision("ready", lease_id=lease.lease_id)
# State is evidence of the Router's current output, not evidence
# that an operator issued a command. It may simply be a periodic
# repeat of the pre-acquisition state, so keep waiting for the
# finite confirmation deadline.
return RouterDecision("observed", lease_id=lease.lease_id)
if lease.state == "held":
if mode != lease.target_mode:
warning = (
f"Audio Router changed from Controller-owned {lease.target_mode!r} "
f"to {mode!r}; automatic restoration was relinquished"
)
self.store.update_audio_lease(
lease.lease_id,
state="ownership_lost",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("ownership_lost", lease.lease_id, warning)
return RouterDecision("observed", lease_id=lease.lease_id)
if lease.state == "unconfirmed":
# A late state event must never resurrect a lease that already
# timed out or deliberately continued without confirmation.
return RouterDecision("observed", lease_id=lease.lease_id)
if lease.state == "restore_pending":
if mode == lease.restoration_mode:
self.store.update_audio_lease(
lease.lease_id,
state="released",
confirmation_deadline=None,
released=True,
)
return RouterDecision("restored", lease_id=lease.lease_id)
# The Router may repeat its current state before applying the
# restoration command. Wait for the bounded deadline; explicit
# mode-set intent is handled separately below.
return RouterDecision("observed", lease_id=lease.lease_id)
return RouterDecision("observed", lease_id=lease.lease_id)
def observe_mode_command(
self,
mode: str,
*,
retained: bool = False,
controller_echo: bool = False,
) -> RouterDecision:
"""Observe a Router mode-set publication without treating state as intent.
The MQTT adapter marks an exact echo received while a Controller QoS 1
publication is in flight. Every other live mode-set publication is
conservatively treated as operator intent. Retained commands are old
broker state and can never establish current operator intent.
"""
mode = mode.strip()
if not mode:
warning = "Audio Router published an empty mode command"
self._record_warning(warning)
return RouterDecision("warning", warning=warning)
lease = self.store.live_audio_lease()
lease_id = None if lease is None else lease.lease_id
if retained:
warning = (
f"Retained Audio Router mode command {mode!r} was ignored as stale intent"
)
self._record_warning(warning)
return RouterDecision("retained_ignored", lease_id, warning)
if controller_echo:
return RouterDecision("self_echo", lease_id=lease_id)
if lease is None:
return RouterDecision("operator_intent")
if lease.state == "acquiring":
return self._acquisition_failed(
lease.lease_id,
(
f"Audio Router received an operator mode command for {mode!r} "
f"while {lease.target_mode!r} confirmation was pending; "
"automatic restoration ownership was relinquished"
),
)
if lease.state in {"held", "unconfirmed", "restore_pending"}:
warning = (
f"Audio Router received an operator mode command for {mode!r}; "
"automatic restoration was relinquished"
)
self.store.update_audio_lease(
lease.lease_id,
state="ownership_lost",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("ownership_lost", lease.lease_id, warning)
return RouterDecision("operator_intent", lease_id=lease.lease_id)
def acquire(self, operation_id: str) -> RouterDecision:
observation = self.store.router_observation()
known = observation.freshness == "current" and observation.mode is not None
target = self.config.router.required_mode
previous = observation.mode if known else None
lease_id = uuid.uuid4().hex
if known and previous == target:
self.store.create_audio_lease(
lease_id=lease_id,
operation_id=operation_id,
state="held",
previous_mode=previous,
target_mode=target,
restoration_mode=None,
restoration_source="none",
controller_changed=False,
confirmation_deadline=None,
)
self.store.update_audio_lease(
lease_id,
state="held",
acquired=True,
)
return RouterDecision("ready", lease_id=lease_id)
if known:
restoration_mode = previous
restoration_source = "observed_previous"
elif self.config.router.fallback_restore_mode is not None:
restoration_mode = self.config.router.fallback_restore_mode
restoration_source = "configured_fallback"
else:
restoration_mode = None
restoration_source = "none"
policy = self.config.router.when_state_unknown
if not known and policy == "reject":
warning = "Audio Router state is unavailable; playback was rejected by policy"
self.store.create_audio_lease(
lease_id=lease_id,
operation_id=operation_id,
state="abandoned",
previous_mode=None,
target_mode=target,
restoration_mode=restoration_mode,
restoration_source=restoration_source,
controller_changed=False,
confirmation_deadline=None,
warning=warning,
)
self._record_warning(warning)
return RouterDecision("rejected", lease_id, warning)
deadline = self._clock() + self.config.router.confirmation_timeout_seconds
initial_state = "unconfirmed" if not known and policy == "continue" else "acquiring"
self.store.create_audio_lease(
lease_id=lease_id,
operation_id=operation_id,
state=initial_state,
previous_mode=previous,
target_mode=target,
restoration_mode=restoration_mode,
restoration_source=restoration_source,
controller_changed=True,
confirmation_deadline=(deadline if initial_state == "acquiring" else None),
)
receipt = self.publish_mode(target)
if not receipt.accepted:
return self._acquisition_failed(
lease_id,
f"could not request Audio Router mode {target!r}: {receipt.reason}",
)
if initial_state == "unconfirmed":
warning = (
f"Audio Router state is unavailable; requested {target!r} and "
"continued immediately without restoration ownership"
)
self.store.update_audio_lease(
lease_id,
state="unconfirmed",
warning=warning,
)
self._record_warning(warning)
return RouterDecision("ready", lease_id, warning, continued=True)
return RouterDecision("waiting", lease_id=lease_id)
def check_deadline(self) -> RouterDecision | None:
lease = self.store.live_audio_lease()
if (
lease is None
or lease.confirmation_deadline is None
or self._clock() < lease.confirmation_deadline
):
return None
if lease.state == "acquiring":
return self._acquisition_failed(
lease.lease_id,
(
f"Audio Router did not confirm {lease.target_mode!r} within "
f"{self.config.router.confirmation_timeout_seconds:g} seconds"
),
)
if lease.state == "restore_pending":
warning = (
f"Audio Router did not confirm restoration to {lease.restoration_mode!r}; "
"no further automatic change will be attempted"
)
self.store.update_audio_lease(
lease.lease_id,
state="abandoned",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("restore_failed", lease.lease_id, warning)
return None
def _acquisition_failed(self, lease_id: str, warning: str) -> RouterDecision:
continue_playback = self.config.router.continue_on_confirmation_failure
self.store.update_audio_lease(
lease_id,
state="unconfirmed" if continue_playback else "abandoned",
confirmation_deadline=None,
warning=warning,
released=not continue_playback,
)
self._record_warning(warning)
return RouterDecision(
"ready" if continue_playback else "rejected",
lease_id,
warning,
continued=continue_playback,
)
def release(self, lease_id: str | None) -> RouterDecision:
if lease_id is None:
return RouterDecision("released")
lease = self.store.get_audio_lease(lease_id)
if lease is None:
return RouterDecision("released")
if lease.state == "held" and not lease.controller_changed:
self.store.update_audio_lease(
lease_id,
state="released",
confirmation_deadline=None,
released=True,
)
return RouterDecision("released", lease_id)
if lease.state != "held":
if lease.state in {"unconfirmed", "acquiring"}:
self.store.update_audio_lease(
lease_id,
state="abandoned",
confirmation_deadline=None,
warning="Audio Router was never confirmed; restoration was skipped",
released=True,
)
return RouterDecision("released", lease_id)
observation = self.store.router_observation()
if observation.freshness != "current" or observation.mode != lease.target_mode:
warning = "Audio Router ownership is no longer current; restoration was skipped"
self.store.update_audio_lease(
lease_id,
state="ownership_lost",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("ownership_lost", lease_id, warning)
if lease.restoration_mode is None:
self.store.update_audio_lease(
lease_id,
state="released",
confirmation_deadline=None,
released=True,
)
return RouterDecision("released", lease_id)
receipt = self.publish_mode(lease.restoration_mode)
if not receipt.accepted:
warning = f"could not request Audio Router restoration: {receipt.reason}"
self.store.update_audio_lease(
lease_id,
state="abandoned",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("restore_failed", lease_id, warning)
self.store.update_audio_lease(
lease_id,
state="restore_pending",
confirmation_deadline=(
self._clock() + self.config.router.confirmation_timeout_seconds
),
)
return RouterDecision("restore_waiting", lease_id)
def connection_lost(self) -> RouterDecision | None:
observation = self.store.router_observation()
self.store.set_router_observation(
mode=observation.mode,
freshness="stale",
warning="MQTT continuity was lost",
)
lease = self.store.live_audio_lease()
if lease is None:
return None
if lease.state == "acquiring":
return self._acquisition_failed(
lease.lease_id,
"MQTT disconnected while waiting for Audio Router confirmation",
)
warning = "MQTT continuity was lost; automatic Audio Router restoration was relinquished"
self.store.update_audio_lease(
lease.lease_id,
state="ownership_lost",
confirmation_deadline=None,
warning=warning,
released=True,
)
self._record_warning(warning)
return RouterDecision("ownership_lost", lease.lease_id, warning)
def abandon(self, lease_id: str | None, reason: str) -> None:
if lease_id is None:
return
lease = self.store.get_audio_lease(lease_id)
if lease is None or lease.state in {"released", "abandoned", "ownership_lost"}:
return
self.store.update_audio_lease(
lease_id,
state="abandoned",
confirmation_deadline=None,
warning=reason,
released=True,
)
self._record_warning(reason)
def _record_warning(self, warning: str) -> None:
observation = self.store.router_observation()
self.store.set_router_observation(
mode=observation.mode,
freshness=observation.freshness,
warning=warning,
)
self.store.record_audit_event(
"audio_router_warning",
actor="controller",
details={"message": warning},
)
File diff suppressed because it is too large Load Diff
+2501
View File
File diff suppressed because it is too large Load Diff
+64 -818
View File
@@ -1,838 +1,84 @@
import os
import time
import json
"""Deprecated, lazy compatibility facade for the Controller v2 service.
New deployments use :mod:`wsgi`. These functions remain for local callers
that imported the original module, but importing this module never opens the
database, creates directories, starts threads, or connects to MQTT.
"""
from __future__ import annotations
import threading
import subprocess
from pathlib import Path
from dataclasses import dataclass, asdict
from typing import Dict, List, Optional
from typing import Any
import yaml
import paho.mqtt.client as mqtt
from controller.config import ControllerConfig, load_config
from controller.service import ControllerService, ControllerUnavailableError
# --- Config / paths ---
MEDIA_BASE = Path(os.environ.get("YTPLAYER_MEDIA_BASE", "/mnt/Media/YTPlayer"))
DATA_BASE = Path(os.environ.get("YTPLAYER_DATA_BASE", "/controller-data"))
STATE_PATH = DATA_BASE / "playlists.yaml"
_lifecycle_lock = threading.Lock()
_service: ControllerService | None = None
_config: ControllerConfig | None = None
MAX_SLOTS = int(os.environ.get("YTPLAYER_MAX_SLOTS", "10"))
RETENTION_SECONDS = int(os.environ.get("YTPLAYER_RETENTION_SECONDS", str(48 * 3600)))
# Kept as transition indicators for integrations that inspected the old module.
mqtt_client: Any | None = None
mqtt_started = False
MQTT_HOST = os.environ.get("MQTT_HOST", "10.40.0.2")
MQTT_PORT = int(os.environ.get("MQTT_PORT", "1883"))
MQTT_CLIENT_ID = os.environ.get("MQTT_CLIENT_ID", "yt-controller")
TOPIC_INGEST_PROGRESS = "youtube/ingest/progress"
TOPIC_PLAYBACK_BASE = "youtube/media/playback" # e.g. youtube/media/playback/1/title
TOPIC_PLAYER_STATUS = "youtube/player/status" # JSON from player_service
TOPIC_PLAYER_CMD = "youtube/player/cmd" # commands to player_service
TOPIC_CONTROLLER_CMD = "youtube/controller/cmd" # commands from HA / StreamDeck
def start_controller() -> None:
"""Construct and start the singleton compatibility service explicitly."""
MEDIA_BASE.mkdir(parents=True, exist_ok=True)
DATA_BASE.mkdir(parents=True, exist_ok=True)
# --- MQTT client ---
import logging, sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
def _on_connect(client, userdata, flags, rc, properties=None):
print(f"[mqtt] connected rc={rc}")
if rc == 0:
client.subscribe(TOPIC_PLAYER_STATUS, qos=1)
client.subscribe(TOPIC_CONTROLLER_CMD, qos=1)
print(f"[controller] subscribed to {TOPIC_PLAYER_STATUS} and {TOPIC_CONTROLLER_CMD}")
else:
print(f"[mqtt] connect failed rc={rc}")
def _on_message(client, userdata, msg):
if msg.topic == TOPIC_PLAYER_STATUS:
try:
data = json.loads(msg.payload.decode("utf-8") or "{}")
except Exception:
global _config, _service, mqtt_client, mqtt_started
with _lifecycle_lock:
if _service is not None and _service.started:
return
state_str = data.get("state")
slot_val = data.get("slot")
if not state_str or slot_val is None:
return
try:
slot_num = int(slot_val)
except (TypeError, ValueError):
return
print(f"[controller] player status: slot={slot_num} state={state_str}")
_update_slot_state_from_player(slot_num, state_str)
elif msg.topic == TOPIC_CONTROLLER_CMD:
_handle_control_cmd(msg.payload.decode("utf-8", errors="ignore"))
def _handle_control_cmd(payload: str):
"""
Handle messages on TOPIC_CONTROLLER_CMD, e.g.:
{"action": "delete_slot", "slot": 2, "url": "..."}
{"action": "play", "slot": 1}
{"action": "stop"}
"""
try:
data = json.loads(payload or "{}")
except json.JSONDecodeError:
print(f"[controller] bad JSON on control cmd: {payload!r}")
return
action = data.get("action")
if not action:
print("[controller] control cmd missing 'action'")
return
# delete slot
if action == "delete_slot":
slot = data.get("slot")
try:
slot_num = int(slot)
except (TypeError, ValueError):
print(f"[controller] delete_slot: bad slot {slot!r}")
return
expected_url = data.get("url") # optional safety
delete_slot(slot_num, expected_url=expected_url)
return
# play / stop → hand off to the orchestrator
if action in ("play", "stop"):
handle_play_request(data)
return
print(f"[controller] unknown control action: {action!r}")
def _on_disconnect(client, userdata, rc, properties=None):
print(f"[mqtt] disconnected rc={rc}")
mqtt_client = mqtt.Client(client_id=MQTT_CLIENT_ID, clean_session=True)
# If your broker ever needs auth, you'd uncomment:
# mqtt_client.username_pw_set(os.environ.get("MQTT_USERNAME"), os.environ.get("MQTT_PASSWORD"))
mqtt_client.enable_logger() # send Paho logs to Python logging
mqtt_client.on_connect = _on_connect
mqtt_client.on_disconnect = _on_disconnect
mqtt_client.on_message = _on_message
print(f"[controller] attempting MQTT connect to {MQTT_HOST}:{MQTT_PORT} …")
mqtt_client.connect(MQTT_HOST, MQTT_PORT, keepalive=30)
mqtt_client.loop_start()
print("[controller] MQTT loop started")
mqtt_client.on_message = _on_message
mqtt_client.subscribe(TOPIC_PLAYER_STATUS, qos=1)
mqtt_client.subscribe(TOPIC_CONTROLLER_CMD, qos=1)
print(f"[controller] subscribed to {TOPIC_PLAYER_STATUS} and {TOPIC_CONTROLLER_CMD}")
config = load_config()
service = ControllerService(config)
service.start()
_config = config
_service = service
mqtt_client = service.mqtt
mqtt_started = True
def mqtt_publish(topic: str, payload, **kwargs):
"""
Small wrapper that JSON-encodes non-strings and prints what we publish.
Keeps logs useful and avoids repeating code.
"""
if not isinstance(payload, str):
payload = json.dumps(payload)
print(f"[mqtt] publish {topic} -> {payload}")
mqtt_client.publish(topic, payload, **kwargs)
def stop_controller() -> None:
"""Stop and discard the singleton compatibility service."""
global _config, _service, mqtt_client, mqtt_started
with _lifecycle_lock:
service = _service
if service is not None:
service.stop()
_service = None
_config = None
mqtt_client = None
mqtt_started = False
# --- State model ---
def get_service() -> ControllerService:
"""Return the explicitly started compatibility service."""
@dataclass
class PlaylistSlot:
slot: int
url: str
yt_id: Optional[str]
title: str
path: str
m3u: str
created_at: float
expires_at: float
item_count: int
state: str # "downloading" | "ready" | "error" | "playing" | "busy"
def to_dict(self):
d = asdict(self)
# timestamps as ISO-ish strings
d["created_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(self.created_at))
d["expires_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(self.expires_at))
return d
@staticmethod
def from_dict(d: Dict):
def parse_ts(s: str) -> float:
try:
# naive ISO Z
return time.mktime(time.strptime(s, "%Y-%m-%dT%H:%M:%SZ"))
except Exception:
return float(d.get("created_at_ts", time.time()))
created_at = parse_ts(d.get("created_at", "1970-01-01T00:00:00Z"))
expires_at = parse_ts(d.get("expires_at", "1970-01-01T00:00:00Z"))
return PlaylistSlot(
slot=int(d["slot"]),
url=d["url"],
yt_id=d.get("yt_id"),
title=d.get("title", ""),
path=d["path"],
m3u=d["m3u"],
created_at=created_at,
expires_at=expires_at,
item_count=int(d.get("item_count", 0)),
state=d.get("state", "ready"),
with _lifecycle_lock:
if _service is None or not _service.started:
raise ControllerUnavailableError(
"Controller is not started; call start_controller() first"
)
return _service
class ControllerState:
def __init__(self):
self.slots: Dict[int, PlaylistSlot] = {}
def enqueue_download(
url: str,
submitter: str | None = None,
note: str | None = None,
) -> dict[str, Any]:
"""Queue a download through Controller v2."""
def load(self):
if not STATE_PATH.exists():
return
data = yaml.safe_load(STATE_PATH.read_text()) or {}
slots = data.get("slots", {})
self.slots.clear()
for k, v in slots.items():
try:
slot = PlaylistSlot.from_dict(v)
self.slots[slot.slot] = slot
except Exception:
continue
return get_service().enqueue_download(url, submitter, note)
def save(self):
data = {"slots": {str(s): slot.to_dict() for s, slot in self.slots.items()}}
STATE_PATH.write_text(yaml.safe_dump(data, sort_keys=True))
def prune_expired(self):
"""
Remove any slots whose expires_at is in the past, delete their folders,
compact remaining slots, save, and reannounce MQTT so ghosts disappear.
"""
now = time.time()
changed = False
to_delete: List[int] = []
for s, slot in list(self.slots.items()):
if now >= slot.expires_at:
to_delete.append(s)
for s in to_delete:
slot = self.slots.pop(s, None)
if not slot:
continue
try:
p = Path(slot.path)
if p.exists():
for child in p.iterdir():
try:
child.unlink()
except IsADirectoryError:
# ignore unexpected subdirs
pass
p.rmdir()
except Exception:
pass
changed = True
if changed:
# compact slot numbers (1..N), save, then resync MQTT topics
self.compact_slots()
self.save()
# this is the helper you already have below
reannounce_slots()
def compact_slots(self):
"""Shift slots down so there are no gaps: 1..N"""
if not self.slots:
return
new_slots: Dict[int, PlaylistSlot] = {}
for new_index, slot in enumerate(sorted(self.slots.values(), key=lambda sl: sl.slot), start=1):
if slot.slot != new_index:
old_dir = Path(slot.path)
new_dir = MEDIA_BASE / f"slot{new_index}"
# rename folder if it exists
if old_dir.exists() and old_dir != new_dir:
new_dir.parent.mkdir(parents=True, exist_ok=True)
old_dir.rename(new_dir)
slot.slot = new_index
slot.path = str(new_dir)
slot.m3u = str(new_dir / "playlist.m3u")
new_slots[new_index] = slot
self.slots = new_slots
def get_next_free_slot(self) -> Optional[int]:
self.prune_expired()
used = set(self.slots.keys())
for i in range(1, MAX_SLOTS + 1):
if i not in used:
return i
return None
state = ControllerState()
state.load()
state.prune_expired()
# --- Helpers ---
def delete_slot(slot_num: int, expected_url: Optional[str] = None):
"""
Delete a slot from state + filesystem, then compact and republish.
expected_url is optional: if provided and doesn't match, we ignore
the delete (safety against stale retained messages if someone ever
accidentally turns retain on).
"""
state.load()
slot = state.slots.get(slot_num)
if not slot:
print(f"[controller] delete_slot: slot {slot_num} not found, ignoring")
return
if expected_url and slot.url != expected_url:
print(f"[controller] delete_slot: url mismatch for slot {slot_num}, ignoring")
return
print(f"[controller] deleting slot {slot_num}: {slot.title!r}")
# Remove files on disk
slot_dir = Path(slot.path)
if slot_dir.exists():
for child in slot_dir.iterdir():
try:
child.unlink()
except IsADirectoryError:
# no subdirs expected, but don't crash
pass
try:
slot_dir.rmdir()
except OSError:
pass
# Remove from state
state.slots.pop(slot_num, None)
# Compact and save
state.compact_slots()
state.save()
# Re-publish MQTT for all remaining slots, so HA/Deck are in sync
reannounce_slots()
def reannounce_slots():
"""
Re-publish MQTT state/title/count for all existing slots,
and mark any unused slots as empty.
"""
state.load()
# Re-announce real slots
for s, slot in sorted(state.slots.items()):
publish_slot_topics(slot)
# Clear any slots above the current highest used index
highest = max(state.slots.keys(), default=0)
for s in range(highest + 1, MAX_SLOTS + 1):
base = f"{TOPIC_PLAYBACK_BASE}/{s}"
mqtt_publish(f"{base}/title", "", retain=True)
mqtt_publish(f"{base}/state", "empty", retain=True)
mqtt_publish(f"{base}/count", "0", retain=True)
# also clear item names
for i in range(1, 100):
mqtt_publish(f"{base}/items/{i}", "", retain=True)
def publish_slot_topics(slot: PlaylistSlot):
"""Publish the title/state/count for one slot to its MQTT topics."""
base = f"{TOPIC_PLAYBACK_BASE}/{slot.slot}"
mqtt_publish(f"{base}/title", slot.title, retain=True)
mqtt_publish(f"{base}/state", slot.state, retain=True)
mqtt_publish(f"{base}/count", slot.item_count, retain=True)
def set_slot_busy_temporarily(slot: PlaylistSlot, seconds: int = 5):
"""
Briefly mark a slot as 'busy', then restore after a delay.
"""
base = f"{TOPIC_PLAYBACK_BASE}/{slot.slot}"
mqtt_publish(f"{base}/state", "busy", retain=True)
def _clear_busy_later(slot_number: int, delay: int):
time.sleep(delay)
state.load()
sl = state.slots.get(slot_number)
if sl is None:
base2 = f"{TOPIC_PLAYBACK_BASE}/{slot_number}"
mqtt_publish(f"{base2}/title", "", retain=True)
mqtt_publish(f"{base2}/state", "empty", retain=True)
mqtt_publish(f"{base2}/count", 0, retain=True)
return
publish_slot_topics(sl)
t = threading.Thread(
target=_clear_busy_later,
args=(slot.slot, seconds),
daemon=True,
)
t.start()
def _parse_track_title_from_filename(path: Path) -> str:
stem = path.stem # "000 - Song Name [id]"
# strip leading "000 - "
if " - " in stem:
stem = stem.split(" - ", 1)[1]
# strip trailing " [id]"
if " [" in stem:
stem = stem.rsplit(" [", 1)[0]
return stem
def _update_slot_state_from_player(slot_num: int, new_state: str):
"""
Mirror player state ("playing", "busy", "idle", "error", etc.)
into our per-slot state + MQTT topics.
"""
slot_obj = state.slots.get(slot_num)
if not slot_obj:
return
# Map "idle" / "finished" from player back to "ready" for the slot UI
if new_state in ("idle", "finished"):
slot_obj.state = "ready"
mqtt_state = "ready"
else:
slot_obj.state = new_state
mqtt_state = new_state
state.save()
publish_slot_topics(slot_obj)
def _get_title_for_url(url: str) -> str:
"""
Try to detect a human-friendly title for either:
- a playlist (use playlist_title)
- a single video (fall back to video title)
If we can't figure it out, just return the URL.
"""
try:
# One line: playlist_title|||video_title
out = subprocess.check_output(
[
"yt-dlp",
"--flat-playlist",
"--print", "%(playlist_title)s|||%(title)s",
"--playlist-end", "1",
url,
],
stderr=subprocess.DEVNULL,
text=True,
).strip()
if not out:
return url
line = out.splitlines()[0]
playlist_title, _, video_title = line.partition("|||")
playlist_title = (playlist_title or "").strip()
video_title = (video_title or "").strip()
# If playlist_title looks real (not empty/NA), use it
if playlist_title and playlist_title.upper() != "NA":
return playlist_title
# Otherwise fall back to video title
if video_title:
return video_title
return url
except Exception:
return url
def _download_playlist_to_slot(url: str, slot_num: int):
"""
Runs yt-dlp into /mnt/Media/YTPlayer/slotN, builds playlist.m3u,
updates YAML, and publishes MQTT topics.
"""
slot_dir = MEDIA_BASE / f"slot{slot_num}"
slot_dir.mkdir(parents=True, exist_ok=True)
title = _get_title_for_url(url)
now = time.time()
expires_at = now + RETENTION_SECONDS
# mark slot as downloading
slot = PlaylistSlot(
slot=slot_num,
url=url,
yt_id=None,
title=title,
path=str(slot_dir),
m3u=str(slot_dir / "playlist.m3u"),
created_at=now,
expires_at=expires_at,
item_count=0,
state="downloading",
)
state.slots[slot_num] = slot
state.save()
# Tell HA/Deck: this slot is downloading now
publish_slot_topics(slot)
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{"slot": slot_num, "state": "downloading", "title": title},
)
# run yt-dlp
try:
subprocess.run(
[
"yt-dlp",
"--cookies",
"/cookies.txt", # inside the container (mount your exported cookies here)
"-o",
str(slot_dir / "%(playlist_index)03d - %(title)s [%(id)s].%(ext)s"),
"--yes-playlist",
"--no-overwrites", # keep existing files, only fetch new ones
"-f",
"bv*[height<=1080]+ba/b[height<=1080]/b[height<=1080]/b",
url,
],
check=True,
)
except subprocess.CalledProcessError as e:
slot.state = "error"
state.save()
publish_slot_topics(slot)
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{"slot": slot_num, "state": "error", "message": f"yt-dlp failed: {e}"},
)
return
# --- build .m3u in sorted order ---
media_files = sorted(
[
p
for p in slot_dir.iterdir()
if p.is_file() and p.suffix.lower() in {".mp4", ".mkv", ".webm"}
]
)
if not media_files:
slot.state = "error"
state.save()
publish_slot_topics(slot)
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{"slot": slot_num, "state": "error", "message": "no media files downloaded"},
)
return
item_titles: List[str] = []
m3u_path = slot_dir / "playlist.m3u"
with m3u_path.open("w", encoding="utf-8") as f:
for p in media_files:
# Use a path relative to the slot directory so that if the
# whole slotN folder gets renamed (compaction), the playlist
# still works.
rel = os.path.relpath(p, slot_dir)
f.write(rel + "\n")
item_titles.append(_parse_track_title_from_filename(p))
slot.item_count = len(media_files)
slot.state = "ready"
state.slots[slot_num] = slot
state.save()
# publish MQTT info for HA + StreamDeck
publish_slot_topics(slot)
# items: 1-based index
base = f"{TOPIC_PLAYBACK_BASE}/{slot_num}"
for i, name in enumerate(item_titles, start=1):
mqtt_publish(f"{base}/items/{i}", name, retain=True)
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{"slot": slot_num, "state": "ready", "title": title, "count": slot.item_count},
)
def publish_slot_topics(slot: PlaylistSlot):
"""Publish the title/state/count for one slot to its MQTT topics."""
base = f"{TOPIC_PLAYBACK_BASE}/{slot.slot}"
mqtt_publish(f"{base}/title", slot.title, retain=True)
mqtt_publish(f"{base}/state", slot.state, retain=True)
mqtt_publish(f"{base}/count", slot.item_count, retain=True)
# --- Controller → Player orchestration ---
def handle_play_request(cmd: Dict) -> None:
"""
Handle commands from HA/Companion on TOPIC_CONTROLLER_CMD.
Supported payloads:
- {"action": "play", "slot": N}
- {"action": "play", "playlist": "/some/path.m3u"} # future perm playlist
- {"action": "stop"}
"""
action = cmd.get("action")
# Which slot is currently playing?
playing_slot: Optional[int] = None
for s, slot in state.slots.items():
if slot.state == "playing":
playing_slot = s
break
# --- STOP command ---
if action == "stop":
# Forward stop to player
mqtt_publish(TOPIC_PLAYER_CMD, {"action": "stop"})
# Any slot marked playing goes back to ready
changed = False
for s, slot in state.slots.items():
if slot.state == "playing":
slot.state = "ready"
base = f"{TOPIC_PLAYBACK_BASE}/{s}"
mqtt_publish(f"{base}/state", "ready", retain=True)
changed = True
if changed:
state.save()
return
# --- PLAY command ---
if action != "play":
# ignore unknown for now
return
slot_num_raw = cmd.get("slot")
playlist = cmd.get("playlist") or cmd.get("path")
label = cmd.get("label")
slot_obj: Optional[PlaylistSlot] = None
if slot_num_raw is not None:
try:
slot_num = int(slot_num_raw)
except (TypeError, ValueError):
print(f"[controller] bad slot in play cmd: {slot_num_raw!r}")
return
slot_obj = state.slots.get(slot_num)
if not slot_obj:
print(f"[controller] no such slot {slot_num}")
return
# If we have a slot but no playlist, derive path from the slot.
if slot_obj is not None and not playlist:
playlist = slot_obj.m3u
label = label or slot_obj.title
# If we still have no playlist/path, bail.
if not playlist:
print("[controller] play cmd missing playlist/path and no valid slot provided")
return
# Enforce only one slot at a time for now
if playing_slot is not None and slot_obj is not None and playing_slot != slot_obj.slot:
# briefly mark this requested slot as busy, then restore it
set_slot_busy_temporarily(slot_obj, seconds=5)
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{
"slot": slot_obj.slot,
"state": "busy",
"message": f"slot {playing_slot} already playing; stop first",
},
)
return
# If we're starting a slot-backed playlist, mark that slot as playing
if slot_obj is not None:
slot_obj.state = "playing"
state.save()
base = f"{TOPIC_PLAYBACK_BASE}/{slot_obj.slot}"
mqtt_publish(f"{base}/state", "playing", retain=True)
# Build command for the player
play_cmd = {
"action": "play",
"playlist": playlist,
}
if slot_obj is not None:
play_cmd["slot"] = slot_obj.slot
if label:
play_cmd["label"] = label
mqtt_publish(TOPIC_PLAYER_CMD, play_cmd)
print(f"[controller] sent play cmd to player: {play_cmd}")
# --- HA Discovery ---
HA_DISCOVERY_PREFIX = os.environ.get("HA_DISCOVERY_PREFIX", "homeassistant")
HA_DEVICE = {
"identifiers": ["ytplayer-controller"],
"name": "YouTube Controller",
"manufacturer": "River of Glory",
"model": "YT Controller",
}
def ha_discovery_publish():
"""
Publish MQTT Discovery configs for:
- sensor.yt_slot_<n>_title
- sensor.yt_slot_<n>_state
- sensor.yt_slot_<n>_count
"""
for n in range(1, MAX_SLOTS + 1):
base = f"{TOPIC_PLAYBACK_BASE}/{n}"
obj_id = f"yt_slot_{n}"
# Title
cfg_title = {
"name": f"YT Slot {n} Title",
"state_topic": f"{base}/title",
"unique_id": f"{obj_id}_title",
"icon": "mdi:playlist-music",
"device": HA_DEVICE,
}
mqtt_publish(
f"{HA_DISCOVERY_PREFIX}/sensor/{obj_id}_title/config",
cfg_title,
retain=True,
)
# State (ready/downloading/playing/busy/error/idle)
cfg_state = {
"name": f"YT Slot {n} State",
"state_topic": f"{base}/state",
"unique_id": f"{obj_id}_state",
"icon": "mdi:circle-small",
"device": HA_DEVICE,
}
mqtt_publish(
f"{HA_DISCOVERY_PREFIX}/sensor/{obj_id}_state/config",
cfg_state,
retain=True,
)
# Count
cfg_count = {
"name": f"YT Slot {n} Count",
"state_topic": f"{base}/count",
"unique_id": f"{obj_id}_count",
"icon": "mdi:numeric",
"device": HA_DEVICE,
}
mqtt_publish(
f"{HA_DISCOVERY_PREFIX}/sensor/{obj_id}_count/config",
cfg_count,
retain=True,
)
# --- Public API used by Flask app ---
def enqueue_download(url: str, submitter: Optional[str], note: Optional[str]) -> Dict:
"""
Called by Flask when a new playlist is submitted.
- If this URL is already in a slot, reuse that slot and only grab new videos.
- Otherwise, allocate a new slot.
Returns a small dict summarizing what happened, for UI or HA.
"""
state.load()
state.prune_expired()
# 1) Check if this URL already exists in a slot
existing_slot_obj = None
for sl in state.slots.values():
if sl.url == url:
existing_slot_obj = sl
break
if existing_slot_obj:
slot_num = existing_slot_obj.slot
print(f"[controller] enqueue_download: reusing existing slot {slot_num} for URL {url!r}")
reused = True
else:
# 2) Allocate a new slot
slot_num = state.get_next_free_slot()
reused = False
if slot_num is None:
# queue full
expires = [sl.expires_at for sl in state.slots.values()]
oldest = min(expires) if expires else time.time()
return {
"status": "queue_full",
"oldest_expires_at": oldest,
"oldest_expires_in_seconds": int(oldest - time.time()),
}
# 3) Spawn background thread so Flask doesn't block
t = threading.Thread(
target=_download_playlist_to_slot,
args=(url, slot_num),
daemon=True,
)
t.start()
# 4) Simple MQTT log of submission
mqtt_publish(
TOPIC_INGEST_PROGRESS,
{
"slot": slot_num,
"state": "queued",
"url": url,
"submitter": submitter,
"note": note,
"reused_slot": reused,
},
)
return {
"status": "queued",
"slot": slot_num,
"reused_slot": reused,
}
# --- Post-define initialization that needs functions above ---
print("[controller] running HA discovery publish…")
ha_discovery_publish()
print("[controller] ha_discovery_publish() done")
# Re-announce any existing slots so HA / Deck see them after a restart
for s, slot in state.slots.items():
publish_slot_topics(slot)
print("[controller] re-published slot states")
__all__ = [
"enqueue_download",
"get_service",
"mqtt_client",
"mqtt_started",
"start_controller",
"stop_controller",
]
+51 -9
View File
@@ -1,22 +1,64 @@
services:
controller:
build: .
build:
context: .
args:
CONTROLLER_UID: ${CONTROLLER_UID:-1000}
CONTROLLER_GID: ${CONTROLLER_GID:-1000}
image: video-player-controller:local
container_name: yt-controller
restart: unless-stopped
init: true
read_only: true
ports:
- "7531:7531"
- "${CONTROLLER_WEB_PORT:-7531}:7531"
environment:
- FLASK_SECRET_KEY=change-me
- MQTT_HOST=10.40.0.2
- MQTT_PORT=1883
FLASK_SECRET_KEY: ${FLASK_SECRET_KEY:?Set FLASK_SECRET_KEY in .env}
MQTT_HOST: ${MQTT_HOST:-10.40.0.2}
MQTT_PORT: ${MQTT_PORT:-1883}
MQTT_USERNAME: ${MQTT_USERNAME:-}
MQTT_PASSWORD: ${MQTT_PASSWORD:-}
MQTT_CLIENT_ID: ${MQTT_CLIENT_ID:-yt-controller}
PORT: 7531
YTPLAYER_DATA_BASE: /controller-data
YTPLAYER_MEDIA_BASE: /mnt/Media/YTPlayer
YTPLAYER_COOKIE_PATH: /run/secrets/yt-dlp-cookies.txt
volumes:
- /mnt/Media:/mnt/Media # same path inside container
- /home/media/Scripts/Video-Player:/controller-data
- /home/media/Scripts/Video-Player/cookies.txt:/cookies_ro.txt:ro
- ${CONTROLLER_DATA_DIR:?Set CONTROLLER_DATA_DIR in .env}:/controller-data
- ${YTPLAYER_MEDIA_DIR:-/mnt/Media/YTPlayer}:/mnt/Media/YTPlayer
- ./config.yaml:/app/config.yaml:ro
# Optional: place yt-dlp-cookies.txt in this host directory.
- type: bind
source: ${CONTROLLER_SECRETS_DIR:-./secrets}
target: /run/secrets
read_only: true
tmpfs:
- /tmp:rw,nosuid,nodev,noexec,mode=1777,size=256m
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
healthcheck:
test:
- CMD
- python
- -c
- >-
import urllib.request;
urllib.request.urlopen('http://127.0.0.1:7531/healthz', timeout=3).read()
interval: 30s
timeout: 5s
start_period: 30s
retries: 3
stop_grace_period: 45s
logging:
driver: json-file
options:
max-size: 10m
max-file: "3"
networks:
- tabernet
networks:
tabernet:
external: true
+30
View File
@@ -0,0 +1,30 @@
"""Single-process runtime: ControllerService owns one local coordinator."""
from __future__ import annotations
import sys
from controller.config import load_config
_web_config = load_config().web
bind = f"{_web_config.host}:{_web_config.port}"
workers = 1
worker_class = "gthread"
threads = 4
preload_app = False
accesslog = "-"
errorlog = "-"
capture_output = True
timeout = 30
graceful_timeout = 30
worker_tmp_dir = "/tmp"
control_socket_disable = True
def worker_exit(server, worker): # pragma: no cover - exercised by Gunicorn
module = sys.modules.get("wsgi")
service = getattr(module, "service", None)
if service is None:
return
service.stop()
+14 -4
View File
@@ -1,4 +1,14 @@
flask
paho-mqtt
yt-dlp
pyyaml
Flask==3.1.3
gunicorn==26.0.0
paho-mqtt==2.1.0
PyYAML==6.0.3
yt-dlp[default,pin]==2026.7.4
# Exact Flask/Gunicorn transitive set used by the production image.
blinker==1.9.0
click==8.4.2
itsdangerous==2.2.0
Jinja2==3.1.6
MarkupSafe==3.0.3
packaging==26.3
Werkzeug==3.1.8
+1
View File
@@ -0,0 +1 @@
"""Controller regression tests."""
+494
View File
@@ -0,0 +1,494 @@
import importlib
import os
import sys
import tempfile
import threading
import unittest
from pathlib import Path
from unittest import mock
from controller.config import ControllerConfig
from controller.models import StoreError
from controller.mqtt_adapter import MqttAdapter
from controller.service import ControllerService, ControllerUnavailableError
class FakeControllerService:
def __init__(
self,
*,
enqueue_result=None,
enqueue_error=None,
health=None,
export_jobs=None,
):
self.enqueue_result = enqueue_result or {
"status": "queued",
"slot": 3,
"reused_slot": False,
}
self.enqueue_error = enqueue_error
self.health_result = health or {
"status": "ok",
"started": True,
"coordinator_alive": True,
"mqtt_connected": True,
"router_state": "video",
"degraded_reasons": [],
}
self.export_jobs = list(export_jobs or [])
self.enqueue_calls = []
self.export_calls = []
def enqueue_download(self, url, submitter, note):
self.enqueue_calls.append((url, submitter, note))
if self.enqueue_error is not None:
raise self.enqueue_error
return dict(self.enqueue_result)
def enqueue_export(self, url, mode, submitter, note):
self.export_calls.append((url, mode, submitter, note))
if self.enqueue_error is not None:
raise self.enqueue_error
return {
"kind": "export",
"status": "queued",
"job_id": "export-job",
"mode": mode,
}
def list_export_jobs(self, *, limit=10):
return [dict(job) for job in self.export_jobs[:limit]]
def health(self):
return dict(self.health_result)
class FlaskFactoryTests(unittest.TestCase):
def setUp(self):
self.config = ControllerConfig()
@staticmethod
def _factory():
module = importlib.import_module("app")
return module.create_app
def _app(self, service):
return self._factory()(self.config, service, testing=True)
def _post(
self,
service,
*,
url="https://www.youtube.com/playlist?list=PL1234567890",
**fields,
):
app = self._app(service)
return app.test_client().post(
"/submit",
data={"url": url, **fields},
follow_redirects=True,
)
def test_get_index_preserves_submission_form(self):
response = self._app(FakeControllerService()).test_client().get("/")
self.assertEqual(response.status_code, 200)
self.assertIn(b"Submit YouTube Media", response.data)
self.assertIn(b'action="/submit"', response.data)
self.assertIn(b'name="url"', response.data)
self.assertIn(b'name="submitter"', response.data)
self.assertIn(b'name="note"', response.data)
self.assertIn(b'name="download_mode"', response.data)
self.assertIn(b'value="player_slot" checked', response.data)
self.assertIn(b"MP4 Export", response.data)
self.assertIn(b"M4A Audio", response.data)
self.assertIn(b"MP3 Audio", response.data)
def test_older_clients_omitting_mode_still_use_player_slot(self):
service = FakeControllerService()
response = self._post(service, submitter=" Alice ")
self.assertEqual(response.status_code, 200)
self.assertEqual(
service.enqueue_calls,
[("https://www.youtube.com/playlist?list=PL1234567890", "Alice", None)],
)
self.assertEqual(service.export_calls, [])
def test_each_utility_mode_uses_export_path_without_player_slot(self):
for mode, label in (
("mp4", b"MP4 Export queued"),
("m4a", b"M4A Audio queued"),
("mp3", b"MP3 Audio queued"),
):
with self.subTest(mode=mode):
service = FakeControllerService()
response = self._post(
service,
download_mode=mode,
submitter="Operator",
note="Utility",
)
self.assertEqual(response.status_code, 200)
self.assertIn(label, response.data)
self.assertEqual(service.enqueue_calls, [])
self.assertEqual(
service.export_calls,
[
(
"https://www.youtube.com/playlist?list=PL1234567890",
mode,
"Operator",
"Utility",
)
],
)
def test_invalid_download_mode_is_rejected_server_side(self):
service = FakeControllerService()
response = self._post(service, download_mode="../../slot1")
self.assertEqual(response.status_code, 200)
self.assertIn(b"Invalid download type", response.data)
self.assertEqual(service.enqueue_calls, [])
self.assertEqual(service.export_calls, [])
def test_recent_export_status_shows_results_and_failures(self):
service = FakeControllerService(
export_jobs=[
{
"job_id": "one",
"mode": "mp4",
"status": "completed",
"output_paths": ["exports/video/one/Service.mp4"],
"error": None,
},
{
"job_id": "two",
"mode": "mp3",
"status": "failed",
"output_paths": [],
"error": "ffmpeg failed",
},
]
)
response = self._app(service).test_client().get("/")
self.assertEqual(response.status_code, 200)
self.assertIn(b"Recent utility exports", response.data)
self.assertIn(b"exports/video/one/Service.mp4", response.data)
self.assertIn(b"ffmpeg failed", response.data)
def test_submit_rejects_non_youtube_and_malformed_urls(self):
invalid_urls = (
"",
"http",
"https:///missing-host",
"httpfoo://youtube.com/video",
"ftp://youtube.com/video",
"javascript:alert(1)",
"/relative/path",
# Arbitrary external hosts, including lookalikes and substring
# tricks, must be rejected by exact normalized hostname match.
"https://example.com/video",
"https://notyoutube.com/video",
"https://youtube.com.evil.com/video",
"https://evil.com/?u=youtube.com",
"https://xyoutube.com/watch?v=abc",
"https://youtube.com.attacker.net/watch?v=abc",
# Embedded userinfo must be rejected even when the host matches.
"https://user:[email protected]/watch?v=abc",
"https://[email protected]/watch?v=abc",
"https://[email protected]/watch?v=abc",
# Oversized URL.
"https://www.youtube.com/watch?v=" + ("a" * 3000),
)
for url in invalid_urls:
with self.subTest(url=url):
service = FakeControllerService()
response = self._post(service, url=url)
self.assertEqual(response.status_code, 200)
self.assertIn(b"Invalid URL", response.data)
self.assertEqual(service.enqueue_calls, [])
def test_submit_accepts_only_explicit_official_youtube_hosts(self):
valid_urls = (
"https://youtube.com/watch?v=abc",
"https://www.youtube.com/watch?v=abc",
"http://www.youtube.com/watch?v=abc",
"https://m.youtube.com/watch?v=abc",
"https://music.youtube.com/watch?v=abc",
"https://youtu.be/abc",
"https://YouTube.com/watch?v=abc",
"https://WWW.YOUTUBE.COM/watch?v=abc",
)
for url in valid_urls:
with self.subTest(url=url):
service = FakeControllerService()
response = self._post(service, url=url)
self.assertEqual(response.status_code, 200)
self.assertEqual(service.enqueue_calls, [(url, None, None)])
def test_queued_submission_preserves_message_and_trims_optional_fields(self):
service = FakeControllerService(
enqueue_result={"status": "queued", "slot": 3, "reused_slot": False}
)
response = self._post(
service,
url="https://www.youtube.com/playlist?list=PL1234567890",
submitter=" Alice ",
note=" Before service ",
)
self.assertEqual(response.status_code, 200)
self.assertIn(b"Playlist queued in slot 3", response.data)
self.assertIn(b"start downloading shortly", response.data)
self.assertEqual(
service.enqueue_calls,
[
(
"https://www.youtube.com/playlist?list=PL1234567890",
"Alice",
"Before service",
)
],
)
def test_submit_rejects_oversized_submitter_and_note(self):
module = importlib.import_module("app")
service = FakeControllerService()
response = self._post(
service,
submitter="x" * (module.MAX_SUBMITTER_LENGTH + 1),
)
self.assertEqual(response.status_code, 200)
self.assertIn(b"must be", response.data)
self.assertEqual(service.enqueue_calls, [])
service = FakeControllerService()
response = self._post(
service,
note="x" * (module.MAX_NOTE_LENGTH + 1),
)
self.assertEqual(response.status_code, 200)
self.assertIn(b"must be", response.data)
self.assertEqual(service.enqueue_calls, [])
def test_submit_rejects_oversized_request_body(self):
service = FakeControllerService()
app = self._app(service)
response = app.test_client().post(
"/submit",
data={
"url": "https://www.youtube.com/watch?v=abc",
"note": "x" * 64_000,
},
follow_redirects=True,
)
self.assertEqual(response.status_code, 200)
self.assertIn(b"too large", response.data.lower())
self.assertEqual(service.enqueue_calls, [])
def test_reused_submission_preserves_existing_slot_message(self):
service = FakeControllerService(
enqueue_result={"status": "queued", "slot": 7, "reused_slot": True}
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertIn(b"Playlist queued in slot 7", response.data)
self.assertIn(b"Existing slot", response.data)
self.assertIn(b"only new videos will be added", response.data)
def test_queue_full_has_a_useful_capacity_message(self):
service = FakeControllerService(
enqueue_result={
"status": "queue_full",
"slot": None,
"reused_slot": False,
"oldest_expires_in_seconds": 7200,
}
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertIn(b"Queue is full", response.data)
self.assertIn(b"about 2 hour(s)", response.data)
self.assertNotIn(b"slot None", response.data)
def test_shutting_down_has_a_useful_retry_message(self):
service = FakeControllerService(
enqueue_result={
"status": "shutting_down",
"slot": None,
"reused_slot": False,
}
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertIn(b"shutting down", response.data.lower())
self.assertIn(b"try again", response.data.lower())
self.assertNotIn(b"slot None", response.data)
def test_worker_error_has_a_useful_failure_message(self):
service = FakeControllerService(
enqueue_result={
"status": "worker_error",
"slot": 4,
"reused_slot": False,
}
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertIn(b"download worker", response.data.lower())
self.assertTrue(
b"could not" in response.data.lower()
or b"failed" in response.data.lower()
)
self.assertNotIn(b"Playlist queued", response.data)
def test_controller_unavailable_is_handled_without_a_500(self):
service = FakeControllerService(
enqueue_error=ControllerUnavailableError(
"Controller coordinator did not respond"
)
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertNotIn(b"Internal Server Error", response.data)
self.assertIn(b"Controller", response.data)
self.assertIn(b"unavailable", response.data.lower())
def test_store_error_is_handled_without_a_500(self):
service = FakeControllerService(
enqueue_error=StoreError("Controller database is unavailable")
)
response = self._post(service)
self.assertEqual(response.status_code, 200)
self.assertNotIn(b"Internal Server Error", response.data)
self.assertIn(b"Controller", response.data)
self.assertIn(b"unavailable", response.data.lower())
def test_health_is_healthy_when_ok_or_degraded(self):
cases = (
{
"status": "ok",
"started": True,
"coordinator_alive": True,
"mqtt_connected": True,
"degraded_reasons": [],
},
{
"status": "degraded",
"started": True,
"coordinator_alive": True,
"mqtt_connected": False,
"degraded_reasons": ["mqtt_disconnected"],
},
)
for health in cases:
with self.subTest(status=health["status"]):
app = self._app(FakeControllerService(health=health))
response = app.test_client().get("/healthz")
self.assertEqual(response.status_code, 200)
self.assertEqual(response.get_json(), health)
def test_health_is_unhealthy_when_stopped_or_coordinator_is_dead(self):
cases = (
{
"status": "stopped",
"started": False,
"coordinator_alive": False,
"mqtt_connected": False,
"degraded_reasons": [],
},
{
"status": "degraded",
"started": True,
"coordinator_alive": False,
"mqtt_connected": False,
"degraded_reasons": ["coordinator_dead"],
},
)
for health in cases:
with self.subTest(status=health["status"]):
app = self._app(FakeControllerService(health=health))
response = app.test_client().get("/healthz")
self.assertEqual(response.status_code, 503)
self.assertEqual(response.get_json(), health)
def test_production_factory_requires_flask_secret_key(self):
with mock.patch.dict(os.environ, {}, clear=True):
with self.assertRaisesRegex(RuntimeError, "FLASK_SECRET_KEY"):
self._factory()(self.config, FakeControllerService())
def test_testing_factory_can_use_a_test_only_secret(self):
with mock.patch.dict(os.environ, {}, clear=True):
app = self._factory()(
self.config,
FakeControllerService(),
testing=True,
)
self.assertIs(app.testing, True)
self.assertTrue(app.secret_key)
self.assertNotEqual(app.secret_key, "dev-secret")
class AppImportLifecycleTests(unittest.TestCase):
def test_import_does_not_construct_or_start_controller_runtime(self):
with tempfile.TemporaryDirectory() as directory:
media = Path(directory) / "media"
data = Path(directory) / "data"
environment = {
**os.environ,
"YTPLAYER_MEDIA_BASE": str(media),
"YTPLAYER_DATA_BASE": str(data),
}
environment.pop("FLASK_SECRET_KEY", None)
before_threads = {thread.ident for thread in threading.enumerate()}
sys.modules.pop("app", None)
with mock.patch.dict(os.environ, environment, clear=True):
with mock.patch.object(
ControllerService,
"__init__",
side_effect=AssertionError("import constructed ControllerService"),
), mock.patch.object(
ControllerService,
"start",
side_effect=AssertionError("import started ControllerService"),
), mock.patch.object(
MqttAdapter,
"start",
side_effect=AssertionError("import started MQTT"),
):
imported = importlib.import_module("app")
self.assertTrue(callable(imported.create_app))
self.assertFalse(media.exists())
self.assertFalse(data.exists())
self.assertEqual(
{thread.ident for thread in threading.enumerate()},
before_threads,
)
sys.modules.pop("app", None)
if __name__ == "__main__":
unittest.main()
+246
View File
@@ -0,0 +1,246 @@
import importlib
import json
import os
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from controller.config import ConfigError, load_config
from controller.contracts import (
PLAYER_COMMAND_QOS,
PLAYER_COMMAND_RETAIN,
SLOT_IDS,
Topics,
)
from controller.ha import HA_DEVICE, SlotAnnouncement, discovery_messages, slot_messages
class ContractCharacterizationTests(unittest.TestCase):
def test_player_and_legacy_topic_defaults_are_preserved(self):
topics = Topics()
self.assertEqual(topics.player_command, "youtube/player/cmd")
self.assertEqual(topics.player_status, "youtube/player/status")
self.assertEqual(topics.player_state, "youtube/player/state")
self.assertEqual(topics.player_telemetry, "youtube/player/telemetry")
self.assertEqual(topics.controller_command, "youtube/controller/cmd")
self.assertEqual(topics.ingest_progress, "youtube/ingest/progress")
self.assertEqual(topics.playback_base, "youtube/media/playback")
self.assertEqual(topics.router_mode_set, "audio/player/mode/set")
self.assertEqual(topics.router_mode_state, "audio/player/mode/state")
self.assertEqual(topics.controller_result, "youtube/controller/v2/events")
self.assertEqual(PLAYER_COMMAND_QOS, 1)
self.assertIs(PLAYER_COMMAND_RETAIN, False)
self.assertEqual(SLOT_IDS, tuple(range(1, 11)))
def test_existing_ha_discovery_ids_payloads_and_retain_are_preserved(self):
config = load_config(environ={})
messages = list(discovery_messages(config.topics, config.homeassistant))
self.assertEqual(len(messages), 30)
topic, payload, qos, retain = messages[0]
self.assertEqual(topic, "homeassistant/sensor/yt_slot_1_title/config")
self.assertEqual(qos, 0)
self.assertIs(retain, True)
decoded = json.loads(payload)
self.assertEqual(decoded["unique_id"], "yt_slot_1_title")
self.assertEqual(decoded["state_topic"], "youtube/media/playback/1/title")
self.assertEqual(decoded["device"], HA_DEVICE)
self.assertEqual(messages[-1][0], "homeassistant/sensor/yt_slot_10_count/config")
def test_slot_topics_remain_plain_retained_values_and_clear_item_tail(self):
announcement = SlotAnnouncement(
slot=4,
title="Service",
state="ready",
count=2,
items=("One", "Two"),
)
messages = list(slot_messages(announcement, Topics(), clear_items_through=4))
self.assertEqual(
messages,
[
("youtube/media/playback/4/title", "Service", 0, True),
("youtube/media/playback/4/state", "ready", 0, True),
("youtube/media/playback/4/count", "2", 0, True),
("youtube/media/playback/4/items/1", "One", 0, True),
("youtube/media/playback/4/items/2", "Two", 0, True),
("youtube/media/playback/4/items/3", "", 0, True),
("youtube/media/playback/4/items/4", "", 0, True),
],
)
class ConfigurationTests(unittest.TestCase):
def test_safe_router_defaults_are_finite_and_service_usable(self):
config = load_config(environ={})
self.assertEqual(config.router.required_mode, "video")
self.assertEqual(config.router.confirmation_timeout_seconds, 3.0)
self.assertEqual(config.router.when_state_unknown, "attempt")
self.assertIs(config.router.continue_on_confirmation_failure, True)
self.assertIsNone(config.router.fallback_restore_mode)
self.assertEqual(config.storage.uncertain_review_after_seconds, 900.0)
self.assertEqual(config.mqtt.publish_ack_timeout_seconds, 2.0)
self.assertEqual(config.mqtt.router_echo_grace_seconds, 2.0)
self.assertEqual(config.downloads.export_directory, Path("exports"))
self.assertEqual(config.downloads.export_max_height, 1080)
self.assertEqual(config.downloads.export_max_workers, 1)
self.assertEqual(config.downloads.max_queued_exports, 20)
def test_documented_environment_overrides_apply(self):
env = {
"MQTT_HOST": "broker",
"MQTT_PORT": "2883",
"MQTT_USERNAME": "user",
"MQTT_PASSWORD": "pass",
"YTPLAYER_MEDIA_BASE": "/media-test",
"YTPLAYER_DATA_BASE": "/data-test",
"YTPLAYER_COOKIE_PATH": "",
"PORT": "9000",
}
config = load_config(environ=env)
self.assertEqual(config.mqtt.host, "broker")
self.assertEqual(config.mqtt.port, 2883)
self.assertEqual(config.mqtt.username, "user")
self.assertEqual(config.mqtt.password, "pass")
self.assertEqual(config.storage.media_base, Path("/media-test"))
self.assertEqual(config.storage.data_base, Path("/data-test"))
self.assertIsNone(config.downloads.cookie_path)
self.assertEqual(config.web.port, 9000)
def test_blank_optional_mqtt_credentials_are_treated_as_unset(self):
config = load_config(
environ={
"MQTT_USERNAME": "",
"MQTT_PASSWORD": " ",
}
)
self.assertIsNone(config.mqtt.username)
self.assertIsNone(config.mqtt.password)
def test_blank_compose_credentials_do_not_mask_yaml_credentials(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
"mqtt:\n username: yaml-user\n password: yaml-pass\n",
encoding="utf-8",
)
config = load_config(
path,
environ={"MQTT_USERNAME": "", "MQTT_PASSWORD": ""},
)
self.assertEqual(config.mqtt.username, "yaml-user")
self.assertEqual(config.mqtt.password, "yaml-pass")
def test_slot_count_cannot_break_stable_identities(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text("storage:\n slot_count: 9\n", encoding="utf-8")
with self.assertRaisesRegex(ConfigError, "fixed at 10"):
load_config(path, environ={})
def test_invalid_router_unknown_state_policy_is_rejected(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text("router:\n when_state_unknown: guess\n", encoding="utf-8")
with self.assertRaisesRegex(ConfigError, "attempt, continue, or reject"):
load_config(path, environ={})
def test_non_finite_timeouts_are_rejected(self):
for value in (".nan", ".inf", "-.inf"):
with self.subTest(value=value), tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
f"router:\n confirmation_timeout_seconds: {value}\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ConfigError, "must be finite"):
load_config(path, environ={})
def test_negative_router_echo_grace_is_rejected(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
"mqtt:\n router_echo_grace_seconds: -1\n", encoding="utf-8"
)
with self.assertRaisesRegex(ConfigError, "router_echo_grace_seconds"):
load_config(path, environ={})
def test_runtime_paths_must_be_absolute(self):
cases = (
("storage:\n media_base: relative/media\n", "storage.media_base"),
("storage:\n data_base: relative/data\n", "storage.data_base"),
("downloads:\n cookie_path: relative/cookies.txt\n", "downloads.cookie_path"),
)
for contents, field in cases:
with self.subTest(field=field), tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(contents, encoding="utf-8")
with self.assertRaisesRegex(ConfigError, f"{field} must be an absolute path"):
load_config(path, environ={})
def test_export_directory_must_stay_relative_to_media_root(self):
invalid = ("/tmp/exports", "../exports", "nested/../exports", r"nested\\exports")
for value in invalid:
with self.subTest(value=value), tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
f"downloads:\n export_directory: {value!r}\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ConfigError, "safe relative directory"):
load_config(path, environ={})
def test_export_directory_cannot_overlap_player_slot_storage(self):
invalid = ("slot1", "SLOT10/exports", ".controller-trash/exports")
for value in invalid:
with self.subTest(value=value), tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
f"downloads:\n export_directory: {value!r}\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ConfigError, "slot storage"):
load_config(path, environ={})
def test_export_limits_are_bounded_positive_integers(self):
cases = (
("export_max_height", 0),
("export_max_workers", 0),
("export_max_workers", 11),
("max_queued_exports", 0),
("max_queued_exports", 101),
)
for field, value in cases:
with self.subTest(field=field), tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "config.yaml"
path.write_text(
f"downloads:\n {field}: {value}\n",
encoding="utf-8",
)
with self.assertRaisesRegex(ConfigError, field):
load_config(path, environ={})
class ImportLifecycleTests(unittest.TestCase):
def test_controller_core_import_does_not_connect_or_create_runtime_paths(self):
with tempfile.TemporaryDirectory() as directory:
media = Path(directory) / "media"
data = Path(directory) / "data"
env = {
**os.environ,
"YTPLAYER_MEDIA_BASE": str(media),
"YTPLAYER_DATA_BASE": str(data),
}
sys.modules.pop("controller_core", None)
with mock.patch.dict(os.environ, env, clear=True):
module = importlib.import_module("controller_core")
self.assertFalse(media.exists())
self.assertFalse(data.exists())
self.assertIsNone(module.mqtt_client)
self.assertIs(module.mqtt_started, False)
sys.modules.pop("controller_core", None)
if __name__ == "__main__":
unittest.main()
+594
View File
@@ -0,0 +1,594 @@
import os
import stat
import subprocess
import tempfile
import threading
import time
import unittest
from concurrent.futures import Future
from pathlib import Path
from yt_dlp import YoutubeDL
from controller.config import ControllerConfig, DownloadConfig, StorageConfig
from controller.downloads import (
DownloadManager,
canonical_playlist_path,
player_format_selector,
)
from controller.models import SlotProtectedError
from controller.store import ControllerStore
class ImmediateExecutor:
def submit(self, function, *args, **kwargs):
future = Future()
try:
future.set_result(function(*args, **kwargs))
except BaseException as exc:
future.set_exception(exc)
return future
class QueuedExecutor:
def __init__(self):
self.calls = []
def submit(self, function, *args, **kwargs):
future = Future()
self.calls.append((future, function, args, kwargs))
return future
def run_next(self):
future, function, args, kwargs = self.calls.pop(0)
try:
future.set_result(function(*args, **kwargs))
except BaseException as exc:
future.set_exception(exc)
return future
class FakeYtDlp:
def __init__(
self,
media_base: Path,
*,
fail_download=False,
no_media=False,
rewrite_cookie=False,
):
self.media_base = media_base
self.fail_download = fail_download
self.no_media = no_media
self.rewrite_cookie = rewrite_cookie
self.calls = []
self.cookie_observations = []
def __call__(self, arguments, **kwargs):
self.calls.append((list(arguments), dict(kwargs)))
if "--cookies" in arguments:
cookie_path = Path(arguments[arguments.index("--cookies") + 1])
self.cookie_observations.append(
(
cookie_path,
cookie_path.read_text(encoding="utf-8"),
stat.S_IMODE(cookie_path.stat().st_mode),
)
)
if self.rewrite_cookie:
cookie_path.write_text("updated by yt-dlp", encoding="utf-8")
if "--flat-playlist" in arguments:
return subprocess.CompletedProcess(
arguments,
0,
stdout="Test Playlist|||First Video|||playlist-id\n",
stderr="",
)
if self.fail_download:
raise subprocess.CalledProcessError(7, arguments)
if not self.no_media:
output_index = arguments.index("-o") + 1
output_template = Path(arguments[output_index])
directory = output_template.parent
directory.mkdir(parents=True, exist_ok=True)
(directory / "001 - First [one].mp4").write_bytes(b"video-one")
(directory / "002 - Second [two].webm").write_bytes(b"video-two")
return subprocess.CompletedProcess(arguments, 0, stdout="", stderr="")
class DownloadManagerTests(unittest.TestCase):
def setUp(self):
self.tempdir = tempfile.TemporaryDirectory()
base = Path(self.tempdir.name)
self.media = base / "media"
self.data = base / "data"
self.cookie = base / "cookies.txt"
self.cookie.write_text("cookie data", encoding="utf-8")
self.config = ControllerConfig(
storage=StorageConfig(
media_base=self.media,
data_base=self.data,
retention_seconds=100,
),
downloads=DownloadConfig(
yt_dlp_binary="yt-dlp-test",
cookie_path=self.cookie,
cookie_required=True,
max_workers=1,
process_timeout_seconds=30,
max_height=720,
),
)
self.store = ControllerStore(self.config.storage).open()
def tearDown(self):
self.store.close()
self.tempdir.cleanup()
def manager(self, runner, events=None):
event_list = events if events is not None else []
return DownloadManager(
self.config,
self.store,
runner=runner,
executor=ImmediateExecutor(),
on_event=event_list.append,
)
def test_success_uses_configured_cookie_and_atomically_builds_playlist(self):
events = []
self.cookie.chmod(0o444)
runner = FakeYtDlp(self.media, rewrite_cookie=True)
manager = self.manager(runner, events)
result = manager.enqueue("https://example.test/playlist", "Alice", "Service")
self.assertEqual(result.status, "reserved")
self.assertEqual([event.state for event in events], ["queued", "downloading", "ready"])
slot = self.store.get_slot(1)
self.assertEqual(slot.media_state, "ready")
self.assertEqual(slot.title, "Test Playlist")
self.assertEqual(slot.item_count, 2)
playlist = canonical_playlist_path(self.media, 1)
self.assertEqual(
playlist.read_text(encoding="utf-8"),
"001 - First [one].mp4\n002 - Second [two].webm\n",
)
self.assertEqual(list(playlist.parent.glob(".playlist.*.tmp")), [])
for arguments, _ in runner.calls:
self.assertIn("--ignore-config", arguments)
self.assertIn("--no-js-runtimes", arguments)
self.assertEqual(arguments[arguments.index("--js-runtimes") + 1], "node")
self.assertEqual(
arguments[arguments.index("--cache-dir") + 1],
"/tmp/yt-dlp-cache",
)
self.assertIn("--cookies", arguments)
self.assertNotIn(str(self.cookie), arguments)
self.assertEqual(len(runner.cookie_observations), 2)
working_paths = [item[0] for item in runner.cookie_observations]
self.assertEqual(len(set(working_paths)), 2)
self.assertTrue(
all(item[1] == "cookie data" for item in runner.cookie_observations)
)
self.assertTrue(
all(item[2] == 0o600 for item in runner.cookie_observations)
)
self.assertTrue(all(not path.exists() for path in working_paths))
self.assertEqual(self.cookie.read_text(encoding="utf-8"), "cookie data")
download_arguments = runner.calls[-1][0]
self.assertIn("bv*[height<=720]+ba/b[height<=720]", download_arguments)
@staticmethod
def _select_player_format(max_height, formats):
with YoutubeDL({"quiet": True}) as ydl:
selector = ydl.build_format_selector(
player_format_selector(max_height)
)
selected = list(ydl._select_formats(formats, selector))
return selected
def test_player_selector_chooses_separate_true_1080_video_and_audio(self):
formats = [
{
"format_id": "22",
"url": "https://example.test/720.mp4",
"ext": "mp4",
"height": 720,
"width": 1280,
"vcodec": "avc1.64001f",
"acodec": "mp4a.40.2",
"tbr": 2500,
"filesize": 1,
},
{
"format_id": "399",
"url": "https://example.test/1080.mp4",
"ext": "mp4",
"height": 1080,
"width": 1920,
"vcodec": "av01.0.08M.08",
"acodec": "none",
"tbr": 4500,
"filesize": 1,
},
{
"format_id": "251",
"url": "https://example.test/audio.webm",
"ext": "webm",
"vcodec": "none",
"acodec": "opus",
"tbr": 160,
"filesize": 1,
},
]
selected = self._select_player_format(1080, formats)
self.assertEqual(len(selected), 1)
self.assertEqual(selected[0]["height"], 1080)
self.assertEqual(
[item["format_id"] for item in selected[0]["requested_formats"]],
["399", "251"],
)
self.assertEqual(selected[0]["vcodec"], "av01.0.08M.08")
self.assertEqual(selected[0]["acodec"], "opus")
def test_player_selector_preserves_lower_source_resolution(self):
formats = [
{
"format_id": "18",
"url": "https://example.test/360.mp4",
"ext": "mp4",
"height": 360,
"width": 640,
"vcodec": "avc1.42001e",
"acodec": "mp4a.40.2",
"tbr": 700,
"filesize": 1,
},
{
"format_id": "22",
"url": "https://example.test/720.mp4",
"ext": "mp4",
"height": 720,
"width": 1280,
"vcodec": "avc1.64001f",
"acodec": "mp4a.40.2",
"tbr": 2500,
"filesize": 1,
},
]
selected = self._select_player_format(1080, formats)
self.assertEqual(len(selected), 1)
self.assertEqual(selected[0]["format_id"], "22")
self.assertEqual(selected[0]["height"], 720)
def test_player_selector_never_uses_unfiltered_over_height_fallback(self):
formats = [
{
"format_id": "401",
"url": "https://example.test/2160.mp4",
"ext": "mp4",
"height": 2160,
"width": 3840,
"vcodec": "av01.0.12M.08",
"acodec": "none",
"tbr": 12000,
"filesize": 1,
},
{
"format_id": "251",
"url": "https://example.test/audio.webm",
"ext": "webm",
"vcodec": "none",
"acodec": "opus",
"tbr": 160,
"filesize": 1,
},
]
self.assertEqual(self._select_player_format(1080, formats), [])
def test_worker_failure_is_contained_and_persisted(self):
events = []
runner = FakeYtDlp(self.media, fail_download=True)
manager = self.manager(runner, events)
result = manager.enqueue("https://example.test/failure", None, None)
self.assertEqual(result.status, "reserved")
slot = self.store.get_slot(1)
self.assertEqual(slot.media_state, "error")
self.assertIn("exit code 7", slot.last_error)
self.assertEqual(events[-1].state, "error")
self.assertTrue(
all(not observation[0].exists() for observation in runner.cookie_observations)
)
def test_missing_required_cookie_fails_job_without_escaping(self):
self.cookie.unlink()
events = []
runner = FakeYtDlp(self.media)
manager = self.manager(runner, events)
manager.enqueue("https://example.test/no-cookie", None, None)
self.assertEqual(self.store.get_slot(1).media_state, "error")
self.assertIn("required yt-dlp cookie", self.store.get_slot(1).last_error)
self.assertEqual(runner.calls, [])
def test_delete_moves_only_canonical_slot_and_keeps_other_identity(self):
first = self.store.reserve_download("https://example.test/one", None, None, job_id="one")
second = self.store.reserve_download("https://example.test/two", None, None, job_id="two")
self.store.fail_download("one", 1, first.generation, "failed")
self.store.fail_download("two", 2, second.generation, "failed")
slot_one = self.media / "slot1"
slot_two = self.media / "slot2"
slot_one.mkdir(parents=True)
slot_two.mkdir(parents=True)
(slot_one / "one.mp4").write_bytes(b"one")
(slot_two / "two.mp4").write_bytes(b"two")
manager = self.manager(FakeYtDlp(self.media))
self.assertTrue(manager.delete_slot(1, reason="manual"))
self.assertFalse(slot_one.exists())
self.assertTrue((slot_two / "two.mp4").exists())
self.assertEqual(self.store.get_slot(1).media_state, "empty")
self.assertEqual(self.store.get_slot(2).url, "https://example.test/two")
def test_delete_refuses_live_download_and_playback_guard(self):
reservation = self.store.reserve_download(
"https://example.test/live", None, None, job_id="live"
)
manager = self.manager(FakeYtDlp(self.media))
with self.assertRaises(SlotProtectedError):
manager.delete_slot(1, reason="manual")
self.store.fail_download("live", 1, reservation.generation, "cancelled")
self.store.protect_slot(1, "player_active", guard_id="active")
with self.assertRaises(SlotProtectedError):
manager.delete_slot(1, reason="manual")
def test_symlink_slot_is_unlinked_without_touching_target(self):
reservation = self.store.reserve_download(
"https://example.test/link", None, None, job_id="link"
)
self.store.fail_download("link", 1, reservation.generation, "failed")
target = Path(self.tempdir.name) / "outside"
target.mkdir()
protected = target / "keep.txt"
protected.write_text("keep", encoding="utf-8")
self.media.mkdir(parents=True)
os.symlink(target, self.media / "slot1")
manager = self.manager(FakeYtDlp(self.media))
self.assertTrue(manager.delete_slot(1, reason="manual"))
self.assertEqual(protected.read_text(encoding="utf-8"), "keep")
def test_interrupted_deletion_is_recoverable(self):
reservation = self.store.reserve_download(
"https://example.test/recover", None, None, job_id="recover"
)
self.store.fail_download("recover", 1, reservation.generation, "failed")
token = self.store.begin_delete(1, reason="manual")
slot = self.media / "slot1"
slot.mkdir(parents=True)
(slot / "partial.mp4").write_bytes(b"partial")
self.assertEqual(self.store.pending_deletions(), [
type(token)(slot=1, generation=token.generation, reason="recovered_interrupted_deletion")
])
manager = self.manager(FakeYtDlp(self.media))
self.assertEqual(manager.recover_pending_deletions(), [1])
self.assertFalse(slot.exists())
self.assertEqual(self.store.get_slot(1).media_state, "empty")
def test_interrupted_deletion_recovery_defers_a_new_playback_guard(self):
reservation = self.store.reserve_download(
"https://example.test/protected-recovery",
None,
None,
job_id="protected-recovery",
)
self.store.fail_download(
"protected-recovery",
1,
reservation.generation,
"failed",
)
self.store.begin_delete(1, reason="manual")
slot = self.media / "slot1"
slot.mkdir(parents=True)
(slot / "still-playing.mp4").write_bytes(b"playing")
self.store.protect_slot(1, "player_active", guard_id="fresh-player")
manager = self.manager(FakeYtDlp(self.media))
self.assertEqual(manager.recover_pending_deletions(), [])
self.assertTrue(slot.exists())
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
self.store.clear_playback_guards(reason="fresh terminal idle evidence")
self.assertEqual(manager.recover_pending_deletions(), [1])
self.assertFalse(slot.exists())
def test_async_deletion_failure_remains_durable_and_is_retryable(self):
reservation = self.store.reserve_download(
"https://example.test/retry-delete",
None,
None,
job_id="retry-delete",
)
self.store.fail_download(
"retry-delete",
1,
reservation.generation,
"failed",
)
slot = self.media / "slot1"
slot.mkdir(parents=True)
(slot / "retry.mp4").write_bytes(b"retry")
events = []
manager = self.manager(FakeYtDlp(self.media), events)
original_recover = manager._recover_deletion
attempts = 0
def fail_once(token):
nonlocal attempts
attempts += 1
if attempts == 1:
raise OSError("transient media failure")
return original_recover(token)
manager._recover_deletion = fail_once
token = manager.schedule_delete_slot(1, reason="manual")
self.assertEqual(token.slot, 1)
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
self.assertEqual(events[-1].state, "delete_error")
self.assertEqual(len(self.store.pending_deletions()), 1)
self.assertTrue(manager.schedule_pending_deletion_recovery())
self.assertEqual(attempts, 2)
self.assertEqual(events[-1].state, "deleted")
self.assertEqual(self.store.get_slot(1).media_state, "empty")
self.assertFalse(slot.exists())
def test_async_pending_recovery_retries_after_guard_is_cleared(self):
reservation = self.store.reserve_download(
"https://example.test/deferred-async-delete",
None,
None,
job_id="deferred-async-delete",
)
self.store.fail_download(
"deferred-async-delete",
1,
reservation.generation,
"failed",
)
self.store.begin_delete(1, reason="manual")
slot = self.media / "slot1"
slot.mkdir(parents=True)
(slot / "active.mp4").write_bytes(b"active")
self.store.protect_slot(1, "player_active", guard_id="active-player")
events = []
manager = self.manager(FakeYtDlp(self.media), events)
self.assertTrue(manager.schedule_pending_deletion_recovery())
self.assertEqual(events[-1].state, "delete_deferred")
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
self.assertTrue(slot.exists())
self.store.clear_playback_guards(reason="fresh terminal idle evidence")
self.assertTrue(manager.schedule_pending_deletion_recovery())
self.assertEqual(events[-1].state, "deleted")
self.assertEqual(self.store.get_slot(1).media_state, "empty")
self.assertFalse(slot.exists())
def test_guarded_recovery_restores_crash_staged_trash_to_canonical_slot(self):
reservation = self.store.reserve_download(
"https://example.test/restore-trash",
None,
None,
job_id="restore-trash",
)
self.store.fail_download(
"restore-trash",
1,
reservation.generation,
"failed",
)
token = self.store.begin_delete(1, reason="expired")
trash = (
self.media
/ ".controller-trash"
/ f"slot1.generation{token.generation}.crash"
)
trash.mkdir(parents=True)
(trash / "still-active.mp4").write_bytes(b"active")
self.store.protect_slot(1, "player_active", guard_id="fresh-active")
events = []
manager = self.manager(FakeYtDlp(self.media), events)
self.assertTrue(manager.schedule_pending_deletion_recovery())
canonical = self.media / "slot1"
self.assertEqual(events[-1].state, "delete_deferred")
self.assertTrue((canonical / "still-active.mp4").exists())
self.assertFalse(trash.exists())
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
def test_queued_deletion_rechecks_guard_before_filesystem_mutation(self):
reservation = self.store.reserve_download(
"https://example.test/queued-delete-guard",
None,
None,
job_id="queued-delete-guard",
)
self.store.fail_download(
"queued-delete-guard",
1,
reservation.generation,
"failed",
)
self.store.begin_delete(1, reason="manual")
slot = self.media / "slot1"
slot.mkdir(parents=True)
(slot / "must-survive.mp4").write_bytes(b"active")
events = []
executor = QueuedExecutor()
manager = DownloadManager(
self.config,
self.store,
runner=FakeYtDlp(self.media),
executor=executor,
on_event=events.append,
)
self.assertTrue(manager.schedule_pending_deletion_recovery())
self.store.protect_slot(1, "player_active", guard_id="late-player")
executor.run_next()
self.assertEqual(events[-1].state, "delete_deferred")
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
self.assertTrue((slot / "must-survive.mp4").exists())
self.store.clear_playback_guards(reason="fresh terminal idle evidence")
self.assertTrue(manager.schedule_pending_deletion_recovery())
executor.run_next()
self.assertEqual(events[-1].state, "deleted")
self.assertEqual(self.store.get_slot(1).media_state, "empty")
self.assertFalse(slot.exists())
def test_shutdown_terminates_tracked_process_without_waiting_for_job_timeout(self):
manager = DownloadManager(self.config, self.store)
errors = []
def run_long_process():
try:
manager._run_command(
["/bin/sh", "-c", "sleep 60"],
check=True,
text=True,
timeout=60,
capture_output=True,
)
except BaseException as exc:
errors.append(exc)
worker = threading.Thread(target=run_long_process)
worker.start()
deadline = time.monotonic() + 2
while time.monotonic() < deadline:
with manager._lock:
if manager._processes:
break
time.sleep(0.01)
else:
self.fail("test process did not start")
started = time.monotonic()
manager.shutdown(wait=False)
worker.join(timeout=3)
self.assertFalse(worker.is_alive())
self.assertLess(time.monotonic() - started, 3)
self.assertTrue(errors)
with manager._lock:
self.assertEqual(manager._processes, set())
if __name__ == "__main__":
unittest.main()
+655
View File
@@ -0,0 +1,655 @@
import json
import subprocess
import tempfile
import unittest
from concurrent.futures import Future
from pathlib import Path
from yt_dlp import YoutubeDL
from controller.config import ControllerConfig, DownloadConfig, StorageConfig
from controller.downloads import (
DownloadManager,
mp4_export_format_selector,
safe_export_filename,
)
from controller.store import ControllerStore
class ImmediateExecutor:
def submit(self, function, *args, **kwargs):
future = Future()
try:
future.set_result(function(*args, **kwargs))
except BaseException as exc:
future.set_exception(exc)
return future
class QueuedExecutor:
def __init__(self):
self.calls = []
def submit(self, function, *args, **kwargs):
future = Future()
self.calls.append((future, function, args, kwargs))
return future
class FakeExportRunner:
def __init__(
self,
*,
video_codec="h264",
audio_codec="aac",
pixel_format="yuv420p",
container="mp4",
height=1080,
playlist_items=1,
fail_download=False,
):
self.video_codec = video_codec
self.audio_codec = audio_codec
self.pixel_format = pixel_format
self.container = container
self.height = height
self.playlist_items = playlist_items
self.fail_download = fail_download
self.calls = []
def __call__(self, arguments, **kwargs):
arguments = list(arguments)
self.calls.append((arguments, dict(kwargs)))
program = Path(arguments[0]).name
if program == "yt-dlp-test":
if self.fail_download:
raise subprocess.CalledProcessError(9, arguments)
template = Path(arguments[arguments.index("-o") + 1])
template.parent.mkdir(parents=True, exist_ok=True)
if "--audio-format" in arguments:
extension = "." + arguments[arguments.index("--audio-format") + 1]
else:
extension = {
"mp4": ".mp4",
"matroska": ".mkv",
"webm": ".webm",
"mov": ".mov",
}[self.container]
for index in range(1, self.playlist_items + 1):
(template.parent / f"{index:03d} - Tést Item {index} [id{index}]{extension}").write_bytes(
f"source-{index}".encode()
)
return subprocess.CompletedProcess(arguments, 0, stdout="", stderr="")
if program == "ffmpeg":
Path(arguments[-1]).write_bytes(b"converted")
return subprocess.CompletedProcess(arguments, 0, stdout="", stderr="")
if program == "ffprobe":
path = Path(arguments[-1])
converted = path.is_file() and path.read_bytes() == b"converted"
if path.suffix == ".m4a":
payload = {
"streams": [{"codec_type": "audio", "codec_name": "aac"}],
"format": {"format_name": "mov,mp4,m4a,3gp,3g2,mj2"},
}
elif path.suffix == ".mp3":
payload = {
"streams": [{"codec_type": "audio", "codec_name": "mp3"}],
"format": {"format_name": "mp3"},
}
else:
payload = {
"streams": [
{
"codec_type": "video",
"codec_name": "h264" if converted else self.video_codec,
"pix_fmt": "yuv420p" if converted else self.pixel_format,
"width": 1920 if self.height == 1080 else 1280,
"height": self.height,
},
{
"codec_type": "audio",
"codec_name": "aac" if converted else self.audio_codec,
},
],
"format": {
"format_name": "mov,mp4,m4a,3gp,3g2,mj2"
if converted or self.container in {"mp4", "mov"}
else self.container
},
}
return subprocess.CompletedProcess(
arguments,
0,
stdout=json.dumps(payload),
stderr="",
)
raise AssertionError(f"unexpected command: {arguments!r}")
@property
def yt_dlp_calls(self):
return [arguments for arguments, _ in self.calls if Path(arguments[0]).name == "yt-dlp-test"]
@property
def ffmpeg_calls(self):
return [arguments for arguments, _ in self.calls if Path(arguments[0]).name == "ffmpeg"]
class UtilityExportTests(unittest.TestCase):
def setUp(self):
self.tempdir = tempfile.TemporaryDirectory()
base = Path(self.tempdir.name)
self.media = base / "media"
self.data = base / "data"
self.config = ControllerConfig(
storage=StorageConfig(media_base=self.media, data_base=self.data),
downloads=DownloadConfig(
yt_dlp_binary="yt-dlp-test",
cookie_path=None,
max_workers=1,
process_timeout_seconds=30,
max_height=1080,
export_directory=Path("exports"),
export_max_height=1080,
max_queued_exports=5,
),
)
self.store = ControllerStore(self.config.storage).open()
def tearDown(self):
self.store.close()
self.tempdir.cleanup()
def manager(self, runner, events=None):
return DownloadManager(
self.config,
self.store,
runner=runner,
executor=ImmediateExecutor(),
export_executor=ImmediateExecutor(),
on_event=(events if events is not None else []).append,
)
def assert_player_slots_empty(self):
self.assertTrue(
all(slot.media_state == "empty" for slot in self.store.list_slots())
)
def test_compatible_mp4_exports_without_slot_or_transcode(self):
events = []
runner = FakeExportRunner()
result = self.manager(runner, events).enqueue_export(
"https://www.youtube.com/watch?v=one", "mp4", "Alice", "Slides"
)
self.assertEqual(result.status, "queued")
job = self.store.get_export_job(result.job_id)
self.assertEqual(job.status, "completed")
self.assertEqual(len(job.output_paths), 1)
self.assertTrue(job.output_paths[0].startswith(f"exports/video/{result.job_id}/"))
self.assertTrue(job.output_paths[0].endswith(".mp4"))
self.assertTrue((self.media / job.output_paths[0]).is_file())
self.assertEqual(events, [])
self.assertEqual(runner.ffmpeg_calls, [])
self.assert_player_slots_empty()
arguments = runner.yt_dlp_calls[0]
self.assertIn("--yes-playlist", arguments)
self.assertIn("--windows-filenames", arguments)
self.assertIn("--no-overwrites", arguments)
self.assertEqual(
arguments[arguments.index("-f") + 1],
"bv*[height<=1080]+ba/b[height<=1080]",
)
self.assertEqual(
arguments[arguments.index("-S") + 1],
"res:1080,vcodec:h264,acodec:aac",
)
def test_incompatible_mp4_transcodes_only_incompatible_codecs(self):
runner = FakeExportRunner(
video_codec="vp9",
audio_codec="opus",
pixel_format="yuv420p",
container="webm",
)
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/watch?v=vp9", "mp4", None, None
)
self.assertEqual(self.store.get_export_job(result.job_id).status, "completed")
self.assertEqual(len(runner.ffmpeg_calls), 1)
arguments = runner.ffmpeg_calls[0]
self.assertIn("libx264", arguments)
self.assertIn("-crf", arguments)
self.assertEqual(arguments[arguments.index("-crf") + 1], "18")
self.assertIn("yuv420p", arguments)
self.assertIn("aac", arguments)
self.assertIn("192k", arguments)
self.assertNotIn("-vf", arguments)
self.assertNotIn("scale", " ".join(arguments))
self.assert_player_slots_empty()
def test_compatible_h264_aac_in_other_container_is_remuxed_only(self):
runner = FakeExportRunner(container="matroska")
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/watch?v=mkv", "mp4", None, None
)
self.assertEqual(self.store.get_export_job(result.job_id).status, "completed")
arguments = runner.ffmpeg_calls[0]
self.assertEqual(arguments[arguments.index("-c:v") + 1], "copy")
self.assertEqual(arguments[arguments.index("-c:a") + 1], "copy")
self.assertNotIn("libx264", arguments)
def test_quicktime_alias_is_remuxed_instead_of_renamed_to_mp4(self):
runner = FakeExportRunner(container="mov")
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/watch?v=mov", "mp4", None, None
)
self.assertEqual(self.store.get_export_job(result.job_id).status, "completed")
arguments = runner.ffmpeg_calls[0]
self.assertEqual(arguments[arguments.index("-c:v") + 1], "copy")
self.assertEqual(arguments[arguments.index("-c:a") + 1], "copy")
self.assertNotIn("libx264", arguments)
def test_export_reconciliation_is_submitted_to_dedicated_executor(self):
export_executor = QueuedExecutor()
manager = DownloadManager(
self.config,
self.store,
runner=FakeExportRunner(),
executor=ImmediateExecutor(),
export_executor=export_executor,
)
self.assertTrue(manager.schedule_export_reconciliation())
self.assertEqual(len(export_executor.calls), 1)
_, function, arguments, keyword_arguments = export_executor.calls[0]
self.assertEqual(function, manager.reconcile_export_artifacts)
self.assertEqual(arguments, ())
self.assertEqual(keyword_arguments, {})
def test_lower_resolution_mp4_is_preserved_without_scaling(self):
runner = FakeExportRunner(height=720)
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/watch?v=720", "mp4", None, None
)
self.assertEqual(self.store.get_export_job(result.job_id).status, "completed")
self.assertEqual(runner.ffmpeg_calls, [])
def test_over_height_mp4_is_rejected_even_if_downloader_misbehaves(self):
runner = FakeExportRunner(height=2160)
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/watch?v=2160", "mp4", None, None
)
job = self.store.get_export_job(result.job_id)
self.assertEqual(job.status, "failed")
self.assertIn("maximum height", job.error)
self.assertFalse((self.media / "exports" / "video" / result.job_id).exists())
self.assert_player_slots_empty()
def test_audio_exports_use_intentional_audio_only_strategies(self):
for mode, expected_selector in (
("m4a", "ba/b"),
("mp3", "ba/b"),
):
with self.subTest(mode=mode):
events = []
runner = FakeExportRunner()
result = self.manager(runner, events).enqueue_export(
f"https://www.youtube.com/watch?v={mode}", mode, None, None
)
job = self.store.get_export_job(result.job_id)
self.assertEqual(job.status, "completed")
self.assertEqual(len(job.output_paths), 1)
self.assertTrue(job.output_paths[0].endswith(f".{mode}"))
arguments = runner.yt_dlp_calls[0]
self.assertEqual(arguments[arguments.index("-f") + 1], expected_selector)
self.assertIn("-x", arguments)
self.assertEqual(
arguments[arguments.index("--audio-format") + 1], mode
)
self.assertEqual(
arguments[arguments.index("--audio-quality") + 1], "0"
)
if mode == "m4a":
self.assertEqual(
arguments[arguments.index("-S") + 1],
"abr,acodec:aac",
)
self.assertEqual(runner.ffmpeg_calls, [])
self.assertEqual(events, [])
self.assert_player_slots_empty()
def test_playlist_items_publish_as_individual_collision_safe_files(self):
runner = FakeExportRunner(playlist_items=3)
result = self.manager(runner).enqueue_export(
"https://www.youtube.com/playlist?list=three", "m4a", None, None
)
job = self.store.get_export_job(result.job_id)
self.assertEqual(job.status, "completed")
self.assertEqual(len(job.output_paths), 3)
self.assertEqual(len(set(job.output_paths)), 3)
self.assertTrue(all((self.media / path).is_file() for path in job.output_paths))
def test_export_failure_and_expiry_never_mutate_player_or_export_files(self):
slot_one = self.media / "slot1"
slot_one.mkdir(parents=True)
protected_file = slot_one / "active.mp4"
protected_file.write_bytes(b"active")
failed = self.manager(FakeExportRunner(fail_download=True)).enqueue_export(
"https://www.youtube.com/watch?v=fail", "mp3", None, None
)
self.assertEqual(self.store.get_export_job(failed.job_id).status, "failed")
self.assertEqual(protected_file.read_bytes(), b"active")
self.assert_player_slots_empty()
completed = self.manager(FakeExportRunner()).enqueue_export(
"https://www.youtube.com/watch?v=keep", "m4a", None, None
)
export_path = self.media / self.store.get_export_job(completed.job_id).output_paths[0]
self.manager(FakeExportRunner()).expire_due_slots()
self.assertTrue(export_path.is_file())
self.assertEqual(protected_file.read_bytes(), b"active")
def test_symlinked_export_root_cannot_redirect_into_player_slot(self):
slot_one = self.media / "slot1"
slot_one.mkdir(parents=True)
protected_file = slot_one / "active.mp4"
protected_file.write_bytes(b"active")
(self.media / "exports").symlink_to(slot_one, target_is_directory=True)
result = self.manager(FakeExportRunner()).enqueue_export(
"https://www.youtube.com/watch?v=escape", "m4a", None, None
)
self.assertEqual(self.store.get_export_job(result.job_id).status, "failed")
self.assertIn("unsafe export directory", self.store.get_export_job(result.job_id).error)
self.assertEqual(protected_file.read_bytes(), b"active")
def test_export_work_uses_dedicated_executor_capacity(self):
player_executor = QueuedExecutor()
export_executor = QueuedExecutor()
manager = DownloadManager(
self.config,
self.store,
runner=FakeExportRunner(),
executor=player_executor,
export_executor=export_executor,
)
manager.enqueue_export(
"https://www.youtube.com/watch?v=export", "mp4", None, None
)
manager.enqueue(
"https://www.youtube.com/watch?v=player", None, None
)
self.assertEqual(len(export_executor.calls), 1)
self.assertEqual(len(player_executor.calls), 1)
def test_restart_reconciles_published_export_and_cleans_manifest(self):
runner = FakeExportRunner()
manager = self.manager(runner)
job_id = "a" * 32
self.store.reserve_export_job(
"https://www.youtube.com/watch?v=recover",
"mp4",
None,
None,
max_active=5,
job_id=job_id,
)
self.store.mark_export_running(job_id)
self.store.fail_export_job(job_id, "database completion failed")
work = self.media / "exports" / ".work" / job_id
published = self.media / "exports" / "video" / job_id
work.mkdir(parents=True)
published.mkdir(parents=True)
filename = "Recovered.mp4"
(published / filename).write_bytes(b"compatible")
manager._write_export_manifest(
work,
job_id=job_id,
mode="mp4",
filenames=[filename],
)
manager.reconcile_export_artifacts()
job = self.store.get_export_job(job_id)
self.assertEqual(job.status, "completed")
self.assertEqual(
job.output_paths,
(f"exports/video/{job_id}/{filename}",),
)
self.assertFalse(work.exists())
self.assertTrue((published / filename).is_file())
def test_restart_atomically_publishes_manifested_ready_export(self):
runner = FakeExportRunner()
manager = self.manager(runner)
job_id = "b" * 32
self.store.reserve_export_job(
"https://www.youtube.com/watch?v=staged",
"m4a",
None,
None,
max_active=5,
job_id=job_id,
)
self.store.mark_export_running(job_id)
self.store.fail_export_job(job_id, "interrupted after conversion")
work = self.media / "exports" / ".work" / job_id
ready = work / "ready"
ready.mkdir(parents=True)
filename = "Recovered.m4a"
(ready / filename).write_bytes(b"aac")
manager._write_export_manifest(
work,
job_id=job_id,
mode="m4a",
filenames=[filename],
)
manager.reconcile_export_artifacts()
job = self.store.get_export_job(job_id)
output = self.media / job.output_paths[0]
self.assertEqual(job.status, "completed")
self.assertTrue(output.is_file())
self.assertFalse(work.exists())
def test_restart_discards_only_known_incomplete_export_staging(self):
manager = self.manager(FakeExportRunner())
job_id = "c" * 32
self.store.reserve_export_job(
"https://www.youtube.com/watch?v=incomplete",
"mp3",
None,
None,
max_active=5,
job_id=job_id,
)
self.store.fail_export_job(job_id, "interrupted download")
work = self.media / "exports" / ".work" / job_id
work.mkdir(parents=True)
(work / "partial.part").write_bytes(b"partial")
unknown = self.media / "exports" / ".work" / ("d" * 32)
unknown.mkdir()
(unknown / "operator-file").write_bytes(b"keep")
manager.reconcile_export_artifacts()
self.assertFalse(work.exists())
self.assertTrue((unknown / "operator-file").is_file())
def test_safe_filename_blocks_traversal_preserves_unicode_and_deduplicates(self):
used = set()
first = safe_export_filename(
"../../slot1/CON:Tést\x00?", ".mp4", used_casefolded=used
)
second = safe_export_filename(
"../../slot1/CON:Tést\x00?", ".mp4", used_casefolded=used
)
long_name = safe_export_filename("🎬" * 500, ".mp3")
self.assertNotIn("/", first)
self.assertNotIn("\\", first)
self.assertNotIn("..", Path(first).parts)
self.assertIn("Tést", first)
self.assertNotEqual(first.casefold(), second.casefold())
self.assertTrue(second.endswith(" (2).mp4"))
self.assertLessEqual(len(Path(long_name).stem.encode("utf-8")), 180)
def test_mp4_sorting_keeps_resolution_ahead_of_codec_preference(self):
formats = [
{
"format_id": "av1-1080",
"url": "https://example.test/av1",
"ext": "webm",
"height": 1080,
"width": 1920,
"vcodec": "av01.0.08M.08",
"acodec": "none",
"tbr": 4000,
"filesize": 1,
},
{
"format_id": "h264-720",
"url": "https://example.test/h264",
"ext": "mp4",
"height": 720,
"width": 1280,
"vcodec": "avc1.64001f",
"acodec": "none",
"tbr": 3000,
"filesize": 1,
},
{
"format_id": "aac",
"url": "https://example.test/aac",
"ext": "m4a",
"vcodec": "none",
"acodec": "mp4a.40.2",
"tbr": 128,
"filesize": 1,
},
]
with YoutubeDL(
{
"quiet": True,
"format_sort": ["res:1080", "vcodec:h264", "acodec:aac"],
}
) as ydl:
info = {"formats": formats}
ydl.sort_formats(info)
selector = ydl.build_format_selector(mp4_export_format_selector(1080))
selected = list(ydl._select_formats(info["formats"], selector))
self.assertEqual(selected[0]["height"], 1080)
self.assertEqual(selected[0]["requested_formats"][0]["format_id"], "av1-1080")
def test_mp4_sorting_prefers_native_h264_aac_at_equal_quality(self):
formats = [
{
"format_id": "vp9-1080",
"url": "https://example.test/vp9",
"ext": "webm",
"height": 1080,
"width": 1920,
"vcodec": "vp9",
"acodec": "none",
"tbr": 4200,
"filesize": 1,
},
{
"format_id": "h264-1080",
"url": "https://example.test/h264",
"ext": "mp4",
"height": 1080,
"width": 1920,
"vcodec": "avc1.640028",
"acodec": "none",
"tbr": 4000,
"filesize": 1,
},
{
"format_id": "opus",
"url": "https://example.test/opus",
"ext": "webm",
"vcodec": "none",
"acodec": "opus",
"tbr": 160,
"filesize": 1,
},
{
"format_id": "aac",
"url": "https://example.test/aac",
"ext": "m4a",
"vcodec": "none",
"acodec": "mp4a.40.2",
"tbr": 128,
"filesize": 1,
},
]
with YoutubeDL(
{
"quiet": True,
"format_sort": ["res:1080", "vcodec:h264", "acodec:aac"],
}
) as ydl:
info = {"formats": formats}
ydl.sort_formats(info)
selector = ydl.build_format_selector(mp4_export_format_selector(1080))
selected = list(ydl._select_formats(info["formats"], selector))
self.assertEqual(
[item["format_id"] for item in selected[0]["requested_formats"]],
["h264-1080", "aac"],
)
def test_m4a_sorting_keeps_audio_quality_ahead_of_native_aac_preference(self):
formats = [
{
"format_id": "aac-low",
"url": "https://example.test/aac-low",
"ext": "m4a",
"vcodec": "none",
"acodec": "mp4a.40.2",
"abr": 48,
"tbr": 48,
"filesize": 1,
},
{
"format_id": "opus-high",
"url": "https://example.test/opus-high",
"ext": "webm",
"vcodec": "none",
"acodec": "opus",
"abr": 160,
"tbr": 160,
"filesize": 1,
},
]
with YoutubeDL(
{"quiet": True, "format_sort": ["abr", "acodec:aac"]}
) as ydl:
info = {"formats": formats}
ydl.sort_formats(info)
selected = list(
ydl._select_formats(
info["formats"],
ydl.build_format_selector("ba/b"),
)
)
self.assertEqual(selected[0]["format_id"], "opus-high")
if __name__ == "__main__":
unittest.main()
+504
View File
@@ -0,0 +1,504 @@
import json
import queue
import threading
import unittest
from types import SimpleNamespace
import paho.mqtt.client as mqtt
from controller.config import ControllerConfig, MqttConfig
from controller.mqtt_adapter import MqttAdapter
class MutableClock:
def __init__(self, value=0.0):
self.value = value
def __call__(self):
return self.value
class FakeMessageInfo:
def __init__(self, mid, published):
self.rc = mqtt.MQTT_ERR_SUCCESS
self.mid = mid
self.published = published
self.waited = []
def wait_for_publish(self, timeout=None):
self.waited.append(timeout)
def is_published(self):
return self.published
class FakeClient:
def __init__(self):
self.calls = []
self.published = []
self.subscriptions = []
self.acks = []
self.suppress_exceptions = False
self.connect_error = None
self.connect_result = None
self.loop_result = mqtt.MQTT_ERR_SUCCESS
self.publish_acknowledged = None
self.publish_hook = None
def username_pw_set(self, username, password):
self.calls.append(("username_pw_set", username, password))
def manual_ack_set(self, enabled):
self.calls.append(("manual_ack_set", enabled))
def max_queued_messages_set(self, count):
self.calls.append(("max_queued_messages_set", count))
def reconnect_delay_set(self, **kwargs):
self.calls.append(("reconnect_delay_set", kwargs))
def connect_async(self, *args, **kwargs):
self.calls.append(("connect_async", args, kwargs))
if self.connect_error is not None:
raise self.connect_error
return self.connect_result
def connect(self, *args, **kwargs):
self.calls.append(("connect", args, kwargs))
raise AssertionError("synchronous connect must never be used")
def loop_start(self):
self.calls.append(("loop_start",))
return self.loop_result
def loop_stop(self):
self.calls.append(("loop_stop",))
def disconnect(self):
self.calls.append(("disconnect",))
def subscribe(self, topic, qos=0):
self.subscriptions.append((topic, qos))
return mqtt.MQTT_ERR_SUCCESS, len(self.subscriptions)
def publish(self, topic, payload, qos=0, retain=False):
self.published.append((topic, payload, qos, retain))
if self.publish_hook is not None:
self.publish_hook(topic, payload, qos, retain)
if self.publish_acknowledged is not None:
return FakeMessageInfo(
len(self.published), self.publish_acknowledged
)
return SimpleNamespace(rc=mqtt.MQTT_ERR_SUCCESS, mid=len(self.published))
def ack(self, message_id, qos):
self.acks.append((message_id, qos))
return mqtt.MQTT_ERR_SUCCESS
class MqttAdapterTests(unittest.TestCase):
def setUp(self):
self.client = FakeClient()
self.events = queue.Queue()
self.config = ControllerConfig(
mqtt=MqttConfig(
host="broker",
port=1884,
username="user",
password="pass",
client_id="controller-test",
reconnect_min_seconds=1,
reconnect_max_seconds=2,
command_queue_size=8,
)
)
self.adapter = MqttAdapter(
self.config,
client=self.client,
events=self.events,
)
def tearDown(self):
self.adapter.stop()
def test_start_is_asynchronous_and_configures_manual_ack(self):
self.adapter.start()
names = [call[0] for call in self.client.calls]
self.assertIn("manual_ack_set", names)
self.assertIn("connect_async", names)
self.assertIn("loop_start", names)
self.assertNotIn("connect", names)
connect = next(call for call in self.client.calls if call[0] == "connect_async")
self.assertEqual(connect[1], ("broker", 1884))
self.assertEqual(connect[2], {"keepalive": 30})
def test_paho_two_connect_async_none_starts_network_loop(self):
self.client.connect_result = None
self.adapter.start()
self.assertTrue(self.adapter.started)
self.assertIn("loop_start", [call[0] for call in self.client.calls])
self.assertIsNone(self.adapter.last_error)
def test_initial_setup_oserror_does_not_escape_startup(self):
self.client.connect_error = OSError("network unreachable")
self.adapter.start()
self.assertTrue(self.adapter.started)
self.assertIn("network unreachable", self.adapter.last_error)
self.assertNotIn("loop_start", [call[0] for call in self.client.calls])
def test_loop_start_failure_does_not_escape_startup(self):
self.client.loop_result = mqtt.MQTT_ERR_UNKNOWN
self.adapter.start()
self.assertTrue(self.adapter.started)
self.assertIn("failed to start MQTT network loop", self.adapter.last_error)
def test_stop_during_blocked_connect_never_starts_network_loop_afterward(self):
entered = threading.Event()
release = threading.Event()
original_connect = self.client.connect_async
def blocked_connect(*args, **kwargs):
entered.set()
release.wait(timeout=2)
return original_connect(*args, **kwargs)
self.client.connect_async = blocked_connect
starter = threading.Thread(target=self.adapter.start)
starter.start()
self.assertTrue(entered.wait(timeout=1))
self.adapter.stop()
release.set()
starter.join(timeout=2)
self.assertFalse(starter.is_alive())
self.assertNotIn("loop_start", [call[0] for call in self.client.calls])
self.assertFalse(self.adapter.started)
def test_stop_wins_against_connect_callback_blocked_during_subscribe(self):
self.adapter.start()
entered = threading.Event()
release = threading.Event()
original_subscribe = self.client.subscribe
def blocked_subscribe(topic, qos=0):
entered.set()
release.wait(timeout=2)
return original_subscribe(topic, qos=qos)
self.client.subscribe = blocked_subscribe
callback = threading.Thread(
target=self.adapter._on_connect,
args=(self.client, None, None, 0),
)
callback.start()
self.assertTrue(entered.wait(timeout=1))
stopper = threading.Thread(target=self.adapter.stop)
stopper.start()
release.set()
callback.join(timeout=2)
stopper.join(timeout=2)
self.assertFalse(callback.is_alive())
self.assertFalse(stopper.is_alive())
self.assertFalse(self.adapter.started)
self.assertFalse(self.adapter.connected)
self.assertLess(len(self.client.subscriptions), 6)
self.assertTrue(self.events.empty())
self.assertFalse(
self.adapter.publish_player_command({"action": "stop"}).accepted
)
def test_connect_subscribes_to_all_authoritative_inputs(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.assertTrue(self.adapter.connected)
self.assertEqual(
self.client.subscriptions,
[
("youtube/player/status", 1),
("youtube/player/telemetry", 0),
("youtube/player/state", 0),
("youtube/controller/cmd", 1),
("audio/player/mode/set", 1),
("audio/player/mode/state", 1),
],
)
event = self.events.get_nowait()
self.assertEqual(event.kind, "connected")
self.assertEqual(event.connection_generation, 1)
def test_player_commands_are_exact_qos_one_and_nonretained(self):
disconnected = self.adapter.publish_player_command({"action": "stop"})
self.assertFalse(disconnected.accepted)
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
receipt = self.adapter.publish_player_command(
{"action": "play", "slot": 3, "label": "Opening"}
)
self.assertTrue(receipt.accepted)
topic, payload, qos, retain = self.client.published[-1]
self.assertEqual(topic, "youtube/player/cmd")
self.assertEqual(json.loads(payload), {"action": "play", "slot": 3, "label": "Opening"})
self.assertEqual(qos, 1)
self.assertIs(retain, False)
def test_router_commands_are_qos_one_nonretained_plain_strings(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
receipt = self.adapter.publish_router_mode("video")
self.assertTrue(receipt.accepted)
self.assertEqual(
self.client.published[-1],
("audio/player/mode/set", "video", 1, False),
)
def test_router_self_echo_is_marked_while_publish_is_in_flight(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
def echo(topic, payload, qos, retain):
self.adapter._on_message(
self.client,
None,
SimpleNamespace(
topic=topic,
payload=payload.encode("utf-8"),
qos=qos,
retain=retain,
dup=False,
mid=31,
),
)
self.client.publish_hook = echo
self.assertTrue(self.adapter.publish_router_mode("video").accepted)
self.assertTrue(self.events.get_nowait().controller_echo)
def test_router_self_echo_delivered_after_puback_is_still_claimed_within_grace(
self,
):
# Regression: a broker may legally deliver our own subscription echo
# after the PUBACK that already acknowledged the same publish. If the
# echo marker were cleared the instant publish() observes that
# PUBACK, this late echo would be mistaken for live operator intent
# and cause automatic Router restoration ownership to be relinquished.
clock = MutableClock(1000.0)
client = FakeClient()
events = queue.Queue()
adapter = MqttAdapter(self.config, client=client, events=events, monotonic=clock)
try:
adapter.start()
adapter._on_connect(client, None, None, 0)
events.get_nowait()
self.assertTrue(adapter.publish_router_mode("video").accepted)
clock.value += self.config.mqtt.router_echo_grace_seconds - 0.5
adapter._on_message(
client,
None,
SimpleNamespace(
topic="audio/player/mode/set",
payload=b"video",
qos=1,
retain=False,
dup=False,
mid=32,
),
)
self.assertTrue(events.get_nowait().controller_echo)
finally:
adapter.stop()
def test_router_self_echo_marker_expires_after_grace_window(self):
# A genuine external/operator command that happens to repeat the same
# value must still be recognized as operator intent once the bounded
# grace window has elapsed; the marker must not linger forever.
clock = MutableClock(1000.0)
client = FakeClient()
events = queue.Queue()
adapter = MqttAdapter(self.config, client=client, events=events, monotonic=clock)
try:
adapter.start()
adapter._on_connect(client, None, None, 0)
events.get_nowait()
self.assertTrue(adapter.publish_router_mode("video").accepted)
clock.value += self.config.mqtt.router_echo_grace_seconds + 0.5
adapter._on_message(
client,
None,
SimpleNamespace(
topic="audio/player/mode/set",
payload=b"video",
qos=1,
retain=False,
dup=False,
mid=32,
),
)
self.assertFalse(events.get_nowait().controller_echo)
finally:
adapter.stop()
def test_retained_router_set_never_consumes_controller_echo_marker(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
def retained_then_live_echo(topic, payload, qos, retain):
for message_retain, mid in ((True, 41), (False, 42)):
self.adapter._on_message(
self.client,
None,
SimpleNamespace(
topic=topic,
payload=payload.encode("utf-8"),
qos=qos,
retain=message_retain,
dup=False,
mid=mid,
),
)
self.client.publish_hook = retained_then_live_echo
self.assertTrue(self.adapter.publish_router_mode("video").accepted)
retained_event = self.events.get_nowait()
live_event = self.events.get_nowait()
self.assertTrue(retained_event.retain)
self.assertFalse(retained_event.controller_echo)
self.assertFalse(live_event.retain)
self.assertTrue(live_event.controller_echo)
def test_qos_message_is_acked_only_after_consumer_calls_acknowledge(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
message = SimpleNamespace(
topic="youtube/controller/cmd",
payload=b'{"action":"stop"}',
qos=1,
retain=False,
mid=42,
)
self.adapter._on_message(self.client, None, message)
event = self.events.get_nowait()
self.assertEqual(self.client.acks, [])
self.assertEqual(event.message_id, 42)
self.assertTrue(self.adapter.acknowledge(event))
self.assertEqual(self.client.acks, [(42, 1)])
def test_stale_mid_from_previous_connection_is_not_acked(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
message = SimpleNamespace(
topic="youtube/controller/cmd",
payload=b'{"action":"stop"}',
qos=1,
retain=False,
mid=7,
)
self.adapter._on_message(self.client, None, message)
old_event = self.events.get_nowait()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
self.assertFalse(self.adapter.acknowledge(old_event))
self.assertEqual(self.client.acks, [])
def test_retained_flag_is_preserved_for_coordinator_rejection(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
message = SimpleNamespace(
topic="youtube/controller/cmd",
payload=b'{"action":"delete_slot","slot":1}',
qos=1,
retain=True,
mid=8,
)
self.adapter._on_message(self.client, None, message)
self.assertIs(self.events.get_nowait().retain, True)
def test_duplicate_flag_is_preserved_for_coordinator_deduplication(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
message = SimpleNamespace(
topic="youtube/controller/cmd",
payload=b'{"action":"toggle"}',
qos=1,
retain=False,
dup=True,
mid=9,
)
self.adapter._on_message(self.client, None, message)
self.assertIs(self.events.get_nowait().duplicate, True)
def test_missing_puback_is_uncertain_and_invalidates_connection(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
self.client.publish_acknowledged = False
receipt = self.adapter.publish_player_command({"action": "stop"})
self.assertFalse(receipt.accepted)
self.assertTrue(receipt.uncertain)
self.assertIn("no broker PUBACK", receipt.reason)
self.assertFalse(self.adapter.connected)
def test_optional_qos_event_puback_failure_does_not_invalidate_transport(self):
self.adapter.start()
self.adapter._on_connect(self.client, None, None, 0)
self.events.get_nowait()
self.client.publish_acknowledged = False
receipt = self.adapter.publish_json(
"youtube/controller/v2/events",
{"version": 2, "type": "warning"},
qos=1,
retain=False,
)
self.assertFalse(receipt.accepted)
self.assertTrue(receipt.uncertain)
self.assertTrue(self.adapter.connected)
def test_discarded_transport_releases_callback_blocked_on_full_queue(self):
events = queue.Queue(maxsize=1)
client = FakeClient()
adapter = MqttAdapter(self.config, client=client, events=events)
try:
adapter.start()
adapter._on_connect(client, None, None, 0)
self.assertTrue(events.full())
callback = threading.Thread(
target=adapter._on_message,
args=(
client,
None,
SimpleNamespace(
topic="youtube/controller/cmd",
payload=b'{"action":"stop"}',
qos=1,
retain=False,
dup=False,
mid=99,
),
),
)
callback.start()
callback.join(timeout=0.05)
self.assertTrue(callback.is_alive())
adapter._discard_unacknowledged_transport("test transport discard")
callback.join(timeout=1)
self.assertFalse(callback.is_alive())
self.assertEqual(events.qsize(), 1)
self.assertEqual(events.get_nowait().kind, "connected")
finally:
adapter.stop()
if __name__ == "__main__":
unittest.main()
+515
View File
@@ -0,0 +1,515 @@
import tempfile
import unittest
from pathlib import Path
from controller.config import ControllerConfig, StorageConfig
from controller.playback import OperationInProgressError, PlaybackReducer
from controller.store import ControllerStore
class Clock:
def __init__(self, value):
self.value = value
def __call__(self):
return self.value
class PlaybackReducerTests(unittest.TestCase):
def setUp(self):
self.tempdir = tempfile.TemporaryDirectory()
base = Path(self.tempdir.name)
self.wall = Clock(1_700_000_000.0)
self.monotonic = Clock(100.0)
self.config = ControllerConfig(
storage=StorageConfig(
media_base=base / "media",
data_base=base / "data",
retention_seconds=100,
uncertain_review_after_seconds=30,
)
)
self.store = ControllerStore(self.config.storage, now=self.wall).open()
self.reducer = PlaybackReducer(
self.config,
self.store,
monotonic=self.monotonic,
wall_clock=self.wall,
operation_timeout_seconds=5,
)
self._ready(1, "one")
self._ready(2, "two")
def tearDown(self):
self.store.close()
self.tempdir.cleanup()
def _ready(self, slot, suffix):
reservation = self.store.reserve_download(
f"https://example.test/{suffix}", None, None, job_id=f"job-{suffix}"
)
self.assertEqual(reservation.slot, slot)
self.store.mark_download_running(
reservation.job_id, reservation.slot, reservation.generation
)
self.store.complete_download(
reservation.job_id,
reservation.slot,
reservation.generation,
title=suffix.title(),
items=((suffix.title(), f"{suffix}.mp4"),),
)
def test_play_intent_is_not_optimistically_reported_playing(self):
operation = self.reducer.begin_operation(
"play",
slot=1,
expected_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
)
self.assertEqual(operation.state, "awaiting_player")
self.assertEqual(self.store.player_observation().phase, "unknown")
self.assertEqual(self.reducer.projected_slot_state(1), "ready")
with self.assertRaises(OperationInProgressError):
self.reducer.begin_operation("stop")
def test_matching_player_playing_confirms_play_and_protects_slot(self):
self.reducer.begin_operation(
"play",
slot=1,
expected_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
)
effects = self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
}
)
self.assertIn("operation_confirmed", [effect.kind for effect in effects])
self.assertIsNone(self.reducer.pending)
self.assertEqual(self.store.player_observation().phase, "playing")
self.assertEqual(self.reducer.projected_slot_state(1), "playing")
guards = self.store.list_live_guards()
self.assertEqual([(guard.slot, guard.kind) for guard in guards], [(1, "player_active")])
def test_reconnect_online_idle_with_path_is_not_terminal(self):
self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
}
)
effects = self.reducer.handle_status(
{
"state": "idle",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
"message": "player online",
}
)
self.assertEqual(effects[0].kind, "player_reconnected_active")
self.assertEqual(self.store.player_observation().phase, "playing")
self.assertEqual(self.store.list_live_guards()[0].slot, 1)
def test_reconnect_online_idle_without_path_is_fresh_idle_evidence(self):
self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
}
)
self.reducer.handle_status({"state": "idle", "message": "player online"})
self.assertEqual(self.store.player_observation().phase, "idle")
self.assertEqual(self.store.list_live_guards(), [])
def test_offline_without_slot_preserves_active_protection_as_unknown(self):
self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
}
)
effects = self.reducer.handle_status({"state": "offline"})
observation = self.store.player_observation()
self.assertEqual(effects[0].kind, "player_offline")
self.assertEqual(observation.active_slot, 1)
self.assertEqual(observation.phase, "unknown")
self.assertEqual(self.reducer.projected_slot_state(1), "unknown")
self.assertEqual(self.store.list_live_guards()[0].kind, "restart_uncertain")
def test_busy_identifies_existing_playback_not_requested_slot(self):
self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": "/mnt/Media/YTPlayer/slot1/playlist.m3u",
}
)
self.reducer.begin_operation(
"play",
slot=2,
expected_path="/mnt/Media/YTPlayer/slot2/playlist.m3u",
)
effects = self.reducer.handle_status(
{
"state": "busy",
"message": "mpv already playing; stop first",
}
)
self.assertIn("operation_failed", [effect.kind for effect in effects])
self.assertEqual(self.store.player_observation().active_slot, 1)
self.assertEqual(self.reducer.projected_slot_state(1), "playing")
self.assertEqual(self.reducer.projected_slot_state(2), "ready")
def test_real_pathless_busy_without_prior_identity_becomes_unknown(self):
self.reducer.begin_operation(
"play",
slot=2,
expected_path="/mnt/Media/YTPlayer/slot2/playlist.m3u",
)
effects = self.reducer.handle_status(
{
"state": "busy",
"message": "mpv already playing; stop first",
}
)
self.assertIn("operation_failed", [effect.kind for effect in effects])
observation = self.store.player_observation()
self.assertEqual(observation.phase, "unknown")
self.assertIsNone(observation.active_slot)
self.assertIsNone(observation.active_path)
def test_pause_status_does_not_override_actual_telemetry(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": path, "paused": False}
)
self.reducer.begin_operation("pause", slot=1, expected_path=path)
effects = self.reducer.handle_status(
{"state": "paused", "slot": 1, "path": path}
)
self.assertIn("control_issued", [effect.kind for effect in effects])
self.assertIsNotNone(self.reducer.pending)
self.assertEqual(self.reducer.projected_slot_state(1), "playing")
effects = self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": path, "paused": True}
)
self.assertIn("operation_confirmed", [effect.kind for effect in effects])
self.assertEqual(self.reducer.projected_slot_state(1), "paused")
def test_terminal_idle_and_cleared_telemetry_release_protection(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": path, "paused": False}
)
self.reducer.begin_operation("stop", slot=1, expected_path=path)
effects = self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": None, "paused": False}
)
self.assertIn("operation_confirmed", [effect.kind for effect in effects])
self.assertEqual(self.store.player_observation().phase, "idle")
self.assertEqual(self.store.list_live_guards(), [])
self.reducer.handle_status({"state": "idle", "slot": 1, "reason": "stop"})
self.assertEqual(self.reducer.projected_slot_state(1), "ready")
def test_error_does_not_release_owned_playback(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_status({"state": "playing", "slot": 1, "path": path})
self.reducer.handle_status(
{"state": "error", "slot": 1, "path": path, "message": "cannot stop"}
)
self.assertEqual(self.store.player_observation().phase, "error")
guard = self.store.list_live_guards()[0]
self.assertEqual(guard.slot, 1)
self.assertEqual(guard.kind, "restart_uncertain")
self.assertEqual(self.reducer.projected_slot_state(1), "unknown")
def test_pathless_error_converts_previous_active_slot_to_reviewable_unknown(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_status({"state": "playing", "slot": 1, "path": path})
self.reducer.handle_status(
{"state": "error", "message": "nothing playing to pause"}
)
observation = self.store.player_observation()
guard = self.store.list_live_guards()[0]
self.assertEqual(observation.active_slot, 1)
self.assertEqual(observation.reconciliation_id, guard.guard_id)
self.assertEqual(observation.reconciliation_state, "awaiting_evidence")
self.assertEqual(guard.kind, "restart_uncertain")
def test_terminal_startup_error_clears_identity_even_if_clear_telemetry_was_lost(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.begin_operation("play", slot=1, expected_path=path)
effects = self.reducer.handle_status(
{
"state": "error",
"slot": 1,
"path": path,
"reason": "startup_error",
"message": "failed to start mpv",
}
)
self.assertIn("player_idle", [effect.kind for effect in effects])
self.assertIn("operation_failed", [effect.kind for effect in effects])
observation = self.store.player_observation()
self.assertEqual(observation.phase, "idle")
self.assertIsNone(observation.active_slot)
self.assertIsNone(observation.active_path)
self.assertEqual(self.store.list_live_guards(), [])
def test_terminal_worker_error_after_cleared_telemetry_remains_idle(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.begin_operation("play", slot=1, expected_path=path)
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": None, "paused": False}
)
effects = self.reducer.handle_status(
{
"state": "error",
"slot": 1,
"path": path,
"reason": "worker_start_error",
"message": "failed to start playback worker",
}
)
self.assertIn("player_idle", [effect.kind for effect in effects])
observation = self.store.player_observation()
self.assertEqual(observation.phase, "idle")
self.assertIsNone(observation.active_path)
self.assertEqual(self.store.list_live_guards(), [])
def test_player_state_takes_precedence_over_deferred_storage_deletion(self):
reservation = self.store.reserve_download(
"https://example.test/late-player",
None,
None,
job_id="late-player",
)
self.store.fail_download(
"late-player",
1,
reservation.generation,
"failed",
)
self.store.begin_delete(1, reason="expired")
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_status(
{"state": "playing", "slot": 1, "path": path}
)
self.assertEqual(self.store.get_slot(1).media_state, "deleting")
self.assertEqual(self.reducer.projected_slot_state(1), "playing")
def test_play_timeout_becomes_unknown_then_operator_review_without_unprotecting(self):
self.reducer.begin_operation(
"play",
slot=1,
expected_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
)
self.monotonic.value += 6
effect = self.reducer.check_timeout()
self.assertEqual(effect.kind, "operation_timed_out")
self.assertEqual(self.reducer.projected_slot_state(1), "unknown")
self.wall.value += 31
review = self.reducer.mark_due_reconciliation()
self.assertEqual(review[0].kind, "operator_review")
self.assertEqual(self.reducer.projected_slot_state(1), "operator_review")
self.assertEqual(self.store.list_live_guards()[0].state, "review_required")
def test_pause_timeout_invalidates_stale_playing_projection(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": path, "paused": False}
)
self.reducer.begin_operation("pause", slot=1, expected_path=path)
self.monotonic.value += 6
effect = self.reducer.check_timeout()
self.assertEqual(effect.kind, "operation_timed_out")
observation = self.store.player_observation()
self.assertEqual(observation.phase, "unknown")
self.assertIsNone(observation.paused)
self.assertEqual(self.reducer.projected_slot_state(1), "unknown")
self.assertEqual(self.store.list_live_guards()[0].kind, "player_active")
def test_resume_timeout_invalidates_stale_paused_projection(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": 1, "path": path, "paused": True}
)
self.reducer.begin_operation("resume", slot=1, expected_path=path)
self.monotonic.value += 6
effect = self.reducer.check_timeout()
self.assertEqual(effect.kind, "operation_timed_out")
observation = self.store.player_observation()
self.assertEqual(observation.phase, "unknown")
self.assertIsNone(observation.paused)
self.assertEqual(self.reducer.projected_slot_state(1), "unknown")
def test_malformed_and_unknown_statuses_are_isolated(self):
for payload in (None, [], {}, {"state": "mystery"}):
effects = self.reducer.handle_status(payload)
self.assertEqual(effects[0].kind, "warning")
def test_contradictory_slot_and_path_cannot_confirm_play(self):
expected = str(self.config.storage.media_base / "slot1" / "playlist.m3u")
self.reducer.begin_operation("play", slot=1, expected_path=expected)
effects = self.reducer.handle_status(
{
"state": "playing",
"slot": 1,
"path": str(
self.config.storage.media_base / "slot2" / "playlist.m3u"
),
}
)
self.assertEqual(effects[0].kind, "warning")
self.assertIsNotNone(self.reducer.pending)
self.assertEqual(self.store.player_observation().phase, "unknown")
def test_mismatched_active_evidence_fails_intent_and_is_not_overwritten(self):
self.reducer.begin_operation(
"play",
slot=1,
expected_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
)
effects = self.reducer.handle_status(
{
"state": "playing",
"slot": 2,
"path": "/mnt/Media/YTPlayer/slot2/playlist.m3u",
}
)
self.assertIn("operation_failed", [effect.kind for effect in effects])
self.assertIsNone(self.reducer.pending)
self.monotonic.value += 100
self.assertIsNone(self.reducer.check_timeout())
observation = self.store.player_observation()
self.assertEqual(observation.active_slot, 2)
self.assertEqual(observation.phase, "playing")
self.assertEqual(self.store.list_live_guards()[0].slot, 2)
def test_bare_idle_does_not_clear_live_playback(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_status({"state": "playing", "slot": 1, "path": path})
effects = self.reducer.handle_status({"state": "idle", "slot": 1})
self.assertEqual(effects[0].kind, "warning")
self.assertEqual(self.store.player_observation().phase, "playing")
self.assertEqual(self.store.list_live_guards()[0].slot, 1)
def test_path_only_evidence_derives_and_protects_managed_slot(self):
path = str(self.config.storage.media_base / "slot2" / "playlist.m3u")
self.reducer.handle_telemetry(
{"state": "telemetry", "slot": None, "path": path, "paused": False}
)
observation = self.store.player_observation()
self.assertEqual(observation.active_slot, 2)
self.assertEqual(self.store.list_live_guards()[0].slot, 2)
def test_duplicate_offline_evidence_does_not_postpone_review(self):
path = "/mnt/Media/YTPlayer/slot1/playlist.m3u"
self.reducer.handle_status({"state": "playing", "slot": 1, "path": path})
self.reducer.handle_status({"state": "offline"})
first_deadline = self.store.list_live_guards()[0].review_after
self.wall.value += 20
self.reducer.handle_status({"state": "offline"})
self.assertEqual(
self.store.list_live_guards()[0].review_after,
first_deadline,
)
def test_repeated_active_telemetry_reuses_guard_without_history_churn(self):
path = str(self.config.storage.media_base / "slot1" / "playlist.m3u")
payload = {
"state": "telemetry",
"slot": 1,
"path": path,
"paused": False,
}
self.reducer.handle_telemetry(payload)
first_guard = self.store.list_live_guards()[0]
for _ in range(20):
self.reducer.handle_telemetry(payload)
self.assertEqual(self.store.list_live_guards()[0].guard_id, first_guard.guard_id)
count = self.store.connection.execute(
"SELECT COUNT(*) FROM slot_guards"
).fetchone()[0]
self.assertEqual(count, 1)
def test_external_play_timeout_transitions_to_review_without_assuming_stop(self):
path = "/srv/operator-playlists/special-event.m3u"
self.reducer.begin_operation("play", expected_path=path)
self.monotonic.value += 6
timeout = self.reducer.check_timeout()
self.assertEqual(timeout.kind, "operation_timed_out")
uncertain = self.store.player_observation()
self.assertEqual(uncertain.active_path, path)
self.assertEqual(uncertain.reconciliation_state, "awaiting_evidence")
self.assertIsNotNone(uncertain.reconciliation_id)
self.wall.value += 31
effects = self.reducer.mark_due_reconciliation()
self.assertEqual(len(effects), 1)
self.assertEqual(effects[0].kind, "operator_review")
self.assertEqual(effects[0].operation_id, uncertain.reconciliation_id)
review = self.store.player_observation()
self.assertEqual(review.reconciliation_state, "operator_review")
self.assertEqual(review.active_path, path)
def test_external_pause_does_not_bleed_to_a_different_path(self):
first_path = "/srv/operator-playlists/first.m3u"
second_path = "/srv/operator-playlists/second.m3u"
self.reducer.handle_telemetry(
{
"state": "telemetry",
"path": first_path,
"paused": True,
}
)
self.assertIs(self.store.player_observation().paused, True)
self.reducer.handle_status(
{"state": "playing", "path": second_path}
)
observation = self.store.player_observation()
self.assertEqual(observation.active_path, second_path)
self.assertIsNone(observation.paused)
self.assertEqual(observation.phase, "playing")
def test_stop_timeout_preserves_existing_operator_review_and_token(self):
path = str(self.config.storage.media_base / "slot1" / "playlist.m3u")
self.reducer.handle_status(
{"state": "playing", "slot": 1, "path": path}
)
self.reducer.handle_status({"state": "offline"})
self.wall.value += 31
self.reducer.mark_due_reconciliation()
before = self.store.player_observation()
guard_before = self.store.list_live_guards()[0]
self.assertEqual(before.reconciliation_state, "operator_review")
self.reducer.begin_operation("stop", slot=1, expected_path=path)
self.monotonic.value += 6
effect = self.reducer.check_timeout()
self.assertEqual(effect.kind, "operation_timed_out")
after = self.store.player_observation()
guard_after = self.store.list_live_guards()[0]
self.assertEqual(after.reconciliation_state, "operator_review")
self.assertEqual(after.reconciliation_id, before.reconciliation_id)
self.assertEqual(guard_after.guard_id, guard_before.guard_id)
self.assertEqual(guard_after.state, "review_required")
if __name__ == "__main__":
unittest.main()
+316
View File
@@ -0,0 +1,316 @@
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from controller.config import ControllerConfig, RouterConfig, StorageConfig
from controller.mqtt_adapter import PublishReceipt
from controller.router import AudioRouterCoordinator
from controller.store import ControllerStore
class Clock:
def __init__(self, value=1000.0):
self.value = value
def __call__(self):
return self.value
class ModePublisher:
def __init__(self):
self.modes = []
self.accepted = True
self.reason = None
def __call__(self, mode):
self.modes.append(mode)
return PublishReceipt(self.accepted, message_id=len(self.modes), reason=self.reason)
class AudioRouterCoordinatorTests(unittest.TestCase):
def setUp(self):
self.tempdir = tempfile.TemporaryDirectory()
base = Path(self.tempdir.name)
self.clock = Clock()
self.base_config = ControllerConfig(
storage=StorageConfig(
media_base=base / "media",
data_base=base / "data",
),
router=RouterConfig(
required_mode="video",
confirmation_timeout_seconds=3,
when_state_unknown="attempt",
continue_on_confirmation_failure=True,
fallback_restore_mode=None,
),
)
self.store = ControllerStore(self.base_config.storage, now=self.clock).open()
self.publisher = ModePublisher()
def tearDown(self):
self.store.close()
self.tempdir.cleanup()
def coordinator(self, router_config=None):
config = self.base_config
if router_config is not None:
config = replace(config, router=router_config)
return AudioRouterCoordinator(
config,
self.store,
self.publisher,
clock=self.clock,
)
def test_known_mode_is_confirmed_before_play_and_restored_afterward(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-one")
self.assertEqual(decision.action, "waiting")
self.assertEqual(self.publisher.modes, ["video"])
self.assertEqual(router.observe_mode("video").action, "ready")
lease = self.store.get_audio_lease(decision.lease_id)
self.assertEqual(lease.state, "held")
self.assertEqual(lease.previous_mode, "service")
restore = router.release(decision.lease_id)
self.assertEqual(restore.action, "restore_waiting")
self.assertEqual(self.publisher.modes, ["video", "service"])
self.assertEqual(router.observe_mode("service").action, "restored")
self.assertEqual(self.store.get_audio_lease(decision.lease_id).state, "released")
def test_default_unknown_state_timeout_continues_with_warning_not_hang(self):
router = self.coordinator()
decision = router.acquire("play-timeout")
self.assertEqual(decision.action, "waiting")
self.assertEqual(self.publisher.modes, ["video"])
self.clock.value += 2.9
self.assertIsNone(router.check_deadline())
self.clock.value += 0.2
decision = router.check_deadline()
self.assertEqual(decision.action, "ready")
self.assertTrue(decision.continued)
self.assertIn("did not confirm", decision.warning)
lease = self.store.get_audio_lease(decision.lease_id)
self.assertEqual(lease.state, "unconfirmed")
self.assertEqual(router.release(lease.lease_id).action, "released")
self.assertEqual(self.publisher.modes, ["video"])
def test_timeout_can_reject_when_continue_policy_is_disabled(self):
router = self.coordinator(
replace(
self.base_config.router,
continue_on_confirmation_failure=False,
)
)
decision = router.acquire("play-reject")
self.clock.value += 4
decision = router.check_deadline()
self.assertEqual(decision.action, "rejected")
self.assertFalse(decision.continued)
self.assertEqual(self.store.get_audio_lease(decision.lease_id).state, "abandoned")
def test_unknown_state_continue_policy_does_not_wait_or_gain_late_ownership(self):
router = self.coordinator(
replace(self.base_config.router, when_state_unknown="continue")
)
decision = router.acquire("play-immediate")
self.assertEqual(decision.action, "ready")
self.assertTrue(decision.continued)
self.assertEqual(self.store.get_audio_lease(decision.lease_id).state, "unconfirmed")
self.assertEqual(router.observe_mode("video").action, "observed")
self.assertEqual(self.store.get_audio_lease(decision.lease_id).state, "unconfirmed")
def test_unknown_state_reject_policy_does_not_publish(self):
router = self.coordinator(
replace(self.base_config.router, when_state_unknown="reject")
)
decision = router.acquire("play-reject")
self.assertEqual(decision.action, "rejected")
self.assertEqual(self.publisher.modes, [])
def test_already_video_requires_no_change_and_no_restore(self):
router = self.coordinator()
router.observe_mode("video")
decision = router.acquire("play-video")
self.assertEqual(decision.action, "ready")
self.assertEqual(self.publisher.modes, [])
self.assertEqual(router.release(decision.lease_id).action, "released")
self.assertEqual(self.publisher.modes, [])
def test_confirmed_unknown_acquisition_can_use_configured_fallback(self):
router = self.coordinator(
replace(self.base_config.router, fallback_restore_mode="service")
)
decision = router.acquire("play-fallback")
self.assertEqual(router.observe_mode("video").action, "ready")
lease = self.store.get_audio_lease(decision.lease_id)
self.assertEqual(lease.restoration_source, "configured_fallback")
self.assertEqual(router.release(decision.lease_id).action, "restore_waiting")
self.assertEqual(self.publisher.modes, ["video", "service"])
def test_operator_change_is_sticky_even_if_mode_later_returns_to_video(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-operator")
router.observe_mode("video")
lost = router.observe_mode("announcements")
self.assertEqual(lost.action, "ownership_lost")
router.observe_mode("video")
self.assertEqual(router.release(decision.lease_id).action, "released")
self.assertEqual(self.publisher.modes, ["video"])
self.assertEqual(
self.store.get_audio_lease(decision.lease_id).state,
"ownership_lost",
)
def test_connection_loss_relinquishes_confirmed_lease(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-disconnect")
router.observe_mode("video")
lost = router.connection_lost()
self.assertEqual(lost.action, "ownership_lost")
self.assertEqual(router.release(decision.lease_id).action, "released")
self.assertEqual(self.publisher.modes, ["video"])
def test_publish_failure_uses_confirmation_failure_policy(self):
self.publisher.accepted = False
self.publisher.reason = "MQTT disconnected"
router = self.coordinator()
decision = router.acquire("play-publish-fail")
self.assertEqual(decision.action, "ready")
self.assertTrue(decision.continued)
self.assertIn("MQTT disconnected", decision.warning)
def test_restoration_timeout_is_bounded_and_not_retried(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-restore-timeout")
router.observe_mode("video")
router.release(decision.lease_id)
self.clock.value += 4
timeout = router.check_deadline()
self.assertEqual(timeout.action, "restore_failed")
self.assertEqual(self.publisher.modes, ["video", "service"])
self.assertIsNone(router.check_deadline())
def test_repeated_previous_state_during_acquisition_waits_for_deadline(self):
router = self.coordinator()
router.observe_mode("service")
waiting = router.acquire("play-repeated-state")
self.assertEqual(router.observe_mode("service").action, "observed")
self.assertEqual(
self.store.get_audio_lease(waiting.lease_id).state,
"acquiring",
)
self.clock.value += 4
decision = router.check_deadline()
self.assertEqual(decision.action, "ready")
self.assertTrue(decision.continued)
self.assertEqual(
self.store.get_audio_lease(waiting.lease_id).state,
"unconfirmed",
)
router.release(waiting.lease_id)
self.assertEqual(self.publisher.modes, ["video"])
def test_repeated_target_state_is_observation_not_operator_takeover(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-repeated-state")
self.assertEqual(router.observe_mode("video").action, "ready")
self.assertEqual(router.observe_mode("video").action, "observed")
self.assertEqual(
self.store.get_audio_lease(decision.lease_id).state,
"held",
)
self.assertEqual(router.release(decision.lease_id).action, "restore_waiting")
self.assertEqual(self.publisher.modes, ["video", "service"])
def test_controller_mode_set_echo_does_not_relinquish_ownership(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-self-echo")
self.assertEqual(
router.observe_mode_command("video", controller_echo=True).action,
"self_echo",
)
self.assertEqual(router.observe_mode("video").action, "ready")
self.assertEqual(router.release(decision.lease_id).action, "restore_waiting")
self.assertEqual(
router.observe_mode_command("service", controller_echo=True).action,
"self_echo",
)
self.assertEqual(router.observe_mode("service").action, "restored")
def test_manual_mode_set_during_acquisition_uses_continue_policy(self):
router = self.coordinator()
router.observe_mode("service")
waiting = router.acquire("play-manual-acquire")
decision = router.observe_mode_command("announcements")
self.assertEqual(decision.action, "ready")
self.assertTrue(decision.continued)
self.assertEqual(
self.store.get_audio_lease(waiting.lease_id).state,
"unconfirmed",
)
self.assertIsNone(self.store.get_audio_lease(waiting.lease_id).confirmation_deadline)
def test_manual_mode_set_during_acquisition_can_reject(self):
router = self.coordinator(
replace(
self.base_config.router,
continue_on_confirmation_failure=False,
)
)
router.observe_mode("service")
waiting = router.acquire("play-manual-reject")
decision = router.observe_mode_command("announcements")
self.assertEqual(decision.action, "rejected")
self.assertFalse(decision.continued)
self.assertEqual(
self.store.get_audio_lease(waiting.lease_id).state,
"abandoned",
)
def test_manual_mode_set_while_held_relinquishes_restoration(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-manual-held")
router.observe_mode("video")
lost = router.observe_mode_command("video")
self.assertEqual(lost.action, "ownership_lost")
self.assertEqual(router.release(decision.lease_id).action, "released")
self.assertEqual(self.publisher.modes, ["video"])
def test_manual_mode_set_while_restoring_relinquishes_restoration(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-manual-restore")
router.observe_mode("video")
self.assertEqual(router.release(decision.lease_id).action, "restore_waiting")
lost = router.observe_mode_command("announcements")
self.assertEqual(lost.action, "ownership_lost")
self.assertEqual(
self.store.get_audio_lease(decision.lease_id).state,
"ownership_lost",
)
self.assertEqual(self.publisher.modes, ["video", "service"])
def test_retained_mode_set_does_not_claim_live_operator_intent(self):
router = self.coordinator()
router.observe_mode("service")
decision = router.acquire("play-retained-set")
router.observe_mode("video")
ignored = router.observe_mode_command("announcements", retained=True)
self.assertEqual(ignored.action, "retained_ignored")
self.assertEqual(
self.store.get_audio_lease(decision.lease_id).state,
"held",
)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,206 @@
"""Real-broker smoke test for Audio Router self-echo correlation.
The fake MQTT client used by ``tests/test_mqtt_adapter.py`` delivers a
publish's own echo synchronously, inside the ``publish()`` call itself, so it
cannot exercise the actual ordering risk: a real broker may legally deliver a
Controller-originated Router mode-set message back through the Controller's
own subscription either before or after the PUBACK that already acknowledged
the same publish. This test runs the real ``MqttAdapter`` against a real
Mosquitto broker in Docker to prove that ordering no longer matters.
Skipped automatically when Docker (or a Mosquitto broker container) is not
available, so it never blocks the primary regression suite.
"""
from __future__ import annotations
import queue
import shutil
import socket
import subprocess
import tempfile
import time
import unittest
import uuid
from pathlib import Path
import paho.mqtt.client as mqtt
from controller.config import ControllerConfig, MqttConfig
from controller.mqtt_adapter import MqttAdapter
def _docker_available() -> bool:
return shutil.which("docker") is not None
def _free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
@unittest.skipUnless(
_docker_available(), "docker is not available for a real Mosquitto broker"
)
class RouterSelfEchoMosquittoIntegrationTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.port = _free_port()
cls.container_name = f"controller-echo-test-{uuid.uuid4().hex[:10]}"
cls.tempdir = tempfile.TemporaryDirectory()
conf_path = Path(cls.tempdir.name) / "mosquitto.conf"
conf_path.write_text("listener 1883\nallow_anonymous true\n", encoding="utf-8")
try:
subprocess.run(
[
"docker",
"run",
"-d",
"--rm",
"--name",
cls.container_name,
"-p",
f"127.0.0.1:{cls.port}:1883",
"-v",
f"{conf_path}:/mosquitto/config/mosquitto.conf:ro",
"eclipse-mosquitto",
],
check=True,
capture_output=True,
text=True,
timeout=30,
)
except (OSError, subprocess.CalledProcessError, subprocess.TimeoutExpired) as exc:
cls.tempdir.cleanup()
raise unittest.SkipTest(
f"could not start a real Mosquitto broker container: {exc}"
)
try:
cls._wait_for_broker(cls.port)
except AssertionError as exc:
subprocess.run(
["docker", "rm", "-f", cls.container_name],
capture_output=True,
timeout=15,
)
cls.tempdir.cleanup()
raise unittest.SkipTest(str(exc))
@classmethod
def tearDownClass(cls) -> None:
subprocess.run(
["docker", "rm", "-f", cls.container_name],
capture_output=True,
timeout=15,
)
cls.tempdir.cleanup()
@staticmethod
def _wait_for_broker(port: int, timeout: float = 15.0) -> None:
deadline = time.monotonic() + timeout
last_exc: OSError | None = None
while time.monotonic() < deadline:
try:
with socket.create_connection(("127.0.0.1", port), timeout=0.5):
return
except OSError as exc:
last_exc = exc
time.sleep(0.2)
raise AssertionError(f"Mosquitto broker never became reachable: {last_exc}")
def _config(self) -> ControllerConfig:
return ControllerConfig(
mqtt=MqttConfig(
host="127.0.0.1",
port=self.port,
client_id=f"controller-echo-{uuid.uuid4().hex[:10]}",
reconnect_min_seconds=1,
reconnect_max_seconds=1,
command_queue_size=32,
publish_ack_timeout_seconds=5.0,
router_echo_grace_seconds=2.0,
)
)
def test_controller_recognizes_its_own_router_echo_over_a_real_broker(self) -> None:
config = self._config()
adapter = MqttAdapter(config)
adapter.start()
try:
self._wait_until(
lambda: adapter.connected,
"Controller MQTT adapter never connected to the real broker",
)
receipt = adapter.publish_router_mode("video")
self.assertTrue(receipt.accepted, receipt.reason)
event = self._next_message_event(adapter, timeout=5.0)
self.assertEqual(event.topic, config.topics.router_mode_set)
self.assertEqual(event.payload, b"video")
self.assertTrue(
event.controller_echo,
"Controller did not recognize its own Router mode-set echo "
"delivered by a real broker; the PUBACK-vs-subscription-echo "
"race was not closed",
)
finally:
adapter.stop()
def test_genuine_external_router_command_is_not_treated_as_self_echo(self) -> None:
config = self._config()
adapter = MqttAdapter(config)
adapter.start()
operator = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2,
client_id=f"operator-{uuid.uuid4().hex[:10]}",
)
try:
self._wait_until(
lambda: adapter.connected,
"Controller MQTT adapter never connected to the real broker",
)
operator.connect(config.mqtt.host, config.mqtt.port, keepalive=30)
operator.loop_start()
info = operator.publish(
config.topics.router_mode_set, "service", qos=1, retain=False
)
info.wait_for_publish(timeout=5.0)
event = self._next_message_event(adapter, timeout=5.0)
self.assertEqual(event.payload, b"service")
self.assertFalse(
event.controller_echo,
"A genuine external Router command was mistaken for a "
"Controller echo; automatic restoration ownership would be "
"incorrectly preserved instead of relinquished",
)
finally:
operator.loop_stop()
operator.disconnect()
adapter.stop()
@staticmethod
def _wait_until(predicate, message: str, timeout: float = 10.0) -> None:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if predicate():
return
time.sleep(0.05)
raise AssertionError(message)
@staticmethod
def _next_message_event(adapter: MqttAdapter, *, timeout: float):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
event = adapter.events.get(timeout=0.2)
except queue.Empty:
continue
if event.kind == "message":
return event
raise AssertionError("no MQTT message event was observed within the timeout")
if __name__ == "__main__":
unittest.main()
+314
View File
@@ -0,0 +1,314 @@
import ast
import re
import unittest
from pathlib import Path
from typing import Any
import yaml
ROOT = Path(__file__).resolve().parents[1]
def _read(name: str) -> str:
return (ROOT / name).read_text(encoding="utf-8")
def _docker_logical_lines(text: str) -> list[str]:
logical: list[str] = []
pending = ""
for raw_line in text.splitlines():
stripped = raw_line.strip()
if not stripped or (not pending and stripped.startswith("#")):
continue
pending = f"{pending} {stripped}".strip()
if pending.endswith("\\"):
pending = pending[:-1].rstrip()
continue
logical.append(pending)
pending = ""
if pending:
logical.append(pending)
return logical
def _environment_map(value: Any) -> dict[str, str | None]:
if isinstance(value, dict):
return {str(key): None if item is None else str(item) for key, item in value.items()}
result: dict[str, str | None] = {}
for entry in value or []:
key, separator, item = str(entry).partition("=")
result[key] = item if separator else None
return result
def _volume_mounts(value: Any) -> list[tuple[str | None, str, bool]]:
mounts: list[tuple[str | None, str, bool]] = []
for entry in value or []:
if isinstance(entry, dict):
target = str(entry.get("target") or "")
if target:
mounts.append(
(
None if entry.get("source") is None else str(entry["source"]),
target,
bool(entry.get("read_only", False)),
)
)
continue
# Greedy source matching tolerates Compose substitutions containing
# `:-` or `:?` while keeping the absolute container target distinct.
match = re.match(r"^(.*):(/[^:]+)(?::([^:]+))?$", str(entry))
if match:
mode = match.group(3) or ""
mounts.append((match.group(1), match.group(2), "ro" in mode.split(",")))
return mounts
def _secret_targets(value: Any) -> set[str]:
targets: set[str] = set()
for entry in value or []:
if isinstance(entry, dict):
source = str(entry.get("source") or "")
target = str(entry.get("target") or source)
else:
source = str(entry)
target = source
if not target:
continue
targets.add(target if target.startswith("/") else f"/run/secrets/{target}")
return targets
class RequirementPinTests(unittest.TestCase):
def test_runtime_requirements_are_exactly_pinned(self):
path = ROOT / "requirements.txt"
self.assertTrue(path.is_file(), "requirements.txt is missing")
requirements: dict[str, str] = {}
pattern = re.compile(
r"^([A-Za-z0-9_.-]+)(?:\[[A-Za-z0-9_,.-]+\])?==([^;\s]+)(?:\s*;.*)?$"
)
for line_number, raw_line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
line = raw_line.split("#", 1)[0].strip()
if not line:
continue
match = pattern.fullmatch(line)
self.assertIsNotNone(
match,
f"requirements.txt:{line_number} is not an exact == pin: {line!r}",
)
assert match is not None
normalized = re.sub(r"[-_.]+", "-", match.group(1)).lower()
requirements[normalized] = match.group(2)
for project in ("flask", "gunicorn", "paho-mqtt", "pyyaml", "yt-dlp"):
self.assertIn(project, requirements, f"missing runtime dependency {project}")
self.assertNotIn("*", requirements[project])
class DockerfileTests(unittest.TestCase):
def setUp(self):
self.path = ROOT / "Dockerfile"
self.assertTrue(self.path.is_file(), "Dockerfile is missing")
self.text = self.path.read_text(encoding="utf-8")
self.lines = _docker_logical_lines(self.text)
def test_image_copies_the_v2_runtime(self):
copies = "\n".join(
line for line in self.lines if line.upper().startswith("COPY ")
)
for required in (
"controller",
"config.yaml",
"app.py",
"wsgi.py",
"gunicorn.conf.py",
):
self.assertIn(required, copies, f"Dockerfile does not copy {required}")
def test_image_runs_non_root_on_the_controller_port(self):
users = [line.split(None, 1)[1] for line in self.lines if line.upper().startswith("USER ")]
self.assertTrue(users, "Dockerfile must set a non-root USER")
self.assertNotIn(users[-1].strip().lower(), {"root", "0", "0:0"})
self.assertRegex(self.text, r"(?im)^\s*EXPOSE\s+7531(?:/tcp)?\s*$")
def test_image_has_an_http_healthcheck_and_runs_gunicorn(self):
self.assertRegex(self.text, r"(?im)^\s*HEALTHCHECK\b")
self.assertIn("/healthz", self.text)
launch = "\n".join(
line
for line in self.lines
if line.upper().startswith(("CMD ", "ENTRYPOINT "))
).lower()
self.assertIn("gunicorn", launch)
def test_image_has_no_unbounded_yt_dlp_upgrade_or_hidden_global_config(self):
self.assertNotRegex(
self.text,
r"(?i)(?:--upgrade|-U)\s+[\"']?(?:yt-dlp|yt_dlp)",
)
self.assertNotIn("/etc/yt-dlp.conf", self.text)
def test_image_verifies_a_supported_node_runtime_for_yt_dlp(self):
self.assertRegex(
self.text,
r"(?im)^FROM\s+node:22\.\d+\.\d+-bookworm-slim\s+AS\s+",
)
self.assertIn("NodeJsRuntime", self.text)
self.assertIn("info.supported", self.text)
class GunicornConfigTests(unittest.TestCase):
def test_gunicorn_uses_exactly_one_worker_without_preload(self):
path = ROOT / "gunicorn.conf.py"
self.assertTrue(path.is_file(), "gunicorn.conf.py is missing")
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
assignments: dict[str, Any] = {}
for node in tree.body:
if not isinstance(node, (ast.Assign, ast.AnnAssign)):
continue
targets = node.targets if isinstance(node, ast.Assign) else [node.target]
try:
value = ast.literal_eval(node.value)
except (TypeError, ValueError):
continue
for target in targets:
if isinstance(target, ast.Name):
assignments[target.id] = value
self.assertEqual(assignments.get("workers"), 1)
self.assertIs(assignments.get("preload_app"), False)
self.assertIs(assignments.get("control_socket_disable"), True)
self.assertEqual(assignments.get("worker_tmp_dir"), "/tmp")
class ComposeRuntimeTests(unittest.TestCase):
def setUp(self):
self.path = ROOT / "docker-compose.yml"
self.assertTrue(self.path.is_file(), "docker-compose.yml is missing")
self.text = self.path.read_text(encoding="utf-8")
document = yaml.safe_load(self.text)
self.assertIsInstance(document, dict)
self.document = document
self.service = document.get("services", {}).get("controller")
self.assertIsInstance(self.service, dict, "Compose controller service is missing")
def test_compose_requires_a_real_flask_secret_and_publishes_port_7531(self):
environment = _environment_map(self.service.get("environment"))
secret = environment.get("FLASK_SECRET_KEY")
self.assertIsNotNone(secret, "FLASK_SECRET_KEY environment setting is missing")
assert secret is not None
self.assertTrue(secret.startswith("${FLASK_SECRET_KEY"))
self.assertIn("?", secret, "FLASK_SECRET_KEY must use required Compose substitution")
self.assertNotIn("change-me", secret.lower())
self.assertNotIn("dev-secret", secret.lower())
published_targets: set[str] = set()
for entry in self.service.get("ports") or []:
if isinstance(entry, dict):
published_targets.add(str(entry.get("target")))
else:
match = re.search(r":(\d+)(?:/(?:tcp|udp))?$", str(entry))
if match:
published_targets.add(match.group(1))
self.assertIn("7531", published_targets)
def test_compose_passes_all_supported_mqtt_environment_overrides(self):
environment = _environment_map(self.service.get("environment"))
for name in (
"MQTT_HOST",
"MQTT_PORT",
"MQTT_USERNAME",
"MQTT_PASSWORD",
"MQTT_CLIENT_ID",
):
with self.subTest(name=name):
self.assertIn(name, environment)
def test_compose_uses_dedicated_data_media_and_cookie_mounts(self):
mounts = _volume_mounts(self.service.get("volumes"))
by_target = {target: (source, read_only) for source, target, read_only in mounts}
self.assertIn("/controller-data", by_target)
self.assertIn("/mnt/Media/YTPlayer", by_target)
self.assertNotEqual(by_target["/controller-data"][0], "/home/media/Scripts/Video-Player")
cookie_path = "/run/secrets/yt-dlp-cookies.txt"
mounted_cookie = cookie_path in by_target and by_target[cookie_path][1]
mounted_secret_directory = (
"/run/secrets" in by_target and by_target["/run/secrets"][1]
)
secret_cookie = cookie_path in _secret_targets(self.service.get("secrets"))
self.assertTrue(
mounted_cookie or mounted_secret_directory or secret_cookie,
f"cookies must be read-only at exactly {cookie_path}",
)
self.assertNotIn("/cookies_ro.txt", self.text)
self.assertNotRegex(
self.text,
r"/home/media/Scripts/Video-Player(?=[:\s\"'])",
)
def test_compose_enables_container_hardening_and_bounded_operations(self):
self.assertIs(self.service.get("read_only"), True)
self.assertIs(self.service.get("init"), True)
cap_drop = self.service.get("cap_drop") or []
if isinstance(cap_drop, str):
cap_drop = [cap_drop]
self.assertIn("ALL", {str(value).upper() for value in cap_drop})
security_options = self.service.get("security_opt") or []
if isinstance(security_options, str):
security_options = [security_options]
self.assertTrue(
any("no-new-privileges" in str(value).lower() for value in security_options)
)
tmpfs = self.service.get("tmpfs") or []
if isinstance(tmpfs, str):
tmpfs = [tmpfs]
self.assertTrue(any(str(value).split(":", 1)[0] == "/tmp" for value in tmpfs))
self.assertTrue(any("noexec" in str(value) for value in tmpfs))
healthcheck = self.service.get("healthcheck")
self.assertIsInstance(healthcheck, dict)
self.assertIn("/healthz", str(healthcheck.get("test")))
logging_config = self.service.get("logging")
self.assertIsInstance(logging_config, dict)
options = logging_config.get("options") or {}
self.assertTrue(options.get("max-size"), "logging max-size is missing")
self.assertTrue(options.get("max-file"), "logging max-file is missing")
grace = str(self.service.get("stop_grace_period") or "").rstrip("s")
self.assertGreaterEqual(float(grace), 30.0)
def test_compose_preserves_the_external_tabernet_network(self):
service_networks = self.service.get("networks") or []
if isinstance(service_networks, dict):
service_networks = service_networks.keys()
self.assertIn("tabernet", service_networks)
tabernet = self.document.get("networks", {}).get("tabernet")
self.assertIsInstance(tabernet, dict)
self.assertIs(tabernet.get("external"), True)
class DockerIgnoreTests(unittest.TestCase):
def test_dockerignore_excludes_secrets_and_python_caches(self):
path = ROOT / ".dockerignore"
self.assertTrue(path.is_file(), ".dockerignore is missing")
patterns = {
line.strip().lower()
for line in path.read_text(encoding="utf-8").splitlines()
if line.strip() and not line.lstrip().startswith("#")
}
self.assertTrue(any("__pycache__" in pattern for pattern in patterns))
self.assertTrue(
any("pyc" in pattern or "py[cod]" in pattern for pattern in patterns)
)
self.assertTrue(any(pattern in {".env", ".env*", "*.env"} for pattern in patterns))
self.assertTrue(any("secret" in pattern for pattern in patterns))
self.assertTrue(any("cookie" in pattern for pattern in patterns))
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
+728
View File
@@ -0,0 +1,728 @@
import concurrent.futures
import hashlib
import os
import sqlite3
import tempfile
import unittest
from pathlib import Path
import yaml
from controller.config import StorageConfig
from controller.models import MigrationError, SlotProtectedError
from controller.store import ControllerStore, LEGACY_MIGRATION_NAME
class MutableClock:
def __init__(self, value=1_700_000_000.0):
self.value = value
def __call__(self):
return self.value
class StoreTestCase(unittest.TestCase):
def setUp(self):
self.tempdir = tempfile.TemporaryDirectory()
self.base = Path(self.tempdir.name)
self.media = self.base / "media"
self.data = self.base / "data"
self.clock = MutableClock()
self.config = StorageConfig(
media_base=self.media,
data_base=self.data,
retention_seconds=100,
uncertain_review_after_seconds=30,
)
def tearDown(self):
self.tempdir.cleanup()
def open_store(self):
return ControllerStore(self.config, now=self.clock).open()
class FreshStoreTests(StoreTestCase):
def test_fresh_database_seeds_exact_stable_slots(self):
with self.open_store() as store:
slots = store.list_slots()
self.assertEqual([slot.slot for slot in slots], list(range(1, 11)))
self.assertTrue(all(slot.media_state == "empty" for slot in slots))
self.assertEqual(store.player_observation().phase, "unknown")
def test_schema_one_database_upgrades_reconciliation_and_exports_atomically(self):
with self.open_store() as store:
database_path = store.path
connection = sqlite3.connect(database_path)
try:
connection.execute(
"ALTER TABLE player_observation DROP COLUMN reconciliation_id"
)
connection.execute("DELETE FROM schema_migrations WHERE version = 2")
connection.execute("PRAGMA user_version = 1")
connection.commit()
finally:
connection.close()
with self.open_store() as upgraded:
self.assertEqual(
upgraded.connection.execute("PRAGMA user_version").fetchone()[0],
3,
)
columns = {
row["name"]
for row in upgraded.connection.execute(
"PRAGMA table_info(player_observation)"
).fetchall()
}
self.assertIn("reconciliation_id", columns)
migration = upgraded.connection.execute(
"SELECT name FROM schema_migrations WHERE version = 2"
).fetchone()
self.assertEqual(migration["name"], "external_playback_reconciliation")
export_migration = upgraded.connection.execute(
"SELECT name FROM schema_migrations WHERE version = 3"
).fetchone()
self.assertEqual(export_migration["name"], "utility_export_jobs")
tables = {
row["name"]
for row in upgraded.connection.execute(
"SELECT name FROM sqlite_master WHERE type = 'table'"
).fetchall()
}
self.assertIn("export_jobs", tables)
def test_schema_two_database_adds_only_utility_export_jobs(self):
with self.open_store() as store:
database_path = store.path
connection = sqlite3.connect(database_path)
try:
connection.execute("DROP TABLE export_jobs")
connection.execute("DELETE FROM schema_migrations WHERE version = 3")
connection.execute("PRAGMA user_version = 2")
connection.commit()
finally:
connection.close()
with self.open_store() as upgraded:
self.assertEqual(
upgraded.connection.execute("PRAGMA user_version").fetchone()[0],
3,
)
self.assertEqual(upgraded.list_export_jobs(), [])
def test_export_jobs_are_slot_free_bounded_and_durable(self):
with self.open_store() as store:
first = store.reserve_export_job(
"https://www.youtube.com/watch?v=one",
"mp4",
"Alice",
"Sunday",
max_active=1,
job_id="export-one",
)
self.assertEqual(first.status, "queued")
self.assertEqual(first.job_id, "export-one")
self.assertEqual(
store.reserve_export_job(
"https://www.youtube.com/watch?v=two",
"mp3",
None,
None,
max_active=1,
).status,
"queue_full",
)
self.assertTrue(store.mark_export_running("export-one"))
self.assertTrue(
store.complete_export_job(
"export-one",
("exports/video/export-one/Service.mp4",),
)
)
job = store.get_export_job("export-one")
self.assertEqual(job.mode, "mp4")
self.assertEqual(job.status, "completed")
self.assertEqual(job.submitter, "Alice")
self.assertEqual(
job.output_paths,
("exports/video/export-one/Service.mp4",),
)
self.assertTrue(all(slot.media_state == "empty" for slot in store.list_slots()))
def test_export_listing_keeps_live_jobs_visible_before_recent_history(self):
with self.open_store() as store:
store.reserve_export_job(
"https://www.youtube.com/watch?v=active",
"mp4",
None,
None,
max_active=5,
job_id="active-old",
)
self.clock.value += 1
for index in range(3):
job_id = f"completed-{index}"
store.reserve_export_job(
f"https://www.youtube.com/watch?v={index}",
"mp3",
None,
None,
max_active=5,
job_id=job_id,
)
store.mark_export_running(job_id)
store.complete_export_job(
job_id,
(f"exports/audio/{job_id}/item.mp3",),
)
self.clock.value += 1
listed = store.list_export_jobs(limit=2)
self.assertEqual(listed[0].job_id, "active-old")
self.assertEqual(listed[0].status, "queued")
self.assertEqual(listed[1].job_id, "completed-2")
def test_restart_marks_live_exports_interrupted_without_touching_slots(self):
store = self.open_store()
store.reserve_export_job(
"https://www.youtube.com/playlist?list=one",
"m4a",
None,
None,
max_active=2,
job_id="queued-export",
)
store.reserve_export_job(
"https://www.youtube.com/watch?v=two",
"mp3",
None,
None,
max_active=2,
job_id="running-export",
)
store.mark_export_running("running-export")
store.close()
with self.open_store() as reopened:
jobs = {job.job_id: job for job in reopened.list_export_jobs()}
self.assertEqual(jobs["queued-export"].status, "interrupted")
self.assertEqual(jobs["running-export"].status, "interrupted")
self.assertTrue(
all("Controller restart" in job.error for job in jobs.values())
)
self.assertTrue(
all(slot.media_state == "empty" for slot in reopened.list_slots())
)
def test_export_failures_do_not_corrupt_player_inventory(self):
with self.open_store() as store:
slot = store.reserve_download(
"https://www.youtube.com/watch?v=managed",
None,
None,
job_id="managed",
)
store.fail_download(
"managed", slot.slot, slot.generation, "managed failure"
)
before = store.get_slot(slot.slot)
export = store.reserve_export_job(
"https://www.youtube.com/watch?v=export",
"mp3",
None,
None,
max_active=5,
job_id="failed-export",
)
self.assertEqual(export.status, "queued")
self.assertTrue(store.fail_export_job("failed-export", "ffmpeg failed"))
self.assertEqual(store.get_slot(slot.slot), before)
def test_slot_reservation_is_committed_before_worker_transition(self):
with self.open_store() as store:
reservation = store.reserve_download(
"https://www.youtube.com/playlist?list=one",
"Alice",
"Service",
job_id="job-one",
)
self.assertEqual(reservation.status, "reserved")
self.assertEqual(reservation.slot, 1)
self.assertEqual(store.get_slot(1).media_state, "queued")
job = store.get_job("job-one")
self.assertIsNotNone(job)
self.assertEqual(job.status, "reserved")
self.assertTrue(store.mark_download_running("job-one", 1, 1))
self.assertEqual(store.get_slot(1).media_state, "downloading")
def test_find_occupied_slot_for_url_only_matches_non_empty_slots(self):
with self.open_store() as store:
url = "https://www.youtube.com/playlist?list=one"
self.assertIsNone(store.find_occupied_slot_for_url(url))
self.assertIsNone(store.find_occupied_slot_for_url(" "))
self.assertIsNone(store.find_occupied_slot_for_url(None))
store.reserve_download(url, None, None, job_id="job-one")
self.assertEqual(store.find_occupied_slot_for_url(url), 1)
self.assertEqual(store.find_occupied_slot_for_url(f" {url} "), 1)
self.assertIsNone(
store.find_occupied_slot_for_url("https://example.test/other")
)
def test_concurrent_requests_get_only_one_remaining_slot(self):
with self.open_store() as store:
for index in range(1, 10):
result = store.reserve_download(
f"https://example.test/{index}",
None,
None,
job_id=f"seed-{index}",
)
self.assertEqual(result.slot, index)
def reserve(index):
return store.reserve_download(
f"https://example.test/concurrent-{index}",
None,
None,
job_id=f"concurrent-{index}",
)
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(reserve, range(8)))
winners = [result for result in results if result.status == "reserved"]
self.assertEqual(len(winners), 1)
self.assertEqual(winners[0].slot, 10)
self.assertEqual(
sum(result.status == "queue_full" for result in results),
7,
)
def test_failed_downloads_receive_expiry_and_cannot_exhaust_slots_forever(self):
with self.open_store() as store:
for slot in range(1, 11):
reservation = store.reserve_download(
f"https://example.test/failure-{slot}",
None,
None,
job_id=f"failure-{slot}",
)
self.assertEqual(reservation.slot, slot)
self.assertTrue(
store.fail_download(
reservation.job_id,
slot,
reservation.generation,
"download failed",
)
)
self.assertEqual(
store.get_slot(slot).expires_at,
self.clock.value + self.config.retention_seconds,
)
self.assertEqual(
store.reserve_download(
"https://example.test/temporarily-full",
None,
None,
).status,
"queue_full",
)
self.clock.value += self.config.retention_seconds + 1
tokens = store.begin_expired_deletions()
self.assertEqual([token.slot for token in tokens], list(range(1, 11)))
for token in tokens:
self.assertTrue(store.finish_delete(token))
replacement = store.reserve_download(
"https://example.test/recovered-capacity",
None,
None,
)
self.assertEqual(replacement.slot, 1)
def test_stale_worker_cannot_overwrite_reused_slot(self):
with self.open_store() as store:
first = store.reserve_download("https://example.test/one", None, None, job_id="old")
store.mark_download_running("old", 1, first.generation)
store.fail_download("old", 1, first.generation, "failed")
deletion = store.begin_delete(1, reason="test")
self.assertTrue(store.finish_delete(deletion))
second = store.reserve_download("https://example.test/two", None, None, job_id="new")
self.assertEqual(second.slot, 1)
self.assertFalse(
store.complete_download(
"old",
1,
first.generation,
title="stale",
items=(("Stale", "stale.mp4"),),
)
)
self.assertEqual(store.get_slot(1).url, "https://example.test/two")
def test_deleting_slot_does_not_renumber_other_slots(self):
with self.open_store() as store:
first = store.reserve_download("https://example.test/one", None, None, job_id="one")
second = store.reserve_download("https://example.test/two", None, None, job_id="two")
store.fail_download("one", 1, first.generation, "failed")
store.fail_download("two", 2, second.generation, "failed")
token = store.begin_delete(1, reason="manual")
store.finish_delete(token)
self.assertEqual(store.get_slot(1).media_state, "empty")
self.assertEqual(store.get_slot(2).slot, 2)
self.assertEqual(store.get_slot(2).url, "https://example.test/two")
def test_concurrent_first_open_installs_only_one_valid_database(self):
stores = [ControllerStore(self.config, now=self.clock) for _ in range(2)]
def open_store(store):
store.open()
return [slot.slot for slot in store.list_slots()]
try:
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as pool:
results = list(pool.map(open_store, stores))
self.assertEqual(results, [list(range(1, 11)), list(range(1, 11))])
markers = stores[0].connection.execute(
"SELECT name FROM data_migrations ORDER BY name"
).fetchall()
self.assertEqual([row["name"] for row in markers], ["fresh_database_v1"])
finally:
for store in stores:
store.close()
class GuardAndExpiryTests(StoreTestCase):
def _ready_slot(self, store, *, slot_expected=1):
reservation = store.reserve_download("https://example.test/ready", None, None, job_id="ready")
self.assertEqual(reservation.slot, slot_expected)
store.mark_download_running("ready", reservation.slot, reservation.generation)
store.complete_download(
"ready",
reservation.slot,
reservation.generation,
title="Ready",
items=(("One", "one.mp4"),),
)
return reservation
def test_active_and_review_guards_block_delete_and_expiry(self):
with self.open_store() as store:
reservation = self._ready_slot(store)
guard = store.protect_slot(
1,
"restart_uncertain",
guard_id="review-me",
review_after=self.clock.value + 10,
)
self.clock.value += 200
due = store.mark_due_guards_for_review()
self.assertEqual(due[0].state, "review_required")
with self.assertRaises(SlotProtectedError):
store.begin_delete(1, reason="manual")
self.assertEqual(store.begin_expired_deletions(), [])
self.assertTrue(
store.clear_guard(
1,
guard.guard_id,
actor="operator",
reason="Player host replaced",
)
)
self.assertEqual(store.player_observation().phase, "unknown")
expired = store.begin_expired_deletions()
self.assertEqual([token.slot for token in expired], [reservation.slot])
def test_admin_clear_requires_current_token_actor_and_reason(self):
with self.open_store() as store:
self._ready_slot(store)
guard = store.protect_slot(1, "player_active", guard_id="current")
with self.assertRaises(ValueError):
store.clear_guard(1, guard.guard_id, actor="", reason="because")
self.assertFalse(
store.clear_guard(
1,
"stale-token",
actor="operator",
reason="confirmed",
)
)
self.assertEqual(store.list_live_guards()[0].guard_id, "current")
def test_restart_marks_last_active_slot_uncertain_without_claiming_idle(self):
with self.open_store() as store:
self._ready_slot(store)
store.set_player_observation(
availability="online",
phase="playing",
active_slot=1,
active_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
paused=False,
reconciliation_state="confirmed",
reconciliation_due_at=None,
reconciliation_id=None,
)
guard = store.mark_restart_uncertain()
self.assertEqual(guard.slot, 1)
observation = store.player_observation()
self.assertEqual(observation.phase, "unknown")
self.assertEqual(observation.reconciliation_state, "awaiting_evidence")
self.assertEqual(observation.active_slot, 1)
def test_external_playback_restart_gets_stable_review_token_and_exact_clear(self):
external_path = "/srv/operator-playlists/live.m3u"
with self.open_store() as store:
store.set_player_observation(
availability="online",
phase="playing",
active_slot=None,
active_path=external_path,
paused=False,
reconciliation_state="confirmed",
reconciliation_due_at=None,
reconciliation_id=None,
)
self.assertIsNone(store.mark_restart_uncertain())
first = store.player_observation()
self.assertEqual(first.reconciliation_state, "awaiting_evidence")
self.assertIsNotNone(first.reconciliation_id)
self.assertEqual(first.reconciliation_due_at, self.clock.value + 30)
self.clock.value += 10
self.assertIsNone(store.mark_restart_uncertain())
repeated = store.player_observation()
self.assertEqual(repeated.reconciliation_id, first.reconciliation_id)
self.assertEqual(repeated.reconciliation_due_at, first.reconciliation_due_at)
self.clock.value = first.reconciliation_due_at + 1
due = store.mark_external_reconciliation_due()
self.assertEqual(due, (first.reconciliation_id, external_path))
self.assertIsNone(store.mark_external_reconciliation_due())
review = store.player_observation()
self.assertEqual(review.reconciliation_state, "operator_review")
self.assertEqual(review.active_path, external_path)
self.assertFalse(
store.clear_external_reconciliation(
"stale-review",
actor="operator",
reason="Player was replaced",
)
)
self.assertTrue(
store.clear_external_reconciliation(
first.reconciliation_id,
actor="operator",
reason="Player was replaced",
)
)
cleared = store.player_observation()
self.assertEqual(cleared.reconciliation_state, "admin_cleared")
self.assertIsNone(cleared.active_path)
self.assertIsNone(cleared.reconciliation_id)
def test_restart_converts_existing_active_guard_and_preserves_deadline(self):
with self.open_store() as store:
self._ready_slot(store)
store.protect_slot(1, "player_active", guard_id="active-before-restart")
store.set_player_observation(
availability="online",
phase="playing",
active_slot=1,
active_path="/mnt/Media/YTPlayer/slot1/playlist.m3u",
paused=False,
reconciliation_state="confirmed",
reconciliation_due_at=None,
reconciliation_id=None,
)
guard = store.mark_restart_uncertain()
self.assertEqual(guard.guard_id, "active-before-restart")
self.assertEqual(guard.kind, "restart_uncertain")
first_deadline = guard.review_after
self.clock.value += 10
repeated = store.mark_restart_uncertain()
self.assertEqual(repeated.guard_id, guard.guard_id)
self.assertEqual(repeated.review_after, first_deadline)
def test_restart_converts_pre_evidence_pending_guard_for_admin_review(self):
with self.open_store() as store:
self._ready_slot(store)
store.protect_slot(1, "pending_play", guard_id="published-before-crash")
guard = store.mark_restart_uncertain()
self.assertEqual(guard.guard_id, "published-before-crash")
self.assertEqual(guard.kind, "restart_uncertain")
self.assertEqual(store.player_observation().active_slot, 1)
self.assertTrue(
store.clear_guard(
1,
guard.guard_id,
actor="operator",
reason="Player replaced",
)
)
def test_guard_review_transition_is_reported_only_once(self):
with self.open_store() as store:
self._ready_slot(store)
store.protect_slot(
1,
"restart_uncertain",
guard_id="one-warning",
review_after=self.clock.value,
)
self.assertEqual(len(store.mark_due_guards_for_review()), 1)
self.assertEqual(store.mark_due_guards_for_review(), [])
def test_clearing_one_legacy_guard_does_not_erase_other_observation(self):
with self.open_store() as store:
self._ready_slot(store)
second = store.reserve_download(
"https://example.test/second", None, None, job_id="second"
)
store.mark_download_running("second", 2, second.generation)
store.complete_download(
"second",
2,
second.generation,
title="Second",
items=(("Second", "second.mp4"),),
)
first_guard = store.protect_slot(
1, "legacy_uncertain", guard_id="legacy-one"
)
store.protect_slot(2, "player_active", guard_id="active-two")
store.set_player_observation(
availability="online",
phase="playing",
active_slot=2,
active_path="/mnt/Media/YTPlayer/slot2/playlist.m3u",
paused=False,
reconciliation_state="confirmed",
reconciliation_due_at=None,
reconciliation_id=None,
)
self.assertTrue(
store.clear_guard(
1,
first_guard.guard_id,
actor="operator",
reason="legacy evidence resolved",
)
)
observation = store.player_observation()
self.assertEqual(observation.active_slot, 2)
self.assertEqual(observation.phase, "playing")
class LegacyMigrationTests(StoreTestCase):
def write_legacy(self, data):
self.data.mkdir(parents=True, exist_ok=True)
raw = yaml.safe_dump(data, sort_keys=True).encode("utf-8")
self.config.legacy_yaml_path.write_bytes(raw)
return raw
def test_migration_preserves_holes_timestamps_original_and_backup(self):
raw = self.write_legacy(
{
"slots": {
"2": {
"slot": 2,
"url": "https://example.test/two",
"yt_id": "abc",
"title": "Two",
"path": "/unsafe/ignored",
"m3u": "/unsafe/ignored/playlist.m3u",
"created_at": "2025-01-02T03:04:05Z",
"expires_at": "2025-01-04T03:04:05Z",
"item_count": 3,
"state": "playing",
},
"7": {
"slot": 7,
"url": "https://example.test/seven",
"title": "Seven",
"path": "/unsafe/seven",
"m3u": "/unsafe/seven/playlist.m3u",
"created_at": "2025-02-01T00:00:00Z",
"expires_at": "2025-02-03T00:00:00Z",
"item_count": 1,
"state": "ready",
},
}
}
)
digest = hashlib.sha256(raw).hexdigest()
with self.open_store() as store:
self.assertTrue(store.migration_report.migrated)
self.assertEqual(store.migration_report.source_sha256, digest)
self.assertEqual(store.get_slot(1).media_state, "empty")
self.assertEqual(store.get_slot(2).title, "Two")
self.assertEqual(store.get_slot(7).slot, 7)
self.assertEqual(store.get_slot(2).created_at, 1_735_787_045.0)
guards = store.list_live_guards()
self.assertEqual([(guard.slot, guard.state) for guard in guards], [(2, "review_required")])
marker = store.connection.execute(
"SELECT * FROM data_migrations WHERE name = ?",
(LEGACY_MIGRATION_NAME,),
).fetchone()
self.assertEqual(marker["source_sha256"], digest)
backup = Path(marker["backup_path"])
self.assertEqual(backup.read_bytes(), raw)
self.assertEqual(self.config.legacy_yaml_path.read_bytes(), raw)
def test_completed_migration_is_idempotent_and_ignores_changed_yaml(self):
original = self.write_legacy(
{
"slots": {
"3": {
"slot": 3,
"url": "https://example.test/original",
"title": "Original",
"path": "/ignored",
"m3u": "/ignored/playlist.m3u",
"created_at": "2025-01-01T00:00:00Z",
"expires_at": "2025-01-03T00:00:00Z",
"item_count": 1,
"state": "ready",
}
}
}
)
with self.open_store() as store:
generation = store.get_slot(3).generation
backup_path = Path(store.migration_report.backup_path)
self.config.legacy_yaml_path.write_text("not: [the, original]\n", encoding="utf-8")
with self.open_store() as store:
self.assertFalse(store.migration_report.migrated)
self.assertEqual(store.get_slot(3).title, "Original")
self.assertEqual(store.get_slot(3).generation, generation)
self.assertEqual(backup_path.read_bytes(), original)
def test_failed_migration_keeps_original_and_verified_backup_recoverable(self):
self.data.mkdir(parents=True, exist_ok=True)
raw = b"slots:\n 1: [truncated\n"
self.config.legacy_yaml_path.write_bytes(raw)
store = ControllerStore(self.config, now=self.clock)
with self.assertRaises(MigrationError):
store.open()
self.assertFalse(self.config.database_path.exists())
self.assertEqual(self.config.legacy_yaml_path.read_bytes(), raw)
backups = list(self.data.glob("playlists.yaml.pre-v2.*.bak"))
self.assertEqual(len(backups), 1)
self.assertEqual(backups[0].read_bytes(), raw)
def test_normalized_duplicate_slot_keys_fail_all_or_nothing(self):
self.data.mkdir(parents=True, exist_ok=True)
raw = b"slots:\n 1:\n slot: 1\n '01':\n slot: 1\n"
self.config.legacy_yaml_path.write_bytes(raw)
with self.assertRaisesRegex(MigrationError, "normalize to 1"):
self.open_store()
self.assertFalse(self.config.database_path.exists())
if __name__ == "__main__":
unittest.main()
+22
View File
@@ -0,0 +1,22 @@
"""Production WSGI entry point and Controller service lifecycle owner."""
from __future__ import annotations
import atexit
import logging
from app import create_app
from controller.config import load_config
from controller.service import ControllerService
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
config = load_config()
service = ControllerService(config)
application = create_app(config, service)
service.start()
atexit.register(service.stop)