Video-Player-Controller/README.md

Video Player Controller v2

Video Player Controller downloads media into ten stable slots, publishes the existing Home Assistant and Companion state topics, and coordinates playback between the authoritative Video Player Node and the Audio Router.

Controller v2 is deliberately conservative: publishing a command is not proof that playback changed. Only fresh Player Node status or telemetry can establish playing, paused, or stopped state. A command accepted by the MQTT broker remains pending until that evidence arrives.

The Controller does not run or supervise mpv. The Player Node owns the MPV lifecycle and sends audio only to pipewire/ytplayer_bus; the Controller owns the request/confirmation/restoration workflow for Audio Router mode.

Architecture and safety model

Browser / HA / Companion
          |
          | download requests and Controller commands
          v
Flask -> bounded event queue -> single Controller coordinator
                                  |        |          |
                                  |        |          +-> SQLite state/audit
                                  |        +------------> Player-slot workers
                                  |        +------------> utility-export workers
                                  +---------------------> MQTT
                                                            |
                              +-----------------------------+------------------+
                              |                                                |
                              v                                                v
                    Video Player Node                                  Audio Router
                    (authoritative state)                       (physical audio mode)
                              |
                              +-> mpv -> pipewire/ytplayer_bus

MQTT callbacks, web requests, timers, and worker notifications converge on a single coordinator thread for orchestration and publication. Download workers perform their external work independently and commit only generation-fenced SQLite transitions. SQLite transactions are the final persistence boundary. This provides the following invariants:

• Slots 1 through 10 are permanent identities. A hole stays a hole; deleting slot 3 never renames slot 4 or its directory.
• A download slot is reserved in an immediate SQLite transaction before a worker is submitted. A partial unique index prevents two live jobs in one slot.
• Every reuse increments the slot generation. A late worker or deletion token cannot commit against a newer occupant.
• Active, pending, uncertain, or operator-review playback guards block slot replacement and deletion.
• A reserved or running download blocks deletion and expiry.
• Filesystem deletion is limited to the canonical slotN directory. The database first atomically fences the generation as deleting; filesystem work then runs asynchronously. A directory is atomically moved under .controller-trash before recursive removal, and interrupted deletion recovery is deferred until Player safety is established.
• MQTT can be unavailable at startup. The web/coordinator process remains up, reports degraded health, and retries with bounded exponential backoff.

The canonical managed media layout is:

/mnt/Media/YTPlayer/
  slot1/playlist.m3u
  slot2/playlist.m3u
  ...
  slot10/playlist.m3u

The base path is configurable, but both Controller and Player Node must see the same absolute paths.

Player-authoritative operation model

Controller commands are serialized. A second transition is rejected while a play, stop, pause, or resume operation is waiting for evidence.

For a play request, the Controller:

1. Validates the stable slot or explicit local path and protects a managed slot before any external command is sent.
2. Acquires the Audio Router mode using the bounded policy described below.
3. Publishes the Player command at QoS 1 with retain=false and waits up to mqtt.publish_ack_timeout_seconds for the broker PUBACK.
4. Reports the request as pending. It does not publish playing merely because the command was sent.
5. Confirms, rejects, or marks the operation uncertain from matching Player status/telemetry. The normal Player-evidence deadline is 10 seconds.

A PUBACK proves only broker receipt, not Player execution. If the PUBACK is uncertain, the Controller replaces its MQTT transport so an unacknowledged QoS 1 command cannot be replayed later, and it preserves uncertain playback state for reconciliation.

Player evidence is interpreted using the current Player Node contract:

• playing status or telemetry with a path establishes active playback.
• Telemetry's boolean paused field is authoritative for pause state. Player paused and toggled status only show that control was issued; Controller waits for telemetry before confirming the resulting pause value. If pause, resume, or toggle confirmation times out, both phase and pause value project as unknown until fresh evidence arrives.
• busy means the new play did not replace the Player's existing process. When a slot/path is present, or an earlier identity is still known, that existing playback remains authoritative. A real pathless busy with no prior identity becomes unknown; Controller keeps the Router lease and global cleanup fence until identity/terminal evidence is reasserted or an administrator reconciles the uncertainty.
• Terminal idle requires one of stop, finished, mpv_error, sigterm, sigint, or shutdown, or cleared telemetry. An ambiguous idle does not erase active ownership.
• A generic Player error is not proof that MPV stopped. Known managed or external playback becomes reviewable uncertainty; an identity-less error keeps the global cleanup fence closed. Only error reasons startup_error and worker_start_error prove that playback never became active and reduce to idle even if cleared telemetry was lost.
• On reconnect, idle with message:"player online" and a path means the Player is still active. The same announcement without a path means idle.
• The retained flat youtube/player/state topic is never playback or cleanup evidence. Its value is historical across outages, and Player reconnect deliberately emits flat idle even while MPV is active. Controller keeps the subscription for Companion compatibility but does not use it to unlock downloads, expiry, deletion recovery, or playback guards.

Use youtube/controller/cmd for operational control. Publishing directly to youtube/player/cmd bypasses Controller Audio Router acquisition and slot guard creation. Controller will eventually reduce resulting Player evidence, but there is an unavoidable cross-service race before that evidence arrives.

MQTT contracts

Defaults remain compatible with the existing Player Node, Home Assistant, and Companion integrations. Topic names can be configured except where another component treats one as fixed; coordinate any change across all consumers.

Topic matrix

Direction	Default topic	Payload	QoS used by Controller	Retained	Contract status
Controller -> Player	youtube/player/cmd	JSON command	publish 1	no	Preserved; required
Player -> Controller	youtube/player/status	JSON status	subscribe 1	no	Player-authoritative
Player -> Controller	youtube/player/state	plain flat state	subscribe 0	yes, by Player	Preserved Companion contract only; never cleanup evidence
Player -> Controller	youtube/player/telemetry	JSON telemetry	subscribe 0	no	Player-authoritative
HA/Companion -> Controller	youtube/controller/cmd	JSON command	subscribe 1	must be no	Preserved; retained commands are rejected
Controller -> consumers	youtube/ingest/progress	JSON progress	publish 0	no	Preserved
Controller -> consumers	youtube/media/playback/<slot>/title	plain string	publish 0	yes	Preserved
Controller -> consumers	youtube/media/playback/<slot>/state	plain string	publish 0	yes	Preserved
Controller -> consumers	youtube/media/playback/<slot>/count	decimal string	publish 0	yes	Preserved
Controller -> consumers	youtube/media/playback/<slot>/items/<position>	plain string	publish 0	yes	Preserved; empty payload clears stale items
Controller/operator -> Router	audio/player/mode/set	plain mode string	publish and subscribe 1	Controller publishes no	Router contract
Router -> Controller	audio/player/mode/state	plain mode string	subscribe 1	Router-owned	Router confirmation
Controller -> consumers	youtube/controller/v2/events	JSON event	publish 1	no	Optional, additive v2 contract
Controller -> Home Assistant	homeassistant/sensor/yt_slot_<slot>_<field>/config	discovery JSON	publish 0	yes	Preserved IDs

QoS in the incoming rows is the Controller's requested subscription QoS; the publisher and broker determine the delivered QoS. Player commands and Router mode commands are always emitted with QoS 1 and retain=false.

The Controller uses Paho manual acknowledgement for incoming QoS 1 messages. It acknowledges after successful coordinator handling and deduplicates repeated packets by MQTT connection generation, topic, and packet ID. Retained Controller commands are rejected so a reconnect cannot replay an old control operation.

Detailed Player status and telemetry are valid evidence only when retain=false. If either arrives retained, Controller treats it as historical, does not reduce it, does not unlock cleanup, and does not clear guards. It logs and audits retained_player_evidence_rejected, publishing that warning when the optional v2 event topic is enabled; a retained QoS 1 message is deliberately acknowledged after rejection so the same invalid evidence is not redelivered indefinitely.

Controller command payloads

Publish all examples at QoS 1 without retain:

mosquitto_pub -h 10.40.0.2 -t youtube/controller/cmd -q 1 \
  -m '{"action":"play","slot":1,"label":"Opening"}'

Supported JSON objects are:

{"action":"play","slot":1,"label":"Opening"}

{"action":"play","path":"/mnt/Media/YTPlayer/special/welcome.mp4","label":"Welcome"}

{"action":"play","playlist":"/mnt/Media/YTPlayer/special/playlist.m3u","title":"Special"}

{"action":"stop"}

{"action":"pause"}

{"action":"resume"}

{"action":"toggle"}

{"action":"delete_slot","slot":4,"url":"optional stale-command guard"}

play requires a ready managed slot or an existing local path/playlist. An explicit path inside a managed slot derives that stable slot identity; an explicit conflicting slot is rejected. toggle is translated to pause or resume from the latest telemetry and is rejected when pause state is unknown.

When the global cleanup gate lacks fresh Player evidence, manual deletion also requires the exact boolean override:

{"action":"delete_slot","slot":4,"confirm_player_unknown":true}

That override only bypasses the global evidence gate. It does not bypass an active download or a per-slot active/uncertain playback guard. It is separate from administrative reconciliation, authorizes only that manual delete, and does not open automatic expiry/recovery or consume a cleanup token.

An accepted manual delete is atomically fenced in SQLite and returns {"status":"deleting","slot":4}, not deleted. Filesystem removal happens on a worker afterward. MQTT commands have no synchronous response topic, so an MQTT consumer observes the retained slot state (and optional v2 warnings) rather than depending on that in-process return value.

Preserved slot and Home Assistant state

For every stable slot, Controller publishes retained title, state, and count values plus retained item titles. Projected state values include empty, queued, downloading, ready, error, deleting, playing, paused, unknown, and operator_review. A pending play remains ready until Player evidence says otherwise.

The thirty discovery entities keep the established identifiers:

• Discovery topic: homeassistant/sensor/yt_slot_<N>_<title|state|count>/config
• Unique ID: yt_slot_<N>_<title|state|count>
• Device identifier: ytplayer-controller
• Device name/manufacturer/model: YouTube Controller / River of Glory / YT Controller

Changing homeassistant.discovery_prefix changes only the discovery prefix. Set homeassistant.enabled: false to suppress discovery; legacy slot state publication continues.

Ingest progress

youtube/ingest/progress is a non-retained JSON stream. Payloads always contain slot and state, with applicable fields such as title, count, message, url, submitter, and note. A queued event also contains reused_slot. Common states are queued, downloading, ready, and error.

Optional v2 events

topics.controller_result defaults to youtube/controller/v2/events. Set it to YAML null to disable publication. This topic is deliberately additive: legacy MQTT, Home Assistant, and Companion consumers never need it.

Events are QoS 1, non-retained JSON:

{
  "version": 2,
  "event_id": "6ed8f0e49a8b4d598e5285ca14ec89ef",
  "type": "audio_router_warning",
  "severity": "warning",
  "message": "Audio Router did not confirm 'video' within 3 seconds",
  "ts": 1786060800.0,
  "slot": 1,
  "operation_id": "4e32c8d628454426bff69c879e709f33",
  "continued": true
}

slot, operation_id, and continued appear only when relevant. Each event is also written to audit_events and logged, so the non-retained stream is not the durable record. Event publication itself depends on MQTT availability.

Audio Router orchestration

The normal playback target is the plain mode video:

• Command: audio/player/mode/set
• Confirmed state: audio/player/mode/state

Before play, Controller snapshots the latest confirmed Router state and creates one persisted lease for the playback session. Router acquisition and restoration both use the finite router.confirmation_timeout_seconds; neither can hang a play operation indefinitely.

Unknown-state and failure policy

The four Router settings are independent:

Setting	Default	Effect
router.confirmation_timeout_seconds	3	Maximum wait for acquisition or restoration confirmation
router.when_state_unknown	attempt	attempt, continue, or reject when no current state exists
router.continue_on_confirmation_failure	true	Continue playback after publish failure, disconnect, operator conflict, or confirmation timeout
router.fallback_restore_mode	null	Optional restoration target when the prior mode was unknown

when_state_unknown has these meanings:

• attempt: request video, wait for the bounded confirmation, then apply continue_on_confirmation_failure if it cannot be confirmed.
• continue: request video and start playback immediately without claiming restoration ownership. This still records and publishes a warning.
• reject: reject this play request without sending a Player command. The caller may retry after a current Router state becomes available.

The safe service default is attempt plus continue_on_confirmation_failure: true: it tries to set and confirm video for three seconds, but unavailable Router state cannot hang or permanently disable playback. The warning is persisted, logged, and published on the optional v2 event topic when MQTT permits.

fallback_restore_mode is used only when the previous mode was unknown and acquisition subsequently becomes confirmed. A play that continues immediately or after failed confirmation has no safe restoration ownership, so it does not apply the fallback later.

Safe restoration and manual override

At authoritative playback termination, Controller restores the prior mode only when all of these remain true:

• Controller actually changed the mode and received confirmation of video.
• The same lease is still held.
• The latest Router state is still the Controller-owned target.
• MQTT continuity was not lost.
• No live, non-self mode command and no conflicting state change indicated an operator override.

Controller subscribes to both Router mode set and state topics. Its own mode-set publication is correlated by exact payload and stays recognized as a self-echo both while the publish is in flight and, because a broker may legally deliver a subscription echo after the PUBACK that acknowledged the same publish, for a short mqtt.router_echo_grace_seconds window afterward (default 2 seconds). Every other live mode-set publication is treated conservatively as operator intent, even if it repeats video. A retained set message is ignored as stale intent. Controller does not publish Router state; it accepts a received live or retained state message as the Router's current mode. Once ownership is lost, Controller does not take it back merely because state later returns to video.

Restoration is requested once and confirmed within the same bounded timeout. Failure produces a warning and no automatic retry. The Router's plain string protocol has no publisher identity or compare-and-set token, so an exactly simultaneous same-value command cannot be distinguished perfectly; conservative ownership loss avoids overwriting a known manual choice.

Downloads, retention, and stable slots

The web form reserves the first empty stable slot, or reuses the slot already associated with the same URL when it is safe. Reservation and the reserved download job commit before a worker starts. downloads.max_workers bounds concurrent yt-dlp work; it does not change the ten-slot capacity.

Each successful download builds slotN/playlist.m3u atomically and records item metadata in the same slot generation. Downloads run yt-dlp with an explicit Node JavaScript runtime and ffmpeg support. On restart, database jobs left in reserved or running become interrupted, and their slots become error instead of pretending success. Whenever a failed or restart-interrupted job would otherwise leave an error slot without expiry, Controller assigns now + storage.retention_seconds. This fallback does not replace an existing expiry and prevents repeated failed downloads from permanently exhausting all ten slots.

Ready and error media can expire after storage.retention_seconds, checked at storage.expiry_interval_seconds. Cleanup skips any slot with a live job or playback guard. Cleanup and interrupted-deletion recovery are globally deferred until safe detailed Player status/telemetry or an explicit audited reconciliation makes cleanup safe. The retained flat Player state never unlocks this fence.

Manual and expiry deletion use the same asynchronous sequence:

1. A transaction rechecks media state, URL guard, live jobs, and playback guards, increments the generation, and commits state deleting.
2. A bounded worker rechecks playback guards before touching the filesystem, atomically moves the canonical directory to generation-specific trash, and checks again before recursive removal. If a late guard appears after the move, the worker restores the directory and leaves deletion pending.
3. Only successful filesystem completion changes the database row to empty. A failure remains durably deleting, emits slot_deletion_pending, and is retried without blocking the coordinator. Error retries are accelerated to at most 30 seconds; otherwise pending deletions are reconsidered during each cleanup/expiry cycle and whenever safe Player evidence or reconciliation releases the fence.

Only one worker handles a given slot generation at a time. A late guard or a stale generation therefore defers/fences completion instead of deleting a newer occupant. A deleting state can be visible for more than one interval during a filesystem failure or active playback review; that is deliberate, recoverable state rather than a completed delete.

Expiry never compacts slots. An expired slot becomes empty; every other slot keeps its number and canonical directory.

Utility exports

The submission page has four explicit modes. Player Slot is selected by default, and an older client that omits download_mode still takes that path. MP4 Export, M4A Audio, and MP3 Audio use a separate durable job queue and dedicated worker pool. They never reserve or mutate a Player slot, publish ingest/HA/MQTT state, acquire the Audio Router, create Player metadata, or enter Player playback, expiry, deletion, or reconciliation gates.

Player Slot remains quality-first and does not force MP4 or H.264. Its strict yt-dlp selector is bv*[height<=H]+ba/b[height<=H]: separate 1080p video and audio are eligible, a lower-resolution source is not upscaled, and no unfiltered fallback may exceed downloads.max_height.

Utility modes behave as follows:

• MP4 selects the best source at or below downloads.export_max_height, with native H.264/AAC preferred only among otherwise equal-quality choices. It probes every result with ffprobe. An existing MP4/H.264/AAC/yuv420p result is moved without remuxing or transcoding; otherwise ffmpeg copies each compatible stream and converts only an incompatible video stream to H.264 or audio stream to AAC. The output height must equal the selected source height, so export never upscales, and the final file is verified as MP4/H.264/AAC.
• M4A selects best available audio, prefers native AAC among equal-quality choices, extracts audio only, and verifies an AAC stream in an M4A/MP4 container. Native AAC is losslessly repackaged by yt-dlp; another source codec is encoded only when required.
• MP3 selects best available audio, extracts audio only, encodes with yt-dlp's highest-quality VBR setting when required, and verifies an MP3-only result.

Playlist exports preserve item order and use bounded, human-readable Unicode filenames with unsafe path/control characters removed and case-insensitive collision suffixes. Each job publishes atomically into its own directory:

/mnt/Media/YTPlayer/exports/
  video/<job-id>/*.mp4
  audio/<job-id>/*.m4a
  audio/<job-id>/*.mp3

Work is staged under exports/.work/<job-id>. A fsynced manifest makes the publication boundary recoverable: after restart, Controller verifies and finishes a fully prepared or already-published job, while a known partial job is marked interrupted and its private staging directory is removed. It does not resume a partial yt-dlp process. Unknown directories are left untouched. The web page lists live jobs before recent completed/failed/interrupted jobs and shows their output paths or error. It intentionally provides no file-serving, cancellation, or deletion endpoint.

Utility output is deliberately outside slot retention. Controller never expires it; operators should monitor capacity and remove an entire completed video/<job-id> or audio/<job-id> directory according to local policy. Do not write untrusted files or symlinks into the media/export tree.

Restart uncertainty and administrative reconciliation

On startup, a last-known managed active slot or pending play is converted to a restart_uncertain guard. An active path outside managed slots receives its own persisted reconciliation ID. These records remain deletion-protected until fresh Player evidence confirms active playback or a terminal state clears it.

Separately, every Controller start and every MQTT connect/disconnect closes the global cleanup fence until safe detailed Player evidence arrives. These are two different reconciliation scopes: durable playback ownership reviews protect a known slot/path, while a connection-scoped cleanup token handles the narrower case where no active identity or guard is known but Player evidence is absent.

Durable playback ownership reviews

storage.uncertain_review_after_seconds defaults to 900 seconds. Reaching that deadline changes awaiting_evidence to operator_review; it never assumes the Player stopped and never removes protection. Set the value to 0 for no automatic review deadline. Review events are re-announced after each broker connection.

Find durable review tokens in GET /healthz under reconciliation_reviews. When the optional event topic is enabled, the same token appears as operation_id on playback_reconciliation_required.

To clear a managed-slot review after independently verifying that playback is not active:

{
  "action": "reconcile_playback",
  "decision": "clear",
  "confirm": true,
  "slot": 1,
  "review_id": "copy-the-current-review-id",
  "actor": "operator-name",
  "reason": "Player host was replaced and old playback cannot still exist"
}

For uncertain external playback, omit slot:

{
  "action": "reconcile_playback",
  "decision": "clear",
  "confirm": true,
  "review_id": "copy-the-current-review-id",
  "actor": "operator-name",
  "reason": "Retired Player host verified offline"
}

The exact current review ID, decision:"clear", and boolean confirm:true are required. A stale ID is rejected, as is reconciliation while a Player operation is pending. Clearing a review does not send stop to the Player; it states that the operator has established safety and can permit later cleanup. An incorrect clear can therefore allow deletion of media still in use.

Connection-scoped cleanup reconciliation

Detailed evidence unlocks the global cleanup fence only when it proves either terminal idle or an online active identity. Examples are cleared telemetry, terminal/reconnect-idle status without a path, matching active telemetry/status, or a busy/control/reconnect status backed by a known slot/path. Retained flat state, pathless busy, identity-less generic error, malformed messages, and mere MQTT connectivity do not qualify.

When cleanup is fenced but there is no known active identity and no live slot guard, /healthz exposes the current token:

{
  "cleanup_reconciliation": {
    "review_id": "copy-the-current-connection-token",
    "kind": "player_evidence",
    "state": "awaiting_evidence",
    "clearable": true
  }
}

The value is null once detailed evidence/override has opened cleanup, and is also withheld when a known identity or guard requires the durable playback review workflow instead. clearable:false means a Router-gated play or Player operation is still pending.

After independently verifying that no Player can be using managed media, publish this exact administrative command:

{
  "action": "reconcile_playback",
  "scope": "cleanup",
  "decision": "clear",
  "confirm": true,
  "review_id": "copy-the-current-connection-token",
  "actor": "operator-name",
  "reason": "Player appliance was independently verified offline"
}

The accepted in-process result is:

{
  "status": "cleared",
  "review_id": "copy-the-current-connection-token",
  "scope": "cleanup"
}

This token is intentionally not durable: it rotates on Controller startup and every MQTT connection or continuity loss. Fetch /healthz again immediately before publishing; an older token is rejected. The clear is connection-scoped, recorded as player_evidence_admin_override in the audit database, and announced as the optional playback_reconciled event. Later MQTT continuity loss closes the fence and creates a new token.

For a pathless busy, accepting this cleanup reconciliation also abandons the held Router lease instead of guessing that playback ended and restoring an old mode. The operator then owns any subsequent Router change. Fresh reasserted playback followed by terminal evidence uses the normal safe restoration path.

Cleanup scope accepts no slot and cannot clear or bypass a known active path, slot guard, active download, or pending Player/Router transition. Use the durable slot/external review payloads above for known playback uncertainty. This override opens cleanup scheduling; each slot deletion still performs its own transactional job/guard checks and late-guard checks.

actor and reason are durable audit fields, but actor is a caller-supplied label, not authentication or authorization. Enforce administrative access with broker credentials and topic ACLs.

SQLite persistence

Controller v2 uses one embedded SQLite database at <storage.data_base>/<storage.database_name>, by default /controller-data/controller.sqlite3. It introduces no external database service.

Schema version 3 contains:

Table	Purpose and important constraints
schema_migrations	Applied internal schema versions; v1/v2 databases upgrade transactionally to v3
data_migrations	One-time import provenance, source hash, backup path, and warnings
slots	Exactly one row for each stable slot 1-10, generation, media state, identity, timestamps, expiry, and error
slot_items	Ordered item title/path metadata keyed by slot and position
download_jobs	Reservation/worker lifecycle keyed by job ID and slot generation; one live job per slot
export_jobs	Slot-free utility job mode/status, request metadata, output paths, and error; no slot foreign key
slot_guards	Active, pending, restart-uncertain, and legacy-uncertain playback protection; one live guard per slot
player_observation	Singleton last status/telemetry reduction, generation, active identity, pause, and reconciliation state/token
audio_router_observation	Singleton last mode, freshness, timestamp, and warning
audio_leases	Acquisition, ownership, restoration, deadline, and failure history for Router sessions
audit_events	Durable migration, safety, reconciliation, and warning history

Runtime connections enable foreign keys, a 5-second busy timeout, WAL journaling, and synchronous=FULL. Writes use BEGIN IMMEDIATE transactions behind a re-entrant process lock. Opening and migrating the store also takes the cross-process /controller-data/.controller-store.lock file lock. Production must run one application worker; these database protections are not permission to start duplicate coordinators or MQTT clients.

For a consistent manual backup, stop Controller cleanly before copying the entire data directory. If an unclean live copy is unavoidable, preserve controller.sqlite3, controller.sqlite3-wal, and controller.sqlite3-shm as a set.

One-time playlists.yaml migration

Migration runs automatically only when both conditions are true:

1. The final SQLite database does not exist.
2. <storage.data_base>/<storage.legacy_yaml_name> exists, normally /controller-data/playlists.yaml.

The migration is non-destructive and idempotent:

1. It takes the process lock and rechecks database existence.
2. It reads at most 8 MiB and rejects a source whose inode, size, or mtime changes during the read.
3. It computes the full SHA-256 and creates a content-addressed, fsynced backup named playlists.yaml.pre-v2.<full-sha256>.bak. An existing backup is reused only if its hash matches.
4. It validates UTF-8, unique YAML mapping keys, slot keys in 1-10, and agreement between each key and embedded slot identity.
5. It creates a separate staging SQLite file, initializes all ten slots, imports the YAML in one transaction, records migration provenance, and checks schema, foreign keys, integrity, and the exact 1-10 slot set.
6. It fsyncs and atomically replaces the final database path only after all validation succeeds.

The original YAML is never renamed, edited, or removed. The backup is created before YAML parsing, so even malformed YAML can leave a verified recovery copy. A parsing, import, or validation failure rolls back the transaction, removes the staging database, and occurs before final database installation. The final step is an atomic replace followed by a directory fsync; if that last fsync itself fails, a validated final database may already be present and must be inspected rather than overwritten or blindly reimported. Fix or restore the YAML and retry only when the final database is absent, or return to the old deployment; do not discard the original or hash-named backup.

Once controller.sqlite3 exists, startup never imports YAML again and later YAML edits are ignored. That database-presence rule plus the data_migrations record makes the migration idempotent. Existing schema-v1 or schema-v2 Controller SQLite databases are upgraded transactionally to schema v3 rather than reimporting YAML. The v2-to-v3 upgrade adds only the slot-free export_jobs table, its recent-jobs index, and migration record.

Legacy state mapping is explicit:

Legacy state	v2 media state	Extra treatment
ready, idle, finished	ready	Preserves slot number, metadata, and expiry
playing, paused, busy	ready	Creates a legacy_uncertain operator-review guard
queued, downloading	error	Records an interrupted download job and warning
error	error	Preserves or supplies the error message
any unknown value	error	Records a warning rather than guessing

Missing URLs and invalid counts/timestamps are recorded as errors or migration warnings. Parseable legacy UTC timestamps are converted to Unix timestamps. Any migrated error/interrupted slot with no usable legacy expiry receives the same migration-time now + storage.retention_seconds fallback, recorded as a migration warning, so it can later expire when cleanup is safe. An existing legacy expiry is preserved. Migration does not move, rename, compact, or delete media. Legacy YAML does not contain complete per-item paths, so v2 does not invent them; existing playlist files and retained item topics remain untouched until authoritative v2 metadata is available.

Configuration

config.yaml is loaded relative to the application, not the invoking shell's working directory. The production image bind-mounts it read-only at /app/config.yaml. Configuration is validated before service startup and has no hot reload; restart the one-worker service after changes.

Environment overrides

Only the following environment variables override YAML:

Environment variable	YAML setting	Application default
MQTT_HOST	mqtt.host	10.40.0.2
MQTT_PORT	mqtt.port	1883
MQTT_USERNAME	mqtt.username	unset
MQTT_PASSWORD	mqtt.password	unset
MQTT_CLIENT_ID	mqtt.client_id	yt-controller
YTPLAYER_MEDIA_BASE	storage.media_base	/mnt/Media/YTPlayer
YTPLAYER_DATA_BASE	storage.data_base	/controller-data
YTPLAYER_RETENTION_SECONDS	storage.retention_seconds	172800
YTPLAYER_COOKIE_PATH	downloads.cookie_path	/run/secrets/yt-dlp-cookies.txt
PORT	web.port	7531

MQTT username and password must either both be set or both be unset. Media, data, and cookie paths must be absolute. storage.slot_count is validated as exactly 10 and cannot be used to compact or resize the stable namespace.

Compose-only deployment variables are documented in .env.example:

Variable	Purpose
FLASK_SECRET_KEY	Required long random Flask signing secret; no production fallback
CONTROLLER_DATA_DIR	Host directory mounted at /controller-data
YTPLAYER_MEDIA_DIR	Host media directory mounted at /mnt/Media/YTPlayer
CONTROLLER_SECRETS_DIR	Host secrets directory mounted read-only at /run/secrets
CONTROLLER_WEB_PORT	Host-side published HTTP port
CONTROLLER_UID, CONTROLLER_GID	Non-root image user/group matching writable host paths

YAML settings

The checked-in config.yaml is the canonical example. Important defaults are:

• MQTT: 30-second keepalive, 1-to-30-second reconnect backoff, 512-entry event queue, and 2-second QoS 1 PUBACK timeout.
• Storage: 48-hour retention, 60-second expiry scan, and 15-minute uncertainty review deadline.
• Downloads: two Player-slot workers, one dedicated utility-export worker, 2-hour subprocess timeout, strict Player/export maximum height 1080, at most 20 queued/running exports, output below exports, and optional cookies.
• Router: required mode video, 3-second confirmation timeout, unknown-state policy attempt, continue after confirmation failure, and no fallback restore mode.
• Home Assistant: discovery enabled under homeassistant.
• Web: bind 0.0.0.0:7531.

All remaining MQTT topic defaults are listed in the topic matrix. Numeric values must be finite and within their validated ranges.

Cookies

Place an optional Netscape-format cookie file at:

<CONTROLLER_SECRETS_DIR>/yt-dlp-cookies.txt

Compose exposes it as /run/secrets/yt-dlp-cookies.txt. For each yt-dlp command, Controller makes a mode-0600 temporary copy under /tmp, passes that copy to yt-dlp, and removes it afterward. The source is never copied into the image.

With downloads.cookie_required: false (the default), a missing cookie file is logged and download continues without cookies. Set it to true when silent cookie omission is unacceptable. Keep the host file out of Git, restrict its permissions, and rotate it as a credential.

HTTP and health

The HTTP surface is intentionally small:

• GET / renders the submission form and recent utility-export status.
• POST /submit accepts an HTTP(S) URL, optional submitter and note, and download_mode equal to player_slot, mp4, m4a, or mp3. Omitting the field remains equivalent to player_slot.
• GET /healthz returns service and reconciliation state as JSON.

Health includes status, started, coordinator_alive, mqtt_connected, router_state, player, durable reconciliation_reviews, connection-scoped cleanup_reconciliation, and degraded_reasons. Typical degraded reasons are mqtt_disconnected, awaiting_player_evidence, and router_state_unavailable.

The endpoint returns HTTP 200 whenever the service is started and its coordinator thread is alive, including degraded MQTT or Router conditions. It returns 503 when the service is stopped, the coordinator is dead, or health evaluation fails. This means a broker outage does not make Docker restart an otherwise healthy Controller. A configuration, storage, or migration failure prevents startup and therefore cannot produce a healthy endpoint.

Docker deployment

The production image and Compose file intentionally differ from Controller v1:

• Gunicorn runs one gthread worker with four HTTP threads and preload=false. One process must own the singleton coordinator, MQTT client, executor, and SQLite connection. Do not increase workers or enable development hot reload.
• The container runs as configurable non-root UID/GID with all Linux capabilities dropped and no-new-privileges enabled.
• The root filesystem is read-only. Only the data/media bind mounts and bounded /tmp tmpfs are writable.
• The persistent Controller data mount is /controller-data; media is mounted at the same /mnt/Media/YTPlayer path used by Player Node.
• Cookies are supplied through the read-only /run/secrets bind mount instead of the old /cookies_ro.txt path.
• The image includes ffmpeg and a Node 22 runtime required by the pinned yt-dlp YouTube JavaScript support.
• Utility exports use the existing writable media bind mount and ffmpeg/ffprobe installation; they add no service, port, volume, or external database.
• The build currently uses node:22.23.1-bookworm-slim as a JavaScript-runtime stage and python:3.12-slim-bookworm as the final base. Runtime Python requirements are exact pins, including Flask 3.1.3, Gunicorn 26.0.0, paho-mqtt 2.1.0, PyYAML 6.0.3, and yt-dlp 2026.7.4.
• The HTTP health check, 45-second stop grace, bounded JSON logs, init process, and unless-stopped restart policy are explicit.
• tabernet remains an external Docker network and must already exist.

Create a deployment environment file and secret:

cp .env.example .env
python3 -c 'import secrets; print(secrets.token_urlsafe(48))'

Put the generated value in FLASK_SECRET_KEY, set every host path, and ensure CONTROLLER_UID/CONTROLLER_GID can write the data and media directories. Then validate and start:

docker network inspect tabernet
docker compose config --quiet
docker compose build --pull
docker compose up -d
docker compose ps
docker compose logs --tail=200 controller
curl -fsS http://127.0.0.1:7531/healthz

Upgrade from Controller v1

Use a maintenance window. Replace the example paths below with the paths from the existing deployment; the historical Compose deployment commonly used /home/media/Scripts/Video-Player as /controller-data.

1. Record the exact old Git commit or image ID, Compose file, configuration, environment, UID/GID, and MQTT settings. Keep the old image locally.
2. Stop new submissions and let downloads finish. Prefer to stop playback and verify authoritative Player terminal/cleared telemetry before the cutover. If Player must remain active, leave it running; v2 will start conservatively and protect last-known activity until fresh evidence arrives.
3. Stop only the old Controller. Do not modify, upgrade, or replace Player Node as part of this migration.
4. While Controller is stopped, take a matched, read-only rollback snapshot of the old data directory and /mnt/Media/YTPlayer. Preserve permissions and timestamps. Record a SHA-256 of playlists.yaml.
5. Create a dedicated v2 data directory. Copy, do not move, the legacy playlists.yaml into its root before first v2 startup. The v2 directory must not already contain controller.sqlite3, or automatic YAML import will not run.
6. Create the secrets directory and, if needed, install the old cookies in it as yt-dlp-cookies.txt. Do not bake cookies into the image.
7. Copy .env.example to .env; set a new long FLASK_SECRET_KEY, the dedicated CONTROLLER_DATA_DIR, the shared YTPLAYER_MEDIA_DIR, secrets directory, MQTT values, web port, and matching UID/GID. Review config.yaml, especially Router policy and uncertainty deadline.
8. Validate and build the v2 deployment, then start it once. Watch logs for the migration report or any startup failure.
9. Verify that the original YAML still exists, its full-hash .pre-v2.*.bak exists, and controller.sqlite3 exists. Confirm the source and backup hashes match.
10. Check /healthz. Resolve or deliberately reconcile any review token; do not equate a 200/degraded response with current Player or Router evidence.
11. Verify retained slot title/state/count values and all thirty HA discovery IDs. Perform a controlled play/stop through youtube/controller/cmd, confirm Player evidence drives state, and verify Router restoration/manual override behavior.

A concrete copy/start sequence, using the common legacy path, is:

sudo install -d -o 1000 -g 1000 -m 0750 /srv/video-player-controller/data
sudo install -m 0640 -o 1000 -g 1000 \
  /home/media/Scripts/Video-Player/playlists.yaml \
  /srv/video-player-controller/data/playlists.yaml
sudo install -d -o root -g 1000 -m 0750 /srv/video-player-controller/secrets

cp .env.example .env
# Edit .env and config.yaml before continuing.

docker compose config --quiet
docker compose build --pull
docker compose up -d
docker compose logs --tail=200 controller
curl -fsS http://127.0.0.1:7531/healthz

sha256sum \
  /srv/video-player-controller/data/playlists.yaml \
  /srv/video-player-controller/data/playlists.yaml.pre-v2.*.bak
ls -l /srv/video-player-controller/data/controller.sqlite3

If first startup reports migration failure, stop the restart loop, retain the original YAML and generated hash backup, inspect/fix a copy of the source, and retry only while no final SQLite database exists. Returning to the recorded v1 image with the untouched v1 data/media snapshot remains safe because migration does not modify them.

Upgrade from an earlier Controller v2 build

Stop Controller and back up the complete data directory before installing this build. On first start, schema v1 or v2 is upgraded in one transaction to schema v3; existing slots, guards, observations, leases, jobs, and audit rows are not rewritten. Confirm exports/.work, exports/video, and exports/audio can be created by the configured container UID/GID, then submit one controlled export and inspect the path shown on the web page. The legacy playlists.yaml import does not rerun when the SQLite database already exists.

Rollback

Code rollback and state rollback are different operations. Controller v2 does not write changes back to playlists.yaml; v1 does not understand controller.sqlite3. Earlier Controller v2 builds also do not understand schema v3. After any v2 download, delete, expiry, or slot reuse, the pre-migration YAML and current media tree may no longer agree.

1. Stop Controller v2 cleanly. Leave Player Node unchanged.
2. Archive the complete v2 data directory, logs, .env, config.yaml, image ID, and current media tree. Do not delete the SQLite database, original YAML, or hash backup.
3. Restore the matched pre-upgrade v1 playlists.yaml and media snapshot as a pair into the paths used by the old deployment. Do not point v1 at the live v2 SQLite file or mix old YAML with media already mutated by v2.
4. Restore the recorded v1 image/commit, Compose file, cookie mount, environment, and path layout.
5. Start only v1 and verify MQTT connectivity, retained stable-topic consumers, submissions, and a controlled Player operation.
6. Keep the archived v2 database/media pair. A later roll-forward must restore those together before starting v2 again.

If v2 has made no media or inventory mutation, the untouched original YAML and pre-upgrade media may still be a valid rollback pair. Verify that fact rather than assuming it. Controller v1 can compact and rename slots during prune or delete, so starting it against mismatched/current v2 media can immediately undo the stable-slot guarantee.

For a migration-only failure before final SQLite installation, simply keep v2 stopped and restart the recorded v1 deployment against the untouched original data and media; no v2 database state needs conversion.

To roll back only this utility-export extension to an earlier v2 build, stop the service and restore the complete schema-v2 data-directory backup made before upgrade; do not point the older binary at the schema-v3 database. Utility files already published below exports are independent of slots and may be archived or left in place, but the older build will not list or manage them. Rolling forward again requires restoring a matched newer database backup or allowing a fresh schema-v2 backup to upgrade once more.

Development and tests

Application imports are side-effect free. Constructing the Flask app does not open SQLite, start MQTT, create directories, or launch threads. wsgi.py owns the production lifecycle; python app.py explicitly owns the development lifecycle.

Create an isolated environment and run the complete regression suite:

python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install --requirement requirements.txt
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m unittest discover -s tests -v

The suite covers configuration and import side effects, atomic reservation and stale generations, asynchronous deletion/retry/late-guard safety, migration recovery/idempotence, MQTT startup/reconnect/PUBACK/manual acknowledgement, Player generation, pathless busy/error handling and pause-timeout reduction, durable playback plus connection-scoped cleanup reconciliation, Audio Router timeouts and ownership loss, service integration, HA compatibility, HTTP health, and Docker/runtime assets. It uses fakes and temporary directories; a real-broker/container smoke test is still required for a deployment. Utility coverage additionally exercises actual pinned yt-dlp selection semantics, Player/export isolation, queue bounds, codec command paths, safe names, failure/restart publication recovery, and schema-v2-to-v3 migration.

Useful static and container checks are:

docker compose config --quiet
docker compose build
docker compose up -d
curl -fsS http://127.0.0.1:7531/healthz
docker compose logs --tail=200 controller

The authoritative Player Node has its own isolated regression suite. Validate the sibling without installing dependencies globally or modifying its source:

cd ../Video-Player-Node
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install --requirement requirements.txt
PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m unittest discover -s tests -v

Security and operational limitations

• The web form is unauthenticated and listens on 0.0.0.0 by default. Restrict the published port with host firewall/network ACLs or place it behind an authenticated reverse proxy. Submitted URLs must use http/https, carry no embedded userinfo, and resolve (by exact, normalized hostname match) to an official YouTube host: youtube.com, www.youtube.com, m.youtube.com, music.youtube.com, or youtu.be. URL, submitter, note, and total request size are also server-side bounded. yt-dlp performs the outbound request.
• MQTT username/password support is available, but application-level TLS is not configured here. Keep this connection on a trusted/protected network or add a separately managed secure tunnel/proxy, and grant the Controller only the required publish/subscribe topic ACLs. In particular, restrict youtube/controller/cmd and Router mode commands.
• Administrative reconciliation's actor is an audit label, not an identity proof. Authorization comes from who can publish to the command topic.
• Direct commands to youtube/player/cmd bypass Controller orchestration and can race its Router and deletion guards. Operational publishers should use youtube/controller/cmd exclusively.
• Player command PUBACK is transport acknowledgement, not an application-level correlated acknowledgement. Controller infers completion from authoritative status/telemetry and serializes transitions; the Player protocol still has no request ID or generation echoed in evidence.
• Audio Router's plain set/state topics have no publisher identity or atomic compare-and-set. Manual override detection is deliberately conservative but cannot eliminate a perfectly simultaneous same-value race.
• Optional v2 events are non-retained and can be missed while a consumer or MQTT is offline. SQLite audit rows and durable playback reconciliation records are the historical sources; the cleanup-only /healthz token is deliberately connection-scoped and must be refetched. Existing consumers must not depend on the event stream.
• Production is intentionally one Gunicorn worker with no preload or hot reload. More workers would duplicate the coordinator, MQTT client, and downloader.
• Shutdown actively terminates tracked yt-dlp process groups (TERM, then KILL after a short bound) rather than waiting for the configured 2-hour command timeout. The current job fails instead of resuming. A container kill before that bookkeeping completes is recovered as interrupted/error; queued deletion work remains durably deleting for the evidence-gated retry path.
• Utility exports are not subject to slot retention and have no automatic disk quota or cleanup policy. Monitor the media filesystem and remove completed job directories administratively. Interrupted partial jobs are not resumed.
• Export-path checks reject configured slot/trash overlap and encountered symlinks, but the writable media tree is not a hostile multi-user sandbox. Only trusted host accounts should be able to modify it while Controller runs.
• The container is read-only and non-root, but the data and media mounts must be writable. Controller can intentionally delete canonical expired or administratively deleted slot directories; maintain independent backups.
• Python dependencies and yt-dlp are exactly pinned, but Docker base images are tag-pinned rather than digest-pinned and Debian apt packages are not version pinned. Builds are therefore not bit-for-bit reproducible. Pin verified image digests and use a controlled package mirror if that is required.
• YouTube changes regularly. Update the explicit yt-dlp[default,pin] version in requirements.txt, rebuild the image, and run both regression and download smoke tests on a deliberate maintenance cadence. Runtime self-update is not enabled. The copied Node binary must match the target image architecture and remain supported by yt-dlp.
• SQLite and its fcntl process lock are designed for a local persistent filesystem. Do not place /controller-data on storage whose locking or atomic rename semantics are unreliable.