docs: document utility export workflow

README.md

@@ -23,7 +23,8 @@ Browser / HA / Companion
 Flask -> bounded event queue -> single Controller coordinator
                                   |        |          |
                                   |        |          +-> SQLite state/audit
-                                  |        +------------> bounded yt-dlp workers
+                                  |        +------------> Player-slot workers
+                                  |        +------------> utility-export workers
                                   +---------------------> MQTT
                                                             |
                               +-----------------------------+------------------+
@@ -411,6 +412,61 @@ state rather than a completed delete.
 Expiry never compacts slots. An expired slot becomes empty; every other slot
 keeps its number and canonical directory.
 
+## Utility exports
+
+The submission page has four explicit modes. `Player Slot` is selected by
+default, and an older client that omits `download_mode` still takes that path.
+`MP4 Export`, `M4A Audio`, and `MP3 Audio` use a separate durable job queue and
+dedicated worker pool. They never reserve or mutate a Player slot, publish
+ingest/HA/MQTT state, acquire the Audio Router, create Player metadata, or enter
+Player playback, expiry, deletion, or reconciliation gates.
+
+Player Slot remains quality-first and does not force MP4 or H.264. Its strict
+yt-dlp selector is `bv*[height<=H]+ba/b[height<=H]`: separate 1080p video and
+audio are eligible, a lower-resolution source is not upscaled, and no
+unfiltered fallback may exceed `downloads.max_height`.
+
+Utility modes behave as follows:
+
+- MP4 selects the best source at or below `downloads.export_max_height`, with
+  native H.264/AAC preferred only among otherwise equal-quality choices. It
+  probes every result with ffprobe. An existing MP4/H.264/AAC/yuv420p result is
+  moved without remuxing or transcoding; otherwise ffmpeg copies each compatible
+  stream and converts only an incompatible video stream to H.264 or audio stream
+  to AAC. The output height must equal the selected source height, so export
+  never upscales, and the final file is verified as MP4/H.264/AAC.
+- M4A selects best available audio, prefers native AAC among equal-quality
+  choices, extracts audio only, and verifies an AAC stream in an M4A/MP4
+  container. Native AAC is losslessly repackaged by yt-dlp; another source codec
+  is encoded only when required.
+- MP3 selects best available audio, extracts audio only, encodes with yt-dlp's
+  highest-quality VBR setting when required, and verifies an MP3-only result.
+
+Playlist exports preserve item order and use bounded, human-readable Unicode
+filenames with unsafe path/control characters removed and case-insensitive
+collision suffixes. Each job publishes atomically into its own directory:
+
+```text
+/mnt/Media/YTPlayer/exports/
+  video/<job-id>/*.mp4
+  audio/<job-id>/*.m4a
+  audio/<job-id>/*.mp3
+```
+
+Work is staged under `exports/.work/<job-id>`. A fsynced manifest makes the
+publication boundary recoverable: after restart, Controller verifies and
+finishes a fully prepared or already-published job, while a known partial job
+is marked interrupted and its private staging directory is removed. It does
+not resume a partial yt-dlp process. Unknown directories are left untouched.
+The web page lists live jobs before recent completed/failed/interrupted jobs and
+shows their output paths or error. It intentionally provides no file-serving,
+cancellation, or deletion endpoint.
+
+Utility output is deliberately outside slot retention. Controller never
+expires it; operators should monitor capacity and remove an entire completed
+`video/<job-id>` or `audio/<job-id>` directory according to local policy. Do not
+write untrusted files or symlinks into the media/export tree.
+
 ## Restart uncertainty and administrative reconciliation
 
 On startup, a last-known managed active slot or pending play is converted to a
@@ -552,15 +608,16 @@ Controller v2 uses one embedded SQLite database at
 `/controller-data/controller.sqlite3`. It introduces no external database
 service.
 
-Schema version 2 contains:
+Schema version 3 contains:
 
 | Table | Purpose and important constraints |
 |---|---|
-| `schema_migrations` | Applied internal schema versions; v1 databases upgrade transactionally to v2 |
+| `schema_migrations` | Applied internal schema versions; v1/v2 databases upgrade transactionally to v3 |
 | `data_migrations` | One-time import provenance, source hash, backup path, and warnings |
 | `slots` | Exactly one row for each stable slot 1-10, generation, media state, identity, timestamps, expiry, and error |
 | `slot_items` | Ordered item title/path metadata keyed by slot and position |
 | `download_jobs` | Reservation/worker lifecycle keyed by job ID and slot generation; one live job per slot |
+| `export_jobs` | Slot-free utility job mode/status, request metadata, output paths, and error; no slot foreign key |
 | `slot_guards` | Active, pending, restart-uncertain, and legacy-uncertain playback protection; one live guard per slot |
 | `player_observation` | Singleton last status/telemetry reduction, generation, active identity, pause, and reconciliation state/token |
 | `audio_router_observation` | Singleton last mode, freshness, timestamp, and warning |
@@ -615,8 +672,10 @@ discard the original or hash-named backup.
 
 Once `controller.sqlite3` exists, startup never imports YAML again and later
 YAML edits are ignored. That database-presence rule plus the `data_migrations`
-record makes the migration idempotent. An existing schema-v1 Controller SQLite
+record makes the migration idempotent. Existing schema-v1 or schema-v2
-database is upgraded transactionally rather than reimporting YAML.
+Controller SQLite databases are upgraded transactionally to schema v3 rather
+than reimporting YAML. The v2-to-v3 upgrade adds only the slot-free
+`export_jobs` table, its recent-jobs index, and migration record.
 
 Legacy state mapping is explicit:
 
@@ -686,8 +745,9 @@ The checked-in `config.yaml` is the canonical example. Important defaults are:
   queue, and 2-second QoS 1 PUBACK timeout.
 - Storage: 48-hour retention, 60-second expiry scan, and 15-minute uncertainty
   review deadline.
-- Downloads: two workers, 2-hour subprocess timeout, maximum height 1080, and
+- Downloads: two Player-slot workers, one dedicated utility-export worker,
-  optional cookies.
+  2-hour subprocess timeout, strict Player/export maximum height 1080, at most
+  20 queued/running exports, output below `exports`, and optional cookies.
 - Router: required mode `video`, 3-second confirmation timeout, unknown-state
   policy `attempt`, continue after confirmation failure, and no fallback restore
   mode.
@@ -718,8 +778,10 @@ permissions, and rotate it as a credential.
 
 The HTTP surface is intentionally small:
 
-- `GET /` renders the playlist submission form.
+- `GET /` renders the submission form and recent utility-export status.
-- `POST /submit` accepts an HTTP(S) URL plus optional submitter and note.
+- `POST /submit` accepts an HTTP(S) URL, optional submitter and note, and
+  `download_mode` equal to `player_slot`, `mp4`, `m4a`, or `mp3`. Omitting the
+  field remains equivalent to `player_slot`.
 - `GET /healthz` returns service and reconciliation state as JSON.
 
 Health includes `status`, `started`, `coordinator_alive`, `mqtt_connected`,
@@ -752,6 +814,8 @@ The production image and Compose file intentionally differ from Controller v1:
   of the old `/cookies_ro.txt` path.
 - The image includes ffmpeg and a Node 22 runtime required by the pinned yt-dlp
   YouTube JavaScript support.
+- Utility exports use the existing writable media bind mount and ffmpeg/ffprobe
+  installation; they add no service, port, volume, or external database.
 - The build currently uses `node:22.23.1-bookworm-slim` as a JavaScript-runtime
   stage and `python:3.12-slim-bookworm` as the final base. Runtime Python
   requirements are exact pins, including Flask 3.1.3, Gunicorn 26.0.0,
@@ -850,11 +914,22 @@ retry only while no final SQLite database exists. Returning to the recorded v1
 image with the untouched v1 data/media snapshot remains safe because migration
 does not modify them.
 
+### Upgrade from an earlier Controller v2 build
+
+Stop Controller and back up the complete data directory before installing this
+build. On first start, schema v1 or v2 is upgraded in one transaction to schema
+v3; existing slots, guards, observations, leases, jobs, and audit rows are not
+rewritten. Confirm `exports/.work`, `exports/video`, and `exports/audio` can be
+created by the configured container UID/GID, then submit one controlled export
+and inspect the path shown on the web page. The legacy `playlists.yaml` import
+does not rerun when the SQLite database already exists.
+
 ## Rollback
 
 Code rollback and state rollback are different operations. Controller v2 does
 not write changes back to `playlists.yaml`; v1 does not understand
-`controller.sqlite3`. After any v2 download, delete, expiry, or slot reuse, the
+`controller.sqlite3`. Earlier Controller v2 builds also do not understand
+schema v3. After any v2 download, delete, expiry, or slot reuse, the
 pre-migration YAML and current media tree may no longer agree.
 
 1. Stop Controller v2 cleanly. Leave Player Node unchanged.
@@ -881,6 +956,14 @@ For a migration-only failure before final SQLite installation, simply keep v2
 stopped and restart the recorded v1 deployment against the untouched original
 data and media; no v2 database state needs conversion.
 
+To roll back only this utility-export extension to an earlier v2 build, stop the
+service and restore the complete schema-v2 data-directory backup made before
+upgrade; do not point the older binary at the schema-v3 database. Utility files
+already published below `exports` are independent of slots and may be archived
+or left in place, but the older build will not list or manage them. Rolling
+forward again requires restoring a matched newer database backup or allowing a
+fresh schema-v2 backup to upgrade once more.
+
 ## Development and tests
 
 Application imports are side-effect free. Constructing the Flask app does not
@@ -904,7 +987,10 @@ Player generation, pathless busy/error handling and pause-timeout reduction,
 durable playback plus connection-scoped cleanup reconciliation, Audio Router
 timeouts and ownership loss, service integration, HA compatibility, HTTP
 health, and Docker/runtime assets. It uses fakes and temporary directories; a
-real-broker/container smoke test is still required for a deployment.
+real-broker/container smoke test is still required for a deployment. Utility
+coverage additionally exercises actual pinned yt-dlp selection semantics,
+Player/export isolation, queue bounds, codec command paths, safe names,
+failure/restart publication recovery, and schema-v2-to-v3 migration.
 
 Useful static and container checks are:
 
@@ -965,6 +1051,12 @@ PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m unittest discover -s tests -v
   timeout. The current job fails instead of resuming. A container kill before
   that bookkeeping completes is recovered as `interrupted`/`error`; queued
   deletion work remains durably `deleting` for the evidence-gated retry path.
+- Utility exports are not subject to slot retention and have no automatic disk
+  quota or cleanup policy. Monitor the media filesystem and remove completed
+  job directories administratively. Interrupted partial jobs are not resumed.
+- Export-path checks reject configured slot/trash overlap and encountered
+  symlinks, but the writable media tree is not a hostile multi-user sandbox.
+  Only trusted host accounts should be able to modify it while Controller runs.
 - The container is read-only and non-root, but the data and media mounts must be
   writable. Controller can intentionally delete canonical expired or
   administratively deleted slot directories; maintain independent backups.