474 lines
17 KiB
Markdown
474 lines
17 KiB
Markdown
# video-output
|
|
|
|
A small live-video-to-display appliance for Fedora. It receives live video and
|
|
puts it fullscreen on one chosen monitor. That is all it does.
|
|
|
|
Two jobs:
|
|
|
|
| Command | Path |
|
|
| --- | --- |
|
|
| `video-output camera` | ATEM Mini Pro over USB → V4L2 → MJPEG decode → GPU texture → fullscreen HDMI |
|
|
| `video-output ndi` | ProPresenter → NDI → GPU texture → fullscreen HDMI → ATEM chroma key |
|
|
|
|
No MPV, no GStreamer, no OBS, no browser, no MQTT, no audio, no recording, no
|
|
playback controls. One process that a launcher starts and kills.
|
|
|
|
**Status:** both input backends are implemented. The camera path is verified on
|
|
the production hardware. The installed NDI 6 runtime and source discovery have
|
|
also been verified; live ProPresenter video still needs a sender to be
|
|
broadcasting for end-to-end validation.
|
|
|
|
---
|
|
|
|
## Why Rust, SDL3, libjpeg-turbo
|
|
|
|
* **Rust** — the toolchain was already present, `cargo test` gives the test
|
|
suite with no extra machinery, and a long-running unattended process that
|
|
parses frames off a USB bus benefits from bounds checking. The whole
|
|
dependency tree is **two crates**: `sdl3-sys` and `libc`. V4L2, TurboJPEG and
|
|
NDI use small, hand-written FFI surfaces, so there is no binding crate to rot.
|
|
* **SDL3** — the one portable way to get a borderless fullscreen GPU window on
|
|
a *chosen* Wayland output without writing raw `xdg-shell` code. Used through
|
|
`sdl3-sys` (raw FFI) rather than a safe wrapper, for direct control over
|
|
display selection and YUV texture formats.
|
|
* **libjpeg-turbo (TurboJPEG API)** — SIMD MJPEG decode, and its
|
|
`tj3DecompressToYUVPlanes8` hands back raw YUV planes so the CPU never does a
|
|
colour conversion.
|
|
* **NDI SDK** — official discovery and receive APIs, loaded only when an NDI
|
|
command runs. Audio and metadata output buffers are never requested.
|
|
|
|
### The pixel path
|
|
|
|
The ATEM emits **baseline 4:2:2 JPEG**. Rather than decode to RGBA (4 bytes per
|
|
pixel, CPU colour conversion), the planes are interleaved into packed **YUY2**
|
|
(2 bytes per pixel) and uploaded to an SDL YUY2 texture. SDL's selected renderer
|
|
backend performs the YUV-to-output conversion. That avoids a CPU matrix
|
|
conversion and halves upload bandwidth versus RGBA.
|
|
|
|
```
|
|
MJPEG → TurboJPEG → Y/U/V planes → interleave to YUY2 → SDL texture → GPU
|
|
```
|
|
|
|
4:2:0 camera sources skip even the interleave (planes go straight in as I420).
|
|
Anything else (4:4:4, 4:4:0, 4:1:1, grayscale) falls back to packed BGRX.
|
|
|
|
For NDI, the receiver requests `NDIlib_recv_color_format_fastest`. On this
|
|
platform that preserves the SDK's native packed 4:2:2 path: UYVY without alpha,
|
|
or UYVA when a sender advertises alpha. UYVA begins with an ordinary UYVY plane,
|
|
so the unused alpha plane is ignored and the same two-byte-per-pixel region is
|
|
copied into an SDL UYVY texture. The ATEM keys the green background; alpha is
|
|
not needed.
|
|
|
|
```
|
|
ProPresenter NDI → libndi fastest UYVY/UYVA → SDL UYVY texture → selected output
|
|
```
|
|
|
|
SDL accepts the packed UYVY texture and chooses how to convert it for the active
|
|
renderer. The application does not assume that every SDL renderer implements
|
|
that conversion in a particular shader or GPU stage.
|
|
|
|
### Latency
|
|
|
|
Three places prevent stale data from accumulating, all on purpose:
|
|
|
|
1. **Driver queue** — each wakeup drains every queued V4L2 buffer and decodes
|
|
only the newest, returning the rest unread.
|
|
2. **NDI receive queue** — after a blocking capture wakes, immediately available
|
|
video frames are drained and freed; only the newest is copied for display.
|
|
3. **Frame slot** — a one-deep mailbox between the input thread and the
|
|
renderer. Publishing a second frame throws the first away.
|
|
|
|
A backlog cannot grow in application memory. If capture, decoding, or display
|
|
falls behind, stale frames are dropped instead of replayed later.
|
|
|
|
---
|
|
|
|
## Fedora dependencies
|
|
|
|
```bash
|
|
sudo dnf install -y SDL3-devel turbojpeg-devel gcc
|
|
```
|
|
|
|
| Package | Why |
|
|
| --- | --- |
|
|
| `SDL3-devel` | headers and the `libSDL3.so` link symlink (the `SDL3` runtime package alone is not enough to build) |
|
|
| `turbojpeg-devel` | `libturbojpeg.so` for the MJPEG decoder |
|
|
| `gcc` | the linker driver cargo invokes |
|
|
|
|
Rust, if not present:
|
|
|
|
```bash
|
|
sudo dnf install -y cargo rust # or: https://rustup.rs
|
|
```
|
|
|
|
You must be in the `video` group to open capture devices:
|
|
|
|
```bash
|
|
groups | grep -q video || sudo usermod -aG video "$USER" # then log out and back in
|
|
```
|
|
|
|
### NDI runtime
|
|
|
|
Download the **NDI SDK for Linux** from ndi.video and install its runtime
|
|
library on the system loader path. On the production Fedora machine it is:
|
|
|
|
```
|
|
/usr/lib64/libndi.so.6
|
|
```
|
|
|
|
For an unpacked runtime elsewhere, set `NDI_RUNTIME_DIR_V6` to the directory
|
|
containing `libndi.so.6`. The SDK is not redistributable, so neither its headers
|
|
nor runtime are vendored here.
|
|
|
|
The executable does not link against libndi. NDI commands load the runtime with
|
|
`dlopen`/`dlsym` and resolve only the official find/receive entry points they
|
|
use. Camera, display, device, help, and version commands therefore continue to
|
|
work when NDI is absent. An NDI command instead exits with a clear NDI-specific
|
|
loader error.
|
|
|
|
---
|
|
|
|
## Build
|
|
|
|
```bash
|
|
cargo build --release
|
|
sudo install -m755 target/release/video-output /usr/local/bin/
|
|
```
|
|
|
|
The release binary remains small and does not contain or bundle the NDI SDK.
|
|
|
|
---
|
|
|
|
## Discovering inputs and displays
|
|
|
|
### Capture devices
|
|
|
|
```bash
|
|
video-output devices
|
|
```
|
|
|
|
```
|
|
/dev/video0
|
|
card : ATEM Mini Pro: Blackmagic Desig
|
|
driver : uvcvideo
|
|
bus : usb-0000:67:00.3-1
|
|
usable : yes, video capture + streaming
|
|
[0] MJPG (Motion-JPEG) [needs JPEG decode]
|
|
1920x1080 @ 29.97 fps
|
|
|
|
/dev/video1
|
|
card : ATEM Mini Pro: Blackmagic Desig
|
|
(not a video capture node -- nothing to stream from here)
|
|
```
|
|
|
|
This is the same information as `v4l2-ctl --list-formats-ext`, without needing
|
|
`v4l-utils` installed.
|
|
|
|
> **On the ATEM Mini Pro specifically:** `/dev/video1` is a *metadata* node, not
|
|
> a second video node. It exposes no formats at all, so there is no uncompressed
|
|
> (YUYV/NV12) alternative on this hardware — MJPEG is the only option. The code
|
|
> is structured so an uncompressed device would only need a new branch in
|
|
> `pixel_format_for_fourcc`, not a restructure.
|
|
|
|
### NDI sources
|
|
|
|
Ask the NDI finder to collect advertisements for five seconds:
|
|
|
|
```bash
|
|
video-output ndi-sources
|
|
```
|
|
|
|
The command prints each full, human-readable name exactly as NDI advertises it.
|
|
Use that full name for production configuration:
|
|
|
|
```text
|
|
1 source(s):
|
|
[0] PROPRESENTER-PC (ProPresenter Lower Thirds) ndi://...
|
|
```
|
|
|
|
Discovery time can be set from 1 to 300 seconds. Repeat `--extra-ip` when a
|
|
sender is reachable by unicast but mDNS discovery does not cross the network
|
|
boundary:
|
|
|
|
```bash
|
|
video-output ndi-sources --timeout 10 \
|
|
--extra-ip 10.0.0.42 \
|
|
--extra-ip 10.0.1.42
|
|
```
|
|
|
|
`--source` uses the same deterministic matching policy as displays: an exact
|
|
case-insensitive name first, then a unique prefix, then a unique substring.
|
|
Ambiguity is an error rather than a guess. Omitting `--source` selects the first
|
|
visible source, which is convenient for a bench test but not recommended for a
|
|
production profile.
|
|
|
|
### Displays
|
|
|
|
```bash
|
|
video-output displays
|
|
```
|
|
|
|
```
|
|
Video driver: wayland
|
|
2 display(s):
|
|
[0] Sceptre Tech Inc Sceptre F27 1920x1080+0+1080 @ 110.00 Hz (primary)
|
|
[1] HXA BMD HDMI 1920x1080+0+0 @ 29.97 Hz
|
|
```
|
|
|
|
> **Important, and probably not what you expect:** on KDE Plasma / Wayland, SDL
|
|
> reports the **monitor's model name**, not the connector name. You will see
|
|
> `HXA BMD HDMI`, not `HDMI-A-2`. Always run `video-output displays` and use a
|
|
> name from that list.
|
|
|
|
`--display` accepts, in order of precedence:
|
|
|
|
1. an exact name (case-insensitive) — `--display "HXA BMD HDMI"`
|
|
2. a unique prefix — `--display Sceptre`
|
|
3. a unique substring — `--display BMD`
|
|
4. an index — `--display 1` or `--display #1`
|
|
5. `primary`, the default
|
|
|
|
An ambiguous fragment is an **error**, never a guess: picking the wrong screen
|
|
mid-service is the failure this is designed to avoid. Names are strongly
|
|
preferred over indices, which shift when an output is hotplugged.
|
|
|
|
---
|
|
|
|
## Running it
|
|
|
|
Simplest possible form:
|
|
|
|
```bash
|
|
video-output camera --device /dev/video0 --display "Sceptre Tech Inc Sceptre F27"
|
|
video-output ndi \
|
|
--source "<exact name printed by video-output ndi-sources>" \
|
|
--display "HXA BMD HDMI"
|
|
```
|
|
|
|
Full options: `video-output --help`.
|
|
|
|
### Profiles
|
|
|
|
So the launcher only needs a stable word:
|
|
|
|
```bash
|
|
sudo mkdir -p /etc/video-output
|
|
sudo cp examples/multiview.toml /etc/video-output/
|
|
sudo cp examples/lower-thirds.toml /etc/video-output/
|
|
video-output multiview
|
|
video-output lower-thirds
|
|
```
|
|
|
|
A profile is a flat `key = value` file using the same names as the long
|
|
options, plus `mode = "camera"` or `mode = "ndi"`. Searched in order:
|
|
|
|
```
|
|
./<name>.toml
|
|
$XDG_CONFIG_HOME/video-output/<name>.toml
|
|
~/.config/video-output/<name>.toml
|
|
/etc/video-output/<name>.toml
|
|
```
|
|
|
|
Flags given after a profile name override it:
|
|
|
|
```bash
|
|
video-output multiview --display Sceptre
|
|
```
|
|
|
|
Unknown keys are rejected at startup rather than ignored — a typo should fail
|
|
before the service, not during it.
|
|
|
|
### From a launcher
|
|
|
|
It is an ordinary foreground process:
|
|
|
|
* start it, and it opens fullscreen on the chosen output
|
|
* `SIGTERM` (or `SIGINT`/`SIGHUP`) makes it exit cleanly
|
|
* a missing or broken media source never closes the output window
|
|
* fatal configuration, display-initialisation, or NDI-runtime errors return a
|
|
nonzero status instead of retrying something that cannot recover
|
|
|
|
```bash
|
|
video-output multiview &
|
|
LAUNCHER_PID=$!
|
|
# ...
|
|
kill -TERM "$LAUNCHER_PID"
|
|
```
|
|
|
|
Nothing needs to be restarted because the camera was unplugged or ProPresenter
|
|
stopped sending; it reconnects on its own. An optional systemd template is in
|
|
[`packaging/[email protected]`](packaging/[email protected]).
|
|
|
|
---
|
|
|
|
## Display behaviour
|
|
|
|
Borderless, fullscreen, no title bar, no controls, no OSD, cursor hidden,
|
|
screensaver disabled. Fullscreen is *fullscreen-desktop*: it adopts the
|
|
output's current resolution rather than forcing a mode change, so the HDMI link
|
|
is never renegotiated on a live output.
|
|
|
|
Aspect ratio is preserved by SDL's letterbox logical presentation, so a 16:9
|
|
source fills a 16:9 output exactly and is never stretched.
|
|
|
|
Wayland gives a client no way to *place* a window, only to request fullscreen on
|
|
a particular `wl_output`. The window is therefore created hidden, positioned on
|
|
the target display, set fullscreen, and only then shown — and afterwards the
|
|
program re-reads which display it actually landed on. If the compositor
|
|
overrode the request you get a warning naming both displays, rather than a
|
|
silently wrong monitor.
|
|
|
|
---
|
|
|
|
## Failure behaviour
|
|
|
|
| Situation | What happens |
|
|
| --- | --- |
|
|
| Camera missing at startup | one clear error, then quiet retries with backoff (250 ms → 5 s); the window stays up and black |
|
|
| Device busy | says so explicitly, keeps retrying |
|
|
| Permission denied | tells you to join the `video` group |
|
|
| Camera unplugged while running | `signal lost` logged once, black output, reopens automatically when it returns |
|
|
| Named NDI source absent at startup | logs the search condition once, stays black, and keeps discovering in the background |
|
|
| NDI source appears | creates the receiver automatically and begins displaying valid video |
|
|
| NDI source disappears | marks signal lost, clears to black, rebuilds discovery/receive state, and reconnects when it returns |
|
|
| Malformed or unsupported frame | dropped and counted; the receive process continues and stale output is not left on screen indefinitely |
|
|
| NDI runtime missing or incompatible | the NDI command exits nonzero with the attempted runtime path/symbol in its error; non-NDI commands are unaffected |
|
|
| Fatal decoder error | the JPEG handle is rebuilt so one bad frame cannot poison the ones after it |
|
|
| Backend panics | caught, logged, backend restarted; the display never goes down with it |
|
|
| No valid frame | black |
|
|
|
|
Logging is edge-triggered — an input absent for twenty minutes produces a state
|
|
transition, not a line every frame or discovery poll — plus one throughput line
|
|
per minute (`--stats <seconds>`, `0` disables). Set `VIDEO_OUTPUT_LOG=debug` or
|
|
`--log debug` for additional discovery and recovery diagnostics. Frame problems
|
|
are counted and reported periodically instead of logged individually. Under
|
|
systemd, journald severity prefixes are emitted automatically so
|
|
`journalctl -p warning` filters correctly.
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
cargo test
|
|
```
|
|
|
|
The hardware-independent suite covers CLI and profile parsing, invalid
|
|
configuration, deterministic display and NDI-source matching, reconnect policy,
|
|
NDI and V4L2 ABI layouts, malformed NDI layouts, both stale-frame drop points,
|
|
and the one-frame handoff. Process tests force an isolated missing NDI runtime
|
|
and verify both the clear NDI error and that a non-NDI command still succeeds;
|
|
on Linux, `readelf` also verifies that the executable has no `libndi` dynamic
|
|
dependency.
|
|
|
|
Camera coverage remains intact: real JPEG fixtures exercise all four
|
|
subsampling paths, with truncation and bit-flip fuzzing to prove that corrupt
|
|
input cannot panic or wedge the decoder.
|
|
|
|
Regenerate the JPEG fixtures with `python3 tests/data/generate.py` (needs
|
|
Pillow). They are solid green on purpose: green is what the ATEM keys on, so a
|
|
wrong YUV matrix or range shows up in a test rather than on stage.
|
|
|
|
### Manual camera validation on Fedora
|
|
|
|
Hardware paths that cannot be unit tested:
|
|
|
|
```bash
|
|
# 1. Devices and displays are discovered.
|
|
video-output devices
|
|
video-output displays
|
|
|
|
# 2. Windowed, so it does not take over a screen; expect ~29.97 fps and
|
|
# "0 undecodable"
|
|
video-output camera --device /dev/video0 --display <name> --windowed --stats 3
|
|
|
|
# 3. Fullscreen on the intended output. Confirm the "Opened display:" line
|
|
# names the monitor you meant, with no placement warning after it
|
|
video-output camera --device /dev/video0 --display <name> --stats 5
|
|
|
|
# 4. Recovery: unplug the ATEM's USB cable while it runs.
|
|
# Expect "signal lost", black output, then "signal restored" on replug,
|
|
# with no restart and no repeated log lines
|
|
# 5. Clean shutdown.
|
|
kill -TERM <pid> # exits promptly, no error
|
|
```
|
|
|
|
Verified on this machine: Fedora 44, KDE Plasma on Wayland, ATEM Mini Pro on
|
|
USB, 1920x1080 MJPEG @ 29.97 — sustained 30.0 fps decoded, 0 undecodable
|
|
frames, 0 stale buffers, 0 frames dropped before display, ~33% of one CPU core
|
|
and ~98 MB RSS.
|
|
|
|
### Manual NDI validation on Fedora
|
|
|
|
First verify that the runtime loads and record the exact advertised name:
|
|
|
|
```bash
|
|
video-output ndi-sources --timeout 10
|
|
```
|
|
|
|
If the sender requires unicast discovery:
|
|
|
|
```bash
|
|
video-output ndi-sources --timeout 10 --extra-ip <ProPresenter-IP>
|
|
```
|
|
|
|
Then run the exact name on the production lower-thirds output:
|
|
|
|
```bash
|
|
video-output ndi \
|
|
--source "<exact advertised ProPresenter source name>" \
|
|
--display "HXA BMD HDMI" \
|
|
--stats 5
|
|
```
|
|
|
|
Confirm all of the following:
|
|
|
|
1. The log names `HXA BMD HDMI`, the selected NDI source, and packed UYVY.
|
|
2. ProPresenter graphics appear fullscreen with the correct green background
|
|
and no controls or OSD.
|
|
3. Stopping the ProPresenter NDI output logs signal loss once and turns the
|
|
display black rather than freezing the last lower third.
|
|
4. Restarting the same output restores video without restarting
|
|
`video-output`.
|
|
5. Leaving the source unavailable at startup keeps the black window alive and
|
|
connects automatically when broadcasting begins.
|
|
|
|
At the last validation pass, the installed NDI 6.3.2 runtime loaded correctly
|
|
but no source was advertising. Discovery/retry and the live-video path should be
|
|
rechecked with ProPresenter broadcasting.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
**`cannot find -lSDL3` or `cannot find -lturbojpeg` when building**
|
|
`SDL3-devel` / `turbojpeg-devel` are not installed. The runtime packages alone
|
|
do not provide the link symlinks.
|
|
|
|
**`cannot open /dev/video0: Permission denied`**
|
|
Join the `video` group and log out and back in.
|
|
|
|
**`device is busy`**
|
|
Something else is capturing — often another copy of this program, or OBS.
|
|
|
|
**`no display matches "..."`**
|
|
Run `video-output displays`. Remember the names are monitor models, not
|
|
connector names, on KDE Wayland.
|
|
|
|
**`display "HDMI" is ambiguous`**
|
|
Two outputs match that fragment. Use a longer one, or the exact name.
|
|
|
|
**Warning that the compositor placed the window on the wrong display**
|
|
KWin overrode the fullscreen output request. Check for a KWin window rule
|
|
matching app id `video-output`.
|
|
|
|
**Low frame rate, many "stale buffers skipped"**
|
|
You are running a debug build. Use `cargo build --release`; the YUY2 interleave
|
|
is several times slower without optimisation.
|
|
|
|
**Log line says "decoded despite a libjpeg warning"**
|
|
Normal for the ATEM. It pads frames with a few bytes before the JPEG
|
|
end-of-image marker, which makes libjpeg warn on an otherwise perfect image.
|
|
These frames are complete and are kept. Truncated frames are caught separately
|
|
by a structural check and dropped.
|