Zayin_Admin/Video-Output
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix me lol

Browse Source
This commit is contained in:
Joey
2026-08-29 17:11:55 -05:00
parent 4b7ab4f10a
commit 94a2d4fed6
12 changed files with 1912 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+159 -39
View File
@@ -13,8 +13,10 @@ Two jobs:
No MPV, no GStreamer, no OBS, no browser, no MQTT, no audio, no recording, no
playback controls. One process that a launcher starts and kills.
**Status:** the camera path is complete and verified on real hardware. The NDI
backend is stubbed and reports that it is not implemented yet.
**Status:** both input backends are implemented. The camera path is verified on
the production hardware. The installed NDI 6 runtime and source discovery have
also been verified; live ProPresenter video still needs a sender to be
broadcasting for end-to-end validation.
---
@@ -24,7 +26,7 @@ backend is stubbed and reports that it is not implemented yet.
suite with no extra machinery, and a long-running unattended process that
parses frames off a USB bus benefits from bounds checking. The whole
dependency tree is **two crates**: `sdl3-sys` and `libc`. V4L2, TurboJPEG and
(later) NDI are hand-written FFI, so there is no binding crate to rot.
NDI use small, hand-written FFI surfaces, so there is no binding crate to rot.
* **SDL3** — the one portable way to get a borderless fullscreen GPU window on
a *chosen* Wayland output without writing raw `xdg-shell` code. Used through
`sdl3-sys` (raw FFI) rather than a safe wrapper, for direct control over
@@ -32,33 +34,52 @@ backend is stubbed and reports that it is not implemented yet.
* **libjpeg-turbo (TurboJPEG API)** — SIMD MJPEG decode, and its
`tj3DecompressToYUVPlanes8` hands back raw YUV planes so the CPU never does a
colour conversion.
* **NDI SDK** — official discovery and receive APIs, loaded only when an NDI
command runs. Audio and metadata output buffers are never requested.
### The pixel path
The ATEM emits **baseline 4:2:2 JPEG**. Rather than decode to RGBA (4 bytes per
pixel, CPU colour conversion), the planes are interleaved into packed **YUY2**
(2 bytes per pixel) and uploaded to an SDL YUY2 texture, letting the GPU sampler
convert. Half the upload bandwidth and no CPU matrix multiply.
(2 bytes per pixel) and uploaded to an SDL YUY2 texture. SDL's selected renderer
backend performs the YUV-to-output conversion. That avoids a CPU matrix
conversion and halves upload bandwidth versus RGBA.
```
MJPEG → TurboJPEG → Y/U/V planes → interleave to YUY2 → SDL texture → GPU
```
4:2:0 sources skip even the interleave (planes go straight in as I420).
4:2:0 camera sources skip even the interleave (planes go straight in as I420).
Anything else (4:4:4, 4:4:0, 4:1:1, grayscale) falls back to packed BGRX.
For NDI, the receiver requests `NDIlib_recv_color_format_fastest`. On this
platform that preserves the SDK's native packed 4:2:2 path: UYVY without alpha,
or UYVA when a sender advertises alpha. UYVA begins with an ordinary UYVY plane,
so the unused alpha plane is ignored and the same two-byte-per-pixel region is
copied into an SDL UYVY texture. The ATEM keys the green background; alpha is
not needed.
```
ProPresenter NDI → libndi fastest UYVY/UYVA → SDL UYVY texture → selected output
```
SDL accepts the packed UYVY texture and chooses how to convert it for the active
renderer. The application does not assume that every SDL renderer implements
that conversion in a particular shader or GPU stage.
### Latency
Two places drop stale data, both on purpose:
Three places prevent stale data from accumulating, all on purpose:
1. **Driver queue** — each wakeup drains every queued V4L2 buffer and decodes
only the newest, returning the rest unread.
2. **Frame slot** — a one-deep mailbox between the input thread and the
2. **NDI receive queue** — after a blocking capture wakes, immediately available
video frames are drained and freed; only the newest is copied for display.
3. **Frame slot** — a one-deep mailbox between the input thread and the
renderer. Publishing a second frame throws the first away.
A backlog can never form. If decoding falls behind, frames are dropped, not
queued. Latency stays bounded by one capture interval plus one present
interval.
A backlog cannot grow in application memory. If capture, decoding, or display
falls behind, stale frames are dropped instead of replayed later.
---
@@ -86,13 +107,24 @@ You must be in the `video` group to open capture devices:
groups | grep -q video || sudo usermod -aG video "$USER" # then log out and back in
```
### NDI (phase 2, not yet wired up)
### NDI runtime
Download the **NDI SDK for Linux** from ndi.video, then either install its
`libndi.so.6` into `/usr/local/lib` and run `sudo ldconfig`, or point
`LD_LIBRARY_PATH` at it. The SDK is not redistributable, so it is not vendored
here. The receiver will load it at runtime via `dlopen`, so the camera path
keeps working on machines that do not have it.
Download the **NDI SDK for Linux** from ndi.video and install its runtime
library on the system loader path. On the production Fedora machine it is:
```
/usr/lib64/libndi.so.6
```
For an unpacked runtime elsewhere, set `NDI_RUNTIME_DIR_V6` to the directory
containing `libndi.so.6`. The SDK is not redistributable, so neither its headers
nor runtime are vendored here.
The executable does not link against libndi. NDI commands load the runtime with
`dlopen`/`dlsym` and resolve only the official find/receive entry points they
use. Camera, display, device, help, and version commands therefore continue to
work when NDI is absent. An NDI command instead exits with a clear NDI-specific
loader error.
---
@@ -103,11 +135,11 @@ cargo build --release
sudo install -m755 target/release/video-output /usr/local/bin/
```
The release binary is about 560 KB.
The release binary remains small and does not contain or bundle the NDI SDK.
---
## Discovering devices and displays
## Discovering inputs and displays
### Capture devices
@@ -138,6 +170,38 @@ This is the same information as `v4l2-ctl --list-formats-ext`, without needing
> is structured so an uncompressed device would only need a new branch in
> `pixel_format_for_fourcc`, not a restructure.
### NDI sources
Ask the NDI finder to collect advertisements for five seconds:
```bash
video-output ndi-sources
```
The command prints each full, human-readable name exactly as NDI advertises it.
Use that full name for production configuration:
```text
1 source(s):
[0] PROPRESENTER-PC (ProPresenter Lower Thirds) ndi://...
```
Discovery time can be set from 1 to 300 seconds. Repeat `--extra-ip` when a
sender is reachable by unicast but mDNS discovery does not cross the network
boundary:
```bash
video-output ndi-sources --timeout 10 \
--extra-ip 10.0.0.42 \
--extra-ip 10.0.1.42
```
`--source` uses the same deterministic matching policy as displays: an exact
case-insensitive name first, then a unique prefix, then a unique substring.
Ambiguity is an error rather than a guess. Omitting `--source` selects the first
visible source, which is convenient for a bench test but not recommended for a
production profile.
### Displays
```bash
@@ -175,8 +239,10 @@ preferred over indices, which shift when an output is hotplugged.
Simplest possible form:
```bash
video-output camera --device /dev/video0 --display BMD
video-output ndi --source "ProPresenter Lower Thirds" --display HDMI
video-output camera --device /dev/video0 --display "Sceptre Tech Inc Sceptre F27"
video-output ndi \
--source "<exact name printed by video-output ndi-sources>" \
--display "HXA BMD HDMI"
```
Full options: `video-output --help`.
@@ -188,7 +254,9 @@ So the launcher only needs a stable word:
```bash
sudo mkdir -p /etc/video-output
sudo cp examples/multiview.toml /etc/video-output/
sudo cp examples/lower-thirds.toml /etc/video-output/
video-output multiview
video-output lower-thirds
```
A profile is a flat `key = value` file using the same names as the long
@@ -216,7 +284,9 @@ It is an ordinary foreground process:
* start it, and it opens fullscreen on the chosen output
* `SIGTERM` (or `SIGINT`/`SIGHUP`) makes it exit cleanly
* it never exits on its own because of a missing or broken source
* a missing or broken media source never closes the output window
* fatal configuration, display-initialisation, or NDI-runtime errors return a
nonzero status instead of retrying something that cannot recover
```bash
video-output multiview &
@@ -254,20 +324,26 @@ silently wrong monitor.
| Situation | What happens |
| --- | --- |
| Device missing at startup | one clear error, then quiet retries with backoff (250 ms → 5 s); the window stays up and black |
| Camera missing at startup | one clear error, then quiet retries with backoff (250 ms → 5 s); the window stays up and black |
| Device busy | says so explicitly, keeps retrying |
| Permission denied | tells you to join the `video` group |
| Device unplugged while running | `signal lost` logged once, black output, reopens automatically when it returns |
| Malformed frame | dropped, counted, stream continues |
| Camera unplugged while running | `signal lost` logged once, black output, reopens automatically when it returns |
| Named NDI source absent at startup | logs the search condition once, stays black, and keeps discovering in the background |
| NDI source appears | creates the receiver automatically and begins displaying valid video |
| NDI source disappears | marks signal lost, clears to black, rebuilds discovery/receive state, and reconnects when it returns |
| Malformed or unsupported frame | dropped and counted; the receive process continues and stale output is not left on screen indefinitely |
| NDI runtime missing or incompatible | the NDI command exits nonzero with the attempted runtime path/symbol in its error; non-NDI commands are unaffected |
| Fatal decoder error | the JPEG handle is rebuilt so one bad frame cannot poison the ones after it |
| Backend panics | caught, logged, backend restarted; the display never goes down with it |
| No valid frame | black |
Logging is edge-triggered — a camera unplugged for twenty minutes produces one
`signal lost` line, not one per retry — plus one throughput line per minute
(`--stats <seconds>`, `0` disables). Set `VIDEO_OUTPUT_LOG=debug` or
`--log debug` for per-frame diagnostics. Under systemd, journald severity
prefixes are emitted automatically so `journalctl -p warning` filters correctly.
Logging is edge-triggered — an input absent for twenty minutes produces a state
transition, not a line every frame or discovery poll — plus one throughput line
per minute (`--stats <seconds>`, `0` disables). Set `VIDEO_OUTPUT_LOG=debug` or
`--log debug` for additional discovery and recovery diagnostics. Frame problems
are counted and reported periodically instead of logged individually. Under
systemd, journald severity prefixes are emitted automatically so
`journalctl -p warning` filters correctly.
---
@@ -277,23 +353,28 @@ prefixes are emitted automatically so `journalctl -p warning` filters correctly.
cargo test
```
87 tests covering CLI and profile parsing, invalid configuration, display
selection (including the ambiguity rules), the frame slot's drop-stale
guarantee, backoff and signal-transition state, the V4L2 struct ABI against the
kernel headers, and real JPEG decoding of all four subsampling paths — plus
truncation and bit-flip fuzzing to prove a corrupt frame cannot panic or wedge
the decoder.
The hardware-independent suite covers CLI and profile parsing, invalid
configuration, deterministic display and NDI-source matching, reconnect policy,
NDI and V4L2 ABI layouts, malformed NDI layouts, both stale-frame drop points,
and the one-frame handoff. Process tests force an isolated missing NDI runtime
and verify both the clear NDI error and that a non-NDI command still succeeds;
on Linux, `readelf` also verifies that the executable has no `libndi` dynamic
dependency.
Camera coverage remains intact: real JPEG fixtures exercise all four
subsampling paths, with truncation and bit-flip fuzzing to prove that corrupt
input cannot panic or wedge the decoder.
Regenerate the JPEG fixtures with `python3 tests/data/generate.py` (needs
Pillow). They are solid green on purpose: green is what the ATEM keys on, so a
wrong YUV matrix or range shows up in a test rather than on stage.
### Manual validation on Fedora
### Manual camera validation on Fedora
Hardware paths that cannot be unit tested:
```bash
# 1. Devices and displays are discovered
# 1. Devices and displays are discovered.
video-output devices
video-output displays
@@ -308,7 +389,7 @@ video-output camera --device /dev/video0 --display <name> --stats 5
# 4. Recovery: unplug the ATEM's USB cable while it runs.
# Expect "signal lost", black output, then "signal restored" on replug,
# with no restart and no repeated log lines
# 5. Clean shutdown
# 5. Clean shutdown.
kill -TERM <pid> # exits promptly, no error
```
@@ -317,6 +398,45 @@ USB, 1920x1080 MJPEG @ 29.97 — sustained 30.0 fps decoded, 0 undecodable
frames, 0 stale buffers, 0 frames dropped before display, ~33% of one CPU core
and ~98 MB RSS.
### Manual NDI validation on Fedora
First verify that the runtime loads and record the exact advertised name:
```bash
video-output ndi-sources --timeout 10
```
If the sender requires unicast discovery:
```bash
video-output ndi-sources --timeout 10 --extra-ip <ProPresenter-IP>
```
Then run the exact name on the production lower-thirds output:
```bash
video-output ndi \
--source "<exact advertised ProPresenter source name>" \
--display "HXA BMD HDMI" \
--stats 5
```
Confirm all of the following:
1. The log names `HXA BMD HDMI`, the selected NDI source, and packed UYVY.
2. ProPresenter graphics appear fullscreen with the correct green background
and no controls or OSD.
3. Stopping the ProPresenter NDI output logs signal loss once and turns the
display black rather than freezing the last lower third.
4. Restarting the same output restores video without restarting
`video-output`.
5. Leaving the source unavailable at startup keeps the black window alive and
connects automatically when broadcasting begins.
At the last validation pass, the installed NDI 6.3.2 runtime loaded correctly
but no source was advertising. Discovery/retry and the live-video path should be
rechecked with ProPresenter broadcasting.
---
## Troubleshooting