Implement Phase 4 integrations and display telemetry
Add MQTT playback control, durable Home Assistant incident notifications, and display heartbeat telemetry on top of the verified Phase 3B playback authority. No playback logic is duplicated: every MQTT command and every observed condition goes through the existing playback service, and all integration work happens after the authoritative SQLite commit. MQTT (server/src/mqtt) subscribes the exact required command topics under a configurable prefix that defaults to announcements. Durable commands use QoS 1; next and previous stay at QoS 0 because they are not idempotent and carry no command ID. Retained authoritative state is published only on the separate availability, mode/state, and playback/state topics, never onto a command topic, and only after a mutation commits. A registered last will reports an ungraceful disappearance, a clean shutdown retracts it, and a broker outage degrades status while playback, HTTP, and the display protocol continue untouched. Notifications (server/src/notifications) use the Phase 1 incident and outbox tables. A failing condition opens one incident and queues exactly one notification; continued failure only updates its observation time; recovery queues exactly one recovery; a later failure is a new episode. Delivery retries on a documented capped schedule with a stable notification ID, survives restart, and never blocks playback. With no webhook URL configured, incidents are still recorded but no delivery work is queued. Conditions come from storage health transitions, committed playback state, display heartbeat staleness, and newly failed uploads. Displays (server/src/displays) ingest validated, bounded display:heartbeat telemetry keyed by persistent client ID rather than socket ID. Telemetry is read-only and never touches playback or playlist revisions. Offline is decided by heartbeat staleness, not by a closed socket, and a startup grace period keeps an API restart from manufacturing offline alerts. Adds GET /api/displays and meaningful-change display:status events. Health and status now report redacted storage, MQTT, notification, playback, and display state. External integrations degrade status and never affect readiness, so no integration outage can cause a restart loop. No MQTT password, S3 credential, webhook URL, or webhook secret is exposed. Uses the existing Phase 1 schema unchanged; no migration was required. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
@@ -181,6 +181,186 @@ exit static safely, return to normal rotation, and only then delete the asset.
|
||||
Phase 3B does not depend on the refusal, so the change belongs to the later
|
||||
phase that revisits asset lifecycle.
|
||||
|
||||
## MQTT integration
|
||||
|
||||
The API connects to `MQTT_URL` when it is configured and stays disabled when it
|
||||
is not. Commands are dispatched straight into the authoritative playback
|
||||
service, so an MQTT command and its REST equivalent are the same operation with
|
||||
the same semantics; no playback logic is duplicated in the integration.
|
||||
|
||||
With the default `MQTT_TOPIC_PREFIX` of `announcements` the command topics are:
|
||||
|
||||
| Topic | Payload | QoS |
|
||||
| ------------------------- | ---------------------- | --- |
|
||||
| `announcements/mode` | `on` or `off` | 1 |
|
||||
| `announcements/pause` | ignored | 1 |
|
||||
| `announcements/resume` | ignored | 1 |
|
||||
| `announcements/reload` | ignored | 1 |
|
||||
| `announcements/static` | LibraryAsset UUID | 1 |
|
||||
| `announcements/next` | ignored | 0 |
|
||||
| `announcements/previous` | ignored | 0 |
|
||||
|
||||
Mode payloads are trimmed and case-insensitive. `off` is only the visual black
|
||||
gate; it never pauses, restarts, or resets the playlist, cursor, static hold, or
|
||||
timing. Static accepts only a stable LibraryAsset UUID: URLs, filenames, and S3
|
||||
object keys are rejected. The asset must exist, be `ready`, and be unexpired,
|
||||
but it does not need to appear in any playlist. A rejected command is logged as
|
||||
a structured warning, changes nothing, and publishes no state.
|
||||
|
||||
`next` and `previous` stay at QoS 0 on purpose. They are not idempotent and
|
||||
carry no command ID, so an at-least-once redelivery would silently skip a
|
||||
slide. Commands are never retained, and the server never publishes onto a
|
||||
command topic.
|
||||
|
||||
The server publishes retained state on separate topics:
|
||||
|
||||
| Topic | Payload |
|
||||
| --------------------------------- | ---------------------------------- |
|
||||
| `announcements/availability` | `online` / `offline` |
|
||||
| `announcements/mode/state` | `on` / `off` |
|
||||
| `announcements/playback/state` | small versioned JSON snapshot |
|
||||
|
||||
Availability describes this API's MQTT integration, not a display. A registered
|
||||
last will and testament lets the broker publish `offline` when the API
|
||||
disappears without a clean disconnect; a planned shutdown retracts it directly.
|
||||
The compact playback topic carries mode, transport, active playlist, current
|
||||
entry and asset, static asset, and revision. It never carries the Socket.IO
|
||||
manifest or any private storage detail.
|
||||
|
||||
State is published only after a playback mutation has committed, and MQTT
|
||||
publication happens after the SQLite commit. A publish failure degrades the
|
||||
integration and is never allowed to roll playback back. A broker outage leaves
|
||||
the API, HTTP, and playback untouched: the client reconnects on its own, then
|
||||
resubscribes and republishes availability and authoritative state.
|
||||
|
||||
## Home Assistant notifications
|
||||
|
||||
Announcements sends outgoing generic webhook POSTs to `HA_WEBHOOK_URL`. Home
|
||||
Assistant is only the receiver; Announcements never calls Companion directly and
|
||||
needs no Home Assistant authentication beyond the configured webhook. The URL
|
||||
and `HA_WEBHOOK_SECRET` are configuration, never API or status output.
|
||||
|
||||
Notifications are durable and deduplicated through `notification_incidents` and
|
||||
`notification_outbox`. A condition that starts failing opens one incident and
|
||||
queues exactly one failure notification. While it keeps failing only its
|
||||
observation time is updated, so a thirty-minute outage still sends one message.
|
||||
Recovery closes the incident and queues exactly one recovery notification. If
|
||||
the same condition fails again later that is a new episode with its own
|
||||
failure/recovery pair. Open incidents and undelivered notifications survive an
|
||||
API restart.
|
||||
|
||||
Reported conditions are `storage.unavailable` / `storage.recovered`, driven by
|
||||
the Phase 2 storage health transition rather than individual request failures;
|
||||
`display.offline` / `display.recovered` per display client; and
|
||||
`playback.no_playable_slides` / `playback.playable` from committed playback
|
||||
state. A static override does not suppress the playback condition, because
|
||||
holding one asset does not repair an empty playlist. A failed upload opens one
|
||||
`asset.processing_failed` warning per asset; an individual failed upload has no
|
||||
automatic recovery, so the operator resolves it by deleting or re-uploading.
|
||||
|
||||
The payload is versioned:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"notificationId": "stable-delivery-id",
|
||||
"incidentId": "stable-incident-id",
|
||||
"event": "storage.unavailable",
|
||||
"severity": "critical",
|
||||
"state": "failure",
|
||||
"message": "Announcement storage is unavailable.",
|
||||
"timestamp": "2026-08-16T12:00:00.000Z",
|
||||
"context": { "conditionKey": "storage", "episode": 1 }
|
||||
}
|
||||
```
|
||||
|
||||
Severities are `info`, `warning`, `critical`, and `recovery`. Home Assistant can
|
||||
treat them all as ordinary notifications; the field exists so later Ekklesia or
|
||||
HA handling can discriminate.
|
||||
|
||||
Delivery is asynchronous and bounded. A 2xx marks the notification delivered.
|
||||
Any other response and any transport failure leaves it pending and retries after
|
||||
30 seconds, 1 minute, 5 minutes, 15 minutes, and 30 minutes, then every 30
|
||||
minutes. The notification ID is stable across retries so a receiver can
|
||||
deduplicate, retries resume after a restart, and a failing webhook never blocks
|
||||
playback, MQTT, storage, or heartbeats. When no webhook URL is configured,
|
||||
incidents are still recorded but no delivery work is queued, and status reports
|
||||
the integration as disabled.
|
||||
|
||||
An example Home Assistant receiver:
|
||||
|
||||
```yaml
|
||||
automation:
|
||||
- alias: Announcements Notifications
|
||||
triggers:
|
||||
- trigger: webhook
|
||||
webhook_id: announcements
|
||||
allowed_methods:
|
||||
- POST
|
||||
local_only: true
|
||||
|
||||
actions:
|
||||
- action: notify.mobile_app_YOUR_DEVICE
|
||||
data:
|
||||
title: "Announcements"
|
||||
message: "{{ trigger.json.message }}"
|
||||
```
|
||||
|
||||
Substitute any notify target; Companion is not required. Announcements cannot
|
||||
send a webhook while the Announcements API itself is completely down. That is a
|
||||
limitation of self-reporting, not a bug: if server-down alerting is wanted, an
|
||||
external monitor such as Home Assistant should poll `/health/live`.
|
||||
|
||||
## Display telemetry
|
||||
|
||||
The future `/display` client reports read-only telemetry over the Socket.IO
|
||||
event `display:heartbeat`, expected every 10-15 seconds:
|
||||
|
||||
```json
|
||||
{
|
||||
"protocolVersion": 1,
|
||||
"clientId": "persistent-display-id",
|
||||
"outputId": "global",
|
||||
"activePlaylistId": "playlist-id",
|
||||
"playlistRevision": 12,
|
||||
"playbackRevision": 42,
|
||||
"currentEntryId": "entry-id",
|
||||
"currentAssetId": "asset-id",
|
||||
"cachedAssetCount": 10,
|
||||
"playableAssetCount": 10,
|
||||
"mostRecentError": null
|
||||
}
|
||||
```
|
||||
|
||||
Every field is validated and bounded, `mostRecentError` is truncated, and
|
||||
malformed telemetry is ignored without creating a display. Heartbeats can never
|
||||
mutate playback and never increment a playback or playlist revision. Identity is
|
||||
the persistent `clientId` stored in `display_clients`, never a Socket.IO
|
||||
connection ID, so a browser reconnect remains the same display.
|
||||
|
||||
A display is online while heartbeats are arriving and offline once
|
||||
`DISPLAY_OFFLINE_THRESHOLD_MS` (45 seconds, about three missed heartbeats) has
|
||||
passed since the last one. A closed socket alone never marks a display offline,
|
||||
because reconnects and brief network hiccups are normal. Crossing the threshold
|
||||
opens one `display.offline` incident that does not repeat while the display
|
||||
stays away, and the next valid heartbeat recovers it exactly once.
|
||||
|
||||
Display history is persisted, so a restart does not erase monitoring. The API
|
||||
cannot know what happened to a display while the API itself was down, so it is
|
||||
deliberately conservative: nothing is reported offline until a full threshold has
|
||||
elapsed since startup. A healthy display that reconnects during that window is
|
||||
never reported, a display that is still missing afterwards is reported once, and
|
||||
an incident that was already open before the restart is not duplicated.
|
||||
|
||||
`GET /api/displays` returns each known display with a status derived from
|
||||
heartbeat freshness rather than mere database existence, and exposes no socket
|
||||
internals. `display:status` is emitted only for meaningful changes — first seen,
|
||||
offline, recovered, or a changed error — so steady telemetry creates no realtime
|
||||
churn. `mostRecentError` is telemetry only; arbitrary browser strings never
|
||||
become Home Assistant notifications. Structured display-side errors such as
|
||||
decode failures arrive with the Phase 5 renderer, which will report them through
|
||||
this same field and event.
|
||||
|
||||
## Storage and health
|
||||
|
||||
The API uses a small path-style S3 adapter configured by `S3_ENDPOINT`,
|
||||
@@ -193,3 +373,11 @@ Storage outages do not stop the API or fail process readiness. `/health/ready`
|
||||
reports storage as degraded while remaining HTTP 200 when SQLite and the API
|
||||
runtime are healthy. `/api/health` and `/api/status` expose cached, redacted
|
||||
storage state and recover after a successful probe or storage operation.
|
||||
|
||||
`/health/live` reports only that the process is running. Readiness stays limited
|
||||
to SQLite, migrations, and playback initialization. Every external integration
|
||||
degrades the reported status instead of failing readiness, so a broker, webhook
|
||||
receiver, or display problem can never put the API into a restart loop.
|
||||
`/api/health` and `/api/status` report redacted storage, MQTT, notification,
|
||||
playback, and display state. No MQTT password, S3 credential, webhook URL, or
|
||||
webhook secret appears in any response.
|
||||
|
||||
Reference in New Issue
Block a user