Stream monitoring (agent)

Observare's cloud-managed stream monitors can watch any stream the public internet can reach. But plenty of streams never face the internet: an encoder's output on a private VLAN, an origin server sitting behind the CDN, a packager inside a Docker network, a test channel on a staging box. And even for public streams, there's a tier of checking the cloud deliberately never does — downloading actual video and measuring it.

Agent-executed stream monitors cover both. The Linux agent runs the same HLS / DASH / MP4 liveness check as the cloud, from inside your network — and can optionally go much deeper.

Two tiers

Playlist tier (the default)

The agent fetches your playlist or manifest, parses it, and reports the stream's position — media sequence and newest segment for HLS, a manifest fingerprint for live DASH. The server compares that position against the previous check: a live stream whose playlist has stopped advancing is a frozen stream, even though the URL still returns HTTP 200. That's the failure mode no plain uptime check can see, and it works identically whether the check ran from the cloud or from your agent.

At this tier only playlist and manifest text is ever fetched — never media segments. Segment requests are indistinguishable from playback, so on streams with session tokens or concurrent-connection limits they would either fail spuriously or count your monitoring as an extra viewer. A playlist check doesn't count as a play, ever.

Deep inspection (opt-in, per monitor)

When you enable deep inspection on a monitor, the agent additionally downloads the newest segment after each successful playlist check and measures it with ffprobe and ffmpeg:

• Measured resolution, frame rate, and bitrate — what the video actually is, not what the playlist advertises.
• Video and audio codecs.
• Black-frame detection — the stream is "up" but transmitting black.
• Frozen-frame detection — the picture has stopped moving.
• Silent-audio detection — the video plays but the sound is gone.

This is the class of failure that's invisible to every delivery-level check: the encoder is running, segments are flowing, HTTP is green — and the channel is showing a black screen with no audio. Only decoding the video catches it.

The results appear on the monitor's detail page in a Deep Inspection card — measured resolution, fps, bitrate, codecs, and Picture / Audio status, which turn red when a problem is detected. In this release, quality findings are informational: a black or silent segment doesn't mark the monitor down, it flags the card so you can see it at a glance.

What deep inspection requires

ffprobe and ffmpeg on the agent host's PATH — both, since metadata comes from ffprobe but the black/freeze/silence detections are ffmpeg filters. On Debian/Ubuntu:

sudo apt install ffmpeg

The agent looks for both binaries once at startup and logs what it found. If either is missing, monitors with deep inspection enabled simply run at the playlist tier — the agent never fails a check because the tooling is absent, and a probe error on a segment never fails the monitor either. The whole deep pass is budgeted at about ten seconds and one segment per check, and its duration is excluded from the monitor's recorded response time.

Creating an agent stream monitor

Agent stream monitors are created from the agent host, via the interactive config menu:

observare config

Choose Add stream monitor and fill in:

• Friendly name — 1–100 characters.
• URL — the playlist, manifest, or file URL (e.g. https://10.0.0.9/live/master.m3u8). Anything the agent host can reach, including addresses the cloud never could.
• Format — auto (default), hls, dash, or mp4. Auto-detection checks the URL extension, then the response content type, then the body itself.
• Check interval — 5, 10, or 15 minutes.
• Timeout — 5–120 seconds, default 30.
• Deep inspection — yes/no. If you say yes on a host without ffprobe/ffmpeg, the prompt warns you and the monitor runs playlist-only until the tools are installed.
• Alert channels — any channels you've configured in the web UI.

The agent uploads the monitor and starts checking within 30 seconds. From then on it behaves like any other stream monitor: it appears on the Stream monitoring page with an agent badge (and a deep marker when deep inspection is on), can be renamed, re-intervalled, paused, or have its channels and deep-inspection setting changed from the web UI, and can appear on your status pages.

observare list shows stream monitors with their format and a [deep] marker; observare remove <id> deletes one.

How results are evaluated

The agent is deliberately stateless: it reports what it saw, and the server — which remembers the previous check's position — decides whether the stream advanced. That means the server's verdict can differ from what the agent submitted: a live playlist that hasn't moved is submitted as a successful fetch and evaluated as a stale_playlist failure. observare status shows the server's verdict, which is the one that drives incidents and alerts.

Failure handling matches every other monitor type: the agent rechecks locally 30 seconds after a failure and only submits the confirmed result, so one bad fetch doesn't page you. Incidents, recovery alerts, and flapping detection all work exactly as they do for cloud stream monitors.

Failure types

The playlist tier reports the same delivery errors as the cloud checker — http_error, invalid_playlist, empty_playlist, variant_unreachable, invalid_mp4, invalid_manifest, unknown_format, plus the network family (timeout, dns_failure, connection_refused, ssl_error, connection_error). The staleness verdicts (stale_playlist, stale_manifest) come from the server's comparison. See the stream monitoring reference for what each one means.

Cloud or agent?

• Public stream, delivery checking is enough → cloud monitor. Nothing to install.
• Private stream (origin, encoder output, internal packager) → agent monitor. The cloud can't reach it.
• You want quality measurement (real bitrate, black/frozen/silent detection) → agent monitor with deep inspection. The cloud never downloads video by design — the heavy work happens on your hardware, where it belongs.
• Session-locked or viewer-capped stream → either tier's playlist check is safe; just keep deep inspection off.

A monitor belongs to whichever side created it: web UI monitors are checked from the cloud, observare config monitors are checked by that agent. Both count toward the same 30-stream-monitor allowance.