Monitoring

If you have a large Nageru installation, you may want to monitor it remotely, as opposed to just eyeballing the UI and occasionally checking into the stream to verify everything is OK. As of version 1.6.1, Nageru supports native Prometheus metrics.

This section is not intended to explain Prometheus; for that, see the Prometheus documentation. You do not need a separate exporter for Nageru, you simply point Prometheus to the same HTTP port as is used for your stream, and it will fetch all metrics from the /metrics endpoint.

Nageru’s metrics export basically everything you can get from the UI and much more. Unfortunately, Prometheus is relatively inefficient when it comes to dealing with histograms, and many of the values Nageru tries to measure (like latency through the various points in the pipeline) can take on a quite wide range of values. Thus, most values that would normally be exported as Prometheus histograms are exported as summaries over one-minute sliding windows instead. This means they are highly precise, but unfortunately, it means you cannot aggregate them meaningfully.

Grafana dashboard

Nageru ships with an example Grafana dashboard exported in JSON format; it does not graph all metrics, but it should cover most of the interesting parts. Note that there is currently no guarantee of forward or backward compatibility in metrics, so if Nageru internals change significantly, some graphs could stop working. Of course, you can change the dashboard as you see fit; if you don’t use e.g. HDMI/SDI output, you will probably want to remove the panels related to it.

Tally display

Not related to Prometheus per se, but yet an important part of monitoring is the tally light, or just tally for short. The tally is a talkback signal from the mixer to the camera, showing the cameraperson that they are indeed on air (or are about to get to). Professional studio cameras will often not only have a tally light for the operator, but also for the subject (to know which camera to look into). However, most cheaper cameras will have none at all.

Nageru does not have functionality to talk to tally lights directly (there are too many kinds of interfaces and standards), but it does have functionality to expose tally data. Using this, you can build your own tally light, be in through injecting the right bits on an SDI cable, having a single-board computer set some GPIO pins for a LED, or simply show a red or green dot on a mobile phone with a web browser.

Nageru’s tally data is automatically exposed over HTTP, just as any other monitoring. You can get one JSON file with a list of all channels (at “http://yourserver.example.org:9095/channels”), or you can ask for a specific channel and get just its color in plain text (at “http://yourserver.example.org:9095/channels/2”, where 2 is the lowest channel ID, since live and preview don’t have tally). The tally color values are taken automatically from the theme’s channel_color function, so that they match the borders around the input in the producer’s mixer display—typically green for on preview and red for live. This also means they can be CSS textual colors (like “red”), although you can of course make the theme return only “#ff0000” or similar if this makes your tally application simpler.

Note that the tally endpoints have fully open CORS headers, so that they can be queried from anywhere; tally data is not sensitive, and this makes it significantly easier to query them from a web page.