[FFmpeg-devel] [PATCH 0/6] WebRTC sub-second live streaming support

[Michael Riedl]

This patch series adds support for WHIP and WHEP (WebRTC-HTTP ingestion protocol
and WebRTC-HTTP egress protocol). It supersedes the previous patch series for
WHIP.

The WHIP and WHEP are defined in the following draft RFCs:
- WHIP: https://datatracker.ietf.org/doc/draft-ietf-wish-whip
- WHEP: https://datatracker.ietf.org/doc/draft-murillo-whep/

The implementation builds on FFmpeg's existing RTP, HTTP and SDP support as well
as libdatachannel for the rest. This library is much more lightweight than other
libraries (e.g. libwebrtc). At the same time, using this library avoids
reimplementing parts of WebRTC in FFmpeg.

This patch series was tested with WebRTC servers Dolby.io (formerly Millicast)
and SRS (https://github.com/ossrs/srs). Using a local server, an end-to-end
latency of 3 frames (50 ms at 60 fps) was measured with video-only output, and
12 frames (200 ms at 60 fps) with audio and video output. Using a DeckLink input
device, an end-to-end latency of about 4 frames (70 ms at 60 fps) was measured
with both, video-only and audio and video output.

Using a remote server, only the RTT is added to the end-to-end latency. For
example, using a server in Amsterdam (The Netherlands) from a location in
Salzburg (Austria), with RTT=18ms, an end-to-end latency of 50+18=68 ms was
measured for video-only output.