[FFmpeg-devel] Network IO in FFmpeg (was: abstractable threading api)

[Nicolas George]

Le nonidi 29 frimaire, an CCXXII, Nicolas George a écrit :
> In a next mail, I will discuss why we want this thread for UDP sockets, and
> what other options are available.

So, why do we need a thread to receive UDP packets?

When data arrives for a socket, the kernel stores it in a buffer until the
userland application reads it. For UDP, if the buffer is full, packets are
discarded, and can not be recovered since UDP does not have flow control nor
retransmission.

The maximum size of the buffer seems to be around 300k, at least on Linux
(it could be higher with the help of root privileges, but we can not count
on it). That is about half a second of digital TV, unless I am mistaken. A
video processing application can easily be busy for that amount of time,
even if it is on average fast enough to work in real time.

Therefore, we need something whose task is to read from the socket as fast
as possible, storing the data into a larger buffer.

Doing much more work in this thread would clearly defeat the purpose. But
still, there may be room to make a little more work in it. A few ideas.


A thread for TCP too

TCP has the same kind of problem: the kernel buffer is the same size, and
the same treatment happens inside FFmpeg. The difference is that TCP has
flow control, therefore when the buffer is full, the data is not discarded.
But it can lead to other kinds of unpleasantness. The least of it would be
to achieve a lower throughput than actually possible. At worst, the flow
control could reach the sending application and cause it to lose some of its
input.


A thread for devices too

Some devices have the same kind of problem. ALSA may do buffer overruns,
x11grab may skip frames, etc. Unlike the protocols, the devices are at the
demuxer level, so it is not possible to handle everything together, but the
solution will likely be similar.

Also, note that ffmpeg itself implements threads for the demuxers. An API
for that in the library, with clean pause and shutdown, would be useful.


A single thread for all sockets

Having a whole thread for each socket may seem expensive, but the big 5
megabytes circular buffer kind of dwarfs it. Still, it may be beneficial
under heavy load on single-core systems, it would bear some benchmarking.


Integration in a generic event loop

Some projects may need to avoid uncontrollable threads and to integrate the
network IO in their own event loop. Making sure this is possible, at least
for well known event loop APIs (libev2 for example) would be a good idea.
That would probably require making ffurl_get_file_handle() public again (but
probably with some extensions), and adding a public API to tell a protocol
"do the shortest non-blocking operation you can".


Input-driven processing

If we have a thread, then we are notified as soon as data arrive on the
socket. We could add an API to register a callback to forward the
notification to the application immediately; the callback would have to be
fast, of course.

This would be especially beneficial for protocols that use several
underlying sockets: they all implement their own poll()-based stuff. It is
an actual miracle that they work more or less reliably with that poll() and
the UDP thread at the same time.

Input-driven processing with a callback would simplify greatly all that, and
probably make it more efficient too.

(There is the small issue of getting it to work without threads too, but I
believe it can be managed.)

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20131223/08d325ac/attachment.asc>