[FFmpeg-devel] [PATCH] avformat: AMQP: add option delivery_mode

[Andriy Gelman]

On Sat, 20. Jun 12:28, Florian Levis wrote:
> (re-sending, forgot list, sorry Andriy for double sending)
> 
> Hi Andriy,
> 
> > Is the goal of the non-persistent option to reduce disk io, or something
> else?
> > I ran a quick test to look at disk io, but didn't see a difference.
> 
> TL;DR
> -> it have effect on the broker (ex: RabbitMQ)
> -> it should not have effect on the client (ffmpeg)
> 
> Explanations :
> 
> For RabbitMQ
> - if the message is short (default 4 kbytes), it stays in RAM
> - if not, it write it on disk, unless :
> ** the queue is flagged as non durable (queue not recreated if the node
> reboot) (not sure about it, can't find it in the doc)
> ** the message itself is flagged as non-persistent, even if the queue is
> durable
> 
> The informations I gave above are from RabbitMQ documentation
> - part from here https://www.rabbitmq.com/memory-use.html#breakdown-queues
> - here too https://www.rabbitmq.com/persistence-conf.html
> 
> In my case :
> - I read video streams, full fps (25), extracting each frame and sending
> them to queue
> - I can't use something like webp because encoding is too long (no GPU
> available)
> - As a result, I have to use JPG, resulting in serialized message ~40kbytes
> (If i'm right, RMQ store them as base64 encoded)
> - As a result, with delivery_mode=2, the system have a lot of I/O pressure
> (lot of iowait)
> 
> -> delivery_mode=2 (non-persistent) solved the problem

Thanks for explanation.
I also had to set lazy mode on the queue. Otherwise all messages were in memory for
me.

> > >
> > > + at item delivery_mode @var{mode}
> > > +Sets the delivery mode.
> > > +
> >
> > > +The following values are recognized:
> > > + at table @samp
> > > + at item persistent
> > > +Delivery mode set to "persistent" (2). This is the default value.
> > > +
> > > + at item non-persistent
> > > +Delivery mode set to "non-persistent" (1)
> > > +
> > > + at end table
> > > +

Please mention the difference: i.e. for the persistent mode the messages may be written
to disk depending on server's setup. For the non-persistent mode the queued
messages are in memory.

> > > +    { "delivery_mode",  "Delivery mode", OFFSET(delivery_mode),
> > AV_OPT_TYPE_INT, { .i64 = AMQP_DELIVERY_PERSISTENT }, 1, 2, .flags = E,

> > "delivery_mode"},
> > > +    { "persistent",     "persistent delivery mode",     0,
nit:                            ^capitalize                             

> > AV_OPT_TYPE_CONST, {.i64=AMQP_DELIVERY_PERSISTENT }, 0, 0, E,
                        ^add a space. same for below.

> > "delivery_mode" },
> > > +    { "non-persistent", "non-persistent delivery mode", 0,
> > AV_OPT_TYPE_CONST, {.i64=AMQP_DELIVERY_NONPERSISTENT }, 0, 0, E,

Thanks,
-- 
Andriy