[FFmpeg-devel] [PATCH] libavcodec: add bit-rate support to RoQ video encoder

[Victor Luchitz]

On Tue, Jan 23, 2024 at 8:44 PM Tomas Härdin <git at haerdin.se> wrote:

> tis 2024-01-23 klockan 11:33 +0300 skrev Victor Luchitz:
> > Re-posting the patch as an attachment. Sorry for the inconvenience!
>
> Yep, this one applies and also passes FATE.
>
> I notice ffplay_buffer whines a lot when playing RoQ files:
>
> > [swscaler @ 0x7fcb44043240] deprecated pixel format used, make sure you
> did set range correctly
> > [ffplay_buffer @ 0x7fcb44032fc0] filter context - w: 256 h: 256 fmt: 14
> csp: unknown range: unknown, incoming frame - w: 256 h: 256 fmt: 14 csp:
> unknown range: pc pts_time: 0
> > [ffplay_buffer @ 0x7fcb44032fc0] Changing video frame properties on the
> fly is not supported by all filters.
>
> I don't think this problem relates to this patch however.
>

It definitely doesn't. If we decide this needs ironing out, it'd go in a
separate
patch in my opinion.


>
> Anyway, using -b:v 100k causes the encoder to effectively become stuck
> on the first frame, being unable to go below 621 kbps and increasing
> qscale very slowly. But you mentioned this already of course. Perhaps
> there should be a faster "startup" phase? Subsequent frames being P-
> frames may lead to the average hitting the target bitrate.
>

I'm not sure we'd go as far as special-casing the encoder for extremely low
bit-rates at high image resolutions. I mean, if you want low picture
quality,
the proper way for RoQ would be to scale down your video to a lower
resolution
instead of turning the video into a huge ugly mess of macroblocks.

The main issues with allowing a larger keyframe and then trying to make it
up on subsequent frames are: doable for FIRST frame, may cause buffer
underrun on later keyframes; we are targeting CDROM media with this patch,
be it 1X, 2X, whatever. We cannot have too much variance in the frame sizes
(bigger). Smaller is allowable, but not bigger.

It's OK if encoding takes some time :)


> Even when using 1 Mbps the encoding is very slow:
>
> ./ffmpeg -r:v 30 -t 1 -f lavfi -i testsrc2 -s 256x256 -b:v 1000k -y
> foo-1000k.roq
>
> > [roqvideo @ 0x3112980]
> > Generated a frame too big for desired bit rate (1489 kbps), now
> switching to a bigger qscale value (257).
> > [roqvideo @ 0x3112980]
> > Generated a frame too big for desired bit rate (1489 kbps), now
> switching to a bigger qscale value (259).
> > [...]
> > Generated a frame too big for desired bit rate (1100 kbps), now
> switching to a bigger qscale value (196863).
> > [roqvideo @ 0x3112980] .0 size=       0kB time=N/A bitrate=N/A
> speed=N/A
> > Generated a frame too big for desired bit rate (1069 kbps), now
> switching to a bigger qscale value (262399).
> > [roqvideo @ 0x3112980]
> > Generated a frame too small for desired bit rate (601 kbps), reverting
> lambda and using smaller inc on qscale (196863).
> > [roqvideo @ 0x3112980] 0.0 size=      17kB time=00:00:00.16 bitrate=
> 812.7kbits/s speed=0.0476x
> > Generated a frame too small for desired bit rate (915 kbps), qscale
> value cannot be lowered any further (1).
> > [...]
> > [roqvideo @ 0x3112980]
> > Generated a frame too big for desired bit rate (1084 kbps), now
> switching to a bigger qscale value (327680).
> > [roqvideo @ 0x3112980]
> > Generated a frame too big for desired bit rate (1068 kbps), now
> switching to a bigger qscale value (393216).
> > [roqvideo @ 0x3112980] 0.0 size=      42kB time=00:00:00.40 bitrate=
> 866.5kbits/s speed=0.0667x
> > Generated a frame too small for desired bit rate (514 kbps), reverting
> lambda and using smaller inc on qscale (327680).
> > [roqvideo @ 0x3112980] 0.0 size=      62kB time=00:00:00.56 bitrate=
> 896.2kbits/s speed=0.0708x
> > Generated a frame too small for desired bit rate (901 kbps), qscale
> value cannot be lowered any further (1).
>
> This suggests an unstable regulation loop. It also makes the encoder
> slower than the Cinepak encoder, amazingly. 59.6 vs 21.57 seconds for
> encoding 90 frames of dimension 256x256.
>
> -bt should have a default of 0, and similar default logic as -b. The
> default of bitrate/20 also seems excessively small? If I change the
> default to be the same as the bitrate then the encoding time above
> shrinks to 11.7 seconds.
>

The reason for the default tight bitrate tolerance is explained above:
we're targeting slow 1x and 2x speed CD-ROM drives..

However, if you feel we need to change it to something else, that's fine,
as long as we can override it via the command line :)


>
> There is an excessive newline visible here:
>
> > +                   "\nGenerated a frame too big for desired bit rate
> > (%d kbps), "
>
> And elsewhere.
>
> Using non-integer framerates causes the encoder to go bananas. For
> example 24000/1001 fps makes the ratecontrol think it's running at
> 24000 fps. An easy fix is to reject avctx->time_base.num != 1 on init.
> roq_write_header() rejects non-integer framerates.
>

Seems to me like a somewhat generic cleanup change that shouldn't
necessarily go with this patch. I'm OK with bundling it with everything
else though.


>
> I'm also curious about this hunk:
>
> > -        if (enc->lambda > 100000) {
> > +        if (enc->lambda > 100000000) {
> >              av_log(roq->logctx, AV_LOG_ERROR, "Cannot encode video in
> Quake compatible form\n");
> >              return AVERROR(EINVAL);
> >          }
>
> Where in the Quake (3?) source code is this limitation? Seems rather
> that there should be a retry limit. There's probably no harm in keeping
> going so long as the packet doesn't end up above the limit.
>

In our tests we found that lambda can go way higher than the original
100000. Stopping at 100000 is artificially restricting yourself when you
can go much further. 100000000 is high enough to allow 1X  bitrates
even on extreme frames.


>
> A quick logarithmic regression on bitrate vs qscale suggests it scales
> with somewhere between qscale^-0.2833 and qscale^-0.2842. Conversely,
> to hit a specific bitrate, try scaling qscale with the bitrate ratio
> raised to around 3.5. A more conservative exponent like 2 is probably
> also fine. See patch attached.
>

Thanks so much for the patch! I've tested it on a pretty long hires video at
835kbps and everything seems to work fine. The runtime was 2m39s for
both: the original and your version.



>
> A more systematic approach might be to do a binary search for qscale,
> with an initial guess per the formula above.
>
> /Tomas
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>


-- 
Best regards,
 Victor Luchitz