[FFmpeg-devel] [PATCH] libavformat/mpegtsenc.c -- correctly re-emit extradata ahead of IDR pictures

[John Coiner]

On Sun, Feb 5, 2023 at 6:33 PM Marton Balint <cus at passwd.hu> wrote:
> Your code duplicates AUD-s if they are already present and sps is also
> added:
>
> if original packet had
> [AUD][IDR]
> it will become
> [AUD][SPS][PPS][AUD][IDR].
>
> Regards,
> Marton

Hi Marton,

You're right, this is not compliant. The H.264 spec (section
7.4.1.2.3) is clear about this, it says:
 - There should be at most one AUD per access unit
 - Each access unit should include a picture. (We can't produce a fake
"access unit" to carry the SPS and PPS alone; that's not allowed.)
 - When AUD is present, it should be the first NAL in the access unit.
(We can't just prepend SPS and PPS ahead of the AUD.)

Perhaps you knew all this; I am just learning. :-)

So if the original packet has [AUD][IDR], then we should produce
[AUD][SPS][PPS][IDR].

And if the original packet has [SEI][AUD][IDR], then we should produce
[AUD][SPS][PPS][SEI][IDR] -- chopping out the original AUD, and then
inserting the prefix with the new AUD. Tests show x264 producing the
[SEI][AUD][IDR] sequence, despite it being contrary to the spec's
recommendation that AUD should be first.

Does that seem like the correct handling? If so I'll post a v2 patch
for it. Thanks again.

John