[FFmpeg-devel] [PATCH 2/2] avformat/movenc: parse h264 packets to build Sync Sample and Recovery Point tables

[Michael Niedermayer]

On Thu, Jul 29, 2021 at 03:09:03PM -0300, James Almer wrote:
> On 7/29/2021 2:58 PM, Michael Niedermayer wrote:
> > On Tue, Jul 27, 2021 at 10:08:20AM -0300, James Almer wrote:
> > > Since we can't blindly trust the keyframe flag in packets and assume its
> > > contents are a valid Sync Sample, do some basic bitstream parsing to build the
> > > Sync Sample table in addition to a Random Access Recovery Point table.
> > > 
> > > Suggested-by: ffmpeg at fb.com
> > > Signed-off-by: James Almer <jamrial at gmail.com>
> > > ---
> > >   libavformat/movenc.c         | 125 +++++++++++++++++++++++++++++++++--
> > >   libavformat/movenc.h         |   1 +
> > >   tests/ref/lavf-fate/h264.mp4 |   6 +-
> > >   3 files changed, 123 insertions(+), 9 deletions(-)
> > 

A.
> > Will this allow a user to mark a subset of keyframes as random
> > access points ?
> > A user may want seeking to preferably happen to a scene start and not a
> > frame before
> > 

B.
> > Will this allow a user to mark a damaged keyframe as such ?
> > A frame might in fact have been a keyframe and maybe the only in the file
> > but maybe was damaged so a parser might fail to detect it.
> 
> This code will ensure all packets with an IDR picture will be listed in the
> Sync Sample table, and all packets with a Recovery Point SEI message will be
> listed in the Random Access Recovery Point table.
> Whatever is signaled in packets (like the keyframe flag) is ignored to
> prevent creating non-spec compliant output.

The file in case B will be non compliant, it is damaged data, it cannot
be muxed in a compliant way. If we support muxing damaged data then 
parsing that data as a way to unconditionally override the user is
problematic.
If you try to interpret damaged data the result is undefined, the spec
does not tell you how to interpret it and 2 parsers can disagree
if a damaged frame is a keyframe. 

Lets just as 2 examples consider multiple slices some being parsed as IDR
some as non IDR, so what is that ? or what if some packet reodering or
duplication causes parts of a IDR and non IDR frame to be mixed.
where is the IDR frame in that case ? 2 parsers can disagree
still a decoder can with some luck start decoding from such a
point i would guess. But does the user want this marked as a keyframe ?
maybe yes, maybe no. I think taking the users only way to set the keyframe
flag away is a step backward

Thanks

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is what and why we do it that matters, not just one of them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210730/837cf6a6/attachment.sig>