[FFmpeg-devel] headers for gapless playback

[Tomas Härdin]

mån 2019-11-25 klockan 22:53 +0100 skrev Marton Balint:
> 
> On Sun, 24 Nov 2019, Tomas Härdin wrote:
> 
> > tor 2019-11-21 klockan 23:35 +0100 skrev Marton Balint:
> > > On Thu, 21 Nov 2019, Tomas Härdin wrote:
> > > 
> > > > tis 2019-11-19 klockan 19:16 +0000 skrev John Kaplan:
> > > > > Hi Guys,
> > > > > 
> > > > > We just did some research on gapless playback across multiple client
> > > > > devices, and could not find support for adding gapless metadata
> > > > > headers in ffmpeg. Our interest is specifically in AAC/MP4, and we
> > > > > ended up shifting to the fdk-aac package, which exposes options for
> > > > > adding moov/edts/elst headers (which are the ones we used) as well as
> > > > > iTunSMPB.
> > > > > 
> > > > > So, I wanted to open a discussion with the team on ffmpeg support for
> > > > > gapless metadata.
> > > > > I’m not sure if you guys have discussed before, so here are a few
> > > > > opening questions to get me up to speed…
> > > > > 
> > > > > 
> > > > >   1.  Does ffmpeg support addition of gapless metadata in any form
> > > > > currently, and we missed the doc as to how to specify it?
> > > > >   2.  If not, have you discussed previously? If so, what were the
> > > > > results? (I.e. rejected as evil, put on roadmap but in future,
> > > > > currently in development, etc.)
> > > > 
> > > > Something very similar to this is being discussed in 
> > > > https://trac.ffmpeg.org/ticket/8366
> > > > 
> > > > >   3.  If support is still in the future, would you be open to me
> > > > > contributing a patch that implements it?
> > > > 
> > > > Yes please
> > > > 
> > > > I think opening a generic ticket for this would be a good move. Having
> > > > the ability to trim essence would be very useful. Then we could also
> > > > remove that godawful hack in mov.c as well, hopefully. There's more
> > > > than a few formats that need some "warm-up" data or that leave a few
> > > > frames of garbage at the end of the file, like all block transform
> > > > audio codecs
> > > 
> > > For audio we have AV_PKT_DATA_SKIP_SAMPLES packet side data
> > 
> > This is not enough for sample-level cutting as needed for gapless
> > playback
> 
> Why not? The side data contains the number of samples to be skipped from 
> the start and at the end of the packet.

Aha, I missed that. That should work I guess. I still feel we should
have an explicit "the actual media is between these timestamps"

> > > for video AV_PKT_FLAG_DISCARD packet flag. What else is needed?
> > 
> > Why is this on the packets? It's not the demuxer's job to decide what
> > should and should not be cut.
> 
> The demuxer only signals this, frames are dropped after they are decoded 
> in generic libavcodec decode.c.

That's OK at least. But see my previous comment.

/Tomas