[Ffmpeg-devel] Interleaving audio and video

[Michael Niedermayer]

Hi

On Fri, Feb 02, 2007 at 02:40:01PM +0100, Baptiste Coudurier wrote:
> Michael Niedermayer wrote:
> > Hi
> > 
> > On Fri, Feb 02, 2007 at 12:34:10PM +0100, Baptiste Coudurier wrote:
> >> Michael Niedermayer wrote:
> >>> Hi
> >>>
> >>> On Thu, Feb 01, 2007 at 09:57:15AM +0100, Baptiste Coudurier wrote:
> >>>> Michael Niedermayer wrote:
> >>>>> Hi
> >>>>>
> >>>>> On Wed, Jan 31, 2007 at 08:10:28PM -0800, Roman Shaposhnik wrote:
> >>>>>> While implementing an idea suggested by Michael for restructuring DV 
> >>>>>> audio muxer I've come to realize that DV muxer currently expects a 
> >>>>>> caller to take care of interleaving audio and video properly based
> >>>>>> on PTS valued. Now we know that ffmpeg.c does it, but is it fair
> >>>>>> to require that kind of behaviour in general of all users of 
> >>>>>> libavformat ?
> >>>>>>
> >>>>>> Or to put it differently, do I have to care about caller giving 
> >>>>>> me the following pattern of DV audio/video packets:
> >>>>> see av_interleaved_write_frame() (the docs above it)
> >>>>> and AVOutputFormat.interleave_packet()
> >>>>>
> >>>> Would it be same to force using interleave_packet for some formats ?
> >>>>
> >>>> DV,GXF,MXF always require audio to be specially packed, and therefore
> >>>> won't support passing 48000 samples packet like MOV can contain.
> >>> interleave_packet() wasnt designed to split and merge packets which seems
> >>> to be what you want?
> >> well, when I coded gxf muxer, which needs 65536 samples per packet, you
> >> suggested me to use interlave_packet, I assumed that it was the correct
> >> way to interleave samples.
> > 
> > hmm, that must have been a missunderstanding, sorry, anyway i see no
> > serious problem with using interleave_packet() to chopup and merge PCM
> > packets its just not designed for that
> > 
> > 
> >>> and about huge PCM packets, these should be never generated (=the demuxer
> >>> should split them) the reason is simply memory requiremnt, audio buffering
> >>> issues for playback and seeking granularity if the pcm is streamcopied
> >>> if 48k samples is huge or not is another question, 10mb audio packets
> >>> definitly are to large though and yes files which such audio packets
> >>> exist
> >> Maybe lavf need AVPacketizer ?
> >>
> >> MXF specs enforce pcm audio samples packets at 48khz to
> >> 2000/1920/1601,2,1 for 24/25/29.97 fps, and so on, which ensures
> >> perfect(almost for NTSC) sync.
> >>
> >> That should be also perfect for other formats, there is no framer for
> >> pcm atm, what do you suggests ?
> >>
> >> Add option to ffmpeg.c ? Set enc->frame_size to 1920 or so, though that
> >> will perturb mov muxer for example which expect frame_size to 1 for pcm.
> > 
> > if mxf/gxf are the only containers havig such odd limitations then the code
> > to deal with it should be in them (its already there i assume and so its
> > the least work and seems simplest)
> > 
> > also keep in mind that in reality audio and video come from different
> > hardware and their sampling intervals are not synchronized so the idea
> > of X audio samples per video sample is broken by design, its not a
> > problem in reality as audio can be resampled video frames can be droped
> > or duplicated but if your goal is good quality then such a limitation is
> > unaccpetable, iam really curious why formats like that are being designed
> > while much simpler, more advanced and less retarded formats already 
> > exist
> 
> Which ones if you would name them ? Im curious, and audio framing is
> always a serious problem in every workflow.

well it depends on the exact requirements
for hardware stuff and if overhead and muxing complexity arent a problem then
simple mpeg-ps seems like a reasonable choice
otherwise theres nut and matroska or even mov
avi doesnt have timestamps so its no option here (our disscussion after all is
timestamp related ...) and asf is unnecesarily complex though that applies to
mov and matroska too ...

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Breaking DRM is a little like attempting to break through a door even
though the window is wide open and the only thing in the house is a bunch
of things you dont want and which you would get tomorrow for free anyway
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070202/ec855b42/attachment.pgp>