[FFmpeg-devel] [PATCH 000/279] New channel layout API

[Marton Balint]

On Thu, 9 Dec 2021, Anton Khirnov wrote:

> Quoting Nicolas George (2021-12-09 11:31:54)
>> Anton Khirnov (12021-12-09):
>>> I disagree. Technical limitations that were overcome 10 years ago should
>>> not guide new API design.
>>
>> In the case of amerge, it was not a technical limitation, merging
>> several streams into one so that they can be handled by single-stream
>> filters is 100% part of the design.
>
> I fail to see how that is an advantage. You can just as well create
> multiple instances of those single-stream filters instead of adding
> hacks into core APIs.
>
>> I suspect devices that capture several independent channels are
>> designed that way intentionally too, possibly to reduce the risk of
>> desynchronization.
>
> "possibly" is not a strong enough argument. I'd like to hear at least
> one clearly-defined use case that cannot just as well be handled by
> using multiple streams.

I recently worked on the MXF demuxer to recognize channel designations in 
MXF files, and in MXF the designation and grouping of the channels is 
completely separate from the track those channels are muxed in.

So if you have e.g. english stereo sound, and german stereo sound you 
can mux it
- as a single 4 channel track
- as two 2 channel tracks
- as four 1 channel tracks.
Some MXF flavors use the multichannel single track approach, others the 
mono-track-only approach. So the user may not be able to choose the 
optimal muxed track assignment...

So ultimately, if you demux and decode a packet from a track, you will 
have an AVFrame, which might contain a single sound group on its own 
(optimal case), part of a sound group, or multiple sound groups.

To summerize, muxed tracks are not necessarily 1:1 mapping between sound 
groups. And when processing/filtering audio, you typically want sound 
groups, not tracks. And yes, it is very rare to have a soundgroup which 
has channels with the same designation, but for a muxed track, it depends 
on the format.

The goal of the end user is probably to be able to specify sound groups, 
not select muxed tracks. Preferably a demuxer should provide which channel 
is part of which sound group, and you should be able to use a filter or a 
combination of filters select a specific sound group. E.g. amerge all 
tracks, then keep only the channels from all the merged channels which are 
part of a specific sound group.

Regards,
Marton