[FFmpeg-devel] [PATCH 000/279] New channel layout API

[Marton Balint]

On Sun, 12 Dec 2021, Anton Khirnov wrote:

> Quoting Marton Balint (2021-12-10 01:04:57)
>>
>>
>> On Thu, 9 Dec 2021, Anton Khirnov wrote:
>>
>>> Quoting Nicolas George (2021-12-09 11:31:54)
>>>> Anton Khirnov (12021-12-09):
>>>>> I disagree. Technical limitations that were overcome 10 years ago should
>>>>> not guide new API design.
>>>>
>>>> In the case of amerge, it was not a technical limitation, merging
>>>> several streams into one so that they can be handled by single-stream
>>>> filters is 100% part of the design.
>>>
>>> I fail to see how that is an advantage. You can just as well create
>>> multiple instances of those single-stream filters instead of adding
>>> hacks into core APIs.
>>>
>>>> I suspect devices that capture several independent channels are
>>>> designed that way intentionally too, possibly to reduce the risk of
>>>> desynchronization.
>>>
>>> "possibly" is not a strong enough argument. I'd like to hear at least
>>> one clearly-defined use case that cannot just as well be handled by
>>> using multiple streams.
>>
>> I recently worked on the MXF demuxer to recognize channel designations in
>> MXF files, and in MXF the designation and grouping of the channels is
>> completely separate from the track those channels are muxed in.
>>
>> So if you have e.g. english stereo sound, and german stereo sound you
>> can mux it
>> - as a single 4 channel track
>> - as two 2 channel tracks
>> - as four 1 channel tracks.
>> Some MXF flavors use the multichannel single track approach, others the
>> mono-track-only approach. So the user may not be able to choose the
>> optimal muxed track assignment...
>>
>> So ultimately, if you demux and decode a packet from a track, you will
>> have an AVFrame, which might contain a single sound group on its own
>> (optimal case), part of a sound group, or multiple sound groups.
>>
>> To summerize, muxed tracks are not necessarily 1:1 mapping between sound
>> groups. And when processing/filtering audio, you typically want sound
>> groups, not tracks. And yes, it is very rare to have a soundgroup which
>> has channels with the same designation, but for a muxed track, it depends
>> on the format.
>>
>> The goal of the end user is probably to be able to specify sound groups,
>> not select muxed tracks. Preferably a demuxer should provide which channel
>> is part of which sound group, and you should be able to use a filter or a
>> combination of filters select a specific sound group. E.g. amerge all
>> tracks, then keep only the channels from all the merged channels which are
>> part of a specific sound group.
>
> So what are you proposing? In my view, such higher level information
> should live at a higher level - e.g. in the side data. You can then
> have a filter that reads this side data and gets you the group you want.

Does not look that simple to use side data for everything, because when 
you have a frame in a filter, you already should have configured the 
output channel layout... So unless you pass side data to filters somehow 
before initialization, it does not help you.

Regards,
Marton