[FFmpeg-devel] decklink 24/32 bit question

[Devin Heitmueller]

Hello Marton,

> On Oct 18, 2017, at 3:02 PM, Marton Balint <cus at passwd.hu> wrote:
> 
> 
> 
> On Tue, 17 Oct 2017, Devin Heitmueller wrote:
> 
>>> > The decklink sdk only defines two BMDAudioSampleType values: bmdAudioSampleType16bitInteger and bmdAudioSampleType32bitInteger. I don't think there's an easy way to support a 24 bit input here. Generally in this case I've used bmdAudioSampleType32bitInteger and then encode it at pcm_s24le.
>>> Dave Rice
>> 
>> For what it’s worth, I’ve got deinterleaving code in the works to handle capture of multiple pairs of audio (i.e. break 16 channels into 8 pairs and announce them as separate S16LE streams).  If we really thought 24-bit was desirable, that code could be adjusted accordingly (the hardware would still capture 32-bit, but the deinterleaver would put out S24LE).
> 
> Breaking 8/16 channels to stereo streams can be done by an audio filter (by using "asplit" to multiply the source to 4 outputs and then "pan" or "channelmap" on each output to select the proper source channels), so I don't think direct support of splitting channels in the decklink device is acceptable.

So using an audio filter sounds like a great idea and was my initial instinct.  However then I dug into the ffmpeg API interfaces and discovered that audio filters cannot output anything but audio samples.  This prevents me from doing detection of compressed audio over SDI (i.e. S302M) during the probing phase (i.e. for Dolby-E or AC-3), since the output would be something other than uncompressed audio samples.

It also means you cannot do a simple use case for having the decklink demux announce 8 streams which can be easily fed to eight different encoders through the standard map facility.  You would have to probe the input, and use filter_complex to insert an audio filter which deinterleaves the audio, and then manually instantiate a series of audio encoders mapped to each output.

The approach I’ve proposed will “just work” in the ffmpeg command line use case.  Run the command, say “-map 0”, and eight streams will be fed to eight encoders (or pass through with the copy codec for compressed audio) and inserted into the transport stream.

I’m not against refactoring the existing S302 codec to create a demux to detect audio and create the streams with either raw or compressed codec type based on detection.  In that case the decklink demux would spawn an S302 sub-demux for each of the streams.  However in any case it would still require the audio to be de-interleaved in order to detect the S302 packets.

Now I welcome someone to better design the filter interface to allow filters which can take in raw audio and output compressed data.  Likewise I welcome someone to introduce improvements which allow filters to create new streams.  However in the absence of either of these rather large redesigns of ffmpeg’s internals, the approach I’m proposing is the only thing I could come up with which allows for these decisions to be made at the probing phase.

I’m also not against the notion of invoking a filter from inside the demux  if such a deinterleaving filter exists (and then have the decklink demux create the streams based on passing the data through the filter).  That would allow for some code reuse but is still a hack to overcome the limitations of ffmpeg’s probing framework.

I’m also not against the notion of creating a demux which is fed all eight audio streams as one blob and having that demux create the streams (and then invoking that demux from inside the decklink demux).  While that might lend itself to a bit of code sharing if we had other SDI input cards we wanted to support (assuming the provide audio in the same basic format), it still wouldn’t be an audio filter and I’m not confident the extra abstraction is worth the effort.

Devin