[FFmpeg-devel] [PATCH 1/2] avcodec/s302m: enable non-PCM decoding

[Nicolas Gaullier]

>On 2024-01-28 04:24 pm, Anton Khirnov wrote:
>> Quoting Gyan Doshi (2024-01-26 05:23:50)
>>>
>>> On 2024-01-25 06:47 pm, Andreas Rheinhardt wrote:
>>>> Gyan Doshi:
>>>>>> On 2024-01-25 10:29 am, Andreas Rheinhardt wrote:
>>>>>> Gyan Doshi:
>>>>>>> Set up framework for non-PCM decoding in-place and add support 
>>>>>>> for Dolby-E decoding.
>>>>>>>
>>>>>>> Useful for direct transcoding of non-PCM audio in live inputs.
>>>>>>> ---
>>>>>>>     configure          |   1 +
>>>>>>>     doc/decoders.texi  |  40 +++
>>>>>>>     libavcodec/s302m.c | 609 +++++++++++++++++++++++++++++++++++++--------
>>>>>>>     3 files changed, 543 insertions(+), 107 deletions(-)
>>>>>>>
>>>>>>> diff --git a/configure b/configure index c8ae0a061d..8db3fa3f4b 
>>>>>>> 100755
>>>>>>> --- a/configure
>>>>>>> +++ b/configure
>>>>>>> @@ -2979,6 +2979,7 @@ rv20_decoder_select="h263_decoder"
>>>>>>>     rv20_encoder_select="h263_encoder"
>>>>>>>     rv30_decoder_select="golomb h264pred h264qpel mpegvideodec rv34dsp"
>>>>>>>     rv40_decoder_select="golomb h264pred h264qpel mpegvideodec rv34dsp"
>>>>>>> +s302m_decoder_select="dolby_e_decoder"
>>>>>>>     screenpresso_decoder_deps="zlib"
>>>>>>>     shorten_decoder_select="bswapdsp"
>>>>>>>     sipr_decoder_select="lsp"
>>>>>>> diff --git a/doc/decoders.texi b/doc/decoders.texi index 
>>>>>>> 293c82c2ba..9f85c876bf 100644
>>>>>>> --- a/doc/decoders.texi
>>>>>>> +++ b/doc/decoders.texi
>>>>>>> @@ -347,6 +347,46 @@ configuration. You need to explicitly 
>>>>>>> configure the build with
>>>>>>>     An FFmpeg native decoder for Opus exists, so users can decode Opus
>>>>>>>     without this library.
>>>>>>>     + at section s302m
>>>>>>> +
>>>>>>> +SMPTE ST 302 decoder.
>>>>>>> +
>>>>>>> +SMPTE ST 302 is a method for storing AES3 data format within an 
>>>>>>> +MPEG
>>>>>>> Transport
>>>>>>> +Stream. AES3 streams can contain LPCM streams of 2, 4, 6 or 8
>>>>>>> channels with a
>>>>>>> +bit depth of 16, 20 or 24-bits at a sample rate of 48 kHz.
>>>>>>> +They can also contain non-PCM codec streams such as AC-3 or Dolby-E.
>>>>>>> +
>>>>>> This sounds like we should add bitstream filters to extract the 
>>>>>> proper underlying streams instead.
>>>>>> (I see only two problems with this approach: The BSF API needs to 
>>>>>> set the CodecID of the output during init, but at this point no 
>>>>>> packet has reached the BSF to determine it. And changing codec IDs 
>>>>>> mid-stream is also not supported.)
>>>>> In theory, this decoder shouldn't exist, as it is just a carrier, 
>>>>> whether of LPCM or non-PCM.
>>>>> FFmpeg architecture also imposes a fundamental limitation in that 
>>>>> one s302m stream may carry multiple payload streams and we support 
>>>>> only one decoding context per input stream
>>>> Then why does the demuxer not separate the data into multiple streams?
>>> I didn't add demuxing support for this codec in MPEGTS, but I can 
>>> venture
>>>
>>> a) it would mean essentially inlining this decoder in the demuxer.
>> Why is that a problem? This decoder seems like it shouldn't be a 
>> decoder.
>>
>> I agree with Andreas that this seems like it's a demuxer pretending to 
>> be a decoder.
>
>This module transforms the entire raw payload data to generate its output, even if the syntax is simple which essentially makes it a de-coder. The de-multiplexer aspect of multiple streams is an academic possibility allowed by the >standard but not seen in any sample which makes me suspect it's used for carriage between broadcast facilities rather than something ever sent to an OTT provider, let alone an end user.
>
>Regards,
>Gyan

AFAIK, DolbyE may be found on satellite feeds, for carriage between broadcast facilities, and thus it makes them accessible so they may be grabbed by "end users". Otherwise, it is "broadcast professional end users", which are users too. AFAIK, its most common form is 20Bits and you simply "cannot" have a single stream in a 20Bit carrier; but indeed, most of the time only the first stream ("program") is used and the second is a downmix; but not always. For example, you can have a first program which is standard 5.1 and a second program with Audio Description.
Anyway, I like this s302m patch support as it makes the existing DolbyE decoder really available (not requiring two pass).
If this design is not acceptable and you want to get all the picture, then I don't think there is any other option as introducing a new filter, 1 pin in 48KHz, N pins out (XXKhz) with stream metadata inserted in each (downmix meta, dialnorm..). That would be nice, indeed from a user point of view, and this was already discussed in some forum discussions. But are you ready to go for it ? A filter, dedicated to DolbyE, and that would link to lavc ?
Nicolas