[FFmpeg-devel] [PATCH 1/2] avcodec/s302m: enable non-PCM decoding

[Kieran Kunhya]

On Mon, 29 Jan 2024 at 10:17, Gyan Doshi <ffmpeg at gyani.pro> wrote:

>
>
> On 2024-01-29 02:57 pm, Nicolas Gaullier wrote:
> >> On 2024-01-28 04:24 pm, Anton Khirnov wrote:
> >>> Quoting Gyan Doshi (2024-01-26 05:23:50)
> >>>> On 2024-01-25 06:47 pm, Andreas Rheinhardt wrote:
> >>>>> Gyan Doshi:
> >>>>>>> On 2024-01-25 10:29 am, Andreas Rheinhardt wrote:
> >>>>>>> Gyan Doshi:
> >>>>>>>> Set up framework for non-PCM decoding in-place and add support
> >>>>>>>> for Dolby-E decoding.
> >>>>>>>>
> >>>>>>>> Useful for direct transcoding of non-PCM audio in live inputs.
> >>>>>>>> ---
> >>>>>>>>      configure          |   1 +
> >>>>>>>>      doc/decoders.texi  |  40 +++
> >>>>>>>>      libavcodec/s302m.c | 609
> +++++++++++++++++++++++++++++++++++++--------
> >>>>>>>>      3 files changed, 543 insertions(+), 107 deletions(-)
> >>>>>>>>
> >>>>>>>> diff --git a/configure b/configure index c8ae0a061d..8db3fa3f4b
> >>>>>>>> 100755
> >>>>>>>> --- a/configure
> >>>>>>>> +++ b/configure
> >>>>>>>> @@ -2979,6 +2979,7 @@ rv20_decoder_select="h263_decoder"
> >>>>>>>>      rv20_encoder_select="h263_encoder"
> >>>>>>>>      rv30_decoder_select="golomb h264pred h264qpel mpegvideodec
> rv34dsp"
> >>>>>>>>      rv40_decoder_select="golomb h264pred h264qpel mpegvideodec
> rv34dsp"
> >>>>>>>> +s302m_decoder_select="dolby_e_decoder"
> >>>>>>>>      screenpresso_decoder_deps="zlib"
> >>>>>>>>      shorten_decoder_select="bswapdsp"
> >>>>>>>>      sipr_decoder_select="lsp"
> >>>>>>>> diff --git a/doc/decoders.texi b/doc/decoders.texi index
> >>>>>>>> 293c82c2ba..9f85c876bf 100644
> >>>>>>>> --- a/doc/decoders.texi
> >>>>>>>> +++ b/doc/decoders.texi
> >>>>>>>> @@ -347,6 +347,46 @@ configuration. You need to explicitly
> >>>>>>>> configure the build with
> >>>>>>>>      An FFmpeg native decoder for Opus exists, so users can
> decode Opus
> >>>>>>>>      without this library.
> >>>>>>>>      + at section s302m
> >>>>>>>> +
> >>>>>>>> +SMPTE ST 302 decoder.
> >>>>>>>> +
> >>>>>>>> +SMPTE ST 302 is a method for storing AES3 data format within an
> >>>>>>>> +MPEG
> >>>>>>>> Transport
> >>>>>>>> +Stream. AES3 streams can contain LPCM streams of 2, 4, 6 or 8
> >>>>>>>> channels with a
> >>>>>>>> +bit depth of 16, 20 or 24-bits at a sample rate of 48 kHz.
> >>>>>>>> +They can also contain non-PCM codec streams such as AC-3 or
> Dolby-E.
> >>>>>>>> +
> >>>>>>> This sounds like we should add bitstream filters to extract the
> >>>>>>> proper underlying streams instead.
> >>>>>>> (I see only two problems with this approach: The BSF API needs to
> >>>>>>> set the CodecID of the output during init, but at this point no
> >>>>>>> packet has reached the BSF to determine it. And changing codec IDs
> >>>>>>> mid-stream is also not supported.)
> >>>>>> In theory, this decoder shouldn't exist, as it is just a carrier,
> >>>>>> whether of LPCM or non-PCM.
> >>>>>> FFmpeg architecture also imposes a fundamental limitation in that
> >>>>>> one s302m stream may carry multiple payload streams and we support
> >>>>>> only one decoding context per input stream
> >>>>> Then why does the demuxer not separate the data into multiple
> streams?
> >>>> I didn't add demuxing support for this codec in MPEGTS, but I can
> >>>> venture
> >>>>
> >>>> a) it would mean essentially inlining this decoder in the demuxer.
> >>> Why is that a problem? This decoder seems like it shouldn't be a
> >>> decoder.
> >>>
> >>> I agree with Andreas that this seems like it's a demuxer pretending to
> >>> be a decoder.
> >> This module transforms the entire raw payload data to generate its
> output, even if the syntax is simple which essentially makes it a de-coder.
> The de-multiplexer aspect of multiple streams is an academic possibility
> allowed by the >standard but not seen in any sample which makes me suspect
> it's used for carriage between broadcast facilities rather than something
> ever sent to an OTT provider, let alone an end user.
> >>
> >> Regards,
> >> Gyan
> > AFAIK, DolbyE may be found on satellite feeds, for carriage between
> broadcast facilities, and thus it makes them accessible so they may be
> grabbed by "end users". Otherwise, it is "broadcast professional end
> users", which are users too. AFAIK, its most common form is 20Bits and you
> simply "cannot" have a single stream in a 20Bit carrier; but indeed, most
> of the time only the first stream ("program") is used and the second is a
> downmix; but not always. For example, you can have a first program which is
> standard 5.1 and a second program with Audio Description.
> In the samples I have, including yours, the outermost layer is AES3
> which contains one inner stream Dolby-E, which in turn contains 8
> channels constituting multiple programs. Those are programs downstream
> of the dolby_e decoder. That's not what we are talking about. The
> discussion here is about multiple payload streams within the AES3 layer
> - streams stored in subframe mode e.g. a Dolby-E and AC-3 or LPCM in
> alternate subframes/frames. I've no such samples of that sort.
>

I have not seen such streams in the wild.
Kieran