[FFmpeg-devel] [PATCH v3 1/1] avformat/mpegtsenc: Write necessary descriptors into PMT for arib_caption muxing

[Jan Ekström]

On Tue, Apr 20, 2021 at 8:16 PM Jan Ekström <jeebjp at gmail.com> wrote:
>
> On Tue, Apr 20, 2021 at 3:03 PM zheng qian <xqq at xqq.im> wrote:
> >
> > On Tue, Apr 20, 2021 at 6:46 AM Jan Ekström <jeebjp at gmail.com> wrote:
> > >
> > > On Tue, Apr 20, 2021 at 12:11 AM Marton Balint <cus at passwd.hu> wrote:
> > > >
> > > >
> > > >
> > > > On Mon, 19 Apr 2021, zheng qian wrote:
> > > >
> > > > > Is there anyone who could review this patch?
> > > >
> > > > Jan was interested in this, so preferably he should also comment, but it
> > > > looks fine to me.
> > > >
> > >
> > > OK, this explains why I didn't see my response on patchwork.
> > > Apparently he had CC'd me and thus the "reply" button in gmail sent an
> > > e-mail directly to him and I was hurrying due to being on a lunch
> > > break -_- (and thus didn't notice).
> > >
> > > In any case, I did some comments and am now waiting for a second
> > > opinion regarding the usage of stream_identifiers in the ARIB context.
> > > After all, the specifications do let one utilize 0x30-0x37 for profile
> > > A/full-seg ARIB captions, so there must be a reason for them to not be
> > > as limited as profile C/1seg to a single identifier :)
> > >
> >
> > I've found related definitions in ARIB TR-B14, Fascicle 1, 4.2.8.1
> > and you can find it in
> > http://web.archive.org/web/20160319090430/http://arib.or.jp/english/html/overview/doc/8-TR-B14v2_8-1p3-2-E2.pdf
> >
> > 4.2.8.1 section says:
> > "However, for component tag values of default ES of caption,
> > set 0x30 or 0x87, for component tag value of default ES of
> > superimpose, set 0x38 or 0x88."
> >
> > That means 0x30 is considered as the default value for
> > Profile A caption ES. The section didn't describe how to
> > utilize other values rather than 0x30 in the 0x30~0x37 range,
> > and due to the second language caption is designed to be
> > multiplexed in the same ES, seems that it's assumed that
> > there will be usually only one ARIB caption within a program.
> >
>
> Alright, so I took another look as well throughout these Technical
> Recommendations:
>
> TR B14 fascicle 4
> - http://www.arib.or.jp/english/html/overview/doc/8-TR-B14v6_5-4p5-E1.pdf
>
> See:
> 14  Use of “component_tag”
>
> > Table 14-1   Assignment of “component_tag” Values
> > Others
> > 0x30 to 0x7F
> > Please note that 0x40 is assigned to the default ES for data
> > broadcasting. 0x30, 0x31 to 0x37, 0x38 and 0x39 to 0x3F are assigned to subtitle
> > main, subtitle sub, teletext main and teletext sub respectively.
>
> Then in TR B14 fascicle 2:
> - http://www.arib.or.jp/english/html/overview/doc/8-TR-B14v6_5-2p5-E1.pdf
>
> > 4.2.1  Specification for composition and transmission
> > (3) Number of ES
> > The maximum number of ESs that can be transmitted simultaneously for the same service is
> > 1  for  captions  and  1  for  superimpositions  when  the  component  group  descriptor  is  not
> > transmitted. When the component group descriptor is transmitted, the maximum number of ESs
> > for captions is 1 and the maximum number of ESs for superimpositions is 1 for each component
> > group.
>
> So to recap my understanding:
> 1. STD side (specs): It seems like these values have to be unique at
> least at the level of a PMT (thus, per-program) - discussed this with
> a person working on ARIB area of operations :) .
> 2. STD side: We can indeed have multiple caption/subtitle ES in a
> single program (PMT) for Profile A. Profile C is always limited to one
> ES per program since only one content_tag is possible.
> 3. TR side (tech. rec.): 0x30/0x87 should be the default for Profile A/C
> 4. TR side: If we do not code Component Group Descriptor (0xD9 in the
> EIT), we should stick to a single ES (the default one) on both Profile
> A and C.
> 5. TR side: If we code a Component Group Descriptor in the EIT, we can
> have one ES for each composition group.
>
> So basically:
> 1. STD side lets us go for it
> 2. TR side says "nope, you stick to one - unless you're defining
> composition groups". Which is why you do not see multi-subtitle
> streams in the wild, since almost nobody utilizes Content Group
> Descriptors.
>
> We don't and not sure if we will ever support component groups, so the
> alternatives are:
> 1. Follow TR side, and be more strict than the specification. Limit in
> `mpegts_init` each program to a single ARIB caption stream (be it
> Profile A or Profile C). Set component_tag in descriptor to the
> default one (0x30 or 0x87 respectively).
> 2. Follow STD side, and utilize what the specification enables. Limit
> in `mpegts_init` each program to either 8 Profile A ARIB caption
> streams, or 1 Profile C ARIB caption stream. Set component_tag
> iteratively in order.
>

I think in general I am preferring the "follow the TR" way since I
just heard another implementer generally speaking also ignores
everything else than 0x30/0x87.

Did not yet add the per-program limitation but poked a bit at it with
https://github.com/jeeb/ffmpeg/commits/mpegts_arib_caption_muxing .

1. Split the writing of the ARIB caption descriptor as the PMT writing
function is already way too long. Was not sure if exit or plain break
would be sufficient, but nothing else around in that function seems to
return with an error... so break it is.
2. Added logging in case an unset/unknown profile was utilized.
3. Re-added extraction of additional_arib_caption_info into extradata,
and writing it out in the muxer if available.

Feel free to note how you like these changes :)

Jan