[FFmpeg-devel] [RFC] Implementation of closed caption support

[Gisle Sælensminde]

I'm trying to add support for closed captions in ffmpeg, namely cea608 and
cea708. Unlike normal subtitles, these are embedded in the video frames
themselves rather than a separate track or file. As a first step I try 
to write
bitstreams filters for extraction and inserting closed captions into 
h.264 and
mpeg2 videos. For cea608, I use .scc files to represent the cea608 data.
Later an interpreter and producer of scc files can be made as a subtitle
codec.

I have already written a cea 608 tools for my employer, but that is in
python, so not suitable for inclusion in ffmpeg, so this will be a
new implementation.

I'm currently writing the extraction filter and I had hoped that I could use
it as a filter with parameters as follows.

ffmpeg -i vid_with_cc.ts -acodec copy -vcodec copy -bsf 
cea680_extract?scc=out.scc -f null /dev/null

This is not possible, since bitstream filters don't have parameters. The 
main
problems can be summarized as follows:

- The lack of parameters means that I don't have a way to specify where 
to store
   the .scc file. How can I do that?

- I can't find a way to get out the timestamps (pts and dts) in a bitstream
   filter. The cea 608 and 708 data is stored in pts order in the frames, so
   I need to reorder the data before writing them to file. Also, the scc 
files
   have timestamps, so I need timestamps for that too. I can only find the
   timebase in AVCodecContext, and the AVFormatContext or AVPacket is 
not passed
   in to the bitstream filter. The AVPacket is a deprecated field, but it is
   NULL, so of no use. Is there something I have overlooked here?

I may be missing something, and it is of cause possible that the 
bitstream filter
approch is misguided, but as far as I can tell, that is the best option. 
Does anyone
have a clear idea of how this should be done?

-Gisle