[FFmpeg-devel] [PATCH 00/21] Add timed metadata tracks, track references, channel layout

[erkki.seppala.ext at nokia.com]

From: Erkki Seppälä <erkki.seppala.ext at nokia.com>

Hello,

Here is a patch set with the goal of adding the following features to
the ISO base media file format support of FFmpeg (also available at
https://github.com/nokiatech/FFmpeg/tree/timed-metadata):

* Ability to mux and demux timed metadata tracks

  Timed metadata tracks have a sample descriptor describing their
  contents (ie. an URI and a configuration) in the headers. The tracks
  are associated with media tracks by using track references and the
  samples of the metadata track can have arbitrary length in size and
  time.

  The sample type urim is supported. The metadata track has
  codec id AV_CODEC_ID_META and its type is AVMEDIA_TYPE_DATA.

* Ability to set alternate track groups (instead of using the
  automatically assigned ones)

  This functionality is accessed by adding side data
  AV_PKT_DATA_TRACK_ALTERNATE_GROUP to the track.

* Ability to add and retrieve any types of track reference and
  ability to refere multiple tracks instead of only one

  This functionality is accessed by adding/accessing side data
  AV_PKT_DATA_TRACK_REFERENCES to the track.

* Ability to set advanced channel layout to tracks (the chnl box)

  This functionality is separated into the following three side data:

  AV_PKT_DATA_AUDIO_CHANNEL_PREDEFINED_LAYOUT for choosing a
  predefined channel layout (with a list of omitted channels).

  AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT for setting arbitrary speaker
  positions.

  AV_PKT_DATA_AUDIO_CHANNEL_LAYOUT_OBJECT_STRUCTURED for indicating the
  audio track has object structured audio.

* Some smaller funtionality has also been added:

  av_arraydup for duplicating an array (the allocated memory block is
  compatible with av_realloc).

  Ability to use stream ids directly as track numbers (with option
  use_stream_ids_as_track_ids).

  Two examples have been added (based on their non-metadata versions)
  to doc/examples on how to work with timed metadata.

-

Erkki Seppälä (21):
  libavutil/mem: added av_arraydup to accompany av_realloc*_array
    functions
  libavformat/utils: added ability to probe AVMEDIA_TYPE_DATA format
  libavformat/movenc: added ability to use original stream ids as track
    ids instead of regenerating them
  libavcodec: added a structure AVData for decoding timed metadata
  libavformat/movenc: mov_write_ftyp_tag: write the major brand a
    compatible brand
  libavcodec/avcodec.h: AV_CODEC_ID_META added for timed meta data
  libavcodec/metaenc: added an encoder/decoder for timed metadata
  libavformat/movenc: deal with AVMEDIA_TYPE_DATA by using
    AV_CODEC_ID_META
  libavformat/movenc: support for multiple and client-provided track
    references
  libavformat/movenc: add the urim sample descriptor
  libavformat/movenc, isom: support metadata in mp4 files
  libavformat/mov, isom: read (multiple) track references (tag and
    multiple ids)
  libavformat/mov: basic support for identifying (and reading) timed
    metadata
  libavformat/mov: ff_mov_read_stsd_entries now deals with
    AVMEDIA_TYPE_DATA
  libavformat/mov: read urim metadata from meta
  libavcodec/avcodec, libavformat/movenc: support embedding channel
    layout to stream side data
  libavcodec/avcodec, libavformat/movenc: introduced
    AV_PKT_DATA_TRACK_ALTERNATE_GROUP side data for expressing alternate
    groups
  libavformat/movenc: mov_write_audio_tag writes the proper number of
    channels, not the hardcoded 2
  doc/examples/extract_timed_metadata: added a bare-bones metadata
    extractor; find only the frames
  doc/examples/muxing_with_metadata: example for dealing with timed meta
    data
  changelog: updated

 Changelog                             |  21 +-
 configure                             |   4 +-
 doc/examples/Makefile                 |   3 +
 doc/examples/extract_timed_metadata.c | 233 +++++++++
 doc/examples/muxing_with_metadata.c   | 882 ++++++++++++++++++++++++++++++++++
 libavcodec/Makefile                   |   2 +-
 libavcodec/allcodecs.c                |   3 +
 libavcodec/avcodec.h                  | 115 ++++-
 libavcodec/codec_desc.c               |   8 +
 libavcodec/metacodec.c                |  94 ++++
 libavcodec/utils.c                    |  19 +
 libavformat/isom.c                    |   7 +
 libavformat/isom.h                    |   5 +
 libavformat/mov.c                     | 238 ++++++++-
 libavformat/movenc.c                  | 493 +++++++++++++++++--
 libavformat/movenc.h                  |  64 ++-
 libavformat/movenchint.c              |  11 +-
 libavformat/movmeta.h                 |  46 ++
 libavformat/utils.c                   |   7 +-
 libavutil/mem.c                       |  11 +
 libavutil/mem.h                       |  11 +
 21 files changed, 2219 insertions(+), 58 deletions(-)
 create mode 100644 doc/examples/extract_timed_metadata.c
 create mode 100644 doc/examples/muxing_with_metadata.c
 create mode 100644 libavcodec/metacodec.c
 create mode 100644 libavformat/movmeta.h

-- 
2.7.4