[FFmpeg-devel] [PATCH] avformat/mov: Add support for demuxing still HEIC images

[Vignesh Venkat]

On Tue, Oct 3, 2023 at 5:30 PM Steven Liu <lingjiujianke at gmail.com> wrote:
>
> Vignesh Venkat via ffmpeg-devel <ffmpeg-devel at ffmpeg.org>于2023年10月4日
> 周三06:57写道：
>
> > On Tue, Sep 26, 2023 at 10:37 AM Vignesh Venkatasubramanian
> > <vigneshv at google.com> wrote:
> > >
> > > They are similar to AVIF images (both use the HEIF container).
> > > The only additional work needed is to parse the hvcC box and put
> > > it in the extradata.
> > >
> > > With this patch applied, ffmpeg (when built with an HEVC decoder)
> > > is able to decode the files in
> > > https://github.com/nokiatech/heif/tree/gh-pages/content/images
> > >
> > > Partially fixes trac ticket #6521.
> > >
> > > Signed-off-by: Vignesh Venkatasubramanian <vigneshv at google.com>
> > > ---
> > >  libavformat/isom.h |  2 ++
> > >  libavformat/mov.c  | 38 +++++++++++++++++++++++++++++++++++++-
> > >  2 files changed, 39 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/libavformat/isom.h b/libavformat/isom.h
> > > index 3d375d7a46..b30b9da65e 100644
> > > --- a/libavformat/isom.h
> > > +++ b/libavformat/isom.h
> > > @@ -327,6 +327,8 @@ typedef struct MOVContext {
> > >          int64_t extent_offset;
> > >      } *avif_info;
> > >      int avif_info_size;
> > > +    int64_t hvcC_offset;
> > > +    int hvcC_size;
> > >      int interleaved_read;
> > >  } MOVContext;
> > >
> > > diff --git a/libavformat/mov.c b/libavformat/mov.c
> > > index 1996e0028c..cec9cb5fe1 100644
> > > --- a/libavformat/mov.c
> > > +++ b/libavformat/mov.c
> > > @@ -1218,7 +1218,8 @@ static int mov_read_ftyp(MOVContext *c,
> > AVIOContext *pb, MOVAtom atom)
> > >          c->isom = 1;
> > >      av_log(c->fc, AV_LOG_DEBUG, "ISO: File Type Major Brand:
> > %.4s\n",(char *)&type);
> > >      av_dict_set(&c->fc->metadata, "major_brand", type, 0);
> > > -    c->is_still_picture_avif = !strncmp(type, "avif", 4);
> > > +    c->is_still_picture_avif = !strncmp(type, "avif", 4) ||
> > > +                               !strncmp(type, "mif1", 4);
> > >      minor_ver = avio_rb32(pb); /* minor version */
> > >      av_dict_set_int(&c->fc->metadata, "minor_version", minor_ver, 0);
> > >
> > > @@ -4911,6 +4912,16 @@ static int avif_add_stream(MOVContext *c, int
> > item_id)
> > >      st->priv_data = sc;
> > >      st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
> > >      st->codecpar->codec_id = AV_CODEC_ID_AV1;
> > > +    if (c->hvcC_offset >= 0) {
> > > +        int ret;
> > > +        int64_t pos = avio_tell(c->fc->pb);
> > > +        st->codecpar->codec_id = AV_CODEC_ID_HEVC;
> > > +        avio_seek(c->fc->pb, c->hvcC_offset, SEEK_SET);
> > > +        ret = ff_get_extradata(c->fc, st->codecpar, c->fc->pb,
> > c->hvcC_size);
> > > +        if (ret < 0)
> > > +            return ret;
> > > +        avio_seek(c->fc->pb, pos, SEEK_SET);
> > > +    }
> > >      sc->ffindex = st->index;
> > >      c->trak_index = st->index;
> > >      st->avg_frame_rate.num = st->avg_frame_rate.den = 1;
> > > @@ -4953,6 +4964,8 @@ static int avif_add_stream(MOVContext *c, int
> > item_id)
> > >
> > >  static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom)
> > >  {
> > > +    c->hvcC_offset = -1;
> > > +    c->hvcC_size = 0;
> > >      while (atom.size > 8) {
> > >          uint32_t tag;
> > >          if (avio_feof(pb))
> > > @@ -7826,6 +7839,28 @@ static int mov_read_iloc(MOVContext *c,
> > AVIOContext *pb, MOVAtom atom)
> > >      return atom.size;
> > >  }
> > >
> > > +static int mov_read_iprp(MOVContext *c, AVIOContext *pb, MOVAtom atom)
> > > +{
> > > +    int size = avio_rb32(pb);
> > > +    if (avio_rl32(pb) != MKTAG('i','p','c','o'))
> > > +        return AVERROR_INVALIDDATA;
> > > +    size -= 8;
> > > +    while (size > 0) {
> > > +        int sub_size, sub_type;
> > > +        sub_size = avio_rb32(pb);
> > > +        sub_type = avio_rl32(pb);
> > > +        sub_size -= 8;
> > > +        size -= sub_size + 8;
> > > +        if (sub_type == MKTAG('h','v','c','C')) {
> > > +            c->hvcC_offset = avio_tell(pb);
> > > +            c->hvcC_size = sub_size;
> > > +            break;
> > > +        }
> > > +        avio_skip(pb, sub_size);
> > > +    }
> > > +    return atom.size;
> > > +}
> > > +
> > >  static const MOVParseTableEntry mov_default_parse_table[] = {
> > >  { MKTAG('A','C','L','R'), mov_read_aclr },
> > >  { MKTAG('A','P','R','G'), mov_read_avid },
> > > @@ -7933,6 +7968,7 @@ static const MOVParseTableEntry
> > mov_default_parse_table[] = {
> > >  { MKTAG('p','c','m','C'), mov_read_pcmc }, /* PCM configuration box */
> > >  { MKTAG('p','i','t','m'), mov_read_pitm },
> > >  { MKTAG('e','v','c','C'), mov_read_glbl },
> > > +{ MKTAG('i','p','r','p'), mov_read_iprp },
> > >  { 0, NULL }
> > >  };
> > >
> > > --
> > > 2.42.0.515.g380fc7ccd1-goog
> > >
> >
> > Any comments/objections on merging this?
>
>
> Can this patch support tiled hevc coded or sequence heif？
>

Actually, sequence HEIF is very similar to an HEVC video in a MP4
container and all the necessary data is in the 'moov' box. So it
already works with the current tip-of-tree build of ffmpeg. I tried
all the files in
https://github.com/nokiatech/heif/tree/gh-pages/content/image_sequences
and was able to decode all of them.

So once this patch is applied, the only unsupported items would be
Alpha and Grid (Tiles) for both AVIF and HEIC.

> >
> >
> > --
> > Vignesh
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".



-- 
Vignesh