[FFmpeg-devel] [PATCH V2] Patch to add interlaced HEVC decoding to HEVCDEC

[Hendrik Leppkes]

On Wed, Oct 30, 2024 at 4:52 PM Jose Santiago <jsantiago at haivision.com> wrote:
> +static int interlaced_frame_from_fields(AVFrame *dst,
> +                                        const AVFrame *field1,
> +                                        const AVFrame *field2)
> +{
> +    int i, ret = 0;
> +
> +    av_frame_unref(dst);
> +
> +    dst->format         = field1->format;
> +    dst->width          = field1->width;
> +    dst->height         = field1->height * 2;
> +    dst->nb_samples     = field1->nb_samples;
> +    ret = av_channel_layout_copy(&dst->ch_layout, &field1->ch_layout);
> +    if (ret < 0)
> +        return ret;
> +
> +    ret = av_frame_copy_props(dst, field1);
> +    if (ret < 0)
> +        return ret;
> +    if (field1->duration > 0 && field1->duration != AV_NOPTS_VALUE)
> +        dst->duration = field2->duration * 2;
> +    else if (field2->duration > 0 && field2->duration != AV_NOPTS_VALUE)
> +        dst->duration = field2->duration * 2;
> +
> +    for (i = 0; i < field2->nb_side_data; i++) {
> +        const AVFrameSideData *sd_src = field2->side_data[i];
> +        AVFrameSideData *sd_dst;
> +        AVBufferRef *ref = av_buffer_ref(sd_src->buf);
> +        sd_dst = av_frame_new_side_data_from_buf(dst, sd_src->type, ref);
> +        if (!sd_dst) {
> +            av_buffer_unref(&ref);
> +            return AVERROR(ENOMEM);
> +        }
> +    }
> +
> +    for (i = 0; i < AV_NUM_DATA_POINTERS; i++)
> +        dst->linesize[i] = field1->linesize[i]*2;
> +
> +    ret = av_frame_get_buffer(dst, 0);
> +    if (ret < 0)
> +        return ret;
> +
> +    ret = av_frame_copy(dst, field1);
> +    if (ret < 0)
> +        av_frame_unref(dst);
> +
> +    copy_field2(dst, field2);
> +
> +    for (i = 0; i < AV_NUM_DATA_POINTERS; i++)
> +        dst->linesize[i] = field1->linesize[i];
> +
> +    return ret;
> +}


Copying the entire frame is slow, inefficient and fundamentally
incompatible with hardware acceleration.
A correct approach would be doing what other field-based interlaced
codecs do - that is allocate a full frame picture from the start, and
decode both fields into the same frame. This then even has a chance to
work with hardware acceleration, at least in theory, at the very least
it avoids the copy overhead of the software decoder.

- Hendrik