[FFmpeg-devel] [PATCH] libavutil/video_enc_params: add block type

[Mark Thompson]

On 07/07/2020 23:54, Lynne wrote:
> Jul 7, 2020, 23:47 by yongle.lin.94 at gmail.com:
> 
>> add block type field to AVVideoBlockParams so we could either export or visualize it later.
>> ---
>>   libavutil/video_enc_params.h | 16 ++++++++++++++++
>>   1 file changed, 16 insertions(+)
>>
>> diff --git a/libavutil/video_enc_params.h b/libavutil/video_enc_params.h
>> index 43fa443154..52c0058f5b 100644
>> --- a/libavutil/video_enc_params.h
>> +++ b/libavutil/video_enc_params.h
>> @@ -57,6 +57,11 @@ enum AVVideoEncParamsType {
>>   AV_VIDEO_ENC_PARAMS_H264,
>>   };
>>   
>> +enum AVVideoBlockFlags {
>> +    AV_VIDEO_ENC_BLOCK_INTRA = 1ULL <<  0,  /* Indicates block uses intra prediction */
>> +    AV_VIDEO_ENC_BLOCK_SKIP = 1ULL <<  1,   /* Indicates block is not coded (skipped) */
>> +};
>> +
>>   /**
>>   * Video encoding parameters for a given frame. This struct is allocated along
>>   * with an optional array of per-block AVVideoBlockParams descriptors.
>> @@ -126,6 +131,17 @@ typedef struct AVVideoBlockParams {
>>   * corresponding per-frame value.
>>   */
>>   int32_t delta_qp;
>> +
>> +    /**
>> +     * Type flag of the block
>> +     * Each bit field indicates a type flag
>> +     */
>> +    enum AVVideoBlockFlags flags;
>> +
>> +    /**
>> +     * Reference frames used for prediction
>> +     */
>> +    uint8_t ref[8];
>>   } AVVideoBlockParams;
>>
> 
> After some discussion on IRC, could you clarify the ref array description to this:
> 
>> Each entry specifies the first/second/third/etc. reference frame the current frame uses.
>> The value at each entry specifies the index inside the reference frame array for that current frame.
> 
> E.g. your current frame has 6 valid possible references, and your frame header specifies you
> can use ref_frame[3] and ref_frame[5] as a reference.
> So the values of ref[] for each block must be either 3 or 5.
> Its convoluted because the array maps indices to indices but it makes sense.

Please also define it precisely for H.264, the other supported codec.

I came up with:

"""
For H.264, the values in this array are indices into the default RefPicList0 as constructed by 8.2.4.2 (before ref pic list modification has run and without any truncation).
If the block is intra-coded, no entries are valid.
If the block in inter-coded with reference to a single picture, ref[0] containes the index of that picture (which might have come from L0 or L1 list).
If the block is inter-coded using biprediction, ref[0] contains the index of the L0 picture and ref[1] contains the index of the L1 picture.
"""

Not sure if that's doing exactly the right thing or matches what you intended, but this is tricky so it needs that level of detail.

8 distinct reference pictures also seems slightly ambitious for a single lowest-level block, but I guess the future is always about ever-more-complex coding tools...

- Mark