[FFmpeg-devel] [PATCH] dxva2: add AV1 decode support

[James Almer]

On 11/5/2020 1:55 PM, Hendrik Leppkes wrote:
> On Thu, Nov 5, 2020 at 5:45 PM James Almer <jamrial at gmail.com> wrote:
>>> +    pp->coding.reference_frame_update       = !(frame_header->show_existing_frame == 1 && frame_header->frame_type == AV1_FRAME_KEY);
>>
>> hwaccel->start_frame() is not called for
>> frame_header->show_existing_frame == 1 frames (Those are essentially
>> just a header telling the decoder to output a previously decoded frame,
>> and maybe update the reference frame state), so that check is
>> superfluous, and by extension the whole thing. Just hardcode it to 1.
>>
>> Is this field documented anywhere?
>>
> 
> DXVA Spec:
> "Indicates that the reference frame update process as specified by
> section 7.20 of the specification should be performed after decoding
> this frame.  Otherwise section 7.21 should be performed."
> 
> AV1 Spec (7.4):
> "Otherwise (show_existing_frame is equal to 1), if frame_type is equal
> to KEY_FRAME, the reference frame loading process as specified in
> section 7.21 is invoked"
> 
> That was my interpretation of that flag, and also matches the
> implementation that was provided to dav1d a few months ago by one of
> the authors of the DXVA AV1 spec.
> I can hardcode it, but I also don't see the harm in keeping it as-is
> for documentary purposes.
> 
>>> +
>>> +    if (ctx_pic->bitstream_size > dxva_size) {
>>> +        av_log(avctx, AV_LOG_ERROR, "Bitstream size exceeds hardware buffer");
>>> +        return -1;
>>> +    }
>>> +
>>> +    memcpy(dxva_data, ctx_pic->bitstream, ctx_pic->bitstream_size);
>>
>> This is a memcpy after a (potential) memcpy in dxva2_av1_decode_slice().
>> Is there no way to avoid it?
>>
> 
> That depends on the behavior of av1dec. Are all tile groups guaranteed
> to be available at the same time, so that I can simply save pointers
> to them, and copy them into the final buffer later?

Not currently, in theory, but can be easily done by storing a reference 
to the AVBufferRef in each TileGroupInfo.

That being said, i don't know if the decoder can handle split/incomplete 
Temporal Units as is. Guess it's a matter of injecting the 
av1_frame_split bsf and see how badly it explodes.
If we enforce Temporal Units to be fully contained in a packet (As they 
are muxed into ivf, mp4, webm, annexb, etc), or at least individual 
Frames (Header + Tile Group/s combinations), we could also avoid the 
memcpy in dxva2_av1_decode_slice().

> Or could the first one become invalid? In which case I need to copy
> its data. The call order is not strict enough to allow the buffer
> management to keep the hardware buffer mapped persistently.
> 
> - Hendrik
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>