[FFmpeg-devel] [PATCH] lavc/vaapi_h26[45]: add crop info support in vaapi_h26[4, 5]

[wm4]

On Wed, 30 Nov 2016 11:08:10 +0000
Mark Thompson <sw at jkqxz.net> wrote:

> On 30/11/16 02:20, Jun Zhao wrote:
> > From 20bedd18213420c77d5e8a26fbe741d8d204ac10 Mon Sep 17 00:00:00 2001
> > From: Jun Zhao <jun.zhao at intel.com>
> > Date: Tue, 29 Nov 2016 14:14:25 +0800
> > Subject: [PATCH] lavc/vaapi_h26[45]: add crop info support in vaapi_h26[4,5]
> >
> > add crop information support in vaapi_h26[4,5] hwaccel decode,
> > and align h264/hevc software decoder. After this fix, FATE test
> > h264-conformance-cvfc1_sony_c/hevc-conformance-CONFWIN_A_Sony_1
> > will pass in i965/SKL.
> >
> > Signed-off-by: Wang, Yi A <yi.a.wang at intel.com>
> > Signed-off-by: Jun Zhao <jun.zhao at intel.com>
> > ---
> >  libavcodec/h264dec.c        | 13 +++++++++++++
> >  libavcodec/hevc_refs.c      |  7 +++++++
> >  libavutil/hwcontext.h       |  6 ++++++
> >  libavutil/hwcontext_vaapi.c |  1 +
> >  4 files changed, 27 insertions(+)  
> 
> No.  Top/left cropping needs a more general approach, not hacks in specific decoders/hwcontexts.
> 
> Among other things:
> - The hardware frame context is owned by the user, you can't edit it like this from inside the decoder.
> - It can't be per-frame-context anyway, because multiple streams can be decoded into the same frame context.
> - More components need to be aware of it: the scale filters definitely should be (at <https://git.libav.org/?p=libav.git;a=blob;f=libavfilter/vf_scale_vaapi.c;h=50be68eee07374a78bba1eed8850c3ab9e019fe6;hb=HEAD#l291>, in VAAPI, similarly for the other hardware APIs).  Other components can probably ignore it (with docs saying you should put it through the relevant scale filter if you want correctly-cropped images), but we should make that decision explicitly.
> - Messing with pointers/sizes on frame upload/download has issues with APIs which can only transfer whole frames - see the comment at <http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/hwcontext.h;h=785da090b3fc4b4e91f989370ea4606c79bae823;hb=HEAD#l321>.  (That was written with VDPAU in mind, there are likely others.)
> 
> As wm4 notes in another reply, this ends up pointing to some sort of cropping information associated with the AVFrame which all interested components can then read.
> 
> I know several people have expressed interest in this problem: given that, it's probably a good idea to have some discussion first and decide roughly what the result should look like before writing a lot of code for it.

I've always argued for having a cropping rectangle directly in AVFrame,
but the idea was never popular.