[FFmpeg-devel] [PATCH]lavu/hwaccel_vaapi: Add a fixme for the missing byte_order check

[wm4]

On Thu, 6 Oct 2016 10:34:35 +0100
Mark Thompson <sw at jkqxz.net> wrote:

> On 06/10/16 09:05, Carl Eugen Hoyos wrote:
> > 2016-10-05 21:55 GMT+02:00 Mark Thompson <sw at jkqxz.net>:
> > 
> > What I meant is:
> > How can "LE" make sense for an 8bit planar format?  
> 
> It does not.  So, another reason to ignore that field.
> 
> > (And what does it tell us about the author?)
> > This of course assumes that "YV12" is planar, if it
> > isn't, I simply misunderstand the whole code.  
> 
> Yes, YV12 is planar: it is ffmpeg YUV420P with the two chroma planes swapped.
> 
> >> For example, I can do screen capture in X with:
> >>
> >> ffmpeg -y -vaapi_device /dev/dri/renderD128 -video_size 1920x1080 -framerate 30 -f x11grab -i :0 -vf 'hwupload,scale_vaapi=1920:1080:nv12' -c:v h264_vaapi out.mp4
> >>
> >> which captures in bgr0, uploads it to the GPU, converts it to
> >> nv12 and encodes it as H.264 there.  Alternatively, I can add
> >> 'format=rgb0' at the start of the filter chain to convert and
> >> upload in a different RGB format, and that produces the
> >> correct output too.  
> > 
> > Thank you for confirming this.
> > 
> > Do you think vaapi's P010 should be mapped to FFmpeg's
> > P010LE instead of P010?  
> 
> P010 is defined as format with a 16-bit unit size, so the native format on a BE
> system should be P010BE.  I admit that confusion is likely, though, given that
> the actual hardware may be a graphics card which normally works with an LE host.
>  We may need to look somewhere else for the answer at that point (possibly the
> byte_order field, assuming the drivers manage to fill it correctly in such cases).
> 
> I would prefer to only consider this problem once we have some working system to
> test on.  On the other hand, if you wish to submit a patch changing it now I
> would not mind - it would have no effect on current use because the one system
> with working 10-bit support is LE-only.

Is it even theoretically possible to have hardware using vaapi on big
endian systems? vaapi is for modern Intel GPUs, which AFAIK are all
little endian.