[FFmpeg-devel] Broken endian indication in pixfmt list

[wm4]

On Wed, 5 Nov 2014 02:44:12 +0100
Michael Niedermayer <michaelni at gmx.at> wrote:

> On Wed, Nov 05, 2014 at 02:13:29AM +0100, wm4 wrote:
> > The header pixfmt.h contains the following comment in the pixel format
> > list doxygen:
> > 
> >  * @note
> >  * Make sure that all newly added big-endian formats have (pix_fmt & 1) == 1
> >  * and that all newly added little-endian formats have (pix_fmt & 1) == 0.
> >  * This allows simpler detection of big vs little-endian.
> > 
> 
> > This is currently broken for the following formats: nv20le/be gbrap16le/be
> 
> yes
> 
> 
> > 
> > Obviously, this can't be fixed until the next ABI bump. But more
> 
> can you post a patch which does make sure it gets fixed then?
> and or that documentation corrected

As I said, I could write a patch that adds a proper flag, which the
user can use to determine endian properties. I'd also adjust the
documentation and inform the user that the old way doesn't work anymore.

I'm not so much interested in creating big ifdef messes.

> 
> [...]
> > 
> > Also, we should discuss whether swapped-endian formats should even
> > exist. From what I can see, only the raw decoders/encoders make
> > effective use of them. Wouldn't it be simpler to let the
> > decoders/encoders do the byte-swapping?
> 
> they are usefull to avoid doubble swaping if both in and out are
> non native endian.

Which happens how often?

> also they are usefull for the user API of libswscale, which mey be
> used by end user applications which have such non native formats

Again, when is this actually useful?