[FFmpeg-devel] [PATCH 1/2] lavf/avienc: Simplify palette handling

[Michael Niedermayer]

On Sat, Feb 27, 2016 at 05:45:57PM +0100, Reimar Döffinger wrote:
> On Sat, Feb 27, 2016 at 04:15:10PM +0100, Mats Peterson wrote:
> > On 02/27/2016 04:13 PM, Mats Peterson wrote:
> > >On 02/27/2016 04:08 PM, Mats Peterson wrote:
> > >>On 02/27/2016 04:07 PM, Mats Peterson wrote:
> > >>>On 02/27/2016 04:00 PM, Reimar Döffinger wrote:
> > >>>>On Sat, Feb 27, 2016 at 03:57:06PM +0100, Mats Peterson wrote:
> > >>>>>On 02/27/2016 03:37 PM, Mats Peterson wrote:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>_______________________________________________
> > >>>>>>ffmpeg-devel mailing list
> > >>>>>>ffmpeg-devel at ffmpeg.org
> > >>>>>>http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >>>>>>
> > >>>>>
> > >>>>>I suppose this is what you mean, Reimar. Treating the palette, if a
> > >>>>>packet
> > >>>>>contains one at the end of the video data, as being AVPALETTE_SIZE
> > >>>>>bytes
> > >>>>>exclusively.
> > >>>>
> > >>>>Well, actually not really.
> > >>>>If the palette is part of input frame it should be sent as side
> > >>>>data.
> > >>>>I am not sure where this variant comes from.
> > >>>>It might be that it should just be written as is.
> > >>>>Or even if the palette needs to be split it might be
> > >>>>necessary to auto-detect the palette size via
> > >>>>packet size - (width*height*bits per pixel)/8.
> > >>>>But as said, I am fairly unclear on what case that
> > >>>>code is supposed to handle.
> > >>>>_______________________________________________
> > >>>>ffmpeg-devel mailing list
> > >>>>ffmpeg-devel at ffmpeg.org
> > >>>>http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > >>>>
> > >>>
> > >>>I agree that it should be stored in a side data packet myself normally,
> > >>>and that this is a somewhat weird construction. It probably has to do
> > >>>with the nut format originally, which stores raw palettized data after
> > >>>the video data in the packets. Anyway, I have accepted the facts. For
> > >>>the record, the new ff_reshuffle_raw_rgb() function written by Michael
> > >>>in lavf/rawutils.c that aligns strides properly for AVI and QuickTime,
> > >>>will set a CONTAINS_PAL flag if the packet size is larger than the
> > >>>actual video data. He has hardcoded the palette size to 1024 bytes in
> > >>>that file.
> > >>>
> > >>>Mats
> > >>>
> > >>
> > >>The nut format stores the PALETTE after the video data in the packets,
> > >>nothing else :)
> > >>
> > >
> > >In any case, on muxing, the packets will have the palette after the
> > >video data in the packets, whether it's AVI or QuickTime. Neither
> > >avienc.c or movenc.c uses any side data packets for the palette.
> > >
> > >Michael's intention has been to enable palette switching in the middle
> > >of the stream, hence storage of the palette in each packet, and AVI
> > >supports it by using the 'xxpc' chunks in the video data. It is also
> > >implemented by now.
> > >
> > >Mats
> > >
> > 
> > Not that it couldn't be done with side data packets, though.
> 
> If it doesn't support side data then the muxers are plain broken.
> If the nut muxer stores palette by appending it to the frames,
> then the demuxer should split it out into side data.
> Note that I am absolutely not a fan of this side data stuff,
> but since we already decided to do it like that then that's
> the way we need to go, not randomly doing one way in one
> place and differently in another, that just makes for unusable
> API.
> The only reasons to support "palette appended to data" are
> 1) There are some existing users of the FFmpeg API that rely on it.
>    Ideally we should then change it so it works for all muxers, or
>    the other way round warn that this is a deprecated way of doing
>    things.
> 2) There are file formats that store it that way and we cannot easily
>    split it into side data. Not sure that can really happen.

palettes are a bit annoying, there are quite a few things

the chain generally is
demuxer -----> decoder -----> encoder -----> muxer
OR
demuxer -----------------------------------> muxer

Thus there are 2 interfaces, the demuxer -> muxer and the
decoder -> encoder interface
For the decoder -> encoder interface, the palette is in AVFrame.data[1]
the 8bit indexes as a width(stride) x height array in AVFrame.data[0]
that part is still easy

the demuxer -> muxer interface is more complex
in case of non raw, that is compressed codecs the palette can be in
a codec specific and inseprable format in AVPacket.data with the
rest of the compressed image.
but its also possible that there is no palette in AVPacket.data and
instead its stored in AVPackets side data which would be filled from
container specific chunks like avis PCxx or in  the global extradata

So even without rawvideo there already exist both sidedata and non
sidedata cases

additionally key frame AVPackets must together with the global
extradata contain a full palette to be decodeable.

some containers support storing "partial palettes", for example
avis PCxx chunks can do that, so one should at keyframes store a
full one but subequent non keyframes should only store the part that
differs from the previous.

The container specific compression like PCxx would semantically
best fit into the muxer

should rawvideo AVPackets palette use data[] or sidedata, honestly i
do not know, but i dont think it makes a big difference
even supporting both, likely only adds 3-5 lines of code or so
its more a philosophical question
is the palette like chroma or alpha part of the frame?
why is alpha not sidedata if palette is?
or
palette is side data for a few (not many) compressed pal8 formats
so it can be for rawvideo too

there are arguments both way, iam not a philosopher so i dont really
have an oppinion on this ...

about existing API, i suspect there arent many applications that
use ffmpegs demuxers without the decoders  for raw pal8, i might
of course be wrong but this seems a rather uncommon case of a uncommon
case. And muxers side it was all broken before mats ...

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 2
"100% positive feedback" - "All either got their money back or didnt complain"
"Best seller ever, very honest" - "Seller refunded buyer after failed scam"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160227/3edc8284/attachment.sig>