[FFmpeg-devel] Scaling PAL8 images with alpha

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Soft Works
> Sent: Sunday, 26 September 2021 01:00
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> 
> 
> 
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Michael Niedermayer
> > Sent: Saturday, 25 September 2021 22:47
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel at ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> >
> > On Sat, Sep 25, 2021 at 10:34:32PM +0200, Michael Niedermayer
> wrote:
> > > On Sat, Sep 25, 2021 at 03:46:25PM +0000, Soft Works wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> Behalf
> > Of
> > > > > Michael Niedermayer
> > > > > Sent: Saturday, 25 September 2021 16:30
> > > > > To: FFmpeg development discussions and patches <ffmpeg-
> > > > > devel at ffmpeg.org>
> > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with alpha
> > > > >
> > > > > On Sat, Sep 25, 2021 at 10:23:56AM +0200, Hendrik Leppkes
> > wrote:
> > > > > > On Sat, Sep 25, 2021 at 5:00 AM Soft Works
> > <softworkz at hotmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> > Behalf
> > > > > Of
> > > > > > > > Soft Works
> > > > > > > > Sent: Friday, 24 September 2021 19:03
> > > > > > > > To: FFmpeg development discussions and patches <ffmpeg-
> > > > > > > > devel at ffmpeg.org>
> > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with
> > alpha
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org>
> On
> > > > > Behalf Of
> > > > > > > > Michael
> > > > > > > > > Niedermayer
> > > > > > > > > Sent: Friday, 24 September 2021 17:40
> > > > > > > > > To: FFmpeg development discussions and patches
> <ffmpeg-
> > > > > > > > devel at ffmpeg.org>
> > > > > > > > > Subject: Re: [FFmpeg-devel] Scaling PAL8 images with
> > alpha
> > > > > > > > >
> > > > > > > > > On Fri, Sep 24, 2021 at 10:30:31AM +0000, Soft Works
> > wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > for a new filter, I want to rescale PAL8 subtitle
> > bitmaps
> > > > > where
> > > > > > > > the palette
> > > > > > > > > includes
> > > > > > > > > > colors with alpha.
> > > > > > > > > >
> > > > > > > > > > From what I’ve seen, swscale doesn’t support PAL8-
> to-
> > PAL8,
> > > > > only
> > > > > > > > PAL8-to-
> > > > > > > > > BGR8
> > > > > > > > > > which doesn’t support alpha and the palette is
> fixed
> > with
> > > > > 256
> > > > > > > > entries
> > > > > > > > > defined by
> > > > > > > > > > convention, while I would ideally like to be able
> to
> > allow
> > > > > the
> > > > > > > > following:
> > > > > > > > > >
> > > > > > > > > > - constrain the output to use the same palette as
> the
> > input
> > > > > > > > > > - adaptively quantize it to a palette with a
> > configurable
> > > > > number
> > > > > > > > of colors
> > > > > > > > > >
> > > > > > > > > > Thus it's about the palette quantization (with or
> > without
> > > > > > > > dithering) after
> > > > > > > > > > scaling in RGBA. (or some cool algorithmic trick
> I'm
> > not
> > > > > aware
> > > > > > > > of)
> > > > > > > > > >
> > > > > > > > > > Is there any existing code that I could reuse? The
> > closest
> > > > > I
> > > > > > > > could find
> > > > > > > > > > is pngenc, but I'm wondering whether there's
> > something
> > > > > > > > else/better
> > > > > > > > > > somewhere in the ffmpeg libs that I haven't seen?
> > > > > > > > >
> > > > > > > > > There are some non linear scaling filters which may
> > make
> > > > > sense when
> > > > > > > > you do
> > > > > > > > > not have the full linear space available see:
> > > > > > > > > hqx, epx, xbr
> > > > > > > > > you could also try some neural net stuff
> > > > > > > >
> > > > > > > > Photoshop can perfectly do what is needed, but their
> > algorithms
> > > > > are
> > > > > > > > not
> > > > > > > > public.
> > > > > > >
> > > > > > > I've put a few thing together to illustrate what I'm
> > talking
> > > > > about:
> > > > > > >
> > > > > > >
> > > > >
> > https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f
> > > > > > >
> > > > > > >
> > > > > > > Meanwhile I found what I need:
> > > > > https://github.com/ImageOptim/libimagequant
> > > > > > >
> > > > > > > Interestingly, they are comparing their lib specifically
> to
> > the
> > > > > > > Photoshop feature that I mentioned:
> > > > > https://pngquant.org/vsphotoshop.html
> > > > > > > In the 2000's, the PS implementation for image
> > quantization,
> > > > > optimization
> > > > > > > and compression (profanely named "Save for Web") had
> > remained
> > > > > > > unparalleled for years, that's why it was one of my first
> > > > > thoughts.
> > > > > > >
> > > > > > > I hadn't heard about libimagequant before, but it seems
> to
> > do
> > > > > exactly
> > > > > > > what is needed for rescaling PAL8 to PAL8 at a high
> > quality.
> > > > > > >
> > > > > > > Before spending any time on it, I wanted ask whether that
> > library
> > > > > > > would be acceptable for adding it as a (GPLv3) reference
> to
> > the
> > > > > > > project and as a dependency to my filter?
> > > > > > >
> > > > > >
> > > > > > External dependencies for what is supposed to be a rather
> > base
> > > > > feature
> > > > > > are not ideal.
> > > > >
> > > > > +1
> > > > >
> > > > > i see no reason why this would need an external dependancy
> > > > > we have filters implementing much more complex things than
> > scaling a
> > > > > pal8
> > > > > image
> > > >
> > > > I dislike GPL3 as well and needing a library for this might
> > > > seem to be a bit too much for solving the problem.
> > > > I'm just not sure whether it can be solved easily. I mean it
> > > > can easily be solved in a basic way, but I'm not sure whether
> > > > it would be possible to achieve the same quality in results.
> > > >
> > > > I've put up here some examples and also a comparison of results
> > > > from palettegen+paletteuse and Photoshop(which I assume to
> > > > be at the same level of quality like libimagequant).
> > > >
> > > >
> > https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f
> > >
> > > is this comparing RGB vs RGBA palettes ?
> > > I would imagine that for smooth edges of subtitles you need
> partly
> > > transparent pixels. I am not sure palettegen supports that
> > >
> > > please fix palettegen if that is the issue. Limiting a palette to
> > > 0% transparent and having 1 100% transparent color hardcoded may
> be
> > the
> > > problem here. Try handling alpha the same way red / green / blue
> is
> > handled
> 
> I already did that and it improved the output a lot. Not as good
> as PS, but it's OK for that simple font example.
> (I have updated the Gist and added the yellow-font result from
> the changed filter)
> 
> > > Anyway i do not buy that palette generation is a super hard
> > problem.
> 
> Not anymore in the year 2021 as the algorithms are known and
> established,
> but there are still many image editing programs that can't do it
> well.
> 
> Neither does the palettegen implementation btw.
> 
> I have added another conversion example to the Gist, comparing
> the results from PS, PngQuant, paletteuse/gen (normal +
> patched for alpha support).

ELBG is quite good. I have submitted a patch to add alpha support
for this as well. Gist is updated with example results.

softworkz