[FFmpeg-devel] request for feedback on video codec idea

[Hendrik Leppkes]

On Wed, Oct 14, 2015 at 8:08 PM, Roger Pack <rogerdpack2 at gmail.com> wrote:
> Lacking a better place to debate this, I would like to ask some
> questions on a video codec idea...
>
> The goal is basically to create a very fast lossless screen capture
> codec (i.e. in the input there will be lots of repeated "colors" of
> neighboring pixels, not a lot of dynamic content between frames).
>
> I have become aware of some "fast" compression tools like LZO, LZ4,
> density, etc.  It seems like they all basically compress "the first
> 64KB then the next 64KB" or something like that [1].
>
> My idea is to basically put pixels of the same position, from multiple
> frames, "together" in a stream, then apply normal (fast) compression
> algorithms to the stream.  The hope being that if the pixels are the
> "same" between frames (presumed to be so because of not much dynamic
> content), the compression will be able to detect the similarity and
> compress it well.
>
> For instance, given 3 frames of video ("one after another" from the
> incoming video stream), "combine them" into one stream like:
> pixel 1 frame 1, pixel 1 frame 2, pixel 1 frame 3, pixel 2 frame 2,
> pixel 2 frame 2, pixel 2 frame 3 ...
>
> then basically apply LZ4 or density algorithm to those bytes.
>
> The theory being that if there is a lot of repeated content between
> frames, it will compress well.
>
> The basic need/desire for this was that huffyuv, though super fast at
> encoding (it basically zips the frame), seems to create *huge* files,
> I assume because "each frame is an I-frame" so it has to re encode
> everything each frame.   And also the egotistical desire to create the
> "fastest video codec in existence" in case the same were useful in
> other situations (i.e. use very little cpu--even huffyuv uses quite a
> bit of cpu) :)
>
> Any feedback on the concept?
> Also does anything similar to this already exist? (though should I
> create my new codec, it would be open source of course, which is
> already different than many [probably efficient] screen capture codecs
> out there).
>
> Thanks.
> -roger-
>

I can't really comment on the merits of this compression scheme, but
note that you might have trouble with the ffmpeg API when handling
such a codec, since every single data packet would result in X output
frames (3 in your example) - this is not a scheme that avcodec can
really represent well.
On top of that, containers might have troubles timestamping this properly.

- Hendrik