[FFmpeg-devel] request for feedback on video codec idea

[Hendrik Leppkes]

On Wed, Oct 14, 2015 at 9:04 PM, Paul B Mahol <onemda at gmail.com> wrote:
> Dana 14. 10. 2015. 20:42 osoba "Hendrik Leppkes" <h.leppkes at gmail.com>
> napisala je:
>>
>> On Wed, Oct 14, 2015 at 8:08 PM, Roger Pack <rogerdpack2 at gmail.com> wrote:
>> > Lacking a better place to debate this, I would like to ask some
>> > questions on a video codec idea...
>> >
>> > The goal is basically to create a very fast lossless screen capture
>> > codec (i.e. in the input there will be lots of repeated "colors" of
>> > neighboring pixels, not a lot of dynamic content between frames).
>> >
>> > I have become aware of some "fast" compression tools like LZO, LZ4,
>> > density, etc.  It seems like they all basically compress "the first
>> > 64KB then the next 64KB" or something like that [1].
>> >
>> > My idea is to basically put pixels of the same position, from multiple
>> > frames, "together" in a stream, then apply normal (fast) compression
>> > algorithms to the stream.  The hope being that if the pixels are the
>> > "same" between frames (presumed to be so because of not much dynamic
>> > content), the compression will be able to detect the similarity and
>> > compress it well.
>> >
>> > For instance, given 3 frames of video ("one after another" from the
>> > incoming video stream), "combine them" into one stream like:
>> > pixel 1 frame 1, pixel 1 frame 2, pixel 1 frame 3, pixel 2 frame 2,
>> > pixel 2 frame 2, pixel 2 frame 3 ...
>> >
>> > then basically apply LZ4 or density algorithm to those bytes.
>> >
>> > The theory being that if there is a lot of repeated content between
>> > frames, it will compress well.
>> >
>> > The basic need/desire for this was that huffyuv, though super fast at
>> > encoding (it basically zips the frame), seems to create *huge* files,
>> > I assume because "each frame is an I-frame" so it has to re encode
>> > everything each frame.   And also the egotistical desire to create the
>> > "fastest video codec in existence" in case the same were useful in
>> > other situations (i.e. use very little cpu--even huffyuv uses quite a
>> > bit of cpu) :)
>> >
>> > Any feedback on the concept?
>> > Also does anything similar to this already exist? (though should I
>> > create my new codec, it would be open source of course, which is
>> > already different than many [probably efficient] screen capture codecs
>> > out there).
>> >
>> > Thanks.
>> > -roger-
>> >
>>
>> I can't really comment on the merits of this compression scheme, but
>> note that you might have trouble with the ffmpeg API when handling
>> such a codec, since every single data packet would result in X output
>> frames (3 in your example) - this is not a scheme that avcodec can
>> really represent well.
>> On top of that, containers might have troubles timestamping this properly.
>
> Well each packet would have several frames, I assume encode2 can handle it.
>

Encoding can, decoding has trouble.