[FFmpeg-devel] [PATCH] RoQ video encoder (take 4)

[Eric Lasota]

Michael Niedermayer wrote:
> Hi
>
> On Thu, Jun 14, 2007 at 07:44:55AM -0400, Eric Lasota wrote:
>   
>> Michael Niedermayer wrote:
>>     
>>> IIRC theres a used count and if its 0 we drop a codebook
>>> so if we just consider codebooks with a small used value (like 1)
>>> for droping then there shouldnt be much reindexing needed
>>>   
>>>       
>> I still would imagine that would be difficult, a good codebook generator 
>> produces very even distribution.
>>     
>
> a good codebook generator produces an optimal codebook the problem isnt
> the generator but rather that not all blocks will use the 4x4 and 2x2
> codebooks, they will rather just use one or the other or they will use
> motion compensation from the previous frame
> -> the points the codebook generator saw are no longer correct and as
> such the codebook is no longer optimal and should be regenerated and
> or less usefull codebooks droped
>   

They aren't "using one or the other," the 4x4 codebook is built from the 
2x2 codebook.  Any point that is not skipped or motion-compensated is a 
valid input for the 2x2 books.

The 4x4 codebook is frequently non-optimal anyway simply because it's 
indexed to the 2x2 codebook.  The encoder used to use the values from 
the codebook generator to figure out which 4x4 entry to index to, which 
usually resulted in almost the whole thing being used, yet as soon as I 
fixed that so it indexed them to the 4x4 codebook AFTER it was rebuilt 
out of the 2x2 entries, significant portions (25-50%) of the 4x4 
codebook were continually being dropped.

In other words, the 4x4 codebook probably doesn't need to be optimized 
anyway because it's dropping so many entries regardless, and probably 
can't be regenerated because it frequently doesn't match expectations 
when generating.

The 2x2 codebook is a larger concern, but since EVERYTHING is made using 
the 2x2 codebook, it is much harder to drop an entry without a spilling 
over to the 4x4 entries and being forced to do everything over.

I really don't think dropping codebook entries would be as viable as 
just trying to regenerate the codebook at a different size and 
recalculate everything, or even attempting to drop the codebook completely.

As for the 256 4x4-entry phenomena, I think it should just be left the 
way it is now.  It is an extremely rare case for reasons mentioned 
above, most of the time it happens, the number of 2x2 entries is nearly 
256 anyway, and having more 4x4 entries helps improve codebook viability 
in cases where it doesn't trigger.

Another thing that might be fun to try: Error diffusion to mitigate the 
banding artifacts.



By the way, why does the rate control target rate lambda instead of 
total distortion, since base distortion can vary and cause quality to 
shift while lambda remains the same?