[Ffmpeg-devel] assertion failure in epzs_motion_search_internal

[Trent Piepho]

On Tue, 12 Dec 2006, Michael Niedermayer wrote:
> On Tue, Dec 12, 2006 at 12:59:36AM -0800, Trent Piepho wrote:
> > Go back to look at macroblock 73,17:
> >
> > ff_estimate_b_frame_motion: estimate b forward
> > ff_estimate_motion_b: 73,17 P_TOPRIGHT is <-384,-288> qpels
> > sab_diamond_search: minima 0 is 0,0 7371 from map eb800000 = 3ae 0 0, key eba00400 = 3ae 400 400
> > sab_diamond_search: minima 1 is 1023,-200 4739 from map eb79c3ff = 3ad 738 3ff, key eb99c7ff = 3ae 338 7ff
> > sab_diamond_search: minima 2 is -96,-73 6976 from map eb7db7a0 = 3ad 7b6 7a0, key eb9dbba0 = 3ae 3b7 3a0
> > sab_diamond_search: minima 3 is -97,-72 6999 from map eb7dbf9f = 3ad 7b7 79f, key eb9dc39f = 3ae 3b8 39f
> > sab_diamond_search: minima 4 is -96,-72 6991 from map eb7dbfa0 = 3ad 7b7 7a0, key eb9dc3a0 = 3ae 3b8 3a0
> > sab_diamond_search: minima 5 is -95,-72 6983 from map eb7dbfa1 = 3ad 7b7 7a1, key eb9dc3a1 = 3ae 3b8 3a1
> > sab_diamond_search: minima 6 is -89,-72 6935 from map eb7dbfa7 = 3ad 7b7 7a7, key eb9dc3a7 = 3ae 3b8 3a7
> > sab_diamond_search: minima 7 is -96,-71 7018 from map eb7dc7a0 = 3ad 7b8 7a0, key eb9dcba0 = 3ae 3b9 3a0
> > sab_diamond_search: result is <1023,-200> pels, allowed range is -1184..112, -288..448
> > ff_estimate_motion_b: 73,17 got <1023,-200> pels from ezps
> > ff_estimate_motion_b: 73,17 got <4092,-800> qpels from sub motion
>
> IIRC none of the minima should be out of range ever
> the problem likely is that map is filled with
>
> (y<<ME_MAP_MV_BITS) + x + map_generation
>
> in this each x and y have ME_MAP_MV_BITS bits space (11bit each) which is
> enough for -1024 .. 1023, if the motion vector is larger it overflows into
> the next field, and the sab extraction routine gets confused, simply
> adding a clip() in the map -> minima initaization should fix this

This appears to be what's happening.  I've tracked back where the values
outside the range come:

ff_estimate_b_frame_motion: estimate b forward
ff_estimate_motion_b: 73,17 P_TOPRIGHT is <-384,-288> qpels
epzs_motion_search_internal:1047 CHECK_MV: map[32] = eb7dbfa0 from <-96,-72> 6217
epzs_motion_search_internal:1048 CHECK_MV: map[24] = eb7db7a0 from <-96,-73> 6202
epzs_motion_search_internal:1049 CHECK_MV: map[40] = eb7dc7a0 from <-96,-71> 6244
epzs_motion_search_internal:1050 CHECK_MV: map[31] = eb7dbf9f from <-97,-72> 6225
epzs_motion_search_internal:1051 CHECK_MV: map[33] = eb7dbfa1 from <-95,-72> 6209
epzs_motion_search_internal:1054 CHECK_MV: map[7] = eb79c3ff from <-1025,-199> 4229
epzs_motion_search_internal:1055 CHECK_MV: map[39] = eb7dbfa7 from <-89,-72> 6161
sab_diamond_search: minima 0 is 0,0 7371 from map eb800000 = 3ae 0 0, key eba00400 = 3ae 400 400
sab_diamond_search: minima 1 is 1023,-200 4739 from map eb79c3ff = 3ad 738 3ff, key eb99c7ff = 3ae 338 7ff

The problem is in the call to CHECK_MV in epzs_motion_search_internal()
line 1054.  <-1025,-199> is getting changed to <1023,-200>.  Line 1054 is:

        CHECK_MV(P_LEFT[0]    >>shift, P_LEFT[1]    >>shift)

So, look at where mb 72,17 came from:

ff_estimate_motion_b: 72,17 P_TOPRIGHT is <-356,-288> qpels
epzs_motion_search_internal:1047 CHECK_MV: map[24] = eab9c410 from <-1008,-199> 4469
epzs_motion_search_internal:1048 CHECK_MV: map[16] = eab9bc10 from <-1008,-200> 4450
epzs_motion_search_internal:1049 CHECK_MV: map[32] = eab9cc10 from <-1008,-198> 4486
epzs_motion_search_internal:1050 CHECK_MV: map[23] = eab9c40f from <-1009,-199> 4420
epzs_motion_search_internal:1051 CHECK_MV: map[25] = eab9c411 from <-1007,-199> 4521
epzs_motion_search_internal:1055 CHECK_MV: map[0] = eab9c3f8 from <-1032,-199> 4042
epzs_motion_search_internal:1056 CHECK_MV: map[39] = eabdbfa7 from <-89,-72> 6488
epzs_motion_search_internal:1067 CHECK_MV: map[0] = eac00000 from <0,0> 7384
sab_diamond_search: minima 0 is 0,0 7384 from map eac00000 = 3ab 0 0, key eae00400 = 3ab 400 400
sab_diamond_search: minima 1 is -1008,-200 4480 from map eab9bc10 = 3aa 737 410, key ead9c010 = 3ab 338 10
sab_diamond_search: minima 2 is -1009,-199 4450 from map eab9c40f = 3aa 738 40f, key ead9c80f = 3ab 339 f
sab_diamond_search: minima 3 is -1008,-199 4475 from map eab9c410 = 3aa 738 410, key ead9c810 = 3ab 339 10
sab_diamond_search: minima 4 is -1007,-199 4551 from map eab9c411 = 3aa 738 411, key ead9c811 = 3ab 339 11
sab_diamond_search: minima 5 is -1008,-198 4516 from map eab9cc10 = 3aa 739 410, key ead9d010 = 3ab 33a 10
sab_diamond_search: minima 6 is -89,-72 6560 from map eabdbfa7 = 3aa 7b7 7a7, key eaddc3a7 = 3ab 3b8 3a7
ff_estimate_motion_b: 72,17 got <-1024,-199> pels from ezps
ff_estimate_motion_b: 72,17 got <-4098,-796> qpels from sub motion

It's worth nothing that the shifting going on here does something strange.
The result from the full pel search is -1024, after the qpel search it's
-4098 qpels, which is -1024.5 pels.  However, when the value is put into
P_LEFT for the next macroblock, it becomes -1025.  It seems like if
<-1025,-199> was better than <-1024,-199> then it would have been what the
full pel search would have returned in the first place.

> motion vectors >1023 fullpel units are useless anyway ...

If motion verctors >1023 are useless, then why is the motion search code
finding them to be the best motion vectors?  Why is it even searching that
far in the first place, if those kinds of motion vectors are useless?