[MPlayer-dev-eng] [PATCH] hack x264_encoder_encode returns 0 incorrectly when flushing

[Reimar Döffinger]

Hello,
the version of libx264 I have lying around (seems to be from 29.6.09)
seems to have an issue with handling delayed frames and multiple threads: when
x264_encoder_encode is called with NULL picture to get delayed frames, with n
threads it returns n-1 times 0 before getting to the delayed frames (I admit
I only tested with input consisting of 4 pngs and up to threads = 4).
Attached patch hacks around the issue for threads values of up to 255
without too much overhead I hope.
Can someone make sure the x264 developers are informed about this? (and
if this behaviour is intended I guess it should be better documented,
FFmpeg should have the same issue).
-------------- next part --------------
Index: libmpcodecs/ve_x264.c
===================================================================
--- libmpcodecs/ve_x264.c	(revision 29412)
+++ libmpcodecs/ve_x264.c	(working copy)
@@ -218,10 +218,14 @@
 static int control(struct vf_instance_s* vf, int request, void *data)
 {
     h264_module_t *mod=(h264_module_t*)vf->priv;
+    int count = 256; // giant HACK, x264_encoder_encode may incorrectly return 0
+                     // when threads > 1 and delayed frames pending
     switch(request){
         case VFCTRL_FLUSH_FRAMES:
-            if(param.i_bframe)
+            if(param.i_bframe) {
+                while(encode_frame(vf, NULL) == 0 && --count);
                 while(encode_frame(vf, NULL) > 0);
+            }
             return CONTROL_TRUE;
         default:
             return CONTROL_UNKNOWN;