[FFmpeg-user] Command being killed in perceptual hashing / ffmpeg in general

[Aoife Fitzmaurice]

>Does it crash because of oom, or for other reasons?

Yes, it runs out of memory. The memory consumption is huge (80gbs) and swapping 60gbs over. A colleague of mine recreated the problem using test files :

ffmpeg -f lavfi -i testsrc=size=1920x1080:rate=25  -c:v mpeg2video -t 01:03:00 -b:v 35000k -minrate 35000k -maxrate 35000k  -pix_fmt yuv420p -g 12 aftest1.mov && ffmpeg -f lavfi -i testsrc=size=1920x1080:rate=25  -c:v mpeg2video -t 00:28:06 -b:v 35000k -minrate 35000k -maxrate 35000k  -pix_fmt yuv420p -g 12 aftest2.mov


>Could it be that you should use "lldb" instead of gdb?

Thank you, that worked! 

Input: (using files created above)

ifi-mac-pro:ffmpeg admin$ lldb ./ffmpeg_g
(lldb) target create "./ffmpeg_g"
Current executable set to './ffmpeg_g' (x86_64).
(lldb) r -i aftest2.mov  -i  aftest1.mov -filter_complex signature=detectmode=full:nb_inputs=2 -f null -


Output:

(lldb) r -i /Users/admin/aftest2.mov  -i /Users/admin/aftest1.mov -filter_complex signature=detectmode=full:nb_inputs=2 -f null - 
Process 10575 launched: './ffmpeg_g' (x86_64)
ffmpeg version N-87767-g14b1553 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  configuration: --enable-gpl
  libavutil      55. 77.101 / 55. 77.101
  libavcodec     57.106.104 / 57.106.104
  libavformat    57. 82.104 / 57. 82.104
  libavdevice    57.  9.102 / 57.  9.102
  libavfilter     6.106.101 /  6.106.101
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/admin/aftest2.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.71.100
  Duration: 00:28:06.00, start: 0.000000, bitrate: 35000 kb/s
    Stream #0:0(eng): Video: mpeg2video (Main) (xdve / 0x65766478), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 34999 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc57.89.100 mpeg2video
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/admin/aftest1.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.71.100
  Duration: 01:03:00.00, start: 0.000000, bitrate: 35000 kb/s
    Stream #1:0(eng): Video: mpeg2video (Main) (xdve / 0x65766478), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 34999 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc57.89.100 mpeg2video
Stream mapping:
  Stream #0:0 (mpeg2video) -> signature:in0
  Stream #1:0 (mpeg2video) -> signature:in1
  signature -> Stream #0:0 (wrapped_avframe)
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf57.82.104
    Stream #0:0: Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.106.104 wrapped_avframe
Process 10575 stopped=-0.0 size=N/A time=00:28:06.00 bitrate=N/A speed=2.97x    
* thread #1: tid = 0x18e8e, 0x00007fff8a63663f libsystem_malloc.dylib`tiny_free_reattach_region + 214, queue = 'com.apple.main-thread', stop reason = signal SIGKILL
    frame #0: 0x00007fff8a63663f libsystem_malloc.dylib`tiny_free_reattach_region + 214
libsystem_malloc.dylib`tiny_free_reattach_region:
->  0x7fff8a63663f <+214>: movq   %r12, %rdi
    0x7fff8a636642 <+217>: callq  0x7fff8a63247f            ; get_tiny_free_size
    0x7fff8a636647 <+222>: movw   %ax, %r15w
    0x7fff8a63664b <+226>: testw  %r15w, %r15w
(lldb)



Could it be a memory leak, or does perceptual hashing not suit large files at present?

Thank you, 

Aoife
_______________________________________________