[Ffmpeg-devel-irc] ffmpeg-devel.log.20141223

[burek]

[00:05] <kierank> threading doesn't appear to do anything
[00:06] <kierank> in fact it makes it slower
[00:06] <Daemon404> i dont recall it working
[00:06] <Daemon404> i think i tried before
[00:06] <kierank> threads are spawned but they do nothing
[00:07] <nevcairiel> should ask michaelni about it, he wrote the frame-thread-encoder
[00:08] <kierank> michaelni: ping
[00:08] <ubitux> kierank: try to add CODEC_CAP_INTRA_ONLY
[00:09] <kierank> did that
[00:09] <kierank> ./ffmpeg_g  -f rawvideo -s 4096x2160 -pix_fmt yuv422p -i /dev/zero -threads 4 -vcodec v210 -f rawvideo -y /dev/null
[00:10] <michaelni> the following works quite a bit faster here: ./ffmpeg -i matrixbench_mpeg2.mpg -vcodec mjpeg -threads 12 -an -f null -
[00:10] <michaelni> than with threads 1
[00:11] <michaelni> v210 might be limited by memory speed and not cpu dunno
[00:11] <kierank> one thread uses all the cpu
[00:11] <kierank> and others do nothing
[00:13] Action: nevcairiel tries
[00:16] <ubitux> reminds me of when i added threading to interlace
[00:16] <ubitux> or whatever that filter was
[00:17] <michaelni> kierank, i get a huge speedup when i comment the whole v210 encoder out, a massive 15%
[00:18] <michaelni> so thats the max that this can get faster with 0% thread overhead and infinite number of cpus
[00:23] <michaelni> it could need a memcpy more with threads > 1 which would explain the slowdown, not sure but it would be a plausible explanation
[00:24] <ubitux> i go from 300 to 900 fps here is i comment out the processing block of encode_frame()
[00:24] <ubitux> if*
[00:24] <nevcairiel> i get about the same performance with threads 1 or threads 8, although around twice as much cpu usage with the second
[00:24] <ubitux> and threading indeed has no effect
[00:25] <michaelni> ubitux, you get 900fps with 4096x2160 ?
[00:25] <ubitux> yes
[00:25] <ubitux> http://pastie.org/9795063  doing this, right?
[00:26] <nevcairiel> frame_thread_encoder doesnt seem to be up-to-date with proper refcounting yet
[00:26] <nevcairiel> it will copy everytime, it seems
[00:26] <kierank> yes but if you comment out the block you're doing nothing
[00:27] <ubitux> i'm just skipping the cpu time of the encoder, where the parallelism would make sense?
[00:27] <ubitux> or am i missing something?
[00:28] <kierank> dunno
[00:29] <kierank> ah yeah av_image_copy
[00:29] <kierank> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/frame_thread_encoder.c#L269
[00:29] <ubitux> also:
[00:29] <ubitux>   65.89%  ffmpeg_g  ffmpeg_g            [.] ff_v210_planar_pack_8_avx.loop
[00:29] <ubitux>   31.80%  ffmpeg_g  [kernel.kallsyms]   [k] __clear_user
[00:29] <ubitux> for me, the encoder is indeed expensive, which makes sense
[00:29] <wm4> is this clear_user thing caused by mmap memory allocation?
[00:29] <ubitux> no idea
[00:30] <wm4> it sounds like it could be the kernel thing that clears pages for mmap()
[00:32] <ubitux> http://pastie.org/pastes/9795071/text (./ffmpeg -f rawvideo -s 4k -pix_fmt yuv422p -i /dev/zero -c:v v210 -f null -)
[00:32] <kierank> yeah it's that memcpy
[00:33] <nevcairiel> shouldn't be too hard to fix that
[00:33] <ubitux> "memcpy is murder"©®"
[00:33] <nevcairiel> to take refcounting into account
[00:33] <wm4> also malloc is murder?
[00:33] <kierank> yes
[00:33] <kierank> should use memory pools
[00:34] <nevcairiel> it actually does for those frame buffers
[00:34] <wm4> so what does it allocate here? packets?
[00:34] <nevcairiel> frames
[00:34] <kierank> michaelni: based on your knowledge of https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/frame_thread_encoder.c#L269 could it be easily updated to refcounting?
[00:34] <nevcairiel> the framethreadencoder is pre-refcounting, so it allocates and copies every frame before sending it to the worker threads
[00:35] <nevcairiel> kierank: its pretty simple actually, replace the ff_get_buffer with av_frame_ref(new, frame), cleanup the junk thats now redundant, and it should work
[00:35] <michaelni> kierank, i think it should be easy to update
[00:36] <nevcairiel> the whole if thing can go really, the flag it checks for is deprecated anyway
[00:36] Action: michaelni wonders what computer ubitux has that gets 900fps ...
[00:36] <ubitux> you don't get that with my diff?
[00:37] <ubitux> and cli?
[00:37] <ubitux> http://pastie.org/9795063 + ./ffmpeg -f rawvideo -s 4k -pix_fmt yuv422p -i /dev/zero -c:v v210 -f null -
[00:38] <ubitux> 950 fps actually
[00:38] <ubitux> (vs 300 with #if 1)
[00:40] <michaelni> ubitux, i only get 660 with that
[00:40] Action: michaelni needs to buy a new box
[00:40] <ubitux> vs how much?
[00:40] <ubitux> (without diff)
[00:42] <michaelni> 237 now, before it was 170 vs 200 but i had various other changes and dont remember with what it was build
[00:42] <ubitux> so proportionnal results i guess
[00:43] <ubitux> (Intel(R) Core(TM) i7-4770R CPU @ 3.20GHz  i have this btw)
[00:43] <wm4> on my box I get only 27 fps
[00:43] <nevcairiel> kierank: try this http://pastebin.com/7ncTLKyw .. feel free to do whatever with it if it works, i will be leaving for the holidays shortly
[00:44] <ubitux> (it's a minipc crap though)
[00:44] <nevcairiel> actually, that leaks a frame in an error case, should add cleanup there
[00:44] <michaelni> Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
[00:44] <ubitux> gcc 4.9 too
[00:45] <ubitux> if that makes any diff
[00:45] <ubitux> :p
[00:45] <kierank> nevcairiel: a little better, yes
[00:45] <kierank> thanks
[00:45] <ubitux> michaelni: http://ark.intel.com/compare/63697,76642
[00:45] <nevcairiel> kierank: http://pastebin.com/ttWaEGsx now with cleanup in error case
[00:46] <nevcairiel> anyway, good night!
[00:46] <kierank> nevcairiel: cheers
[00:46] <ubitux> michaelni: you probably have a faster cpu but avx2 might make a difference here for $reason
[00:46] <kierank> well a modern cpu helps in general
[00:47] <ubitux> his cpu is superior to mine in many regards
[00:48] <ubitux> (even price!)
[00:48] <nevcairiel> its 2 generations old, the IPC improvements alone will make quite a dent there
[00:49] <nevcairiel> the extra cores dont make it faster if you both test with -threads 4 for example :d
[00:56] <kierank> dammit my git format-patch is broken today so I can't mail the patch =p
[00:57] <kierank> sigh
[00:59] <pross> ubitux: this a dick size measuring contest?
[00:59] <ubitux> haha
[00:59] <wm4> no a contest of measuring a format that was made by dicks
[01:01] <kierank> wm4: lolwut
[01:01] <kierank> wm4: and you would do what exactly?
[01:01] <kierank> this is going to be good
[01:06] <michaelni> nevcairiel, is the mutex actually needed over the ref ?
[01:09] <michaelni> anyway, if noone objects ill apply nevcairiels patch we can remove the mutex later
[01:16] <kierank> i don't think the mutex is necessar
[01:16] <kierank> or hmm
[01:16] <kierank> maybe it is
[01:16] <kierank> not sure
[01:17] <ubitux> let's ask helgrind!
[01:17] <ubitux> oh my bad, we can't
[01:17] <ubitux> :(
[01:19] <ubitux> btw, i think fate doesn't cover threaded encoding
[01:20] <ubitux> (6da57043eabdcde99e8411da3628f2249e951e66)
[01:20] <ubitux> which make all the threading tests for filters pointless too
[01:22] <ubitux> anyway, 'night
[01:22] <michaelni> ubitux, n8
[01:31] <wm4> why would the mutex be necessary?
[01:32] <wm4> if the source frame is accessed concurrently (writing), then the code is wrong or less simply than it needs to be anyway
[01:37] <cone-154> ffmpeg.git 03Hendrik Leppkes 07master:1ffcf6ac9045: frame_thread_encoder: use ref-counting to avoid memcpy of all input frames
[03:49] <cone-154> ffmpeg.git 03Michael Niedermayer 07master:13306dde2240: tools/probetest: support testing a single specified input format
[04:46] <cone-154> ffmpeg.git 03Michael Niedermayer 07release/2.5:959ab06c68f8: Changelog: update for 2.5.2
[04:58] <cone-154> ffmpeg.git 03James Almer 07master:32c836cb1107: x86/vp9: remove duplicate function prototypes
[05:00] <cone-154> ffmpeg.git 03James Almer 07fatal: ambiguous argument 'refs/tags/n2.5.2': unknown revision or path not in the working tree.
[05:00] <cone-154> Use '--' to separate paths from revisions
[05:00] <cone-154> refs/tags/n2.5.2:HEAD: x86/vp9: remove duplicate function prototypes
[05:05] <llogan> michaelni: why another release so close to 2.5.1?
[05:25] <michaelni> llogan, bugfixes like more stuff found by Mateusz "j00ru" Jurczyk and Gynvael Coldwind
[05:26] <michaelni> llogan, also feel free to update the download page or ill do it after i wakeup tomorrow
[07:07] <favonia> Bug Report: configure generates by default "LIBDIR=$(DESTDIR)${prefix}/lib" and "SHLIBDIR=$(DESTDIR)${LIBDIR}" in config.mak which places two copies of $DESTDIR in $SHLIBDIR. Not sure how to fix this without having a special case in $PATHS_LIST. :-(
[07:14] <jamrial> favonia: see if this patch fixes it http://ffmpeg.org/pipermail/ffmpeg-devel/2014-December/166828.html
[07:16] <jamrial> and if it doesn't then consider commenting here https://trac.ffmpeg.org/ticket/4199 or open a new ticket
[07:16] <aetasx> if built as static-only, will that still result in the directories being created?
[07:20] <jamrial> only those needed for the static libs i assume
[07:21] <favonia> jamrial: Works beautifully. Thanks. I should have checked the ticket system.
[07:21] <jamrial> no prob. that's a patch waiting review, so it should be commited to the tree soon
[07:22] <aetasx> hey jamrial, mind a general linking question?
[07:23] <jamrial> ask in general instead of just me. more chances to get an answer if i don't have it :p
[07:23] <aetasx> well maybe I have such high respects for your answers that I'm not willing to accept anything less :p
[07:27] <aetasx> anyway, it was my understanding that when building static libs that included other static libs, the necessary symbols would be included in the new library so that no dependencies were needed from the included ones
[07:41] <aetasx> taking the easy way out of answering me.  tsk tsk
[07:42] <favonia> jamrial: On the second thought, libdir and shlibdir are checked twice through set_default, and the second times are always redundant.  Probably unavoidable. :-/
[07:42] <aetasx> he wussed out and split but Im sure he'll chalk it up to a ping timeout
[07:59] <ubitux> with the patch from nevcairiel, the memcpy disappears and benchmarks between no threading and threading are consistent but above -threads 2 the fps crumble quickly
[08:00] <ubitux> like... -threads: 1 ’ ~300fps, 2 ’ ~350fps, 3 ’ ~230fps, 4 ’ ~140fps, ...
[08:01] <ubitux> (i have 4 core, 8 virtuals)
[08:01] <aetasx> virtual as in HT?
[08:01] <ubitux> yed
[08:01] <ubitux> yes
[08:02] <aetasx> is that encoder-specific or global?
[08:02] <ubitux> i haven't test the others
[08:03] <aetasx> ah.  which was this on?
[08:04] <ubitux> the one we talked about "yesterday", v210
[08:05] <ubitux> on a side note, i'm not exactly sure what's the purpose of CODEC_CAP_INTRA_ONLY, but it seems mandatory to avoid memory failures
[08:05] <ubitux> (at exit)
[08:08] <aetasx> can't imagine what it does based on its name alone
[08:08] <nevcairiel> michaelni: i dont think its still required, but threading stuff can give you headaches, and the mutex was there before, so i just kept it :d
[08:11] <ubitux> aetasx: it means the codec supports intra only, but that is redundant with the codec desc
[08:12] <ubitux> typically i would say it's not necessary if the codec desc says intra only; if the desc supports something else than intra but our encoder only supports intra, then it makes sense
[08:12] <nevcairiel> well you could have a codec that supports both intra and inter frames, and an encoder which does intra only
[08:12] <ubitux> but needing both sounds a bit weird
[08:13] <nevcairiel> the codec descriptors only serve informational purposes, while the codec flags actually control behaviour
[08:14] <aetasx> how did you guys start out contributing to ffmpeg considering the size and history behind such a project?  I imagine a lot of things exist the way they are due to history or simply a matter of style
[08:14] <ubitux> we all come from mplayer, remember?
[08:14] <ubitux> :D
[08:14] <nevcairiel> not me!
[08:14] <aetasx> mplayer's still big too :p
[08:14] <ubitux> didn't you ask that question already btw>
[08:15] <aetasx> someone suggested I asked the actual devs instead
[08:15] <ubitux> i remember someone replying "video players"
[08:16] <nevcairiel> For me, I was fed up with the terrible situation of video players on Windows, so I started working on some solutions myself, and once you do that, all roads lead to ffmpeg eventually
[08:16] <aetasx> I wasn't asking via which avenue they came from though, the question could really be applied to almost any large open-source project
[08:16] <ubitux> broken subtitles in mplayer for me
[08:16] <ubitux> but i probably failed at that mission because ffmpeg still doesn't support them well
[08:17] <aetasx> I can't say I've ever really had a need to deal with them in ffmpeg
[08:17] <ubitux> aetasx: it's often that shit is broken and no one wants to fix it
[08:18] <ubitux> hate driven contributions
[08:18] <aetasx> sounds like my rtmpdump patch except has other people actually working on it ;)
[12:54] <ubitux> saste: i have a patchset pending for making showwaves output the picture of the whole track
[12:54] <ubitux> (like many things, i need to get it done)
[12:55] <ubitux> actually, just one patch remaining after what i did last time
[13:04] <saste> ubitux, nice
[15:07] <cone-192> ffmpeg.git 03Nicolas George 07master:40948819fcfe: doc/faq: explain pkg-config basic setup.
[16:04] <cone-192> ffmpeg.git 03Michael Niedermayer 07master:ba8d2c90cd98: avformat/rtpdec_xiph: use av_freep() to avoid leaving stale pointers in memory
[16:04] <cone-192> ffmpeg.git 03Michael Niedermayer 07master:bb858e67e50a: avformat/rtpdec_latm: Use av_freep() to avoid leaving stale pointers in memory
[16:04] <cone-192> ffmpeg.git 03Michael Niedermayer 07master:f7f4a901748c: avformat/rtpdec: Use av_freep() to avoid leaving stale pointers in memory
[16:23] <YaronT> Hello, does anyone know how to add CORS header (Access-Control-Allow-Origin) to ffserver?
[17:32] <arwa> How to rebase a commit with a particular patch file?
[18:53] <cone-192> ffmpeg.git 03Michael Niedermayer 07master:0d7a14e236eb: avformat/avidec: optimize probe
[18:53] <cone-192> ffmpeg.git 03Michael Niedermayer 07master:e01467241fe7: avformat/aea: reduce false positives in probing
[20:39] <kierank> lol http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=0d7a14e236eb05ae0473d2df3276a26f4b6596c8
[20:41] <Daemon404> ricing
[20:43] <kierank> yup
[20:50] <llogan> crappy tickets
[20:51] <ubitux> llogan: you should not encourage users to request support in the trac
[20:51] <ubitux> please redirect them to irc or the user mailing list
[20:51] <ubitux> (’ #4204 should be closed)
[20:51] <llogan> then close it
[20:52] <llogan> first i wanted to see if there was a small chance in hell that it was valid
[20:52] <ubitux> well you seemed motivated to work process tickets so..
[22:16] <pomaranc> hi, I'm trying to make ffmpeg wait for a signal to start processing input, ie start probing and then the usual stuff after that signal -- is there a better place for it than avformat_open_input or av_probe_input_buffer2?
[22:17] <pomaranc> I need it to continue buffering until the signal is received
[00:00] --- Wed Dec 24 2014