[Ffmpeg-devel-irc] ffmpeg-devel.log.20170406

[burek]

[00:01:00 CEST] <cone-983> ffmpeg 03Michael Niedermayer 07master:b02027c79d4b: avcodec/dvbsubdec: Replace fprintf(stderr) by av_log()
[00:01:01 CEST] <cone-983> ffmpeg 03tiejun.peng at foxmail.com 07master:9fe73b0147d3: avformat/ftp: Solve a crash bug when network occur a exception
[00:54:33 CEST] <jamrial> mateo`: you're listed as movenc maintainer, could you check "Add experimental support for Opus in ISO BMFF (MP4)"?
[00:54:57 CEST] <jamrial> tried creating some files with it and firefox reads them just fine
[00:55:15 CEST] <JEEB> I think Paranoialmaniac had some comments on it the last it was discussed
[00:55:32 CEST] <JEEB> although that IIRC was more towards the spec itself than the code specifically, maybe?
[00:56:36 CEST] <jamrial> JEEB: if it was about the patch then he should reply to the relevant thread
[01:38:26 CEST] <BBB> ubitux: it looks like the huffyuv/ffvhuff warnings in tsan are gone now
[01:38:50 CEST] <BBB> ubitux: so I guess that just leaves the h264 patch and the 6 patches around idctdsp to fix the dnxhr/d warnings ATM
[02:18:23 CEST] <atomnuker> so our vorbis decoder is over 3x faster than libvorbis
[02:19:31 CEST] <atomnuker> and our opus decoder is as fast as decoding 256x224 yuv444p10 h264
[02:20:09 CEST] <atomnuker> which is around the same speed as the libvobris decoder
[02:20:10 CEST] <jamrial> that's a curious comparison :p
[02:20:40 CEST] <atomnuker> which is just a bit faster than opus decoding using libopus
[02:21:54 CEST] <atomnuker> which is around 11200 of those video frames per second
[02:25:06 CEST] <atomnuker> I am surprised by how much SIMD gcc generated by itself for opus decoding
[02:25:59 CEST] <atomnuker> though I can't say if it hasn't screwed up and is just loading xmm regs but doing things like scalars
[02:26:59 CEST] <jamrial> atomnuker: what arch? gcc on x86_64 uses sse for fp math, and yeah, all scalar
[02:27:11 CEST] <jamrial> since we disable tree vectorization
[02:29:45 CEST] <atomnuker> yeah, x86_64
[02:31:01 CEST] <atomnuker> now it makes sense, I knew gcc used sse for floats but I didn't connect that to how much xmm regs I saw being used
[02:32:33 CEST] <jamrial> if you use -mavx it will use vex enconding for that, and with -mfma it may even use fmadd/sub where useful
[02:32:58 CEST] <atomnuker> isn't tree vectorization still enabled? check out cb8646af24b
[02:33:19 CEST] <atomnuker> it says its only disabled for versions less than 4.9, did we disable it later?
[02:34:05 CEST] <atomnuker> oh we did, you did rather
[02:34:16 CEST] <atomnuker> fd6dbc53855, 9th of july
[02:52:45 CEST] <iive> yeh, it broke a bunch of small things
[02:53:45 CEST] <atomnuker> disabling it doesn't improve h264 or opus decoding by any significant amount
[03:21:46 CEST] <iive> auto vectorization is not very smart
[03:22:10 CEST] <iive> it might help with simple loops and there are not many left in h264 :D
[04:11:14 CEST] <cone-982> ffmpeg 03Thierry Foucu 07master:ac24a8202a6f: libavformat/mov: Fix memory leak, need to free the chapter tracks array
[12:59:38 CEST] <J_Darnley> Some time ago I remeber that one of you geniuses said they were going to try writing an audio codec using "better" tools than the likes of flac uses.  Did anything become of that?
[12:59:52 CEST] <kierank> was atomnuker 
[13:55:32 CEST] <BBB> does anyone remember the name of that tool to download youtube videos to disk?
[13:56:12 CEST] <atomnuker> BBB: youtube-dl
[13:56:22 CEST] <BBB> tnx!
[13:56:26 CEST] <BBB> and macports has it! \o/
[13:56:56 CEST] <RiCON> someone  was even thinking of using it in unreal engine 4
[14:08:23 CEST] <BBB> hm 32bit compilation is broken
[14:08:27 CEST] <BBB> (on mac)
[14:08:31 CEST] <kkanungo17> I'm a bit confused, the psychoacoustic model is defined in psymodel.c, whose functions are called in aacenc.c, then what's the use of aacpsy.c?
[14:10:08 CEST] <atomnuker> that's where the code that executes is
[14:19:52 CEST] <kkanungo17> so I have to use aacpsy.c functions or the psymodel.c functions to get the block sizes?
[14:20:42 CEST] <atomnuker> I said before, you need s->psy.model->window to get that
[14:21:21 CEST] <kkanungo17> ok
[15:24:43 CEST] <BBB> so nobody wants to review that frame-mt patch hm ...
[15:24:53 CEST] <BBB> I could just push it and see if hell breaks lose?
[15:25:07 CEST] <BBB> What Could Possibly Go Wrong [tm]
[15:34:44 CEST] <ubitux> that's often the best way to get a review
[15:35:13 CEST] <ubitux> cvslog is the ml for the the real reviews
[15:49:44 CEST] <ubitux> http://b.pkh.me/gen-samples-db.py ’ ./gen-samples-db.py ~/samples samples-db.json; jq ... < samples-db.json
[15:49:53 CEST] Action: ubitux now needs to learn how to use jq properly
[15:50:37 CEST] <ubitux> anyway, if you're like me lost with ETOOMANY samples, that might help locating them based on various props
[15:55:01 CEST] <BBB> ubitux: ok I guess Ill push it then? :-p
[15:55:10 CEST] <BBB> I think it would clean up some tsan warnings
[15:55:24 CEST] <BBB> would be fun to get that near-clean
[15:55:39 CEST] <ubitux> i agree :)
[15:55:59 CEST] <ubitux> so you're pushing h264 + frame-mt?
[15:56:11 CEST] <ubitux> (+dnxhd?)
[16:00:03 CEST] <BBB> I can push dnxhd if you want
[16:00:16 CEST] <BBB> I think I addressed all review concerns but Id have to look over briefly
[16:00:25 CEST] <ubitux> the greener tsan is the happier i am
[16:01:41 CEST] <BBB> Im not sure Ill go for entirely green TBH
[16:01:47 CEST] <ubitux> :(
[16:02:04 CEST] <BBB> the mpegvideo code is not the friendliest ;)
[16:02:24 CEST] <BBB> let me first push these patches
[16:05:16 CEST] <BBB> omg
[16:05:22 CEST] <BBB> Im going through some of the gsoc proposals
[16:05:35 CEST] <BBB> this one is just facepalm-worthy: At the end of the internship I will have a better understanding about professional atmosphere.
[16:05:57 CEST] <BBB> or this one: I have no time to write an abstract since I was just now knowing this project and it's just left 30 min for me to complete application... maybe we can talk?
[16:07:19 CEST] <ubitux> "let's solve this with a phone call, i'm a busy man"
[16:13:16 CEST] <BBB> jya: dont forget the first and second question ;)
[16:13:28 CEST] <BBB> jya: (you only answered the third so far :-p)
[16:17:08 CEST] <jamrial> nevcairiel: will you push the content light level patch, or should i?
[16:17:15 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:083300bea935: pthread_frame: allow per-field ThreadFrame owners.
[16:17:15 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:40cbd686dc84: idct_arm: remove use of ff_put/add_pixels_clamped function pointer.
[16:17:16 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:b51d7d89f8bb: x86/xvididct: remove use of ff_put/add_pixels_clamped function pointer.
[16:17:18 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:c9d98c5649ac: cavs: convert idct from inline asm to yasm.
[16:17:18 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:2f0591cfa3b7: cavs: add a sse2 idct implementation.
[16:17:19 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:e0c205677f6b: x86/simple_idct: add explicit sse2 simple_idct_put/add versions.
[16:17:20 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:32baeafeee4f: jrev/xvid: hardcode use of C put/add_pixels_clamped.
[16:17:21 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:83ae7e6350cf: x86/idctdsp_init: reindent.
[16:17:22 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:9a54c6f24341: vp8: make wait/thread_mb_pos atomic.
[16:17:23 CEST] <cone-839> ffmpeg 03Ronald S. Bultje 07master:fed92adbb3fc: vp8: make mv_min/max thread-local if using partition threading.
[16:17:24 CEST] <cone-839> ffmpeg 03Shivraj Patil 07master:2a512f86c12d: build fix for mips
[16:17:31 CEST] <jamrial> nevcairiel: nobody seems to have any more comments about it
[16:17:35 CEST] <nevcairiel> jamrial: feel free to do it if you want to, it might take me a while to get to it
[16:17:37 CEST] <jamrial> BBB: tsan killing push? :D
[16:17:55 CEST] <jamrial> fixing, rather
[16:17:59 CEST] <jya> BBB: have I ?
[16:18:25 CEST] <BBB> jya: do you use a custom get_buffer callback? and can you reproduce yourself?
[16:19:16 CEST] <BBB> ubitux: wanna re-poke the tsan machine?
[16:19:24 CEST] <ubitux> sure
[16:19:29 CEST] <BBB> lets see where we are now
[16:19:45 CEST] <BBB> I believe this leaves mpeg4 and the frame_thread_encoder
[16:19:57 CEST] <BBB> (rv3/4 etc. are also mpeg4)
[16:21:07 CEST] <nevcairiel> are rv3/4 so close to mpeg that this makes any sense, or is the code just insane?
[16:21:33 CEST] <jya> BBB: we don't use a custom get_buffer_callback anymore
[16:21:43 CEST] <BBB> nevcairiel:  ¯\_(Ä)_/¯
[16:21:44 CEST] <jya> we used to, but i got rid of that code a while ago
[16:22:00 CEST] <jya> BBB: I replied 
[16:22:04 CEST] <BBB> ok, ty
[16:23:07 CEST] <BBB> jya: so did this crash start appearing recently? or has this always been around and it just got above a certain threshold where were being poked?
[16:23:36 CEST] <atomnuker> J_Darnley: haven't abandoned it, I'll get some work done on that eventually
[16:23:37 CEST] <BBB> jya: and can you confirm that the custom callback wasnt present in the version that runs in these reports?
[16:23:55 CEST] <atomnuker> getting a better opus encoder is more fun atm
[16:24:37 CEST] <atomnuker> (I actually have a new psychoacoustic system somewhat working which'll hopefully replace the current filter-based one)
[16:24:38 CEST] <BBB> nevcairiel: to me, all of mpegvideo is  ¯\_(Ä)_/¯ at this point
[16:24:42 CEST] <nevcairiel> hehe
[16:24:43 CEST] <jya> BBB: yes, they are all 52
[16:24:48 CEST] <jya> that's the current version
[16:25:04 CEST] <jya> and some of the reports are 55, that's current nightly build
[16:25:05 CEST] <BBB> jya: and were there crashes in 51?
[16:25:13 CEST] <jya> not in the past 7 days
[16:25:18 CEST] <BBB> historically?
[16:25:30 CEST] <BBB> or is that hard to figure out?
[16:25:30 CEST] <jya> which isn't surprising, 52 has been out for several weeks and people typically udpate
[16:25:37 CEST] <jya> I can extend the search 
[16:26:19 CEST] <jya> in the past 6 months: https://crash-stats.mozilla.com/signature/?signature=ff_vp9_loop_filter_v_16_16_sse2&date=%3E%3D2016-10-06T14%3A25%3A52.000Z&date=%3C2017-04-06T14%3A25%3A52.000Z#summary
[16:26:39 CEST] <jya> so it's present on all versions, just much more on 52
[16:27:22 CEST] <jya> that seems to coincide on when we resync to version n3.2-65-gee56777
[16:27:41 CEST] <jya> before that we used n3.1.1-6-g86f9228
[16:27:48 CEST] <BBB> hm& crap...
[16:28:18 CEST] <jya> it takes about 6 months for change to become public
[16:28:40 CEST] <jya> so n3.2-65-gee56777 is first seen in ... (checking)
[16:29:07 CEST] <jya> ah no that's 52
[16:29:12 CEST] <jya> sorry 53
[16:29:17 CEST] <jya> 52 is n3.1.1-6-g86f9228
[16:29:42 CEST] <jya> was introduced in 51
[16:32:08 CEST] <BBB> so 52/51 had same ffmpeg version?
[16:32:14 CEST] <BBB> so that doesnt really help...
[16:32:23 CEST] <BBB> I wonder why its 32bit only
[16:32:26 CEST] <BBB> that must mean something
[16:32:53 CEST] <BBB> also youre saying its windows only right?
[16:32:59 CEST] <BBB> stack alignment?
[16:33:27 CEST] <jya> for now, it appears to be windows only yes
[16:33:31 CEST] <jya> but that doesn't mean much
[16:33:42 CEST] <jya> I mean, in some version there was only a couple of reports
[16:33:49 CEST] <jya> and firefox is 90% windows
[16:34:07 CEST] <jya> something like 300M users vs 40M on mac
[16:34:10 CEST] <nevcairiel> whats your avcodec dll built with? mingw/gcc, msvc?
[16:34:19 CEST] <jya> msvc
[16:34:57 CEST] <BBB> whats the percentage of 32bit users that are on windows?
[16:35:00 CEST] <nevcairiel> well it could be stack alignment then, but we usually find such problems with fate
[16:35:19 CEST] <BBB> (i.e. what is 100%*win_32bit_users/32bit_users)
[16:35:30 CEST] <BBB> maybe its threading-related
[16:35:50 CEST] <BBB> I dont think we have a fate station that tests sse2/32bit/windows/threading w/ valgrind/asan
[16:35:55 CEST] <jya> that's over the past year: only windows https://crash-stats.mozilla.com/signature/?signature=ff_vp9_loop_filter_v_16_16_sse2&date=%3E%3D2016-04-01T14%3A25%3A00.000Z&date=%3C2017-03-31T14%3A25%3A00.000Z#summary
[16:36:10 CEST] <nevcairiel> BBB: and we never will, valgrind no work on windows :)
[16:36:12 CEST] <jya> but again, I've seen cases where the crash report were only on windows yet the bug was present everywhere
[16:36:32 CEST] <BBB> jya: Im just trying to statistically analyze where I should start looking based on the data we have ;)
[16:36:37 CEST] <BBB> jya: are you on windows?
[16:36:59 CEST] <jya> 1331 crashes over the past year (reported that is) all on x86 platform
[16:37:07 CEST] <jya> BBB: no, I'm mostly on mac
[16:37:21 CEST] <BBB> we need a windows user :-p
[16:37:31 CEST] <jya> nevcairiel: we have now asan working on windows build
[16:37:44 CEST] <BBB> Im going to be suspicious of manual stack alignment
[16:37:46 CEST] <jya> and I don't have a 32 bits machine (e.g. a P4)
[16:37:53 CEST] <BBB> since all crashes are 32bit/windows and in x86inc.asm-derived code
[16:38:03 CEST] <BBB> sadly I wrote that code also, so crap
[16:38:12 CEST] <nevcairiel> you dont need a 32-bit machine or even a 32-bit OS
[16:38:36 CEST] <BBB> I can set HAVE_ALIGNED_STACK to 0 in config.asm, I know
[16:38:37 CEST] <nevcairiel> just need to run 32-bit code on your 64-bit ystem
[16:38:37 CEST] <jya> nevcairiel: sure, but 100% of the crashes were on pentium 4
[16:38:48 CEST] <jya> eg. machine with SSE2 only
[16:38:48 CEST] <nevcairiel> how odd
[16:38:58 CEST] <nevcairiel> i suppose otherwise it uses some other asm function, eh?
[16:39:13 CEST] <jya> not so, has BBB explained, there's a SS3 code path there, so you don't enter the SSE2 code
[16:39:21 CEST] <jya> s/has/as
[16:39:48 CEST] <nevcairiel> SSE2 and SSSE3 are very similar though, only a few instructions extra, so if there is an alginemtn problem one might expect it to affect both
[16:39:54 CEST] <BBB> I can give you a manual patch to disable >sse2
[16:40:08 CEST] <BBB> and yes ssse3 also was present in the dumps
[16:40:12 CEST] <BBB> just in smaller numbers
[16:40:20 CEST] <BBB> but I think thats because most ssse3 machines are 64bit
[16:42:49 CEST] <cone-839> ffmpeg 03Steve Lhomme 07master:b378f5bd6401: lavu: add support for Content Light Level side metadata
[16:42:50 CEST] <cone-839> ffmpeg 03Steve Lhomme 07master:157e57a1810c: lavc: Add Content Light Level side metadata found in HEVC
[16:42:51 CEST] <cone-839> ffmpeg 03Steve Lhomme 07master:f4cee500e969: avformat/dump : Display Content Light Level metadata
[16:43:01 CEST] <BBB> so far HAVE_ALIGNED_STACK=0 seems to still work
[16:43:09 CEST] <BBB> but Ill let it run over all files
[16:44:08 CEST] <ubitux> BBB: 3176/3378 ’ 3285/3382
[16:44:11 CEST] <ubitux> http://fate.ffmpeg.org/history.cgi?slot=x86_64-archlinux-gcc-tsan
[16:44:12 CEST] <ubitux> running slices
[16:46:12 CEST] <BBB> so ~100 failing
[16:46:18 CEST] <ubitux> yep
[16:46:20 CEST] <BBB> hey h264 still fails
[16:46:21 CEST] <BBB> wth
[16:46:25 CEST] <BBB> I thought I fixed that
[16:46:28 CEST] <cone-839> ffmpeg 03James Almer 07master:e5c11ab9cef6: avcodec: fix containts typo
[16:47:02 CEST] <J_Darnley> atomnuker: That's good to hear even if I can't use it now to squeeze my music even more.
[16:47:45 CEST] <BBB> ok so 3 bugs left: h264, mpeg4 and frame_thread_encoder
[16:47:57 CEST] <J_Darnley> BBB: you called for a Windows user?
[16:47:59 CEST] <BBB> Ill look further into the h264 one, I expected it to be clean now
[16:48:03 CEST] <BBB> weeeee
[16:48:15 CEST] <BBB> J_Darnley: yes! wanna crash firefox for me?
[16:48:33 CEST] <J_Darnley> Uh, maybe
[16:48:43 CEST] <J_Darnley> I think I have a recent one installed
[16:48:50 CEST] <BBB> J_Darnley: needs to be a 32bit build
[16:48:58 CEST] <BBB> J_Darnley: so you may have to manually install one
[16:49:01 CEST] <J_Darnley> Hm  do I still have 32 bit?
[16:49:34 CEST] <BBB> jya: maybe we should work something out so you guys provide me wiht some machines
[16:49:55 CEST] <BBB> jya: not that I like having crap around, but being able to borrow a 32bit windows laptop for a few days would be helpful ;)
[16:50:13 CEST] <J_Darnley> Sorry, I only have 64-bit version 47
[16:50:19 CEST] <jya> https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-release-win32/1491300568/firefox-52.0.3.en-US.win32.installer.exe
[16:50:43 CEST] <jya> i only have 32 bits VM
[16:50:47 CEST] <nevcairiel> do you also have a particular file to crash on? ;)
[16:50:50 CEST] <BBB> jya: tnx!
[16:50:53 CEST] <BBB> nevcairiel: yes I do
[16:51:01 CEST] <jya> if by 32 bits you win windows 32 bits
[16:51:10 CEST] <BBB> jya: yes
[16:51:13 CEST] <jya> they are available on microsoft web site
[16:51:24 CEST] <BBB> nevcairiel: want a webm file or a youtube link?
[16:51:27 CEST] <nevcairiel> those development VMs directly from microsoft are handy
[16:51:30 CEST] <jya> its' a pain to develop on them, we can no longer build firefox on 32 bits 
[16:51:37 CEST] <nevcairiel> BBB: whatever crashes with less clicks
[16:51:47 CEST] <BBB> nevcairiel: I dont know, I havent been able to reproduce
[16:51:59 CEST] <jya> and the beauty with a VM, is that you'll get VP9 with youtube guarantee
[16:52:00 CEST] <BBB> Im guessing youtube-dl is easier?
[16:52:01 CEST] <nevcairiel> direct file probably, before i have to convince youtube to actually give me vp9
[16:52:09 CEST] <jya> we prefer h264 on hardware decoder machine
[16:52:45 CEST] <BBB> nevcairiel: do you have youtube-dl?
[16:52:58 CEST] <nevcairiel> i can find it
[16:53:18 CEST] <nevcairiel> now i do
[16:53:41 CEST] <jya> nevcairiel: forcing vp9 is easy. go to about:config, and change the pref media.mediasource.webm.enabled from false to true
[16:54:05 CEST] <BBB> nevcariel: for id in 242 243 244 247 248 271 278 313; do youtube-dl -f $id https://www.youtube.com/watch?v=HrAnOqztv5w && mv Flawless FULL COVERAGE Foundation Routine-HrAnOqztv5w.webm Flawless FULL COVERAGE Foundation Routine-HrAnOqztv5w.${id}.webm; done
[16:55:10 CEST] <BBB> and Im hoping that the crash will reproduce on a 32bit windows ffmpeg build using something like for n in *.webm; do ./ffmpeg -i "${n}" -f null -v error -nostats -; done
[16:55:19 CEST] <BBB> but it may be firefox-specific
[16:56:22 CEST] <jya> BBB: replace system malloc with jemalloc
[16:56:36 CEST] <BBB> we dont use malloc
[16:56:49 CEST] <jya> which has difference in particular on mac. on mac, a malloc is always 16 bytes aligned... 
[16:56:49 CEST] <BBB> on windows, we use _aligned_malloc()
[16:57:18 CEST] <nevcairiel> do we even support replaceable malloc still
[16:57:18 CEST] <jya> jemalloc aim is to minimize fragmentation, so it doesn't align much
[16:57:32 CEST] <nevcairiel> anyway av_malloc has to align to our requirements, or everything will break
[16:57:40 CEST] <BBB> and HAVE_ALIGNED_STACK=0 still doesnt crash for me
[16:57:43 CEST] <jya> indeed
[17:00:25 CEST] <nevcairiel> firefox at least doesnt crash on those clips
[17:00:33 CEST] <nevcairiel> but of course my system has more then sse2
[17:00:47 CEST] <nevcairiel> i can try with ffmpeg directly later and limit the instruction set
[17:01:18 CEST] <nevcairiel> or firefox supports vp9 hwaccel already and just bypasses the software decoder :p
[17:02:11 CEST] <jya> I can build you a firefox with SSE3 disabled
[17:02:14 CEST] <jya> on 32 bits
[17:02:30 CEST] <jya> nevcairiel: not in the version I've copied above
[17:02:56 CEST] <jya> we have serious vp9 performance regression with some nvidia GT card
[17:03:08 CEST] <jya> so we've disabled it for now outside of the nightly channel
[17:04:02 CEST] <nevcairiel> always suffering because of people that spend less money on hardware then others on coffee in a week
[17:04:06 CEST] <nevcairiel> :)
[17:04:27 CEST] <jya> at least those machines have SSE2 :)
[17:04:41 CEST] <jya> if you knew the trouble was had to no longer support machine without SSE2
[17:05:04 CEST] <nevcairiel> i should do that, and kill XP at the same time, oh the tears i  can already see
[17:11:46 CEST] <BBB> holy crap h264 suddenly got noisy again here :-/
[17:16:05 CEST] <BBB> I guess tsan doesnt work well if only half the tree is compiled with it :-/
[17:23:42 CEST] <kierank> jemalloc is really good
[17:23:53 CEST] <kierank> we use it on all production builds
[17:23:57 CEST] <ubitux> BBB: tsan slices available
[17:24:07 CEST] <BBB> any improvement?
[17:24:30 CEST] <ubitux> almost none, probably due to the usual randomness
[17:24:32 CEST] <ubitux> http://fate.ffmpeg.org/history.cgi?slot=x86_64-archlinux-gcc-tsan-slices
[17:24:51 CEST] <BBB> 3313 -> 3317
[17:24:53 CEST] <BBB> thats expected
[17:24:56 CEST] <BBB> 4 vp8 tests failed
[17:25:20 CEST] <BBB> and vp8 is fixed now
[17:25:21 CEST] <BBB> good, good
[17:25:27 CEST] <ubitux> oh yeah the vp8 ones, cool
[17:26:10 CEST] <nevcairiel> how is the ratio between actual potential issues and "safe" races in those fixes?
[17:27:44 CEST] <BBB> nevcairiel: the vp8 was one safe race and one real bug
[17:28:03 CEST] <BBB> nevcairiel: the idct (the 6-patch set) was not a real bug
[17:28:40 CEST] <BBB> I believe some of the intra codec fixes (setting privdata->avctx) were somewhat real bugs, although it really only affects videos where stuff (width/height/pix_fmt) changes mid-stream so very low impact
[17:28:56 CEST] <BBB> but I guess it could be abused in some ways if you were clever
[17:29:14 CEST] <BBB> the hevc ones were safe races
[17:29:43 CEST] <BBB> the h264 ones Im not entirely sure, but Id say by-and-large theyre safe races
[17:30:06 CEST] <BBB> the png one was a safe race
[17:34:35 CEST] <jamrial> BBB: i have an sse2 only windows machine, will try those
[17:34:41 CEST] <BBB> woohoo
[17:34:54 CEST] <jamrial> you just want to know if it crashes then?
[17:35:16 CEST] <nevcairiel> I bet he would also take a "why" if you have any
[17:35:16 CEST] <nevcairiel> :D
[17:46:03 CEST] <wm4> BBB: ok to push?
[17:46:07 CEST] <BBB> yes
[17:46:33 CEST] <wm4> ok I'll just run fate on it to be sure
[17:47:26 CEST] <BBB> fate barely triggers flush() ;-)
[17:47:34 CEST] <BBB> ubitux: ok new patch to hopefully address the h264 issues
[17:47:38 CEST] <wm4> that's probably true
[17:48:53 CEST] <ubitux> BBB: supposed to fix all the remaining h264 issues?
[17:49:01 CEST] <BBB> ubitux: theoretically
[17:49:15 CEST] <jamrial> BBB: can't get them to crash firefox
[17:49:23 CEST] <BBB> jya: ^^ :-(
[17:49:26 CEST] <jamrial> windows 10 32 bits, firefox (obviously) 32 bits, on a K10
[17:49:47 CEST] <jamrial> latest firefox afaik
[17:50:14 CEST] <jamrial> will compile a x86_32 ffmpeg build and try that
[17:53:02 CEST] <cone-839> ffmpeg 03wm4 07master:15a23a83326d: pthread_frame: don't return stale error codes after flush
[18:00:05 CEST] <jya> jamrial: when you go to youtube.com/html5
[18:00:29 CEST] <jya> what do you see next to webm vp9? is it checked in blue?
[18:01:19 CEST] <jamrial> jya: yes, i also force enabled the about:config option you mentioned above just in case
[18:01:37 CEST] <jamrial> in any case, i loaded the webms directly after downloading the with youtube-dl
[18:01:38 CEST] <jya> jamrial: good, just wanted to be sure
[18:01:43 CEST] <jya> like you got mp4 instead
[18:01:56 CEST] <jya> the problem with firefox too is how do you know which streams your getting
[18:02:06 CEST] <jya> you may have to manually select each resolution and playing them
[18:02:16 CEST] <jya> ah ok... good
[18:03:32 CEST] <BBB> I think weve pretty much tried everything to reproduce at this point
[18:03:38 CEST] <BBB> jya: any next step suggestions?
[18:04:23 CEST] <BBB> is it possible stuff breaks after seeking or when closign tabs or so?
[18:04:24 CEST] <jya> well, in the past year there's been "only* 1300 crashes...
[18:04:36 CEST] <jya> considering the millions of videos likely seen
[18:04:43 CEST] <atomnuker> have you set media.benchmark.vp9.threshold to 0?
[18:04:46 CEST] <jya> the likelihood to reproduce it is always low
[18:04:54 CEST] <atomnuker> this always makes firefox use vp9
[18:05:07 CEST] <jya> atomnuker: if you set media.mediasource.webm.enabled to true, that is unnecessary
[18:05:29 CEST] <jya> BBB: that could well be too
[18:07:52 CEST] <atomnuker> btw jya: know if firefox is going to ever get partial opengl damages/redraws? with high resolution displays you really have to worry about always leaving firefox open on a blank page
[18:08:23 CEST] <atomnuker> since even a 1px animated thing redraws the entire window
[18:09:00 CEST] <atomnuker> and at 60fps you get like 100% CPU used just redrawing to update that 1 pixel
[18:09:38 CEST] <jya> atomnuker: I'm not familar with the gfx/compositor layer.. but if opengl it should be HW accelerated, that won't take 60% CPU
[18:12:14 CEST] <jya> as far as media playback is concerned, the work required to check on what pixels have changed between images would likely be just as much as repainting it all
[18:12:43 CEST] <jya> especially as to determine which part of the image has changed, you would have to do GPU readbacks, they are very slow
[18:13:21 CEST] <BBB> wm4: you sent that message to me instead of to the list
[18:16:42 CEST] <heeen> hi, I am looking for libavutil/hwcontext.h, but the ffmpeg tree I'm looking at does not have it
[18:16:49 CEST] <heeen> my ffmpeg tree seems to be 3.0
[18:17:00 CEST] <BtbN> you should update it then
[18:17:15 CEST] <heeen> what is the earliest version
[18:17:18 CEST] <heeen> that has it
[18:17:25 CEST] <BtbN> just use latest master
[18:17:32 CEST] <wm4> BBB: oops, stupid mail client
[18:17:35 CEST] <BtbN> why would you want to deliberatle use an older version?
[18:17:37 CEST] <wm4> must have been because of the CC
[18:17:53 CEST] <heeen> BtbN: because it is part of a poky build and I don't know what else depends on it
[18:18:03 CEST] <heeen> I want the smallest change that works
[18:18:07 CEST] <heeen> hm
[18:20:11 CEST] <heeen> ok it looks like the project I am porting from does not actually build libav
[18:20:28 CEST] <heeen> I can see it has libav
[18:20:45 CEST] <nevcairiel> The hwcontext infrastructure is still under active development, using the very latest will give you the best results
[18:20:56 CEST] <wm4> it's been around for a while
[18:21:00 CEST] <wm4> must be an old ffmpeg version
[18:21:30 CEST] <wm4> BtbN: distros put pressure on developers to use a "release"
[18:21:34 CEST] <wm4> directly or indirectly
[18:21:45 CEST] <wm4> they won't shut up about it and neither will users
[18:21:56 CEST] <wm4> thus developers using ffmpeg have to put up with old ffmpeg versions
[18:22:25 CEST] <wm4> this is made worse than some distros want _all_ packages to use the same ffmpeg version, thus the ffmpeg version is usually pretty old
[18:25:17 CEST] <nevcairiel> Screw distros, let the users put pressure on the distros to include your app, which requires new ffmpeg
[18:26:09 CEST] <wm4> they'll include old versions of way more popular apps, which of course can't be easily ported to newer ffmpeg
[18:26:40 CEST] <wm4> but I stopped caring anyway
[18:27:58 CEST] <nevcairiel> Work distributes its own private copy of FFmpeg, and my oss project is windows only, so I never had to care, luckily.
[18:28:31 CEST] <nevcairiel> Other projects like Kodi shipped their own FFmpeg for years, and they somehow got away with it
[18:30:07 CEST] <wm4> I don't know what distros did about it - some probably hacked it to use "system ffmpeg", and broke various things (that's what typically happens)
[18:32:48 CEST] <BtbN> Kodi can use external ffmpeg without hacking it
[18:32:54 CEST] <BtbN> And if it's recent enough, it just works
[18:33:03 CEST] <BtbN> Using system ffmpeg for everything on Gentoo never was an issue for me
[18:33:10 CEST] <BtbN> system-ffmpeg is latest master in my case though
[18:33:36 CEST] <nevcairiel> Historically they basically denied you support if you ran that at the very least
[18:34:15 CEST] <nevcairiel> And some things probably did break, even if you didn't run into those :)
[18:34:18 CEST] <BtbN> I don't care. Usually I am the support.
[18:34:34 CEST] <BtbN> I'm not using ancient software
[18:36:44 CEST] <kierank> 17 PM <"BtbN> just use latest master --> just play russian roulette
[18:37:49 CEST] <BtbN> Even with the current merges, master is still working fine
[18:39:24 CEST] <nevcairiel> Releases are arbitrary points to please distributions
[18:39:37 CEST] <nevcairiel> Not like they are really that much better
[18:54:36 CEST] <kierank> BtbN: hahahahaha
[18:54:37 CEST] <kierank> ahahahahah
[19:17:03 CEST] <atomnuker> rcombs: what's left to address with the dash decoder patch?
[19:17:16 CEST] <rcombs> I should probably, y'know, actually test it
[19:18:56 CEST] <rcombs> as an optimization, it'd probably be good if both it and hls.c started reusing sockets across segments
[19:19:01 CEST] <rcombs> but that shouldn't block merge
[19:19:07 CEST] <rcombs> that reminds me, wtb public API for that
[19:21:36 CEST] <rcombs> probably ought to be a bit more generic; like, "open new file with the same underlying connection" instead of specifically "GET this new HTTP URL"
[19:21:54 CEST] <rcombs> would be applicable to some other protocols as well
[19:22:03 CEST] <rcombs> also, dat http2
[19:23:05 CEST] <wm4> yeah, these things would be nice
[19:24:07 CEST] <ubitux> BBB: i'll test locally your last patch to see if it helps tsan
[19:24:15 CEST] <BBB> tnx!
[19:27:35 CEST] <ubitux> BBB: so it's supposed to fix fate-h264-conformance-ba1_ft_c with standard threading options?
[19:27:41 CEST] <BBB> yes
[19:28:30 CEST] <ubitux> i'm able to reproduce and your patch fixes it
[19:28:34 CEST] <BBB> \o/
[19:28:59 CEST] <ubitux> i still have some h264 failures though
[19:29:25 CEST] <ubitux> http://b.pkh.me/h264-conformance-cabac_mot_fld0_full.err
[19:29:27 CEST] <ubitux> this one typically
[19:29:34 CEST] <ubitux> is it something else?
[19:30:37 CEST] <BBB> with current git/master?
[19:30:43 CEST] <ubitux> and your patch applied
[19:30:58 CEST] <ubitux> oh wait
[19:31:07 CEST] <ubitux> i think i forgot to pull, i'm stupid
[19:35:06 CEST] <ubitux> so, trying again with git/master (:shame:), without your patch it fails, and your patch fixes it
[19:35:14 CEST] <ubitux> and it passses the test i was refering above
[19:35:16 CEST] <ubitux> but still one failing
[19:35:21 CEST] <BBB> which one?
[19:35:25 CEST] <ubitux> h264-conformance-camanl2_toshiba_b
[19:35:32 CEST] <ubitux> http://b.pkh.me/h264-conformance-camanl2_toshiba_b.err
[19:35:58 CEST] <BBB> hm& thats the same as what you just had
[19:36:13 CEST] <ubitux> not the same test
[19:36:19 CEST] <ubitux> but the previous one works now
[19:36:25 CEST] <BBB> oh wait its await_progress
[19:36:26 CEST] <BBB> hm...
[19:36:29 CEST] <BBB> one sec
[19:36:31 CEST] <BBB> let me check
[19:36:47 CEST] <ubitux> h264-conformance-frext-hpcafl_bcrm_c also fails
[19:37:06 CEST] <BBB> oh well yes of course that is an int that is read everywhere
[19:37:14 CEST] <ubitux> http://b.pkh.me/h264-conformance-frext-hpcafl_bcrm_c.err
[19:37:28 CEST] <BBB> yep await also
[19:37:53 CEST] <ubitux> that's the only ones that failed at this run
[19:37:58 CEST] <ubitux> (on fate-h264)
[19:39:35 CEST] <BBB> hm&
[19:39:38 CEST] <ubitux> a new run passed fine all the tests though&
[19:39:50 CEST] <BBB> I expect that one to fail occasionally
[19:39:56 CEST] <BBB> I dont really know for sure if I want to fix it
[19:40:12 CEST] <ubitux> ok ok
[19:40:17 CEST] <BBB> its not a big race anyway
[19:40:18 CEST] <ubitux> anyway, your patch improves the situation
[19:40:21 CEST] <BBB> I Dont think it can be abused
[19:40:36 CEST] <BBB> but the bigger thing is that changing it means having to chagne the type of debug and I Really dont think thats a good idea
[19:40:39 CEST] <BBB> since we need av_opt etc.
[19:41:04 CEST] <BBB> maybe we need to move the printf under the lock
[19:41:08 CEST] <BBB> but that might not be great
[19:41:22 CEST] <BBB> yeah that should probably be done
[19:41:33 CEST] <ubitux> anyway, patch LGTM as a user
[19:41:35 CEST] <ubitux> :p
[19:41:40 CEST] <ubitux> i have to go afk for a while
[19:41:42 CEST] Action: ubitux &
[19:41:45 CEST] <ubitux> (thx!)
[19:41:50 CEST] <BBB> bye
[19:49:02 CEST] <BBB> ubitux: concept patch for that sent to ML. Im not sure I like it but it seems harmless to fix it this way
[19:49:09 CEST] <BBB> (please test)
[20:03:43 CEST] <ubitux> BBB: passed 100 times h264-conformance-camanl2_toshiba_b without issue 
[20:04:02 CEST] <BBB> \o/
[20:04:11 CEST] <BBB> so I guess nows a good time to switch to slice-mt?
[20:04:20 CEST] <BBB> Im going to assume fate-h264 is clean now
[20:04:50 CEST] <ubitux> let me try the slice mode
[20:05:03 CEST] <ubitux> is it supposed to work or there are known issue on fate-h264?
[20:05:49 CEST] <BBB> known issues
[20:06:01 CEST] <ubitux> yeah, quite a bunch are failing here
[20:06:12 CEST] <BBB> I believe most are accesses to slice_table[]
[20:06:24 CEST] <BBB> this is essentially all safe but its kind of dirty and potentially hides real bugs
[20:06:45 CEST] <BBB> I wonder if most of that can be replaced with just >=slice_start_x/y checks
[20:06:54 CEST] <BBB> maybe thats slower, but it safes a dereference
[20:06:58 CEST] <BBB> I guess it needs some testing
[20:08:10 CEST] <BBB> first_mb_addr I guess
[20:21:23 CEST] <BBB> michaelni: so, slice_table[] is basically one big race with slice threading (not in the sense that it introduces bugs, but in the sense that accesses to it are indeed not protected, rendering tsan useless)& WDYT of replacing slice_table[] with a field called resync_mb_xy, and replacing all uses of slice_table[mb_xy] == slice_num with mb_xy >= resync_mb_xy?
[20:22:09 CEST] <BBB> does that interfere with FMO?
[20:22:52 CEST] <BBB> that may break FMO, but otherwise it should work...
[20:23:36 CEST] <BBB> (FMO doesnt work anyway from what I recall)
[20:34:14 CEST] <J_Darnley> BBB, michaelni: in libswscale/x86/scale.asm what is the movbh macro supposed to do?  "move bytes high"?  "move bytes half"?
[20:35:17 CEST] <kierank> iirc for bitdepth indepdence
[20:35:25 CEST] <J_Darnley> for 8-bit samples I think it is defined as movh which is "move half"
[20:39:00 CEST] <BBB> J_Darnley: I think its what kierank says
[20:40:09 CEST] <J_Darnley> Yeah but I need to replace the movh it uses with an actual instruction available for AVX2.
[20:40:33 CEST] <J_Darnley> I think it is always followed by an punpck so I will try pmovzx
[20:40:45 CEST] <J_Darnley> and remove the unpack
[20:44:48 CEST] <BBB> yes that would be ideal
[21:17:00 CEST] <J_Darnley> What tests are there for swscale?  Specifically I want to test hscale_8to15_X4.
[21:17:25 CEST] <michaelni> BBB, hmmm, cant we simply give each slice thread its own slice_table ? it doesnt need the info from others before loop filtering and for loop filtering it needs to know it exactly anyway and xy compares wont do. 
[21:18:10 CEST] <BBB> oh we need it for loopfiltering?
[21:18:49 CEST] <michaelni> see fill_filter_caches_inter(), not sure if something prevents this
[21:18:58 CEST] <BBB> hrm.. that complicates some things
[21:19:51 CEST] <michaelni> isnt there a way to just silence tsan for a variable?
[21:20:38 CEST] <kierank> J_Darnley: none, just try a yuv image
[21:20:49 CEST] <kierank> probably fate
[21:20:54 CEST] <kierank> but no asm unit tests vs c
[21:21:20 CEST] <BBB> michaelni: there is but then Id rather just leave it
[21:21:48 CEST] <BBB> michaelni: I dont mind it complaining, it doesnt bug me, Id like to fix things if it helps reduce noise without pain, but if theres no point, Ill keep it as-is
[21:22:05 CEST] <durandal_1707> how much time we have to pick gsoc proposals?
[21:22:26 CEST] <BBB> michaelni: does loopfilter happen in the slice threads also? I thought it did, right?
[21:23:04 CEST] <michaelni> it would make sense if it does, if it doesnt we should change it so it does
[21:23:51 CEST] <BBB> I recall that it did
[21:24:06 CEST] <BBB> it also seems that top_type = 0 if top_slice_num != this_slice_num in the caller of fill_filter_caches_inter()
[21:24:27 CEST] <BBB> it appears that thats how we prevent filtering the top/cur edge at slice boundaries
[21:25:14 CEST] <BBB> (not 100% sure, this code is somewhat cryptic)
[21:41:31 CEST] <uau> wm4: doesn't the thread ->result handling also have a problem at the other location where got_frame is set to 0, in ff_thread_decode_frame?
[21:42:02 CEST] <J_Darnley> I found a test case: scaling 4K 422 into sub-SD 420.
[21:42:04 CEST] <uau> as in shouldn't it be zeroed in both flush and there similarly to got_frame
[21:42:19 CEST] <kierank> J_Darnley: yes that's what we're doing here actually
[21:42:27 CEST] <J_Darnley> :)
[21:42:30 CEST] <kierank> well HD to SD 422
[21:42:32 CEST] <kierank> no 420 nonsense
[21:46:25 CEST] <J_Darnley> Oh fuck you ffplay and your different options!
[21:46:44 CEST] Action: JEEB pats J_Darnley 
[21:48:38 CEST] <J_Darnley> okay, it is a horrible green mess, yay
[21:48:49 CEST] <wm4> uau: I don't know
[21:49:28 CEST] <uau> wm4: well you get an error, flush, and get the same error again?
[21:49:44 CEST] <uau> ambiguous flush, i meant "flush" as in 0-sized packet there
[21:49:47 CEST] <kierank> J_Darnley: probably need to fix the alignment constraints in swscale
[21:49:53 CEST] <kierank> it's all hardcoded to 16-byte iirc
[21:50:02 CEST] <kierank> so much magic stuff in there
[21:50:27 CEST] <wm4> uau: the new decode API requires you to call flush after draining
[21:50:46 CEST] <wm4> which partially takes care of this case (at least it doesn't get stuck)
[21:51:02 CEST] <uau> you mean if you want to start feeding new frames? or what? that doesn't seem relevant
[21:51:36 CEST] <J_Darnley> kierank: maybe but I would expect alignmment problems to crash rather than output zeros
[21:51:50 CEST] <uau> the problem being that you get an error code from the 0-sized packet even if you already received that error before, no further packets needed
[21:51:53 CEST] <kierank> true
[21:55:53 CEST] <uau> funny that part of the code is so buggy btw - i fixed 2 bugs about "got_picture", and there are still similar issues with "err"
[21:57:34 CEST] <uau> 8414fc85 (old mt tree only, squash-merged with mt was added to libav) and 05b9a89ed718, which does for got_picture what the latest commit does for err
[22:10:33 CEST] <uau> posted that to the mailing list too
[22:36:19 CEST] <J_Darnley> Oh that's nice.  An if-else section followed immediately by another if-else section with an inverted if condition.
[22:36:54 CEST] <kierank> don't read too much swscale, it will make you ill
[22:37:31 CEST] <J_Darnley> You're paying to make me ill?  You heartless bastard!
[22:37:57 CEST] <kierank> consider yourself a war reporter
[22:38:01 CEST] <kierank> going into dangerous territory
[22:38:28 CEST] <JEEB> J_Darnley: consider trying zimg
[22:38:47 CEST] <kierank> maybe we should write a upipe plugin for it
[22:39:24 CEST] <JEEB> that sounds like a nice idea
[22:39:45 CEST] <durandal_170> upipe?
[22:40:25 CEST] <kierank> https://archive.fosdem.org/2016/schedule/event/upipe/
[22:44:55 CEST] <nevcairiel> I was meaning to checkout zimg
[22:45:04 CEST] <nevcairiel> Been lazy :(
[23:33:59 CEST] <J_Darnley> Why doesn't swscale have sse4 variants of the hscaleXto15 functions?  No different instructions to use?
[23:46:19 CEST] <iive> does it have anything newer than sse2 ?
[23:53:18 CEST] <J_Darnley> yes, ssse3 and sse4
[00:00:00 CEST] --- Fri Apr  7 2017