[Ffmpeg-devel-irc] ffmpeg-devel.log.20160209

[burek]

[00:03:30 CET] <llogan> or make it part of an outreachy/gsoc qualification task
[00:07:10 CET] <llogan> i have no idea which project ideas to add to the new outreachy wiki page
[00:19:13 CET] <kierank> a bit hard for an outreachy/gsoc
[00:19:17 CET] <kierank> qualification
[00:19:22 CET] <kierank> but not big enough for a full task
[00:20:49 CET] <jkqxz> nevcairiel:  Are you sure the mr[345]_tandberg streams really work properly with dxva2 (no error output, and not falling back to software because of the profile)?  As far as I can tell, h264 does call the alloc function more times than it should.
[00:23:01 CET] <Timothy_Gu> kierank: get outreachy girls to do it
[00:23:13 CET] <kierank> see what I just wrote 
[00:23:34 CET] <llogan> not limited to only women anymore.
[00:23:47 CET] <Timothy_Gu> oh
[00:24:23 CET] <Timothy_Gu> Additionally, they are open to residents and nationals of the United States of any gender who are Black/African American, Hispanic/Latin@, American Indian, Alaska Native, Native Hawaiian, or Pacific Islander.
[00:24:42 CET] <Timothy_Gu> im neither residents or national of the U.S. or any of those "minorities"
[00:24:53 CET] <Timothy_Gu> ugh
[00:25:01 CET] <llogan> looks like i qualify as "ambigously somewhat brownish guy"
[00:25:32 CET] <Timothy_Gu> alaska native haha
[00:25:44 CET] <kierank> llogan: yeah but imo we're not outreaching
[00:26:19 CET] <llogan> do you mean we aren't going to tell them we want to participate this round?
[00:27:08 CET] <llogan> Timothy_Gu: "We are planning to expand the program to more participants from underrepresented backgrounds in the future."
[00:27:22 CET] <llogan> perhaps you can join the Minority Club later
[00:28:04 CET] <kierank> llogan: I mean if you (llogan) were to apply to outreachy we wouldn't be doing any outreach since you were here before
[00:28:17 CET] <Timothy_Gu> llogan: im pretty sure the next group will be white rather than Asian
[00:28:29 CET] <Timothy_Gu> technically i'm not being outreached either
[00:28:49 CET] <llogan> kierank: oh, heh. not that i would contribute anything interesting.
[00:30:29 CET] <kierank> llogan: can you start making the 2016 page
[00:30:49 CET] <llogan> working on it now
[00:31:12 CET] <llogan> listing you as admin and michael and me as backup
[00:31:53 CET] <kierank> ah ok
[00:31:58 CET] <kierank> I have one task idea
[00:32:11 CET] <kierank> is there a place you are editing it?
[00:33:37 CET] <kierank> Should I include the url in my email?
[00:33:46 CET] <Timothy_Gu> fyi, id really like to have a new documentation engine
[00:34:31 CET] <llogan> Timothy_Gu: https://trac.ffmpeg.org/wiki/SponsoringPrograms/ProjectIdeas#Automaticdocumentation
[00:34:40 CET] <llogan> feel free to make less bad description, etc
[00:34:44 CET] <Timothy_Gu> ah
[00:34:48 CET] <Timothy_Gu> lol
[00:35:35 CET] <llogan> kierank: fflogger just barfed the url. you can include url although it is incomplete
[00:35:39 CET] <kierank> yeah
[00:36:45 CET] <kierank> I'm going to try and improve the opening marketing
[00:36:49 CET] <Timothy_Gu> i thibk we should make sure what kind of approach we want BEFORE the program itself
[00:37:23 CET] <Timothy_Gu> so that tr ibtern can start doing it as soon as possible w/o all the drama of discussing it
[00:37:35 CET] <Timothy_Gu> *the intern
[00:43:54 CET] <llogan> kierank: currently opw at ffmpeg email goes to michael, timnich, reynaldo, stefano, reimar, nicolas george, klaussfreire, and paul b mahol. i can change it if you like (see /etc/aliases on ffbox0)
[00:44:04 CET] <kierank> can you add to me
[00:44:17 CET] <kierank> no idea if I have ffbox0 access
[00:45:03 CET] <llogan> should i remove the others and just add you me and michael for now?
[00:45:43 CET] <kierank> yeah ok
[00:46:53 CET] <llogan> ok. i used to know how to do this off the top of my head... i forgot how to do anything postfix since it's been so long.
[00:49:32 CET] <michaelni> you edit the aliases file and run newaliases
[00:51:00 CET] <michaelni> llogan, ^
[00:51:39 CET] <llogan> thanks. sounds familiar. looking at postfix man pages remined me why i moved everything to fastmail
[00:53:55 CET] <llogan> kierank: do you prefer kunhya or obe email?
[00:54:00 CET] <kierank> kunhya
[00:56:36 CET] <llogan> i updated the alias and ran newaliases
[00:58:21 CET] <kierank> I'll email with obe email because that's subscribed to outreachy
[00:58:24 CET] <kierank> but it shouldn't matter
[01:19:31 CET] <llogan> why is the trac database locked?
[01:20:26 CET] <kierank> dunno
[01:20:39 CET] <Timothy_Gu> "locked"?
[01:21:43 CET] <kierank> sqlite does that sometimes
[01:21:45 CET] <llogan> just trying to submit chages to the outreachy wiki page. "Trac detected an internal error: OperationalError: database is locked"
[01:23:39 CET] <kierank> can't login
[01:23:56 CET] <kierank> ah can
[01:23:58 CET] <kierank> just a bit slow
[01:24:02 CET] <Timothy_Gu> ubitux: is "tests: Add test for proper header guard" ok to commit?
[01:24:19 CET] <llogan> kierank: might be getting slammed by spambots.
[01:25:00 CET] <llogan> i wonder how gitlab is working out for videolan (code.videolan.org)
[01:25:59 CET] <cone-484> ffmpeg 03Timothy Gu 07master:cb8646af24bd: configure: Enable GCC vectorization on e4.9 on x86
[01:28:08 CET] <llogan> unfortunate timing with the email
[01:28:50 CET] Action: kierank restarts apache
[01:29:11 CET] <kierank> seems happy now
[01:29:17 CET] <llogan> thanks
[01:34:53 CET] <cone-484> ffmpeg 03Thierry Foucu 07master:020b75806f6e: lavf/mov: Extend extracting XMP in mov files using UUID Box
[02:10:34 CET] <llogan> 1500 tapes and only 4 had fungus issues
[03:17:07 CET] <Compn> hmm
[03:17:38 CET] <Compn> wonder if llogan was talking about vhs 
[03:50:54 CET] <jamrial> http://pastebin.com/Revwi2dr these warnings started showing up with gcc 6 after the tree vectorization patch
[03:50:57 CET] <jamrial> doesn't happen with gcc 5
[03:51:05 CET] <jamrial> what are the chances they are bogus?
[03:51:35 CET] <jamrial> lcov reports that the entire file is used in a fate run
[04:01:41 CET] <Timothy_Gu> jamrial: my bet is on gcc bug
[04:01:45 CET] <Timothy_Gu> :sigh:
[04:03:02 CET] <jamrial> most likely, yeah
[04:11:24 CET] <J_Darnley> Ha ha ha.  I was browsing the "C string handling" page on wikipedia.  We get a mention for allegedly reimplementing strlcat/strlcpy.
[11:50:22 CET] <jkqxz> nevcairiel:  The required number of surfaces for H.264 is num_ref_frames * threads in degenerate cases.
[11:51:23 CET] <jkqxz> I made a test file to show the problem in the software decoder: <http://ixia.jkqxz.net/~mrt/evil.264>.
[11:51:40 CET] <nevcairiel> even more reason not to use it =p
[11:52:02 CET] <jkqxz> This has num_ref_frames=16.  The stream is a black IDR frame followed by all-skip inter frames with frame_num decreasing.
[11:52:44 CET] <jkqxz> Then this patch instruments malloc to see the large frames: <http://ixia.jkqxz.net/~mrt/big_mem_test.patch>.
[11:53:41 CET] <jkqxz> It fails in the software decoder with threads == 8 because you allocate more than 100 frame buffers for the stream.
[11:53:47 CET] <rcombs> so I've been thinking about format negotiation with hardware formats
[11:54:28 CET] <rcombs> swscale can't handle hardware surfaces, and the hardware lib tends to support a specific subset of software formats
[11:55:49 CET] <rcombs> so, I'm not sure if they should stay separate filters and have lavfi handle the combinations of both that may sometimes be required, or if the surface converter should be merged into vf_scale and have it handle the combinations internally
[11:56:22 CET] <rcombs> (also, there's the question of picking an intermediate format in some cases, but I _think_ there's already code to do something like that elsewhere)
[11:56:22 CET] <jkqxz> Since you don't ever actually touch the frames lazy allocation hides the problem in real memory use, but that's still bad.
[12:01:25 CET] <jkqxz> I'm somewhat dubious about surface-frame autoconversion.  While it makes filtering setup a bit easier to use, any minor misconfiguration might slow everything down immensely without it being clear why.
[12:02:55 CET] <rcombs> that's true with autoinserted swscale as-is (though not to quite as large a degree) depending on your filters
[12:03:40 CET] <jkqxz> (Though I do admit that autoinserted swscale which barfs on all your hardware surfaces is a right pain to tame in the current setup.)
[12:10:19 CET] <wm4> jkqxz: in what case does it "accidentally" insert swscale?
[12:13:56 CET] <rcombs> accidentally?
[12:15:22 CET] <jkqxz> Output of hardware surfaces from a hwaccel decoder initially sets up the graph for the software output format.  When you then output a hardware format, the reconfigure throws in a swscale instance which then barfs on you.
[12:15:59 CET] <jkqxz> (See discussion and hack at the bottom of <http://ffmpeg.org/pipermail/ffmpeg-devel/2016-January/188356.html>.)
[12:16:01 CET] <rcombs> it throws in a swscale all the time, doesn't it?
[12:16:18 CET] <durandal11707> kierank: have you registered FFmpeg to outreachy?
[12:16:29 CET] <kierank> durandal11707: I sent email
[12:16:31 CET] <wm4> well that's stupid
[12:16:38 CET] <wm4> maybe ffmpeg.c shouldn't do this then
[12:16:58 CET] <rcombs> it's usually harmless, since vf_scale doesn't actually call swscale if params are all identical on input and output
[12:17:42 CET] <wm4> it's all because libavfilter needs the params up-front and can't handle format changes
[12:17:51 CET] <wm4> in mpv I have to go out of my way to emulate support for it
[12:18:48 CET] <durandal11707> cant it check for  hwaccel pix formats
[12:19:12 CET] <rcombs> it's meant to handle `-s` and such
[12:20:20 CET] <jkqxz> The other problem with autoconversion is really the in making negotiation harder.  For "-vf vaapi_scale=... -c:v vaapi_h264 ...", you need a format filter to force it into to do the right thing, and autoconversion would open up much more scope for it to mess up.  I'm not sure how to solve this one.
[12:21:20 CET] <durandal11707> hwaccel filters use hwaccel formats
[12:21:21 CET] <jkqxz> (This is where I had the "force_vaapi_out" parameter in a previous version before I realised that the format filter does exactly what was required there.)
[12:22:04 CET] <rcombs> the format filter shouldn't be necessary there either if the swscale filter isn't present, right?
[12:22:14 CET] <durandal11707> does vaapi_scale takes normal pixel format as input?
[12:23:03 CET] <rcombs> though I think it'd be most convenient to have e.g. `-s <size>` work when doing hardware transcodes (i.e. have it do a vaapi scale)
[12:23:12 CET] <jkqxz> vaapi_scale can take some (hardware-dependent, detected at runtime) set of normal pixel formats as input and output.
[12:24:21 CET] <jkqxz> vaapi_h264 allows nv12 input for convenience, which then unhelpfully gets picked when there are better choices.
[12:25:18 CET] <rcombs> is it listed first?
[12:25:57 CET] <jkqxz> AV_PIX_FMT_VAAPI is always first in these lists.
[12:26:00 CET] <rcombs> oh, heh, avcodec_default_get_format
[12:26:08 CET] <rcombs> `while (*fmt != AV_PIX_FMT_NONE && is_hwaccel_pix_fmt(*fmt))`
[13:57:58 CET] <jkqxz> nevcairiel:  ^ Same problem, hwaccel is not relevant at all.  (Though it does make it fail harder, because virtual memory doesn't save you.  Maybe threading+hwaccel shouldn't be allowed... :)
[13:59:11 CET] <wm4> lol
[14:01:01 CET] <nevcairiel> threading+hwaccel isnt allowed =p
[14:02:34 CET] <nevcairiel> luckily allowing it again isnt actually a problem for the majority of users as they dont use it anyway, and those that do can suffer their own fate
[14:03:07 CET] <nevcairiel> trying to stop users from shooting themself in the foot is apparently not a good thing anymore
[14:15:25 CET] <kierank> can we have some more outreachy tasks please
[14:15:29 CET] <kierank> https://trac.ffmpeg.org/wiki/SponsoringPrograms/Outreachy/2016-05
[14:25:14 CET] <wm4> nevcairiel: it's all about vlc
[14:25:24 CET] <wm4> and the inability of debian maintainers to troll vlc properly
[14:25:44 CET] <wm4> vlc is going to have to solve this, there's no other way, or they'll have to live without vp9 hwaccel forever
[14:25:59 CET] <wm4> apropos, why not push your dxva main10 patch?
[14:26:40 CET] <Daemon404> wm4, packagers truly believe that you should fix everything at the lowest level possible
[14:26:44 CET] <Daemon404> because it's the least work for them
[14:26:49 CET] <Daemon404> and fixes the greatest # of packages
[14:27:18 CET] <wm4> they also think unmaintained software is worth keeping around
[14:27:43 CET] <wm4> so they don't really have much credibility
[14:29:38 CET] <jkqxz> How about recommending that they solve it at a lower level: the kernel should serialise all threading.  I bet that would solve loads of odd race conditions in other programs too!
[14:54:07 CET] <nevcairiel> hwaccels+mt dont even multi-thread really
[14:54:16 CET] <nevcairiel> it just switches which worker thread to serially run on =p
[14:54:21 CET] <nevcairiel> so still consuming all the resources
[15:00:10 CET] <cone-229> ffmpeg 03Michael Niedermayer 07master:43a69655699b: avcodec/dirac: Fix memleak of dsh on error
[15:00:24 CET] <kierank> I have no response from outreachy...
[15:04:58 CET] <BtbN> jkqxz, does your vaapi patch series come with ffmpeg_vaapi.c, so the ffmpeg cli tool can use vaapi acceleration?
[15:07:10 CET] <jkqxz> BtbN:  Yes.
[15:14:11 CET] <durandal11707> ok to push metadata filters?
[15:15:49 CET] <durandal11707> kierank: perhaps last day to register was 8th
[15:15:55 CET] <kierank> I sent on 8th
[15:35:48 CET] <Timothy_Gu> kierank: for your fuzzing testsuite project, how do you determine which samples are "likely to crash"?
[15:36:44 CET] <kierank> Well my point is that for h264 commits you fuzz h264 and not opus
[15:37:02 CET] <Timothy_Gu> oh
[15:37:10 CET] <kierank> At the moment the current strategy doesn't focus on making sure stuff stays fixed
[15:37:27 CET] <Timothy_Gu> ok
[15:37:55 CET] <Timothy_Gu> anybody wants to mentor the documentation project?
[15:41:04 CET] <Timothy_Gu> also anyone against making fatebeta default?
[15:53:24 CET] <BBB> nope
[17:48:25 CET] <durandal11707> anybody against (a)loop filters?
[17:48:49 CET] <nevcairiel> you act as if we know what those are supposed to do
[17:49:57 CET] <nevcairiel> looping in filters is going to be extremely limited however, since you cant issue seek commands, so you have to cache all the looping area, consuming memory to no end =p
[17:50:41 CET] <durandal11707> it would cache limited number of frames and at eof loop them
[18:02:08 CET] <BBB> kierank: re 5224, Im not sure that can be fixed
[18:26:37 CET] <Timothy_Gu> i'm having a hard time trying to simd-ify "2 * a * b / 255," where a and b are bytes unpacked into words
[18:27:20 CET] <Timothy_Gu> i've already done a * b / 255 and that works fine
[18:27:45 CET] <Timothy_Gu> but I can't pre-multiply a or b by 2 since that would overflow
[18:28:11 CET] <Timothy_Gu> i've also tried to multiply the result by 2, but it seems to lose a bit of precision
[18:30:38 CET] <jkqxz> BBB:  5224 fell out of hwaccel/vaapi multithread testing.  When you need to use preallocated hardware surfaces, the limit really does kill you (mr3_tandberg_b in FATE does the same thing in a less-extreme way).
[18:30:39 CET] <durandal11707> yea you need ints
[18:30:46 CET] <jkqxz> In the software-only case it doesn't directly matter, but it's possible that a degenerate stream could do nasty things to memory-limited contexts.  (Maybe you can make something which falls over on a 32-bit player iff it uses threads?)
[18:31:23 CET] <nevcairiel> jkqxz: as long as it just barfs gracefully, its probably not something to worry about if its only on crafted streams
[18:32:56 CET] <Timothy_Gu> durandal11707: ugh
[18:32:58 CET] <nevcairiel> any crashes on out of memory would of course be bad
[18:35:45 CET] <Timothy_Gu> is there a "move quarter" macro in x86inc?
[18:36:10 CET] <nevcairiel> move quarter?
[18:36:30 CET] <jkqxz> Well, even if it does barf gracefully, calling malloc() too much could exhaust something else which barfs nongracefully.  So if no change is to be made it should probably be documented so that memory-limited people know that they should be careful with threads and untrusted video.
[18:41:05 CET] <jamrial> Timothy_Gu: pmaddwd?
[18:42:54 CET] <nevcairiel> i think he wanted to avoid 32-bit intermediates
[19:10:16 CET] <Timothy_Gu> yeah, I don't want to unpack bytes to doublewords
[19:10:51 CET] <Timothy_Gu> is it okay if i commit a non-bitexact version (with proper checks)?
[19:11:23 CET] <Timothy_Gu> wait there isn't a bitexact flag in libavfilter
[19:11:58 CET] <durandal11707> why you don't want doublewords?
[19:13:44 CET] <Timothy_Gu> it's too big :p
[19:14:06 CET] <Timothy_Gu> plus i need to find a way to move a quarter of an xmm to the memory
[19:14:50 CET] <Timothy_Gu> or maybe not. i can probably unpack it to another reg
[19:33:23 CET] <BBB> jkqxz: I think we all agree that you shouldnt allocate more than 1-thread worth of surfaces for MT/hw, since you never have actual concurrency with hw enabled
[19:38:13 CET] <jkqxz> Timothy_Gu:  Something like a * b, then the high half of multipling with 514 (pmulhw)?  That's probably not quite right across the whole range, but you have a few more bits above to play with to multiple by something a bit bigger and shift.
[19:51:33 CET] <jamrial> Timothy_Gu: a single dword? movd 
[19:52:31 CET] <jamrial> if it's not the first dword, use psrldq to shift it
[20:19:56 CET] <kierank> 7:14 PM <kierank> Did my email about FFmpeg actually make it to the list?
[20:19:56 CET] <kierank> 7:19 PM <marina> kierank: it did! thank you for arranging for FFmpeg's participation!
[20:20:45 CET] <Timothy_Gu> jamrial: didn't know movd works on xmm too
[20:22:00 CET] <jamrial> Timothy_Gu: http://www.felixcloutier.com/x86/MOVD:MOVQ.html
[20:25:19 CET] <jamrial> you also have pextrd to avoid having to shift dwords, but that's sse4 only
[21:08:24 CET] <durandal11707> kierank: what happened to adding SIMD to overlay?
[21:09:00 CET] <kierank> Might be a priority soon
[21:35:35 CET] <jkqxz> The first constant that will actually do it exactly is 131587 (a * b * 131587 >> 24 == 2 * a * b / 255), which is unfortunately outside 16 bits :(
[21:42:29 CET] <cone-229> ffmpeg 03Michael Niedermayer 07master:331a33d74a0e: nut: Add PAL8 support
[21:47:11 CET] <Timothy_Gu> jkqxz: oh well. i'll do 32 bits
[21:54:18 CET] <Timothy_Gu> fyi the patches arejust sent are different from the one i was trying to do
[00:00:00 CET] --- Wed Feb 10 2016