[Ffmpeg-devel-irc] ffmpeg-devel.log.20160831

[burek]

[00:20:30 CEST] <Compn> yes that
[00:55:21 CEST] <DHE> from an API standpoint it doesn't really provide a way to do per-thread control. those only provide whole-process management. is there really a benefit to further control though?
[00:56:21 CEST] <DHE> pinning is best used for locality of RAM (numa nodes) and maybe cache. beyond that, let the OS do its job.
[01:10:27 CEST] <iive> i think it was called thread affinity 
[01:12:43 CEST] <iive> man taskset
[01:14:12 CEST] <DHE> right. but for example I might want to have two outputs, one encoder on some CPUs and another on some others. Keeps the cache lines relatively local.
[01:14:21 CEST] <DHE> I guess the question is, does that even is make sense?
[01:14:39 CEST] <iive> man pthread_setaffinity_np
[01:15:37 CEST] <iive> the linux kernel have an option where it would try to minimize the spread of processes, in order to preserve cpu power
[01:15:53 CEST] <iive> it's a recent one.
[01:16:26 CEST] <iive> and keeping cache warm is always good idea for memory heavy operations, like video coding.
[01:20:45 CEST] <DHE> but that isn't used in ffmpeg right now, and no way to pass preferences over to libx264 and the like
[01:21:20 CEST] <BBB> you can pass preferences to x264 using -x264opts no?
[01:21:50 CEST] <BBB> and yes that makes sense
[01:22:10 CEST] <BBB> I cant give you specific numbers on how much youll save, but Ive heard (..) people say this makes some difference
[01:24:41 CEST] <DHE> you can ask x264 for specific numbers of threads for various jobs, and it actually renices some threads (lookahead I think) but not affinity without setting a process rule first and letting them inherit
[02:55:46 CEST] <cone-364> ffmpeg 03Umair Khan 07master:9fbf0660c170: MAINTAINERS: Add myself for alsdec
[02:55:46 CEST] <cone-364> ffmpeg 03Steven Liu 07master:3aab6fa6baf8: avformat/hlsenc: add warning for append_list and hls_init_time option
[10:54:43 CEST] <BtbN> michaelni, I think I see why the pad test fails(The pad filter, or rather drawutils, doesn't handle the weirdness of P010).
[10:54:54 CEST] <BtbN> But no idea why the unrelated pixel formats start failing on mips
[10:56:12 CEST] <BtbN> shifting should allways move in zeroes, even on mips?
[11:06:34 CEST] <BtbN> Just noticed. It's not even the hash changing. It's replacing the gbrp*le formats with their be counterparts.
[11:07:13 CEST] <michaelni> BtbN, just checked, it seems the gbrp fail before your patch too
[11:07:27 CEST] <michaelni> this must have been ver recently broken
[11:08:10 CEST] <michaelni> http://fatebeta.ffmpeg.org/report/mips-ubuntu-qemu-gcc-4.4/20160831070052#failed_tests
[11:09:30 CEST] <BtbN> can you test if this: https://gist.github.com/BtbN/7d6da23bd5b9bec5e5b3127f85b8c3bb  results in "p010le b7eb46af7be9f0acb852499a3dabf57b" for the pixfmts-pad test?
[11:10:27 CEST] <durandal_1707> michaelni: thats my error, lut should only support LE variants in query formats
[11:10:52 CEST] <BtbN> and it instead only supports be formats?
[11:11:29 CEST] <durandal_1707> it just pick native one on mips which is be
[11:11:47 CEST] <BtbN> ah, yeah, makes sense
[11:15:24 CEST] <michaelni> BtbN, +p010le              5c79c1d24f1e0d67e2b4d4a2f40838f7
[11:15:33 CEST] <BtbN> hm
[11:15:44 CEST] <BtbN> so I assume mips behaves diffrently when shifting?
[11:17:07 CEST] <BtbN> The md5 hash also hashes the 6 unused bits i assume?
[11:18:09 CEST] <BtbN> yeah, it does. hmm
[11:18:09 CEST] <michaelni> md5 should hash all it gets
[11:18:32 CEST] <BtbN> Does mips not shift in zeroes? Can't really imagine that.
[11:19:13 CEST] <BtbN> I'm tempted to just return ENOSYS for P010 from drawutils, and fix it at a later point.
[11:19:22 CEST] <michaelni> i would expect zeros
[11:19:31 CEST] <BtbN> It would be very broken if it didn't.
[11:20:46 CEST] <cone-073> ffmpeg 03Paul B Mahol 07master:8175fb03f0ba: avfilter/vf_lut: unbreak planar rgb suppot on big-endian
[11:23:02 CEST] <BtbN> michaelni, yeah, no idea what's going on with drawutils and P010. I don't think anyone will use that anyway.
[11:30:16 CEST] <michaelni> BtbN, the output from the pad test looks wrong beyond the padding, also its Output #0, nut, to 'out.nut': Stream #0:0: Video: rawvideo (RGB[15] / 0xF424752), p010le, 500x400, q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc
[11:30:29 CEST] <michaelni> on reading: Stream #0:0: Video: rawvideo (RGB[15] / 0xF424752), rgb555le, 500x400, 25 tbr, 51200 tbn, 51200 tbc
[11:31:08 CEST] <durandal_1707> michaelni: you are missing entries in lavc/raw.c for p010 mapping into nut
[11:31:18 CEST] <michaelni> yes
[11:33:45 CEST] <durandal_1707> michaelni: i also get: libswscale/input.c:944:1: warning: unused function 'planar_rgb9be_to_a' [-Wunused-function]
[11:35:26 CEST] <durandal_1707> ah thats is because there is no gbrap9
[14:02:44 CEST] <BtbN> I feel like P010 will find new bugs all over the place. Just because of this weird shift
[14:55:04 CEST] <mateo`_> is Thomas Volkert on irc ?
[14:55:54 CEST] <ubitux> rcombs: what's the state of the new bitstream filter api? it seems we're still using the old api in ffmpeg.c?
[14:56:31 CEST] <ubitux> (av_apply_bitstream_filters())
[14:56:58 CEST] <cone-833> ffmpeg 03Timo Rothenpieler 07master:d3a23b67777a: avfilter/drawutils: P010 is not supported
[14:56:58 CEST] <cone-833> ffmpeg 03Timo Rothenpieler 07master:2625b955a31a: avfilter/drawutils: honor shift for color component description
[14:56:58 CEST] <cone-833> ffmpeg 03Timo Rothenpieler 07master:99882d05a663: swscale: add support for P010LE/BE output
[15:11:59 CEST] <durandal_1707> trac is very slow for me
[16:17:58 CEST] <durandal_1707> I propose we ditch avisynth in favour of vapoursynth
[16:19:24 CEST] <BtbN> building with asan takes forever
[16:21:13 CEST] <DHE> running with asan is pretty bad as well. not valgrind bad, but...
[16:21:57 CEST] <BtbN> Well, I'm investigating a linker error that only happens on asan for some reason
[16:22:14 CEST] <BtbN> So luckily I "only" have to compile it
[16:22:31 CEST] <BtbN> Hmm... just completed successfully.
[16:22:49 CEST] <BtbN> So no idea what's up with this: http://fate.ffmpeg.org/log.cgi?time=20160828213006&log=compile&slot=x86_64-archlinux-gcc-asan
[16:23:35 CEST] <DHE> how did that happen?
[16:24:21 CEST] <BtbN> I have no idea
[16:24:30 CEST] <BtbN> All fate asan runs fail since nvenc is enabled by default
[16:24:37 CEST] <BtbN> And I just can't figure out a reason why, and it works on my machine
[16:24:58 CEST] <DHE> there isn't anything silly like a precompiled .o file laying around, is there?
[16:25:03 CEST] <BtbN> no
[16:32:23 CEST] <BtbN> ./configure --disable-everything --enable-nvenc --disable-doc --enable-encoder=h264_nvenc --toolchain=gcc-asan
[16:32:28 CEST] <BtbN> reproduces it
[16:33:08 CEST] <BtbN> it's simply not adding -ldl
[16:36:44 CEST] <BtbN> ok, I see what's going on
[16:36:51 CEST] <BtbN> --enable-encoder=nvenc works
[16:37:53 CEST] <BtbN> hm
[16:37:58 CEST] <BtbN> the fate run has nvenc enabled though
[16:38:13 CEST] <BtbN> and a lot of other stuff that should require ldl
[16:40:18 CEST] <BtbN> Hm, so, I'm at a loss as to why gcc-asan is failing on fate.
[17:25:04 CEST] <jamrial> BtbN: think i found why
[17:25:09 CEST] <jamrial> dlsym needs -ldl whereas dlopen doesn't, and configure only checks for dlopen to choose if it needs to add -ldl to EXTRALIBS or not
[17:59:00 CEST] <BtbN> jamrial, uh, what? That's not the case on all my test system(all gentoo).
[18:04:48 CEST] <jamrial> BtbN: it is on ArchLinux x86_64
[18:05:06 CEST] <BtbN> weird
[18:05:23 CEST] <jamrial> odly enough this is an issue only with gcc-asan
[18:05:51 CEST] <BtbN> libdl probably gets pulled in from some other library at runtime
[18:05:55 CEST] <BtbN> so only asan barfs on it
[18:06:04 CEST] <ubitux> i haven't updated since a while
[18:06:06 CEST] <ubitux> maybe i should?
[18:06:29 CEST] <BtbN> I'd rather have this fixed propperly
[18:06:39 CEST] <BtbN> And I doubt updating it fixes it
[18:06:59 CEST] <jamrial> ubitux: i have an updated arch system and it fails the same way as your outdated one
[18:09:38 CEST] <BtbN> I wonder what arch does diffrently than Gentoo?
[18:09:48 CEST] <BtbN> Tested this on all 5 boxes I had available, and it worked fine
[18:09:48 CEST] <jamrial> BtbN: http://pastebin.com/jQk3YFjS
[18:10:06 CEST] <jamrial> when using asan, -ldl is not added to EXTRALIBS
[18:10:45 CEST] <jamrial> the dlopen check in configure (line 5380) succeeds without using -ldl in that case
[18:11:31 CEST] <BtbN> Well, and it's indeed not complaining about dlopen missing.
[18:12:04 CEST] <jamrial> maybe just adding a similar test for dlsym would do it
[18:49:53 CEST] <BtbN> jamrial, is arch using gold? Which gcc version is it using?
[19:07:32 CEST] <jamrial> BtbN: don't think so, and gcc 6.1 currently
[19:09:39 CEST] <BtbN> hm, it's also happening on 5.3 on arch. 4.9 on Gentoo is fine.
[19:09:51 CEST] <BtbN> Haven't tested 5.4 on Gentoo yet.
[19:09:54 CEST] <BtbN> will do so now
[19:11:39 CEST] <BtbN> The PC with 5.4 is the slowest one, of course.
[19:37:14 CEST] <BtbN> jamrial, ok, happens on gentoo as well, when using gcc 5.4
[19:39:36 CEST] <BtbN> same thing, it -ldl only vanishes from EXTRALIBS in asan mode.
[19:40:31 CEST] <BtbN> Hm, there is _a lot_ of stuff tied to dlopen
[19:41:15 CEST] <BtbN> also, is it only me, or is omx_extralibs missplaced? It's placed before $ldl is even set, and still uses it.
[19:57:12 CEST] <BtbN> jamrial, https://paste.pound-python.org/show/boYHbQTFUffn1rzcyDGb solves it for me
[20:01:09 CEST] <jamrial> BtbN: yeah, -ldl is added after that patch
[20:04:54 CEST] <jamrial> BtbN: and regarding that omx check, i think it's fine since check_deps() is called near the end of configure
[20:05:54 CEST] <jamrial> or maybe not since $ldl would be blank, mmh
[20:05:59 CEST] <BtbN> https://github.com/FFmpeg/FFmpeg/blob/master/configure#L2671
[20:06:00 CEST] <BtbN> this one
[20:06:09 CEST] <BtbN> $ldl is just an empty/unset var at that point
[20:06:15 CEST] <BtbN> so omx_extralibs will be empty
[20:06:26 CEST] <jamrial> yeah, i know
[20:06:59 CEST] <BtbN> I'll prepare another series of patches
[20:07:01 CEST] <jamrial> guess it should be moved then. no idea who maintains that omx code
[20:08:47 CEST] <BtbN> the two patches on the ml should be fine to push i think?
[20:11:56 CEST] <jamrial> BtbN: libav has the line "nvenc_deps_any="dlopen LoadLibrary"
[20:12:11 CEST] <jamrial> and a check for LoadLibrary on windows
[20:12:14 CEST] <jamrial> for some reason it wasn't merged
[20:12:39 CEST] <jamrial> probably back when we noop'd their nvenc commits
[20:13:41 CEST] <BtbN> didn't even know that any-logic existed
[20:14:08 CEST] <jamrial> configure is full of neat things like that :p
[21:08:45 CEST] <durandal_1707> why xbr filter uses 24mb lut table to do rgb<->yuv conversion?
[00:00:00 CEST] --- Thu Sep  1 2016