[Ffmpeg-devel-irc] ffmpeg-devel.log.20170603

[burek]

[00:11:41 CEST] <cone-651> ffmpeg 03Shivraj Patil 07master:6f35c21659f7: Disable MSA optimization for big endian arch
[00:11:42 CEST] <cone-651> ffmpeg 03Michael Niedermayer 07master:14b6adfd4627: avcodec/snowdec: Fix runtime error: signed integer overflow: 1404 * 8388608 cannot be represented in type 'int'
[00:11:43 CEST] <cone-651> ffmpeg 03Michael Niedermayer 07master:9faf098163b3: avcodec/aacps: Fix runtime error: left shift of 1073741824 by 1 places cannot be represented in type 'INTFLOAT' (aka 'int')
[00:14:15 CEST] <kevmark> is there any way to run tests/fate.sh on a specific commit instead of a branch? make fate seems to just run all the tests but I don't see any way to get an overview of the results
[00:14:54 CEST] <kevmark> I essentially want to run fate before and after my commit to see if it causes any tests to fail.
[00:15:01 CEST] <nevcairiel> if everything passes thats your result, there isnt really anything much more meaningful - if something fails, it'll stop and you have errors to view
[00:15:36 CEST] <kevmark> so a successful run will just spit out exit code 0?
[00:52:16 CEST] <divis1969> Hi, I'm trying to understand how the hw accel should be implemented in the ffmpeg. It looks there are 2 ways: to define the accel with REGISTER_HWACCEL or to define it in the ffmpeg_opt.c hwaccels array. What is the correct way?
[00:52:54 CEST] <wm4> REGISTER_HWACCEL is to add an AVHWAccel to the internal list in libavcodec
[00:53:10 CEST] <wm4> the one in ffmpeg_opt.c is for the ffmpeg.c tool
[00:53:24 CEST] <wm4> really depends what you're trying to do
[01:05:16 CEST] <divis1969> I need to improve the video decoding on AllWinner A20 to be able to use it with surveilance tools (ex. kerberosio). There is some vdpau_sunxi implementation, but I would prefer to get rid of dependency on X11. And, according to you, it is probably working in ffmpeg only, but kerberosio uses opencv -> libavcodec.
[01:06:11 CEST] <divis1969> Some additional info: https://forum.armbian.com/index.php?/topic/4060-hw-accelerated-video-decodingencoding-on-bpi-m/
[01:11:11 CEST] <wm4> well vdpau has no non-x11 support specified
[01:11:31 CEST] <wm4> but probably could by posting a proposal on the vdpau mailing list, or something
[01:37:11 CEST] <kierank> durandal_1707: fix my mpeg-4 sstp decoder pls ktx
[01:46:18 CEST] <cone-651> ffmpeg 03James Almer 07master:2ba896fef7ed: avformat/matroskaenc: also write chapters when output is WebM
[02:01:14 CEST] <durandal_1707> kierank: is it on github?
[02:01:26 CEST] <kierank> https://github.com/kierank/ffmpeg-sstp
[02:01:34 CEST] <kierank> yes but requires mpegvideoctx madness to be resolved
[02:02:37 CEST] <kierank> and there's one sample that does stuff that does hybrid dct/rle and that will never fit in mpegvideoctx
[03:26:58 CEST] <kevmark> jamrial, are you around?
[03:31:15 CEST] <jamrial> yes
[03:43:57 CEST] <kevmark> jamrial, thanks for the valgrind test. I'm attempting to boot up an Ubuntu VM now to try and replicate
[03:46:23 CEST] <jamrial> kevmark: you can also use gcc address sanitizer for this if that's easier for you
[03:46:39 CEST] <jamrial> it also catches the same invalid read
[03:54:59 CEST] <DHE> gcc is a lot faster, but I find valgrind to provide better reporting, stack traces, etc
[03:56:25 CEST] <kevmark> I believe I could do that by adding --extra-cflags=-fsanitize=kernel-address
[03:56:48 CEST] <DHE> kernel-sanitize is for linux. just use -fsanitize=address
[03:56:52 CEST] <kevmark> err , yeah
[03:57:30 CEST] <kevmark> I'll give both a go if I can. I'm compiling on macOS with clang and I've run into a few issues now
[03:58:00 CEST] <DHE> I say use gcc to find possible problems, valgrind to track them down if gcc is giving you problems.
[03:58:02 CEST] <kevmark> For instance, clang would gleefully let me compile some invalid C90 stuff whereas gcc on Ubuntu wouldn't
[04:03:51 CEST] <kevmark> Adding that line to configure gives me "gcc is unable to create an executable file." error
[04:07:09 CEST] <kevmark> Undefined symbols for ___asan_init and some other asan stuff
[04:09:03 CEST] <DHE> if you have a custom built gcc installed into /opt or such, you'll need to use -L/opt/gcc-xxx/lib64 or whathaveyou. also make sure your -fsanitize=xx parameter is part of ldflags
[04:11:01 CEST] <kevmark> Looks like adding it to --extra-ldflags did the trick for clang masquerading as gcc.
[04:19:31 CEST] <kevmark> clang doesn't seem to be catching it. Time to gcc then.
[04:19:45 CEST] <kevmark> *time to try
[04:25:36 CEST] <jamrial> kevmark: for asan, just configure ffmpeg with --toolchain=gcc-asan or --toolchain=clang-asan
[04:26:08 CEST] <kevmark> jamrial, I'll give it a try
[04:26:37 CEST] <kevmark> Use that in addition to -fsanitize=address?
[04:37:03 CEST] <jamrial> kevmark: no, the toolchain configure option adds the required gcc/clang options
[04:37:30 CEST] <kevmark> Okay, I tried it by itself and clang still couldn't find the issue so I'm giving gcc a go now
[04:38:52 CEST] <kevmark> Oh wait, I could just be an idiot
[04:39:01 CEST] <kevmark> Give me a minute haha
[04:48:41 CEST] <kevmark> Yep. That was the problem. I was running the single test (mine) that was guaranteed to work because it wouldn't replicate the behavior.
[04:49:18 CEST] <kevmark> scale2ref works just fine, it's when you run the scale filter (single input) that it barfs
[13:00:02 CEST] <atomnuker> J_Darnley: what are the speed improvements on the new simple_idct patches?
[13:34:17 CEST] <KGB> [13FFV1] 15michaelni closed pull request #66: Definitions (06master...06definitions) 02https://git.io/vHt47
[13:46:47 CEST] <KGB> [13FFV1] 15michaelni pushed 1 new commit to 06master: 02https://git.io/vH2Ta
[13:46:47 CEST] <KGB> 13FFV1/06master 14ba80af4 15Dave Rice: upgrading to a working group document...
[14:04:46 CEST] <BBB> nevcairiel: he doesnt mean theres a code issue
[14:04:53 CEST] <BBB> nevcairiel: he means theres a versioning issue in our .v script
[14:04:58 CEST] <BBB> nevcairiel: I think hes right TBH
[14:04:58 CEST] <nevcairiel> he wants us to fix his package manager
[14:05:02 CEST] <BBB> no
[14:05:04 CEST] <nevcairiel> yes
[14:05:11 CEST] <BBB> the symbol versioning is upstream in our .v file
[14:05:13 CEST] <nevcairiel> because his package manager relies on that info
[14:05:25 CEST] <nevcairiel> instead of you know, just putting a version number in there
[14:05:27 CEST] <BBB> that seems reasonable if we express the information?
[14:05:43 CEST] <BBB> if we think the .v file is not trustworthy, why do we have it at all?
[14:05:45 CEST] <BBB> then delete it
[14:05:47 CEST] <BBB> Im fine with that
[14:05:52 CEST] <BBB> but if its there, it should be correct
[14:05:58 CEST] <nevcairiel> the .v file is for  symbol visibility, we dont use it for versioning
[14:06:17 CEST] <BBB> this is like having docs that say we have 1000 of sec vulns and version 1.0 will never be released because we dont support wmv2 yet
[14:06:24 CEST] <BBB> we can say we dont really rely on the doc
[14:06:25 CEST] <BBB> but its there
[14:06:27 CEST] <BBB> and its wrong
[14:06:33 CEST] <BBB> so either fix it or delete it
[14:06:35 CEST] <BBB> both are fine
[14:06:41 CEST] <BBB> but having a broken doc - like a broken .v - is wrong
[14:06:43 CEST] <nevcairiel> look at our .v file, it contains one line that says av*
[14:06:55 CEST] <BBB> so delete it :)
[14:07:03 CEST] <nevcairiel> its required for visibility
[14:08:25 CEST] <nevcairiel> without it, the linker wouldnt know which symbols are actually public
[14:08:31 CEST] <nevcairiel> like we dont want to export ff* functions
[14:08:37 CEST] <BBB> mplayer \o/
[14:09:05 CEST] <BBB> anyway, try to help him along
[14:09:14 CEST] <BBB> if the .v should not be trusted, explain that
[14:09:32 CEST] <BBB> hes a package manager, he thinks were stupid for not using autotools and supermake
[14:09:40 CEST] <BBB> we need to help them
[14:09:51 CEST] <nevcairiel> he's a package manager, i think he is stupid for being that =p
[14:10:13 CEST] <BBB> J_Darnley: about the idct patches, the first 3 can be ignored, right? I dont think we intend to use the old mmx code for sse2-idct?
[14:10:21 CEST] <BBB> nevcairiel: tsk tsk, he means well
[14:17:06 CEST] <BtbN> Wouldn't "correctly" using that file also result in applications breaking on every minor bump, because the symbols moved to a diffrent version, even thouhg it would otherwise work fine?
[14:18:50 CEST] <nevcairiel> BBB: the key problem is that its not only new functions that appear within the  same ABI - if we say extend a struct and an app uses the new field in such a struct, running with an older library still crashes, and no ELF versioning is going to protect you there, its just not compatible in that direction
[14:19:11 CEST] <BtbN> Also, if a public function is missing, shouldn't the whole thing fail on startup, because the linker can't find the symbol?
[14:19:30 CEST] <nevcairiel> i would think so, but who knows what kind of lazy linking tricks they use
[14:19:48 CEST] <nevcairiel> ld has a lazy binding mode i think
[14:19:57 CEST] <BtbN> yeah, that case would just crash and burn. And no symbols are even involved
[14:21:50 CEST] <wm4> lazy linking bullshit will make it worse, yes
[14:22:12 CEST] <BtbN> What I think can be done is include a symbol for every minor version bump
[14:22:18 CEST] <wm4> (it's bullshit because apparently it doesn't help that much or makes it worse, but has a huge impact everywhere... shitty "optimization")
[14:22:22 CEST] <BtbN> and make an application link against the latest one
[14:23:32 CEST] <BtbN> So if an application links against lavc_ver_57_96, and the system only has 57_90, it will refuse to load
[14:23:44 CEST] <BtbN> problem with that is, in a lot of cases, such a constellation would work perfectly fine
[14:23:56 CEST] <nevcairiel> why would distributions ever have this problem anyway, dont they build everything against the same version anyway
[14:24:17 CEST] <BtbN> no idea
[14:29:08 CEST] <BBB> also Im not sure you guys are right
[14:29:21 CEST] <BBB> I think if you only use the public API as documented, our FW API compat is pretty decent
[14:29:35 CEST] <BBB> if you abuse it or do things documented to be bad, youre in for trouble, duh
[14:29:49 CEST] <BBB> but a simple video decoder app compiled against 3.0 will work against 3.1 if the major .so is the same
[14:30:00 CEST] <BBB> we bump major .so version if thats not the case
[14:30:10 CEST] <BBB> and I believe weve been pretty good at doing that
[14:30:15 CEST] <nevcairiel> i dont think anyone claimed that this direction would be problematic
[14:30:24 CEST] <nevcairiel> that guy on the ML is talking about the other direction
[14:30:29 CEST] <BBB> > >Anyway, in general, I recommend not pretending that FFmpeg has ABI
[14:30:29 CEST] <BBB> > >compatibility. Especially not across major releases
[14:30:40 CEST] <BBB> that suggests its bad in both directions
[14:30:52 CEST] <BBB> but I think we agree its ok in the FW direction
[14:31:09 CEST] <BBB> and he suggests we bump majors at each major release
[14:31:24 CEST] <BBB> I dont think we need to do that unless we actually broke ABI/API (removed/renamed symbols etc.)
[14:31:54 CEST] <wm4> why care about ABI
[14:33:14 CEST] <wm4> as someone who has an open source project that uses ffmpeg and which is packaged by some distros, I associate nothing but problems with the supposed ABI "compatibility"
[14:33:34 CEST] <wm4> and I added a check to my code which exits with an error if the version libs don't match exactly between runtime/compile time
[14:54:11 CEST] <Compn> convince the distros ?
[14:54:17 CEST] <Compn> they are the ones who want it
[14:54:59 CEST] <Compn> and by it i mean 2 year old ffmpeg system lib
[14:55:45 CEST] <BtbN> I really think introducing a GLIBC like symbol that prevents running against older ffmpeg libraries, even just for minor bumps
[14:56:00 CEST] <BtbN> , might be a good idea.
[14:56:11 CEST] <wm4> doesn't glibc just use fine grained symbol versioning
[14:56:32 CEST] <BtbN> that's for its backwards compat
[14:56:37 CEST] <DHE> sorta. a function might internally be named sscanf@@GLIBC_2.15
[14:56:52 CEST] <DHE> yeah, but they'll still export sscanf@@GLIBC_2.3 for backwards compatibility
[14:56:54 CEST] <BtbN> But in the other way, if something is linked against a more recent glibc than the system has, it also has a mechanism
[14:57:18 CEST] <BtbN> it exports a symbol for every version bump it ever did. And applications link against the latest one
[14:57:25 CEST] <DHE> no, it just shits itself with a missing symbol. those of us on "old" platforms like centos6 see this regularly.
[14:57:28 CEST] <BtbN> so an older version won't have that symbol, and startup will fail
[14:58:05 CEST] <wm4> (solution: don't use centos)
[14:58:28 CEST] <wm4> BtbN: can't glibc do this only because it's the runtime lib?
[14:58:36 CEST] <DHE> wm4: a 10 year support cycle is VERY appealing
[14:58:51 CEST] <BtbN> I don't see why ffmpeg couldn't do it aswell
[14:59:51 CEST] <wm4> DHE: depends on the kind of misery that it's for, I guess
[15:03:01 CEST] <nevcairiel> the problem starts when people use such a distro and then still expect to run the latest stuff on it
[15:15:06 CEST] <kierank> atomnuker: 40% so far but not fully optimised and not fully correct iirc
[15:26:54 CEST] <atomnuker> very nice
[15:28:39 CEST] <kierank> I don't like seeing that ugly mmx inline thing in our profiles
[16:15:46 CEST] <gh0st__> kierank: what's wrong with mmx?
[16:16:00 CEST] <kierank> 2017 is wrong with mmx
[16:16:39 CEST] <gh0st__> ah, I see. But why not to keep? code size?
[16:18:52 CEST] <wm4> maintenance burden, and the need to deal with emms are good reasons to drop mmx
[16:20:41 CEST] <durandal_1707> drop ths mmx
[16:20:46 CEST] <gh0st__> I see, what is emm?
[17:16:47 CEST] <KGB> [13FFV1] 15JeromeMartinez opened pull request #68: Make Quantization descriptions more coherent (06master...06QuantizationTables) 02https://git.io/vH23k
[17:22:21 CEST] <BBB> gh0st__: emms is an instruction that clears the mmx/floating point (x87) state; its basically pointless on x86-64 since nobody uses x87, but since its there you have to clear it anyway, its slow and stupid
[17:22:39 CEST] <BBB> gh0st__: mmx is fine if it fits specifically only in mmx regs, but if it fits in xmm, use only xmm and ignore mmx for all you can  :)
[17:41:52 CEST] <cone-976> ffmpeg 03James Almer 07master:be3809a521fe: x86/aacpsdsp: optimize ff_ps_stereo_interpolate_sse3
[18:16:28 CEST] <Gramner> mmx isn't necessarily fine just because the size of an mmx register is enough to fit your data. padd(u)s*, psub(u)s*, pcmp*, pmax*, pmin*, pavg*, pabs* and psign* (and maybe something else, don't remember) only execute with half throughput in mmx registers compared to xmm registers on modern intel cpus
[20:40:30 CEST] <nevcairiel> durandal_1707: do you plan more flexible input to the upmix filter? like 3.0 input to 5.1, or something like that?
[20:45:03 CEST] <durandal_1707> nevcairiel: anything that have left and right channels could be upmixed, why?
[20:45:41 CEST] <nevcairiel> durandal_1707: but it would overwrite the existing channel then? and just wondering, it would be nice to have it work on any input
[20:46:14 CEST] <durandal_1707> thats lot of combinations and code
[20:46:31 CEST] <nevcairiel> i guess
[20:47:38 CEST] <nevcairiel> could do it a bit flexible and just treat the input as stereo, but if a channel already exists just dont overwrite it
[21:15:10 CEST] <durandal_1707> channels need to be overwritten for proper separation
[21:15:41 CEST] <durandal_1707> except for center channels
[21:16:02 CEST] <durandal_1707> thay can be left unchanged
[21:18:17 CEST] <nevcairiel> i suppose it would mostly be 3.0 signals (ie. stereo with center), stereo with lfe, or stereo center and lfe
[21:19:48 CEST] <nevcairiel> in any case the existing filter is already pretty good, stereo to 5.1 is what most people have asked me for
[21:19:53 CEST] <wm4> lol at all those API and ABI fuckups
[21:22:10 CEST] <kierank> durandal_1707: so I have an idea to split mpeg-4 from mpeg2
[21:28:43 CEST] <jamrial> wm4: afaik, codecpar being at the end was because downstream projects were accessing private fields, and when it was introduced above the private line the offsets for said private fields changed and those projects started to crash
[21:29:17 CEST] <jamrial> so we compromised, moved it down, and violated our own rules. because downstream projects can't follow them
[21:29:42 CEST] <jamrial> you kinda "fixed" it by adding a "from here on, things are public again" marker that made it on 3.3 i think
[21:29:56 CEST] <jamrial> which is ugly af but will be gone once we bump
[21:30:16 CEST] <nevcairiel> we should start making a bump checklist thing
[21:30:20 CEST] <nevcairiel> so many things we still wanted to move
[21:30:21 CEST] <jamrial> ^
[21:31:15 CEST] <durandal_1707> kierank: really, how much code change?
[21:31:42 CEST] <wm4> we could also move the private fields into an internal struct now, and leave only padding fields to keep current ABI
[21:32:13 CEST] <nevcairiel> nah lets just do it all in one go, no reason to break such apps silently
[21:32:14 CEST] <jamrial> what's the point? we're not going to branch a new release before we bump
[21:32:16 CEST] <jamrial> hopefully
[21:32:17 CEST] <wm4> if downstream apps have real reasons to access some of the fields, new APIs can be added, otherwise fuck them
[21:32:45 CEST] <BtbN> I don't think anything would break when removing the private fields
[21:32:53 CEST] <kierank> durandal_1707: dunno, but only way I can merge my decoder afaik
[21:32:54 CEST] <BtbN> Except something does terrible things
[21:34:39 CEST] <BtbN> What we could do is move them to a private struct, and embed that private struct at the end of the existing struct.
[21:34:44 CEST] <BtbN> So API would break, but ABI won't
[21:35:01 CEST] <nevcairiel> we basically already have a plan, so lets just do it at bump time
[21:35:49 CEST] <BtbN> 3.4 will most likely be a major bump, won't it?
[21:36:01 CEST] <nevcairiel> probably
[21:37:09 CEST] <jamrial> i'd rather call it 4.0. major bump, dropped api, so better if we make it obvious
[22:15:17 CEST] <jamrial> nevcairiel: can you review the vpcc on mov demuxer patch?
[22:15:27 CEST] <jamrial> i basically only read the color info, since the rest can be derived from the bitstream
[22:21:39 CEST] <durandal_1707> nevcairiel: do you have 3.0 samples?
[22:24:11 CEST] <nevcairiel> i have a 3.1 real content, and 3.0 artificial content
[22:26:25 CEST] <durandal_1707> 3.1 should be good, i could just drop lfe and compare
[22:28:17 CEST] <nevcairiel> http://files.1f0.de/samples/3.1-st.mkv
[22:29:28 CEST] <nevcairiel> not sure it actually has any LFE content in the channel
[22:29:36 CEST] <nevcairiel> but it does use L/R/C
[22:29:47 CEST] <nevcairiel> ac3 is typically a bit weak in lfe
[22:31:11 CEST] <rcombs> durandal_1707: want a sample from a BD that's supposed to be 2.1, but was mis-coded as 3.0?
[22:31:19 CEST] <rcombs> (on the disc, afaict)
[22:33:47 CEST] <durandal_1707> 2.1 is kind of nice because it have lfe already, could help to compare
[22:34:49 CEST] <durandal_1707> nevcairiel: it have lfe always silence, for 3.0 to 3.1 i will just pick center for lfe generation
[23:14:46 CEST] <cone-367> ffmpeg 03Kevin Mark 07master:08213e0b7974: libavfilter/scale2ref: Fix out-of-bounds array access
[23:14:47 CEST] <cone-367> ffmpeg 03Michael Niedermayer 07master:53e0d5d72475: avformat/options: log filename on open
[00:00:00 CEST] --- Sun Jun  4 2017