[Ffmpeg-devel-irc] ffmpeg.log.20190811

[burek]

[00:02:29 CEST] <jgb> who do i have to beg if i want to get this fixed: https://trac.ffmpeg.org/ticket/7912
[00:07:09 CEST] <nicolas17> wew
[00:07:31 CEST] <nicolas17> I built ffmpeg git-master with libaom git-master to see if that encoded faster than whatever crap Debian Testing ships
[00:07:34 CEST] <nicolas17> SIGFPE
[00:09:11 CEST] <nicolas17> hm aom is unrelated, the exception happens in the audio encoding
[00:15:58 CEST] <cehoyos> How can I reproduce?
[00:16:30 CEST] <jgb> cehoyos how much do you need?
[00:16:56 CEST] <cehoyos> nicolas17: How can I reproduce the exception?
[00:17:15 CEST] <jgb> cehoyos how much money do you need?
[00:18:30 CEST] <DHE> can you run the ffmpeg_g binary under gdb and get a stack trace or something?
[00:18:32 CEST] <nicolas17> I have a few factors to rule out first... and I need to repro with an input that isn't 3GB
[00:25:02 CEST] <jgb> i will pay $$$ to get this fixed:   https://trac.ffmpeg.org/ticket/7912
[00:25:06 CEST] <nicolas17> this is *weird*
[00:25:21 CEST] <nicolas17> https://pastebin.com/1zR9ykqq the crash is in the aac audio encoder but it only happens if I use aom as video codec?!
[00:26:35 CEST] <jgb> i will pay $$$ to get this fixed:   https://trac.ffmpeg.org/ticket/7912
[00:27:31 CEST] <nicolas17> DHE: ^ stack trace
[00:27:37 CEST] <cehoyos> nicolas17: Please also provide the complete, uncut console output
[00:28:45 CEST] <cehoyos> No division happens in line 680 of libavcodec/aacpsy.c afaict...
[00:29:42 CEST] <DHE> doesn't have to be division
[00:31:07 CEST] <cehoyos> I don't think the others can happen in a C program...
[00:31:20 CEST] <jgb> cehoyos  i will pay $$$ to get this fixed:   https://trac.ffmpeg.org/ticket/7912
[00:32:42 CEST] <nicolas17> DHE cehoyos: https://pastebin.com/7cTddMBn
[00:33:04 CEST] <DHE> nicolas17: disassemble
[00:33:15 CEST] <nicolas17> I did, bottom of paste :)
[00:33:27 CEST] <DHE> ah, so you did
[00:33:37 CEST] <jgb> DHE i will pay $$$ to get this fixed:   https://trac.ffmpeg.org/ticket/7912
[00:34:10 CEST] <DHE> jgb: due to my line of work I must refuse
[00:34:10 CEST] <cehoyos> Please provide 0.ts
[00:34:36 CEST] <nicolas17> https://vod-secure.twitch.tv/9a9548d02b0de016012b_kayradim_34830321904_1245563253/chunked/0.ts
[00:34:47 CEST] <DHE> I also don't do unsolicited report unless I feel like it
[00:34:50 CEST] <DHE> *support
[00:36:00 CEST] <DHE> nicolas17: that's weird. what's "p spread_en[w+g-1]"  (and all the other terms I guess) give? anything NaN or some kind of bonkers number?
[00:36:29 CEST] <nicolas17> I just tried with 5 random .ts files from this HLS stream and they all SIGFPE, so I'm not sure if the input matters
[00:36:50 CEST] <cehoyos> To me this looks like a compiler or hardware problem
[00:37:13 CEST] <cehoyos> Do I understand correctly that it does not happen without video encoding?
[00:37:29 CEST] <DHE> or if you used libx264 the crash would go away?
[00:37:29 CEST] <cehoyos> Does it always happen at exactly the same position?
[00:37:42 CEST] <cehoyos> Does it happen with -threads 1?
[00:38:15 CEST] <nicolas17> it doesn't even get to print the status bar so it looks like the very beginning of the input
[00:38:32 CEST] <nicolas17> still happens with -threads 1
[00:39:24 CEST] <nicolas17> I tried -c:v mpeg1video and it doesn't happen (I didn't build this ffmpeg with libx264 support, I can try that)
[00:39:43 CEST] <DHE> that seems sufficient...
[00:40:31 CEST] <nicolas17> I'll try using clang, maybe we can blame gcc
[00:40:50 CEST] <cehoyos> Does the crash also happen with valgrind?
[00:41:28 CEST] <DHE> I am leaning towards some kind of miscompile or hardware issue involving advanced instructions...
[00:41:59 CEST] <DHE> maybe -threads 1 on both input and output. when debugging you should not see any lines like "[New Thread 0x7ffff757d700 (LWP 17706)]" and put it on a fixed CPU core/thread...
[00:42:24 CEST] <nicolas17> hm, doesn't crash with valgrind (status bar goes to frame=11 and then gets glacial-slow for obvious reasons)
[00:42:30 CEST] <DHE> taskset 0x01 ffmpeg -threads 1 -i input.ts [...] -threads 1 output.ts
[00:42:49 CEST] <nicolas17> DHE: I totally forgot that -threads was a per-file option
[00:42:58 CEST] <DHE> if I got that right. I don't normally use taskset
[00:46:39 CEST] <cehoyos> Your command line is slow here but works / does not crash
[00:46:58 CEST] <DHE> then it'll probably never finish on valgrind...
[00:47:08 CEST] <DHE> did you try the taskset+single-thread command?
[00:47:53 CEST] <DHE> actually taskset alone may force single-threaded operation
[00:48:05 CEST] Action: DHE has some dual 20-core CPUs and ffmpeg goes a little bit crazy on those. :)
[00:48:19 CEST] <DHE> *dual-socket
[00:48:39 CEST] <nicolas17> ok I finally have a significant clue... with libaom 1.0.0 it doesn't crash
[00:48:45 CEST] <nicolas17> this calls for a git-bisect
[00:48:50 CEST] <DHE> oh my...
[00:49:46 CEST] <cehoyos> I only have 1.0.0
[00:49:58 CEST] <DHE> you probably need to delete libavcodec/libaom*.o after each rebuild of libaom to make sure ffmpeg builds okay with the new version.
[00:50:28 CEST] <cehoyos> You could also PRELOAD the dll...
[00:50:38 CEST] <DHE> worried about header file changes in libaom
[00:50:48 CEST] <DHE> is the ABI stable in git?
[00:56:05 CEST] Action: DHE is test building a Git version of aom
[01:05:16 CEST] <DHE> no crash with the commandline from your pastebin and sample 0.ts file
[01:05:32 CEST] <DHE> (also I see the encoder is MUCH faster than it was when I first experimented with AV1)
[01:07:12 CEST] <another> DHE: when i first switched to HEAD from 1.0.0 it was 40% faster
[01:09:12 CEST] <DHE> another: I did a test encode of a 1080p video back when AOM made their big press announcement of the codec's "readiness". iirc it was like 10+ minutes per frame. now this is 720p but I'm getting 0.8fps
[01:10:09 CEST] <nicolas17> when I tried 1.0.0 yesterday it did 35 frames in 6 minutes according to ffmpeg's status bar, but it took its sweet extra time to 'flush' at the end too
[01:10:58 CEST] <DHE> internal buffering of frames is pretty common for rate estimation, motion compensation, and b-frames. even x264 want to buffer 30+ (more?) frames with default settings
[01:13:01 CEST] <nicolas17> wow
[01:13:12 CEST] <nicolas17> bisect is still running but from the list of remaining revisions I have an uhhh strong candidate
[01:13:26 CEST] <nicolas17> https://aomedia.googlesource.com/aom/+/85d00fa8e23ea067c246d45b86f26c78bfcb4fbd
[01:13:36 CEST] <nicolas17> https://aomedia.googlesource.com/aom/+/85d00fa8e23ea067c246d45b86f26c78bfcb4fbd%5E%21/ better link
[01:14:53 CEST] <DHE> that seems plausible
[01:15:09 CEST] <nicolas17> yep, bisect finished and points there
[01:16:50 CEST] <DHE> that seems like it could be a trac ticket. aac encoder may overflow, causes SIGFPE on new versions of AOM-AV1
[01:17:01 CEST] <DHE> (or just custom apps that make use of the same thing)
[01:17:36 CEST] <DHE> actually I think I would blame aom more, a library tampering with global process configuration
[01:17:50 CEST] <nicolas17> hrmmm
[01:17:55 CEST] <nicolas17> is that per process or per thread?
[01:19:13 CEST] <DHE> at this point I don't think it particularly matters.
[01:19:39 CEST] <nicolas17> aom seems to be setting and *restoring* the state of floating point exceptions and precision
[01:19:59 CEST] <nicolas17> within aom_codec_encode
[01:21:43 CEST] <cehoyos> I was able to reproduce
[01:22:01 CEST] <cehoyos> My knowledge of the C standatd is not sufficient to understand if there is a bug in FFmpeg or a bug in libaom
[01:23:18 CEST] <DHE> nicolas17: did you try my taskset commandline?
[01:24:30 CEST] <nicolas17> with taskset, and -threads 1 on both input and output... still FPE
[01:25:02 CEST] <DHE> so that leaves me with the assumption that the "restore state" code isn't working properly
[01:25:20 CEST] <DHE> on the plus side it looks like a debug feature, so doing a non-debug AOM build should eliminate it...
[01:26:07 CEST] <cehoyos> Interestingly, it doesn't matter which define I pass to feenableexception(), everything !=0 crashes
[01:26:20 CEST] <nicolas17> wut
[01:26:30 CEST] <nicolas17> is aac triggering every possible exception? :P
[01:26:59 CEST] <DHE> so it sounds like two different small errors that don't really coincide normally are now colliding hard
[01:27:41 CEST] <cehoyos> Which two errors?
[01:27:58 CEST] <another> could not reproduce
[01:28:05 CEST] <cehoyos> How did you test?
[01:29:24 CEST] <another> commandline in pastebin with {ffmpeg,aom}-git
[01:30:16 CEST] <cehoyos> I put a call to feenableexcept() in aac_encode_init() and just did ffmpeg -i input -acodec aac -f null -
[01:32:07 CEST] <another> debug build?
[01:32:25 CEST] <nicolas17> I also found something questionable in the aom build system, build type "Release with debug info" enables CONFIG_DEBUG... I would expect it to be identical to Release except with -g
[01:32:48 CEST] <nicolas17> another: it seems CMAKE_BUILD_TYPE=Release on AOM will not trigger this problem
[01:33:38 CEST] <another> iC
[01:34:10 CEST] <cehoyos> Crashes for INEXACT, OVERFLOW, UNDERFLOW and DIVBYZERO, not for INVALID
[01:40:18 CEST] <cehoyos> And I can confirm that it is not possible to restore the exception state;-)
[01:40:41 CEST] <nicolas17> hmm
[01:40:47 CEST] <nicolas17> I think they're doing it wrong :)
[01:42:00 CEST] <nicolas17> I think feenableexcept enables the exceptions you give it (|=), rather than set the mask of exceptions to *exactly* the ones you give it (=), so eg. feenableexcept(0) is always a no-op
[01:42:44 CEST] <nicolas17> so passing the old set of exceptions to feenableexcept won't work, you need fedisableexcept
[01:43:14 CEST] <cehoyos> Correct, I just came to the same conclusion
[01:43:23 CEST] <cehoyos> You cannot restore with feenableexcept()
[01:44:14 CEST] <cehoyos> I am nearly sure there will be no change in FFmpeg, but thank you for the report!
[01:44:59 CEST] <nicolas17> yeah it seems the bug is in aom's FLOATING_POINT_RESTORE_EXCEPTIONS
[01:46:04 CEST] <cehoyos> Yes
[02:14:47 CEST] <DHE> avoiding overflow/underflow might be a good idea overall, but the priority would be aom fixing their stuff...
[02:15:17 CEST] <cehoyos> Is floating point overflow / underflow undefined behaviour?
[02:15:44 CEST] <DHE> no, but the state of the FPU exception of the CPU is an unkonwn unless explicitly set
[02:15:50 CEST] <DHE> (which AOM tried to do and screwed up)
[02:16:28 CEST] <DHE> maybe linux guarantees a certain default, but it's possible a custom app import ffmpeg changed it
[02:17:29 CEST] <nicolas17> fedisableexcept((FE_DIVBYZERO | FE_UNDERFLOW | FE_OVERFLOW) & ~original_mask) does this look sane?
[02:17:51 CEST] <DHE> there's a "set exact bits" function, why not use that?
[02:19:05 CEST] <nicolas17> what function is that?
[02:34:38 CEST] <nicolas17> filed aom bug
[02:39:40 CEST] <DHE> there's a function fesetexcept they're all documented in the same man page
[03:18:11 CEST] <nicolas17> next mystery
[03:18:32 CEST] <nicolas17> I tried encoding 15 seconds of video with AV1 with different values of -cpu-used
[03:19:40 CEST] <nicolas17> cpu-used 8: 19m57s; cpu-used 6: 21m30s; cpu-used 3: 22m1s; cpu-used 1: 23m6s and a segfault
[03:23:48 CEST] <nicolas17> running under gdb, see you in 24 minutes
[16:03:26 CEST] <Gdocal> Hello guys, I want to record video from images. But images generates in my own program (C#). I Don't want to save files to disk and read them back via ffmpeg. Is there any way to send images over tcp (or windows pipe) to ffmpeg and generate video ?
[16:08:21 CEST] <pink_mist> incorporate the ffmpeg libraries in your program and do it all inside your program
[16:09:38 CEST] <DHE> or output to a more boring format (eg: yuv420p, or RGB24) and pipe it into ffmpeg
[16:09:50 CEST] <DHE> err, I guess drop the p on yuv420p
[16:11:43 CEST] <lain98> unrelated to ffmpeg but i'm trying to find videos to test with my application that uses ffmpeg. can someone point me to public domain videos or videos without licensing issues that i can add to my git repo ?
[16:12:27 CEST] <lain98> like what's lena.jpg for video ?
[16:12:47 CEST] <TheAMM> I use the blender open movies for a few things
[16:13:00 CEST] <TheAMM> but maybe you can grab a few of the samples ffmpeg's tests have
[16:13:18 CEST] <TheAMM> I've never looked at them but they're out there somewhere
[16:15:09 CEST] <another> there's also: https://media.xiph.org/video/derf/
[16:16:38 CEST] <lain98> i avoided the ffmpeg tests because they probably include iffy edge case videos
[16:19:06 CEST] <Gdocal> but maybe there some way I can setup tcp server and force ffmpeg to read my stream of data, but as I said I have only pictures, is it possible to feed pictures via tcp and get video from it?
[16:19:33 CEST] <another> video is basically just pictures
[16:19:58 CEST] <BtbN> tcp would be horribly inefficient. Best to use the libraries directly
[16:31:00 CEST] <lain98> also concerned  that creative commons might not exactly be equivalent to public domain
[16:35:41 CEST] <pink_mist> of course it isn't - public domain isn't a thing in some countries
[16:35:48 CEST] <pink_mist> while creative commons is
[16:36:22 CEST] <pink_mist> and there are different types of creative commons licenses, so it all depends on which one's in use
[16:36:48 CEST] <pink_mist> but ... you should probably speak to a lawyer about this kind of thing.
[16:57:09 CEST] <furq> CC0 is public domain (in jurisdictions where it exists)
[16:57:26 CEST] <furq> it should also theoretically work in jurisdictions where it isn't but i'm not going to find out
[17:03:09 CEST] <lain98> i thought of a better idea. i bet can generate a blank video in ffmpeg with just the frame number with drawtext
[17:04:13 CEST] <durandal_1707> testsrc, testsrc2, smptebars, pal100bars etc
[17:04:35 CEST] <durandal_1707> all creates testing video
[17:04:51 CEST] <durandal_1707> these are video source filters
[17:05:56 CEST] <lain98> thanks durandal_1707
[17:08:37 CEST] <lain98> this is pretty cool https://www.youtube.com/watch?v=tdTTbfUMn7w
[17:49:42 CEST] <HickHoward> uhh
[17:49:58 CEST] <HickHoward> does ffmpeg support e-ac-3 with dolby atmos decoding or something?
[17:50:11 CEST] <durandal_1707> no atmos
[17:50:22 CEST] <HickHoward> yikes
[17:50:37 CEST] <HickHoward> what am i supposed to do?
[17:52:27 CEST] <cehoyos> lain98: lena is copyrighted
[17:53:47 CEST] <cehoyos> HickHoward: E-AC3 does not have an Atmos side-channel, it's just a marketing name for a bit signal in the bitstream
[17:56:42 CEST] <HickHoward> so i got bamboozled into believing that E-AC-3 supports Dolby Atmos...
[17:58:12 CEST] <HickHoward> oshit
[17:58:32 CEST] <cehoyos> TrueHD does
[18:03:22 CEST] <HickHoward> say, does ffmpeg support dolby atmos decoding *as a separate file* or...
[18:03:25 CEST] <HickHoward> something?
[18:05:20 CEST] <cehoyos> no atmos
[18:05:44 CEST] <friki> Hi, I'm looking for articles, benckmarks, documentation, etc about ffmpeg running on SBC (raspberrypi and such). The purpose is encoding HD video (from an USB video source) or from a network fs (samba, nfs).
[00:00:00 CEST] --- Mon Aug 12 2019