[Ffmpeg-devel-irc] ffmpeg-devel.log.20190602

[burek]

[00:34:20 CEST] <keegans> i assume this is the right place to ask ; i am trying to debug what is seemingly heap corruption caused by http.c when reading from HLS streams with large segments
[00:34:54 CEST] <keegans> it receives data w/ length header='Content-Length: 351823', then says it's opening for reading and immediately segfaults directly after
[00:42:59 CEST] <jdarnley> that does sound bad
[00:43:19 CEST] <jdarnley> you need to run it in a debugger
[00:43:55 CEST] <jdarnley> run a debug build of course
[01:13:36 CEST] <keegans> jdarnley: yeah i have a debug build in a debugger
[01:13:48 CEST] <keegans> when i run it in valgrind, it works as intended
[01:14:05 CEST] <keegans> which complicates things slightly
[01:14:29 CEST] <keegans> where does it allocate memory for the body, can i increase the size and see if that changes anything ?
[02:02:08 CEST] <keegans> are  there docs on how i can compile ffmpeg w/ asan
[02:02:22 CEST] <keegans> " --extra-cflags="-fsanitize=address -fno-omit-frame-pointer" --extra-ldflags="-fsanitize=address -fno-omit-frame-pointer"" is not seeming to work
[02:06:55 CEST] <keegans> --toolchain=clang-asan
[02:18:52 CEST] <keegans> ==720==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000029f79 at pc 0x55aa18c5ae80 bp 0x7fff04569cf0 sp 0x7fff04569ce0
[02:18:59 CEST] <keegans> yeah seems i've found a heap buffer overflow
[02:23:14 CEST] <keegans> no something else is wrong
[02:23:24 CEST] <keegans> it crashes at different points in and out of asan
[02:23:30 CEST] <keegans> asan's stacktrace is wrong
[02:39:45 CEST] <iive> keegans,  valgrind has an option to avoid reusing same heap memory, to easy catching use-after-free
[02:41:10 CEST] <iive> aka, allocate buffer, free buffer, then allocate same memory by another routine. if the first one is still using it, it won't register as use-after-free
[02:41:29 CEST] <keegans> do you know what option it is ? 
[02:44:45 CEST] <iive> -memcheck:freelist-vol=512000000
[02:45:43 CEST] <iive> check also malloc-fill and free-fill 
[02:46:18 CEST] <keegans> seems that when i do that it has the same effect as asan, it causes the crash to be at a different place
[02:46:59 CEST] <iive> that's great, it didn't use to crash :)
[02:48:13 CEST] <keegans> well it means that i cannot use it to find the cause of that crash
[02:48:21 CEST] <iive> depends
[02:48:29 CEST] <iive> valgrind tracks memory too.
[02:49:15 CEST] <iive> not only execution. does it try to access previously used memory region?
[02:51:10 CEST] <keegans> it doesn't look like it
[02:52:58 CEST] <keegans> right
[02:53:04 CEST] <keegans> so it doesn't crash at all in valgrind still
[02:53:12 CEST] <keegans> i was wrong, it triggered an abort
[02:53:21 CEST] <iive> hum?
[02:53:32 CEST] <keegans> so it runs as intended inside of valgrind with the option you provided
[02:53:41 CEST] <keegans> and intended means no SIGSEGV
[02:53:49 CEST] <keegans> whereas when i run it outside of valgrind it crashes
[02:54:08 CEST] <keegans> which is the issue i had before iused that option
[02:54:40 CEST] <iive> have you tried the -fill options?
[02:55:32 CEST] <keegans> no , not yet 
[02:55:51 CEST] <keegans> which should i be using 
[02:56:04 CEST] <keegans> or just both malloc and free fill
[02:56:08 CEST] <iive> all of them
[02:56:44 CEST] <iive> one would fill the memory with pattern when you allocate it, so if you use uninitialized pointer, it would be similar to that pattern.
[02:57:23 CEST] <keegans> yeah that crashes it
[02:57:29 CEST] <keegans> but in the wrong place, still
[02:57:43 CEST] <iive> the other would do the same with the memory when you free it, so use-after-free would show that pattern.
[02:57:49 CEST] <iive> what is the wrong place?
[02:58:00 CEST] <keegans> let me try and illustrate with logs, one second
[02:58:05 CEST] <iive> does abourt come from an assert?
[02:58:24 CEST] <keegans> it was my abort
[02:58:41 CEST] <keegans> it was there to show that the program crashed before the abort
[03:03:12 CEST] <iive> aha
[03:05:34 CEST] <keegans> i am just making a writeup
[03:05:35 CEST] <keegans> so you can see
[03:06:26 CEST] <iive> i'll be leaving in a moment.
[03:07:25 CEST] <keegans> iive: https://gist.github.com/meme/30c5cf45dc9b10761ba53fd068c04b30
[03:07:53 CEST] Action: iive looking
[03:11:10 CEST] <iive> 0x602000029f79 is located 0 bytes to the right of 9-byte region [0x602000029f70,0x602000029f79) allocated by thread T0 here: #0 0x7f1f7d23b289 in operator new[](unsigned long)
[03:12:03 CEST] <iive> ok, this is C++ code, so I assume this is your program, and it is reading right after the buffer, so I guess it is overread.
[03:12:30 CEST] <keegans> right but it crashes before it gets to my code
[03:12:32 CEST] <iive> all ffmpeg memory allocations are done with extra number of bytes, there is some ugly define for it.
[03:12:48 CEST] <keegans> ah yeah if i can find that i'll just bump it up and see if that fixes it 
[03:12:51 CEST] <keegans> "fixes"
[03:13:01 CEST] <keegans> do you know what the define is ?
[03:13:15 CEST] <iive>  #1 0x55aef37abbf9 in Host::DecryptForEach()
[03:13:22 CEST] <iive> i may find it...
[03:17:33 CEST] <iive> AV_INPUT_BUFFER_PADDING_SIZE
[03:17:53 CEST] <keegans> thank you
[03:19:30 CEST] <iive> it is defined in libavcodec/avcodec.h 
[03:19:39 CEST] <iive> good night and good luck.
[11:10:35 CEST] <cone-164> ffmpeg 03Paul B Mahol 07master:cbaa60329a73: avfilter/vf_xmedian: remove limitation of only odd number of inputs
[13:03:38 CEST] <cone-164> ffmpeg 03Paul B Mahol 07master:b5355774652c: avfilter/af_anlmdn: add smooth factor option
[13:03:39 CEST] <cone-164> ffmpeg 03Paul B Mahol 07master:1a266a1ef91d: avfilter/af_anlmdn: try to recover when cache becomes negative
[13:03:40 CEST] <cone-164> ffmpeg 03Paul B Mahol 07master:fcfe85220dbd: avfilter/af_anlmdn: avoid creating frames with zero samples
[13:20:26 CEST] <durandal_1707> mplayer is becoming pathetic project
[13:41:43 CEST] <JEEB> the wget gnutls code is a bit more verbose http://git.savannah.gnu.org/cgit/wget.git/tree/src/gnutls.c#n476
[13:41:54 CEST] <JEEB> but still seems to have the same basic idea
[13:54:54 CEST] <JEEB> and curl also does a similar thing in an eternal loop https://github.com/curl/curl/blob/master/lib/vtls/gtls.c#L332
[14:16:05 CEST] <durandal_1707> JEEB: quick! rewrite every loop in ffmpeg so it does not loop forever
[14:16:23 CEST] <JEEB> stop being facetious, thank you
[14:18:52 CEST] <durandal_1707> JEEB: michaelni is silent, expect from him only patch if you apply that patch
[14:19:39 CEST] <JEEB> I'll just double-check how I/O timeouts are handled. otherwise having that eternally looping is OK
[14:19:56 CEST] <JEEB> mostly it could later be improved with a logging patch a la curl
[14:20:00 CEST] <JEEB> but that's separate from the fix
[14:20:03 CEST] <JEEB> :P
[14:27:27 CEST] <durandal_1707> so one can not rescale dvdsub subtitles at all?
[14:28:04 CEST] <JEEB> you decode the AVSubtitles, then if you want to use filtering  you make AVFrames out of them like with sub2video
[14:28:14 CEST] <JEEB> then you see if you can get those into AVSubtitles again
[14:28:15 CEST] <JEEB> :P
[14:43:29 CEST] <durandal_1707> ubitux: please send patches!
[14:44:54 CEST] <JEEB> 32
[14:45:01 CEST] <durandal_1707> 31
[14:53:52 CEST] <ubitux> sorry i'm overhelmed :(
[15:55:22 CEST] <cone-164> ffmpeg 03Steven Liu 07master:4ef0bea292d2: doc/muxers: fix typo of the hls var_stream_map example
[16:31:12 CEST] <kierank> durandal_1707: why
[16:31:12 CEST] <kierank> is mplayer pathetic
[17:26:42 CEST] <durandal_1707> kierank: patch with intrinsic code for old filters nobody use
[18:16:44 CEST] <JEEB> > MP4RA  Scanning specifications for possibly-unregistered 4CCs
[18:44:44 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:f1b359aaf5c3: vf_crop: Add support for cropping hardware frames
[18:44:45 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:963c4f85fe54: doc/indevs: Add example using cropping to capture part of a plane
[18:44:46 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:6ed34a437925: lavfi/vaapi: Factorise out common code for parameter buffer setup
[18:44:47 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:5fb9eb9ed256: vf_misc_vaapi: Add missing return value checks
[18:44:48 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:5051b7f898ae: lavfi/vaapi: Improve support for colour properties
[18:44:49 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:ef2f89bbccc9: vf_scale_vaapi: Add options to configure output colour properties
[18:44:50 CEST] <cone-164> ffmpeg 03Mark Thompson 07master:909bcedc581a: vaapi_encode: Warn if input has cropping information
[19:19:32 CEST] <cehoyos> JEEB: vlc writes its own internal 4CCs into mov files
[20:14:28 CEST] <durandal_1707> cehoyos: why mplayer now have intrinsic code?, that is pathetic!
[20:14:45 CEST] <cehoyos> Please elaborate!
[20:17:44 CEST] <BradleyS> TIL handbrake is pathetic
[20:18:34 CEST] <durandal_1707> cehoyos: http://lists.mplayerhq.hu/pipermail/mplayer-cvslog/2019-May/047359.html
[20:19:13 CEST] <durandal_1707> BradleyS: handbrake have intrinsic code but not abandoned filters
[20:19:26 CEST] <BradleyS> true
[20:19:37 CEST] <BradleyS> what's wrong with vf_pullup? nobody needs ivtc?
[20:20:02 CEST] <cehoyos> I didn't know this, thank you for the link.
[20:20:21 CEST] <cehoyos> I don't think anything is wrong with vf_pullup, is it?
[20:23:02 CEST] <durandal_1707> he wrote intrinsic code to code nobody touched for ages - that is very pathetic, and at same time refuses to update codecs.conf
[20:25:02 CEST] <JEEB> how and why does that offend you as much? why the name-calling? :P
[20:25:34 CEST] <JEEB> or are you just trying to do the usual thing with trying to get attention?
[20:26:18 CEST] <durandal_1707> JEEB: yes, i need your 24h attention to apply my pull request
[20:27:07 CEST] <durandal_1707> it offend me very much, they are ruining project
[20:35:42 CEST] <BradleyS> can't blame others for how you act ;)
[20:43:23 CEST] <durandal_1707> huh, github allows now to report security stuff for each project
[20:43:48 CEST] <durandal_1707> BradleyS: i did nothing
[20:46:34 CEST] <durandal_1707> what should be name of filter?: _f360_ or _panorama_ or _perigon_
[20:59:24 CEST] <durandal_1707> you get 1k$ if you pick!
[21:04:47 CEST] <durandal_1707> nobody wants free money :-(
[21:15:12 CEST] <durandal_1707> good, i'm staying rich
[21:50:22 CEST] <durandal_1707> gmail is down - i can not work this way
[22:08:13 CEST] <BradleyS> _360pano
[22:08:25 CEST] <BradleyS> or _pano360
[22:16:20 CEST] <durandal_1707> not cool enough
[23:12:34 CEST] <mkver> jkqxz: I completed the cbs-mpeg2 patches, but somehow gmail is making problems.
[23:12:48 CEST] <JEEB> yea they're having issues atm
[23:12:54 CEST] <mkver> You can nevertheless already take a look: https://github.com/mkver/FFmpeg/commits/mpeg2-cbs-2
[23:14:11 CEST] <mkver> Your warnong wrt __VA_ARGS__ was unfounded in this case: SUBSCRIPTS already expands to "(0 > 0 ? ((int[0 + 1]){ 0, }) : NULL)" if there are no further arguments; this does not change and it is legal C.
[23:31:52 CEST] <jkqxz> I was actually thinking of the multiple macro expansions.  But yeah, it's fine because foo(a,) and foo(a) end up with the same empty __VA_ARGS__ (unlike if they were functions).
[23:33:51 CEST] <jkqxz> Is github display just terrible or are there weird misalignments in that?
[23:35:36 CEST] <mkver> Where exactly?
[23:36:10 CEST] <_bluez> Hello, I had some questions regarding the implementation of heif support, should the heif (de)muxer be extended as another member in the mov class or should it be a seperate one?
[23:36:48 CEST] <JEEB> it generally sounds like it'd utilize quite a bit of the mov/mp4 (de)muxer
[23:36:56 CEST] <JEEB> so if not in the same file, but at least very close to mov(enc)
[23:37:18 CEST] <jkqxz> mkver:  E.g. on <https://github.com/mkver/FFmpeg/commit/b42203510f03edc38060e49840b37309db0fef2e#diff-a793d83b7d83ba021228d13d0949bd14>.  Looks like it's github inserting tabs for no reason and messing up the spacing, it's fine in the actual patch.
[23:37:22 CEST] <JEEB> I think we had an initial demuxer patch posted by a person a year or two ago
[23:37:27 CEST] <_bluez> yeah i thought so too
[23:37:38 CEST] <JEEB> the problme mostly was by the fact that it was preliminary, and then the whole issue with multi-part images
[23:38:00 CEST] <_bluez> ohh,where can i find this patch?
[23:38:05 CEST] <JEEB> https://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/215003.html
[23:39:12 CEST] <_bluez> Thank You!
[23:39:18 CEST] <JEEB> but since we effectively cannot handle the stuff nicely on our abstraction level I guess the first step would be to get reading of that stuff in with nice enough code
[23:39:36 CEST] <JEEB> I think we already have a flag for image streams
[23:40:17 CEST] <JEEB> so in a way muxer would be simpler because you can close your ears from all the thoughts of how to expose multiple part images in lavf+lavc
[23:40:46 CEST] <JEEB> and in demuxer the actual demuxer part probably wouldn't be the worst part of it, but rather defining how do we tell the API users that "btw, here's some images which are part of a bigger image"
[23:41:08 CEST] <JEEB> stuff like image part ordering etc
[23:41:24 CEST] <JEEB> the libraries with higher level APIs have it easier :P
[23:41:51 CEST] <JEEB> because f.ex. in imagemagick you just ask for the decoded image and don't really care what it internally does (does it have multiple decoders or what)
[23:42:06 CEST] <JEEB> (or how does it stitch the image together etc)
[23:42:29 CEST] <JEEB> in our case it'd be a HEIF demuxer returning multiple images, somehow signaled to be a part of a bigger thing...
[23:42:44 CEST] <JEEB> so that the API user can then know to stitch them into a single image
[23:42:59 CEST] <jamrial> jkqxz: looks fine here. maybe a monitor resolution thing?
[23:45:02 CEST] <jkqxz> I see <http://ixia.jkqxz.net/~mrt/alignment.png>.
[23:45:12 CEST] <_bluez> isn't it the task of the reader (demuxer) to know how to stich the images?
[23:45:12 CEST] <mkver> I don't get tabs and it looks fine in github's preview, but when I copy it, I somehow end up with a newline that is not in actual patch: "         uir(8, transfer_characteris\ntics);"
[23:45:35 CEST] <JEEB> _bluez: yes, it would still have to be signaled somehow so that the actual stitching can be done after decoding
[23:45:48 CEST] <JEEB> _bluez: otherwise you'd just have to guess as you receive the partial images
[23:46:29 CEST] <_bluez> There are indeed many little complicated things like tiling and all, but i guess i shoud first focus on a basic implementation first ':D
[23:46:38 CEST] <JEEB> sure
[23:49:52 CEST] <_bluez> This field is new to me, i only started a month or two before, so i might have some more silly questions i hope you dont mind, for now i'll have a look at that patch..
[23:50:05 CEST] <JEEB> feel free to ask things :)
[23:50:25 CEST] <_bluez> Thanx :D
[23:50:40 CEST] <jamrial> jkqxz: https://0x0.st/zMxc.jpg
[23:50:54 CEST] <jamrial> maybe it's your browser
[23:51:42 CEST] <jkqxz> Weird.  Oh well, the patch itself is right; it's just rather distracting to read it there.
[23:53:41 CEST] <mkver> Why don't you just download them from github and read them with any tool you like?
[23:56:47 CEST] <jkqxz> Because github looks (apparently misleadingly) like you can sanely read patches on the site.
[00:00:00 CEST] --- Mon Jun  3 2019