[Ffmpeg-devel-irc] ffmpeg-devel.log.20190818

[burek]

[00:23:17 CEST] <tjones> A new question - for X11 screen recording, I've been instructed to --enable-x11grab, but when I do that configure errors in 4.2.  Was this removed/changed?
[00:44:22 CEST] <cehoyos> tjones: Please read the channel description
[00:46:26 CEST] <tjones> I'm pretty sure that what I am asking about is related specifically to developing and building ffmpeg.
[00:48:21 CEST] <jamrial> no, questions about how to compile ffmpeg don't belong here
[00:51:02 CEST] <nicolas17> are you developing ffmpeg?
[00:51:15 CEST] <tjones> I'll go there, but you need to state that in the description.  The current description does not say that questions about building ffmpeg do not belong here..
[00:51:31 CEST] <tjones> @nicolas17 - yes,  that's what I'm trying to do here.
[00:52:52 CEST] <tjones> I have found few issues with timing in the X11 screen recording, but I guess I can take them elsewhere.
[01:14:47 CEST] <thardin> michaelni: the instructions for setting up libfuzzer are wrong
[01:14:54 CEST] <thardin> the project has moved elsewhere in llvm
[02:18:40 CEST] <cone-142> ffmpeg 03Limin Wang 07master:df6e341b0418: lavf/dump: use error log level for invalid size
[02:23:44 CEST] <cone-142> ffmpeg 03Steven Liu 07master:cff309097a3a: avformat/hlsenc: merge mpegts and fmp4 workflow to one workflow
[02:36:54 CEST] <thardin> cinepakdec mystery solved. turns out lavc can only allocate huge frames so fast
[02:37:35 CEST] <thardin> I can literally just comment out all the decoding and it still "only" runs at 200 fps
[02:37:53 CEST] <thardin> for a 65535x200 or so file
[02:38:37 CEST] <thardin> is there a "null" or black decoder? how fast can lavc shuffle frames around when you really get down to it?
[02:40:09 CEST] <thardin> ff_reget_buffer() calls frame_copy_video(), go figure
[03:01:07 CEST] <Compn> thardin, that bug sounds like very important one to fix :)
[03:01:13 CEST] <Compn> a hard limit somewhere in lavc ;)
[03:01:56 CEST] <Compn> fixing it may end up making ffmpeg faster all around :)
[03:02:08 CEST] <Compn> thardin, make sample file ?
[03:15:23 CEST] <kierank> thardin: should be able to use memory pools
[03:15:49 CEST] <kierank> frame_copy_video is just copying struct elements
[08:01:45 CEST] <durandal_1707> Compnn: you still awake?
[08:02:16 CEST] <Compnn> yes
[08:02:17 CEST] <Compnn> wat
[08:02:20 CEST] <Compnn> durandal_1707, 
[08:20:48 CEST] <durandal_1707> i will recert scpr patch by michaelni that makes decoder returns no frames
[08:20:59 CEST] <durandal_1707> *revert
[09:52:58 CEST] <thardin> kierank: are you sure you don't mean av_frame_copy_props() ?
[09:53:44 CEST] <thardin> av_frame_copy() documentation: "Copy the frame data from src to dst."
[10:11:09 CEST] <thardin> ah I think I see. this decoder does the opposite of the win3.1 one. instead of copying selectively from the last frame it just copies everything and overwrites wherever there aren't ENC_SKIP MBs
[10:14:59 CEST] <thardin> question then is why can't the decoder just get the last frame to write over?
[10:22:44 CEST] <thardin> mkay, just using ff_get_buffer() once works just as well
[10:47:29 CEST] <nevcairiel> if the frame is being held by the user, then you cant write over it, and you need to make a copy
[10:47:43 CEST] <thardin> how do I know that?
[10:48:56 CEST] <thardin> why don't avcodec_decode_video*() use const AVFrame* ?
[10:49:24 CEST] <thardin> this feels like a big oof moment
[10:49:58 CEST] <nevcairiel> why would it be const, it obviously writes into the frame
[10:50:24 CEST] <thardin> hum. right,  the user supplies the AVFrame
[10:50:50 CEST] <thardin> that would make this a fundamental architecture problem in lavc
[10:51:15 CEST] <nevcairiel> anyway ff_reget_buffer does just that, it'll check if the frame is valid and writable and then just reuse it, and if its not it'll make a copy
[10:51:32 CEST] <thardin> no, it always makes a copy
[10:51:40 CEST] <nevcairiel> no it doesnt
[10:51:48 CEST] <nevcairiel> i have the code in front of me right now
[10:51:54 CEST] <thardin> hmm
[10:52:03 CEST] <thardin> maybe av_frame_is_writable() returns false here?
[10:52:41 CEST] <nevcairiel> quite likely, possibly because the frame is still in filtering or who knows where its being held
[10:53:40 CEST] <nevcairiel> but if you have inter-coding and you can't modify the original frame to apply the differences since its being used by something else, then you have to make a copy, thats the price you pay for inter-coding
[10:55:35 CEST] <thardin> again, architecture problems
[10:55:44 CEST] <nevcairiel> no, fundamental concept of inter-coding
[10:55:53 CEST] <thardin> no
[10:56:07 CEST] <nevcairiel> unless you dont want the user to be able to hold onto the frame, in which case he has to make a copy, but you always end up w ith a copy
[10:56:16 CEST] <thardin> if lavc were to give the user frames instead of the ass-backward way it is now, this problem would not exist
[10:56:26 CEST] <nevcairiel> sure it would
[10:57:07 CEST] <nevcairiel> and we actually have a new decode api that does just that, but it cannot solve this problem, its fundamentally incompatible
[10:58:00 CEST] <durandal_1707> they cant agree with basic things
[10:58:37 CEST] <thardin> hum. but if the user doesn't want modify the frame then the decoders shouldn't have to copy it around if they don't need to
[10:59:18 CEST] <nevcairiel> As soon as you have a frame with a lifetime that exceeds the time between calls to the decode function, then you can't just modify the existing one anymore, since you would modify the previously returned frame
[10:59:33 CEST] <thardin> right
[11:00:16 CEST] <nevcairiel> the old-old decode API basically was like that, and all it did was move the frame copy from lavc to the user, since players or processing tools generally require more frame lifetime
[11:01:00 CEST] <nevcairiel> (and even worse,  it made the copy mandatory for all cases, instead of just cases like this)
[11:01:23 CEST] <thardin> is it possible that it's just the fuzzing tool no relinquishing ownership of frames?
[11:01:43 CEST] <thardin> tools/target_dec_fuzzer.c
[11:01:56 CEST] <nevcairiel> if its just fuzzing the decoder and nothing else, I would expect it should release the frame right after its returned to it
[11:02:38 CEST] <thardin> there's a av_frame_unref() call just before decoding
[11:02:57 CEST] <nevcairiel> yeah thats definitely there
[11:03:20 CEST] <nevcairiel> did you benchmark with the tool?
[11:03:30 CEST] <thardin> target_dec_fuzzer? yes
[11:03:55 CEST] <nevcairiel> i wonder what else would possibly hold a reference to the data then
[11:04:39 CEST] <thardin> maybe it's passing two/three frames around and nothing else?
[11:05:00 CEST] <thardin> like it's got one allocated frame lined up that of course needs a copy of the old frame
[11:05:19 CEST] <thardin> can a decoder hold onto the last frame? that might be one solution
[11:06:29 CEST] <thardin> then I can just make it behave like the win3.1 decoder, making it similarly not-lenient. any area not covered by a strip gets old garbage
[11:07:45 CEST] <nevcairiel> Oh, the fuzzer is not using reference counted frames
[11:07:49 CEST] <nevcairiel> that explains the extra copy
[11:08:29 CEST] <durandal_1707> why is fuzzing that stupid?
[11:09:01 CEST] <nevcairiel> if you add ctx->refcounted_frames = 1 somewhere in the init of that fuzzing tool, the speed penality might go away
[11:09:07 CEST] <thardin> I wonder how many "speedup" patches are due to that
[11:09:22 CEST] <thardin> I'll give it a shot
[11:09:43 CEST] <durandal_1707> all michael patches
[11:10:51 CEST] <thardin> zoooooom
[11:10:53 CEST] <thardin> yeah that fixed it
[11:11:18 CEST] <thardin> so a "crap behind the controls" situation
[11:12:50 CEST] <nevcairiel> we could possibly even fix that in avcodec in the refcounted = 0 mode, but its probably not really tested much by anyone anymore, since refcounting is generally just superior
[11:16:01 CEST] <durandal_1707> michael does not like that flag though iirc
[11:16:25 CEST] <thardin> shouldn't it be default if it's a good thing?
[11:17:33 CEST] <nevcairiel> it should be, but it requires users to properly free frames or it leaks, hence its an incompatible change
[11:19:13 CEST] <nevcairiel> maybe we can do it on next major bump
[11:26:07 CEST] <thardin> makes sense
[11:51:42 CEST] <durandal_1707> so whats next step? continue to apply patches of dubious quality?
[11:53:09 CEST] <thardin> I'd be in favor of the timeout fuzz reports being more available at least
[12:26:44 CEST] <durandal_1707> nothing of that will happen as Lynne correctly explained
[12:29:04 CEST] <durandal_1707> kierank: can you apply 3 v360 patches?
[12:32:17 CEST] <thardin> I'll start my own fuzzing cluster, with blackjack and so on
[12:34:27 CEST] <durandal_1707> kierank: lets fork FFmpeg
[12:37:39 CEST] <thardin> RWmpeg
[12:39:14 CEST] <thardin> how is libav doing these days?
[12:39:52 CEST] <thardin> no releases in 1½ year
[12:40:36 CEST] <durandal_1707> or commits
[15:08:25 CEST] <durandal_1707> i lost all my motivation to continue working on FFmpeg
[16:37:58 CEST] <kierank> durandal_1707: hi
[16:42:22 CEST] <durandal_1707> i wonder if community really need gsoc patches from two my projects
[16:43:52 CEST] <vel0city> people may just be busy with vacation and such, it _is_ August
[16:44:07 CEST] <vel0city> that's the case for other projects at least
[17:40:11 CEST] <durandal_1707> michaelni: you still ignore my requests! and your patches are becoming worse and worse
[17:41:55 CEST] <cehoyos> I have just added a patch to ticket 8081 that changes the parser so that it does not only output frames with size 10*channels but also 11*channels (depending on codec). The decoder now alternately gets frames of size 22 and 2 (for stereo input), the two bytes are repeated (and ignored by the decoder, but an error is printed). Is there a bug in the parser?
[17:45:02 CEST] <durandal_1707> you skip stuff in decoder?
[17:45:20 CEST] <cehoyos> The leading byte with unknown content
[17:46:11 CEST] <cehoyos> Do you know why the parser does not constantly output block_size bytes?
[17:46:12 CEST] <bencoh> I guess that's the kind of things you'd be supposed to do in parser (or maybe a bsf), but not in decoder
[17:46:26 CEST] <cehoyos> The parser is not allowed to drop bytes, so no
[17:47:00 CEST] <cehoyos> (Basically every decoder drops some bytes, even the modern video codecs can have stuffing elements)
[17:48:57 CEST] <durandal_1707> cehoyos: your patch is wrong
[17:49:09 CEST] <cehoyos> Would you like to elaborate?
[17:49:18 CEST] <durandal_1707> pay me first
[17:49:30 CEST] <cehoyos> I don't pay for reviews, sorry
[17:50:19 CEST] <durandal_1707> than no working patch for you
[17:51:29 CEST] <durandal_1707> you even do not know how to check that parser is working correctly
[17:52:50 CEST] <cehoyos> That's why I am asking here
[17:55:41 CEST] <durandal_1707> you even touched that line when adding stereo support
[17:56:36 CEST] <durandal_1707> decoder will need to consume all bytes
[17:56:50 CEST] <durandal_1707> in your cases it does not
[17:57:21 CEST] <durandal_1707> you forgot to update one line
[17:57:33 CEST] <cehoyos> The decoder does consume 11*channels, no?
[17:57:38 CEST] <cehoyos> The decoder does consume 11*channels bytes, no?
[17:57:38 CEST] <durandal_1707> now pay me your day salary
[17:57:48 CEST] <cehoyos> You wouldn't be happy;-=
[17:57:52 CEST] <cehoyos> You wouldn't be happy;-)
[17:58:11 CEST] <cehoyos> So why does the parser output frames of two bytes?
[17:58:34 CEST] <durandal_1707> parser is outputting what you codded up
[17:59:20 CEST] <durandal_1707> decoder is not consuming all bytes causing 2 extra bytes to be left in limbo
[17:59:56 CEST] <cehoyos> Ahh
[18:00:09 CEST] <durandal_1707> the same two bytes you skip in decoder but your decoder still returns old size
[18:00:19 CEST] <thardin> is michael the only one using/caring about discard_damaged_percentage ?
[18:00:49 CEST] <durandal_1707> yes also making cfr content vfr
[18:01:23 CEST] <thardin> yeah that I noticed
[18:01:23 CEST] <michaelni> thardin, discard_damaged_percentage was added after people asked me to add such a field IIRC
[18:02:15 CEST] <michaelni> i think the field makes sense but it wasnt my idea
[18:02:36 CEST] <thardin> hum hum
[18:04:04 CEST] <thardin> is this perhaps intended for satellite streams?
[18:04:43 CEST] <durandal_1707> lol
[18:05:40 CEST] <durandal_1707> its sole purpose is to buffer more such devastating patches to the project
[18:06:14 CEST] <michaelni> thardin, damaged streams or streams  with missing packets occur in more cases than satelite
[18:06:39 CEST] <thardin> that's true
[18:08:18 CEST] <durandal_1707> this is only to keep fuzz timeout patches going on
[18:09:59 CEST] <durandal_1707> new invention is returning invalid return code for overflows in sar calculations
[18:10:07 CEST] <durandal_1707> for vc1
[18:11:16 CEST] <durandal_1707> i imagine ffmpeg will in next few years accept only qcif resolutions as others allow dos attacks and overflows
[18:13:32 CEST] <thardin> well, I've made my feelings on that quite well-known by now: use a proper job queue system if accepting random crap from the internets
[18:14:56 CEST] <durandal_1707> but if you do not fix DOS issues people will use alternative solutions
[18:15:15 CEST] <thardin> the horror
[18:15:47 CEST] <thardin> it's just make-work imo, and makes the decoders needlessly ugly
[18:15:58 CEST] <kierank> Yes it is "busywork"
[18:16:24 CEST] <kierank> I should delete features based on "security"
[18:16:40 CEST] <kierank> All nonfree should be deleted, we don't know what blob is doing
[18:16:50 CEST] <kierank> All hwdec as well
[18:17:03 CEST] <kierank> Should all be deleted, we don't know what hw is doing
[18:17:06 CEST] <thardin> no support for any cpus except risc-v
[18:17:09 CEST] <kierank> Could be integer overflow
[18:17:30 CEST] <thardin> refuse to run on machines with samsung RAM
[18:17:43 CEST] <kierank> michaelni: do you have even any thought that your aggressive behaviour is causing people to leave the project
[18:18:00 CEST] <kierank> And that your hardlineness has been the cause of issues in the past
[18:18:05 CEST] <kierank> Or will you continue like this forever?
[18:18:19 CEST] <durandal_1707> next step is adding ECC ram checks in software
[18:18:37 CEST] <kierank> durandal_1707: should I write patch to delete all nonfree? And all hwdec?
[18:18:42 CEST] <kierank> On security groundd
[18:19:19 CEST] <durandal_1707> dunno, that stuff is not enabled by default
[18:19:56 CEST] <durandal_1707> write patch to kill useless sonic codec
[18:36:59 CEST] <thardin> one month until IBC
[18:39:01 CEST] <kierank> Yes
[18:39:06 CEST] <kierank> Need to book flight and hotel
[18:41:03 CEST] <kierank> Ibc is not a good place to have a meeting
[18:41:46 CEST] <thardin> there's always after
[18:42:42 CEST] <durandal_1707> will there be vdd and when and where?
[19:00:28 CEST] <j-b> yes
[19:00:33 CEST] <j-b> tokyo
[19:01:50 CEST] <durandal_1707> very expensive
[19:08:40 CEST] <thardin> yes
[19:10:02 CEST] <j-b> durandal_1707: so? why do you care?
[19:13:30 CEST] <durandal_1707> i care for developers
[19:15:00 CEST] <j-b> we pay for developers travels.
[19:17:23 CEST] <thardin> o rly?
[19:19:54 CEST] <j-b> as every year.
[19:39:47 CEST] <durandal_1707> which zork pcm decoder we have, from which game?
[19:42:51 CEST] <kurosu> Will JAL and/or Air France sponsor the flights because of their infotainment ? :o
[20:21:58 CEST] <kierank> durandal_1707: Michaelni won't come
[20:22:03 CEST] <kierank> So it's pointless
[20:23:16 CEST] <durandal_1707> kierank: could you apply 3 v360 patches?
[20:33:23 CEST] <kierank> Yes but building Lego Saturn v now
[20:58:44 CEST] <durandal_1707> so whats reasons that fuzzer does not use refcounted frames?
[21:05:16 CEST] <durandal_1707> im wasting my time with FFmpeg, i hate every moment i spent with it and expecially with fellow leader you know who you are.
[21:38:15 CEST] <durandal_1707> cehoyos: will you be next leader? everything is better than current one.
[21:41:38 CEST] <cehoyos> As said before: I believe the project leader needs a better technical understanding of FFmpeg than I have
[22:08:34 CEST] <durandal_1707> cehoyos: what unused byte looks like with values?
[22:09:15 CEST] <cehoyos> One bit shows "left"/"right", another one is always set: alternating 0x82 and 0x02
[22:09:36 CEST] <cehoyos> It is possible that the byte is used to signal silence but not in the only sample we currently have
[22:12:05 CEST] <durandal_1707> cehoyos: look at #4799 bug
[22:12:42 CEST] <cehoyos> ;-)
[22:18:02 CEST] <durandal_1707> does that files work too?
[22:22:46 CEST] <cehoyos> They work (the old one sounds funny), they do not contain other leading bytes (for mono, 0x80 is never set)
[22:23:15 CEST] <durandal_1707> funny in what sense?
[22:24:01 CEST] <cehoyos> A foreign (not English, French, German, Russian, likely Northern not Eastern European) language call center call with a calm man and a shouting woman
[22:24:47 CEST] <durandal_1707> lol
[22:42:33 CEST] <Compnn> cehoyos, ddi you double check lynne copyrights ?
[22:42:55 CEST] <cehoyos> Sorry, I don't understand the question
[22:42:57 CEST] <Compnn> btw lynne seems anonymous . anon email, anon domain. wont be able to contact him in future if we want to relicense
[22:43:09 CEST] <Compnn> cehoyos, copyright on new files that lynne sent patch for
[22:43:16 CEST] <cehoyos> That is true for a large number of contributors
[22:43:30 CEST] <Compnn> one file was missing anyone copyright. not sure if the file was just a header/  not copyrightabkle
[22:43:44 CEST] <cehoyos> Which mail are you talking about
[22:44:11 CEST] <nevcairiel> the copyright line in the header is pretty meaningless,  many people send patches for new files without them
[22:44:58 CEST] <nevcairiel> and people that change a file dont usually add themselves to it either
[22:45:04 CEST] <cone-966> ffmpeg 03Michael Niedermayer 07master:1965161ef6d2: avcodec/anm: Check input size for a frame with just a stop code
[22:47:40 CEST] <Compnn> cehoyos,  http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavcodec/aarch64/opusdsp_neon.S
[22:48:11 CEST] <cehoyos> What is the issue? Do you suspect a copyright infringement?
[22:48:29 CEST] <Compnn> no 
[22:48:36 CEST] <durandal_1707> Compnn:  why you attack lynne?
[22:48:48 CEST] <Compnn> but i remember people complaining in past about files without copyright header
[22:49:03 CEST] <cehoyos> It's not the only such file
[22:50:07 CEST] <durandal_1707> Compnn: do you want to relicense to AGPL? I do.
[22:52:20 CEST] <thardin> could always go the x264 route, except agpl taking the place of gpl for license trolling purposes
[22:53:11 CEST] <thardin> since a lot of ffmpeg forks end up hidden behind clouds
[23:00:12 CEST] <kierank> So what if Lynne is anonymous
[23:00:18 CEST] <kierank> Nothing wrong with anonymous contributors
[23:00:52 CEST] <thardin> or contributors who have been dead for 40 years
[23:01:08 CEST] <durandal_1707> Who is Compnn? Also anonymous.
[23:01:24 CEST] <Compnn> im in maintainers file same as you durandal_1707 
[23:02:05 CEST] <durandal_1707> thardin: why 40 years?
[23:02:15 CEST] <thardin> the famed elvis presley
[23:03:04 CEST] <durandal_1707> that contibution is invalid anf should be removed from history
[23:04:36 CEST] <durandal_1707> Compnn: is your real name listed in that file?
[23:17:38 CEST] <thardin> hah, my strip height optimizer in cinepakenc works. 10% smaller output for a given lambda
[23:22:27 CEST] <durandal_1707> what i missed?
[23:35:32 CEST] <Compnn> thardin, did you find the bottleneck ?
[23:38:05 CEST] <durandal_1707> yes, but michaelni does not approve it
[23:38:56 CEST] <michaelni> durandal_1707, please stop
[23:39:56 CEST] <thardin> Compnn: it just has to do with how things work when refcounting is off vs on
[23:40:44 CEST] <thardin> it's possible to optimize the case where a frame has no changes at all, but if a frame has only a few changes then a full copy is performed (if refcounting is disabled)
[23:41:26 CEST] <thardin> this applies to whole heap of decoders. more relevant for our current times would be gifdec
[23:41:38 CEST] <durandal_1707> what? leader is more preffered? ok. noted.
[23:42:12 CEST] <thardin> and the (a)png decoder
[23:42:32 CEST] <durandal_1707> and qtrle
[23:42:41 CEST] <durandal_1707> and msrle
[23:42:47 CEST] <durandal_1707> and scpr
[23:42:52 CEST] <thardin> those are hardly used tho, just like cinepak
[23:43:06 CEST] <thardin> whereas gif and png are very much in daily use
[23:43:53 CEST] <durandal_1707> how do you know? maybe its other way around
[23:44:08 CEST] <thardin> I am omniscient
[23:44:30 CEST] <michaelni> durandal_1707, you know what i meant, please stop, randomly pointing at me
[23:44:43 CEST] <michaelni> especially if its untrue
[23:45:49 CEST] <durandal_1707> aha, now i need to ask someone first whats true and false
[23:46:08 CEST] <thardin> I'm a bit rusty on LZW, but it compress large runs to O(sqrt(N)), right?
[23:46:49 CEST] <michaelni> durandal_1707, pointing at me like this is already quite rude even if its true
[23:48:10 CEST] <michaelni> but its not even true, iam not against changing the ref counting rather i think letting the fuzzer choose the ref counting would make sense or more completely simulating a decoder
[23:48:54 CEST] <michaelni> but only changing the ref counting can be done too if people prefer that
[23:52:07 CEST] <kierank> michaelni: do you not think it is rude the way you acted towards durandal_1707 
[23:52:15 CEST] <kierank> When he asked for security access
[23:52:44 CEST] <kierank> Or the way you shout "DOS IS IMPORTANT" when he reports a problem with a patch
[23:53:15 CEST] <kierank> And you can't just act like these timeouts are the be all and end all
[23:53:32 CEST] <durandal_1707> i do not care anymore, let him do whatever he please
[23:59:13 CEST] <durandal_1707> just apply v360 patches and move on with other things
[00:00:00 CEST] --- Mon Aug 19 2019