[Ffmpeg-devel-irc] ffmpeg-devel.log.20151019

[burek]

[00:20:46 CEST] <JEEB> durandal_170: < anon32> http://pastebin.com/ga4dak90 <-- can somebody forward this to the durandal dude?
[00:26:13 CEST] <ubitux> durandal_170: so you are relying on the fact that every input frame is cloned (and thus has data aligned) in w3dif?
[00:26:21 CEST] <ubitux> w3fdif*
[00:26:55 CEST] <ubitux> (i was curious how sth like crop=x=1,w3fdif seemed to work even with mova 
[00:26:57 CEST] <ubitux> )
[00:28:04 CEST] <durandal_170> nonsense, I will not change -1 to int_min, whatever if value is reserved or not
[00:29:26 CEST] <durandal_170> ubitux: workline is always aligned
[00:29:59 CEST] <ubitux> but because of the av_frame_clone()
[00:31:02 CEST] <cone-384> ffmpeg 03Clément BSsch 07master:ce0a117ed4f9: avutil/opt: display a better default value for int/int64 options
[00:32:17 CEST] <ubitux> heh i should have use ffmpeg -h filter=showfreqs as an example, it's more obvious
[00:41:44 CEST] <durandal_170> ubitux: no, movh is used for input and output
[00:43:35 CEST] <kierank> do I need to align variables on the stack
[00:44:12 CEST] <Gramner> stack is aligned to mmsize (assuming you allocate it through cglobal)
[00:44:58 CEST] <kierank> I mean if I have dwords on the stack, is there a benefit to aligning them to mmsize for example?
[00:45:02 CEST] <kierank> not mm
[00:45:06 CEST] <kierank> the gpr size i mean
[00:45:39 CEST] <Gramner> you only need to align elements to their native sizes
[00:45:45 CEST] <Gramner> so 4 bytes for dwords
[00:57:54 CEST] <rcombs> https://gist.github.com/16e91a2c04c541b9014f here have a(n incomplete) matroska thing
[01:02:51 CEST] Action: rcombs looks at ubitux and wm4
[01:21:29 CEST] <wm4> yeah, that's an approach which a much older patch also tried
[01:28:24 CEST] <atomnuker> kierank: would you mind installing autotools (autoreconf & the rest) on avdev?
[01:28:45 CEST] <Timothy_Gu> what is avdev?
[01:31:31 CEST] <kierank> it is a development machine I setup
[01:32:02 CEST] <kierank> atomnuker: done
[01:35:33 CEST] <Timothy_Gu> kierank: ok
[01:35:44 CEST] <kierank> if you want access let me know
[01:37:01 CEST] <atomnuker> kierank: thanks
[01:43:14 CEST] <wm4> rcombs: if it can be made work in all cases, I'm looking forward to dropping mpv's mkv demuxer
[01:43:22 CEST] <rcombs> wm4: I'm thinking of sending AV_PKT_DATA_NEW_EXTRADATA when making a segment switch
[01:43:42 CEST] <wm4> rcombs: it should be possible, because lavfilter uses the same approach by reusing haali's demuxer
[01:43:59 CEST] <wm4> yeah, something like this
[01:44:25 CEST] <rcombs> I should look further into what mpv's does for that sort of case
[01:44:37 CEST] <rcombs> right now I'm not stream matching at all
[01:44:47 CEST] <rcombs> which means if the streams don't match all sorts of bad things can happen
[01:46:00 CEST] <rcombs> I'd expect I want to run find_stream_info when a file's UID matches, then check stream codec count and IDs
[01:46:17 CEST] <rcombs> export any non matching attachments
[01:46:57 CEST] <rcombs> and if the external segment has extra non-attachment streams, drop packets from them
[01:47:13 CEST] <wm4> you should do what haali does
[01:47:28 CEST] <rcombs> also, scanning for potential segments takes longer than in mpv and I'm not sure why, so I'll need to look into that
[01:47:41 CEST] <rcombs> probably extra demuxer overhead I can disable
[01:47:50 CEST] <wm4> well, mpv just scans the ebml elements at th start, and gives up as soon as possible
[01:48:09 CEST] <wm4> maybe you'll have to open and parse the file separately for scanning
[01:48:16 CEST] <rcombs> could be
[02:06:27 CEST] <rcombs> wm4: where's the relevant haali code anyway
[02:07:31 CEST] <JEEB> I'd rather look at nev's code than Haali's at this point
[02:08:21 CEST] <JEEB> http://git.1f0.de/gitweb?p=ffmpeg.git;a=blob;f=libavformat/matroskadec_haali.c;h=ba6b07a4ba51e90db9a5141c076a00c6ad01dde4;hb=HEAD;js=1
[02:08:47 CEST] <rcombs> also I don't allow recursion at all
[02:08:50 CEST] <JEEB> and then that is abused by the LAV Splitter to then use all of the stuff
[02:09:00 CEST] <rcombs> which is probably fine since it's compatible with haali
[02:09:53 CEST] <wm4> yeah, that code
[02:11:12 CEST] <rcombs> also, is there any way to name an edition?
[02:12:19 CEST] <JEEB> as far as I can tell, yes
[02:12:28 CEST] <JEEB> http://cccp-project.net/beta/test_files/%5bMKV%5d_Editions,_Linked_Segments,_&_Tracksets.mkv
[02:12:37 CEST] <JEEB> http://cccp-project.net/beta/test_files/%5bMKV%5d_Editions_Sample.mkv
[02:12:40 CEST] <JEEB> for two samples
[02:12:41 CEST] <wm4> via tags afaik
[02:12:52 CEST] <rcombs> of course it's tags
[02:58:13 CEST] <wm4> ubitux: remember the mp4 EDL stuff, where we were thinking about implementing it on the demuxer level? one problem was that we'd have overlapping packet DTS... but maybe this is quite simple to solve with some sort of discontinuity flag
[02:59:03 CEST] <wm4> everything would have to specifically support such a flag to prevent itself from getting confused, but after all it seems like a solution
[03:00:22 CEST] <wm4> maybe I better go to bed
[03:17:57 CEST] <atomnuker> is it too late to bump lavc minor version?
[03:18:20 CEST] <atomnuker> just curious
[03:19:34 CEST] <wm4> for what?
[03:20:15 CEST] <atomnuker> a new codec id
[03:20:53 CEST] <wm4> this can be done any time
[04:03:33 CEST] <rcombs> JEEB: where's that code handle stream matching?
[04:09:26 CEST] <rcombs> wow, those samples are lovely
[04:09:31 CEST] <rcombs> multiple segments in the same file
[04:09:33 CEST] <rcombs> why even
[04:43:10 CEST] <cone-685> ffmpeg 03Matt Oliver 07master:216cc1f6fe33: lavu/intmath.h: Add msvc/icl ctzll optimisations.
[04:43:10 CEST] <cone-685> ffmpeg 03Matt Oliver 07master:b0bb1dc62d40: lavu/intmath.h: Move x86 only msvc/icl functions to x86 specific header.
[04:59:51 CEST] <wm4> rcombs: fortunately nobody uses this
[04:59:58 CEST] <rcombs> good
[05:00:02 CEST] Action: rcombs ignores that case for now
[05:05:52 CEST] <wm4> the only reason mpv supports this case is because I found a relatively simple and unintrusive way to do so
[05:06:14 CEST] <wm4> but I've found literally only 1 sample which uses it (the cccp test file)
[05:06:32 CEST] <wm4> writing lots of code to make 1 sample work is fun
[05:08:44 CEST] <rcombs> yeah, I won't work on it unless I actually get requests from my users
[05:08:57 CEST] <rcombs> but what about stream matching?
[05:09:28 CEST] <rcombs> I see where new_extradata side-data gets sent after switches
[05:10:01 CEST] <wm4> in theory, all mkv segments are supposed to have exactly the same configuration
[05:10:14 CEST] <wm4> same track order, IDs, codecs, extradata, etc.
[05:10:20 CEST] <rcombs> but I'm missing the checks to make sure the stream counts and codecs are the same
[05:10:22 CEST] <wm4> the extradata bit didn't quite work out
[05:10:24 CEST] <rcombs> yeah, in theory :P
[05:10:44 CEST] <wm4> and the extradata bit is also why the original patch (years ago) to add OC to lavf failed
[05:10:50 CEST] <wm4> I think
[05:11:10 CEST] <wm4> and also the design of this approach was controverse
[05:11:21 CEST] <wm4> but by now I don't think I give a shit
[05:11:56 CEST] <wm4> I'd rather be freed of the evil to handle segment switching on playback
[05:12:37 CEST] <rcombs> seems to work well enough for LAV at least
[05:13:10 CEST] <wm4> the lav thing was always a giant hack (which is also why nobody ever attempted to upstream it), but yeah
[05:14:19 CEST] <rcombs> I'm pleased that I got basic support working in <400 lines
[05:15:03 CEST] <Timothy_Gu> would it be acceptable if I submit a patch that chooses between sse2 and avx2 depending on the width of the picture?
[05:15:22 CEST] <Timothy_Gu> and a threshold
[05:15:30 CEST] <rcombs> does it help perf?
[05:15:45 CEST] <Timothy_Gu> cuz avx2 is ~25% faster above that threshold and 2x slower below it
[05:15:54 CEST] <rcombs> then sure, why not?
[05:16:11 CEST] <rcombs> I did that for something in libass
[05:16:23 CEST] <Timothy_Gu> ok cuz i'm not sure if the threshold is the same across all cpus
[05:16:31 CEST] <Timothy_Gu> all models of cpus rather
[05:16:34 CEST] <rcombs> hopefully people will bench it
[05:40:06 CEST] <wm4> rcombs: I might be confused, but does the chapter_finished pts check really work?
[05:41:26 CEST] <wm4> because pts is out of order
[05:43:31 CEST] <rcombs> works on the samples I'm testing, at least
[05:43:53 CEST] <rcombs> what's wrong?
[05:47:51 CEST] <rcombs> side by side with mpv's demuxer, mine's a little bit smoother on the transition here
[05:47:56 CEST] <rcombs> mpv's makes a popping sound
[08:22:46 CEST] <ubitux> rcombs: omg
[08:22:48 CEST] <ubitux> e
[08:43:47 CEST] <Timothy_Gu> Is there a way to make sure AVPicture.data pointers are aligned to 32?
[08:47:35 CEST] <rcombs> shouldn't they always be when AVX is available, since av_malloc does 32-byte alignment then?
[08:48:22 CEST] <Timothy_Gu> they should be but I'm getting { 0x7ffff7fa3010, 0x7ffff7fd4810, 0x7ffff7fe0e10 } as frame->data
[08:48:48 CEST] <Timothy_Gu> and since I'm writing AVX2 asm that means I can't use mova :(
[08:55:32 CEST] <rcombs> missing memalign? (if not, guess you'll have to track down whatever's allocating or offsetting buffers by a non-32-byte-aligned amount)
[08:59:28 CEST] <Timothy_Gu> `b malloc` :(
[09:10:40 CEST] <wm4> rcombs: mpv has an additional wait for the video frame, which may underrun the audio buffer, or maybe the audio segment switching code is just broken (both problems with how segment switching is implemented on the higher levels, and goes away completely if implemented on the demuxer level)
[09:11:06 CEST] <rcombs>  yeah, figured it was something like that
[09:15:08 CEST] <wm4> and the pts issue is that the first packet's (IDR) pts doesn't have to be the lowest, if I got this right
[09:23:13 CEST] <rcombs> hmm?
[10:28:32 CEST] <ubitux> wm4: the problem with pts/dts was something along the line "how do i adjust dts when cutting the video randomly"
[10:28:46 CEST] <ubitux> but also something like "how will this work when it's going to support loop stuff"
[10:29:19 CEST] <ubitux> and "how is this going to achieve a sample and frame accurate cutting" (maybe skip for audio?)
[10:29:54 CEST] <ubitux> (and a skip decode flag for video?) 
[10:30:58 CEST] <nevcairiel> Timothy_Gu: if avx is enabled you should get 32-byte aligned allocations from av_malloc
[10:31:57 CEST] <ubitux> rcombs: so when are you submitting aesni, crypto_bench new ssl code, and mkv ordered chapters? @_@
[10:58:23 CEST] <rcombs> ubitux: I'll resend the aesni series in a minute with Gramner's tweaks (which improve perf a bit further)
[10:58:25 CEST] <rcombs> https://gist.github.com/a02c280533ad11e72876
[10:58:51 CEST] <ubitux> lavu_c is funny
[10:58:53 CEST] <ubitux> :))
[10:58:58 CEST] <ubitux> s/funny/dramatic/
[10:59:10 CEST] <ubitux> nice numbers btw
[10:59:24 CEST] <nevcairiel> crypto seems to scale much better with increasing key size
[10:59:27 CEST] <nevcairiel> you should get on that
[11:00:11 CEST] <rcombs> they parallelize modes where you can do that
[11:00:28 CEST] <rcombs> ubitux: the lavu_c numbers are exaggerated
[11:00:42 CEST] <rcombs> because that's without -O3
[11:00:49 CEST] <rcombs> https://gist.github.com/c7c4c693769abec50ccb with
[11:01:02 CEST] <wm4> ubitux: yes, would need flags for skipping video frames (decoded ones)
[11:01:33 CEST] <ubitux> rcombs: mmh, no -O* at all?
[11:01:38 CEST] <rcombs> yeah
[11:01:43 CEST] <ubitux> why? oO
[11:01:54 CEST] <rcombs> because that's how I normally build when working on non-perf stuff
[11:02:01 CEST] <nevcairiel> do people consider O3 safe for ffmpeg anyway? I've always stuck to O2
[11:02:25 CEST] <Timothy_Gu> nevcairiel: I do get 32-byte alignment from av_malloc but for some reason not as AVFrame.data pointers
[11:02:29 CEST] <ubitux> nevcairiel: i'd remove -fno-tree-vectorize first
[11:02:34 CEST] <Timothy_Gu> anyway, too late, need to sleep now
[11:02:42 CEST] <rcombs> because --disable-optimizations results in easier debugging by a factor of $a lot
[11:02:49 CEST] <ubitux> rcombs: ok
[11:03:08 CEST] <rcombs> Timothy_Gu: break on alloc_buffers or whatever that function's called; set watchpoints on ->data; see who's fucking with them
[11:03:27 CEST] <rcombs> nevcairiel: the funny part is that libcrypto was the slowest of the bunch when I started testing
[11:03:39 CEST] <rcombs> well it was faster than lavu_c without optimizations but iirc slower than with
[11:03:59 CEST] <rcombs> because crypto_bench was using the old API for doing AES
[11:04:08 CEST] <rcombs> and apparently that uses a completely different and much worse implementation
[11:05:57 CEST] <rcombs> I still don't actually understand why CBC is so much slower then ECB
[11:06:18 CEST] <rcombs> there's a comment in the libcrypto code about it
[11:06:25 CEST] <rcombs> apparently they don't know either
[11:07:45 CEST] <ubitux> haha
[11:08:54 CEST] <rcombs> it's literally a 1-instruction difference
[11:10:02 CEST] <rcombs> apparently feeding the output back in incurs a penalty of some sort
[11:11:08 CEST] <rcombs> crypto_bench openssl and other AES key sizes are ready to go
[11:11:49 CEST] <rcombs> it'd probably be good to de-duplicate it a bit with macros though
[11:12:15 CEST] <rcombs> but no reason that can't be done after applying
[11:13:08 CEST] <rcombs> I should probably add decrypting as well, since it performs differently
[11:13:55 CEST] <rcombs> but that can also be later
[11:31:56 CEST] <rcombs> ubitux: well sent all the crypto stuff
[11:32:13 CEST] <rcombs> as for mkv ordered chapters, it's not done yet :P
[11:32:21 CEST] <ubitux> ok :)
[11:32:28 CEST] <rcombs> needs handling for when extradata changes
[11:32:36 CEST] <rcombs> and stream match checking
[11:32:42 CEST] <rcombs> and some more error conditions
[11:32:44 CEST] <rcombs> and testing
[11:32:58 CEST] <rcombs> and checking file UIDs needs to be faster
[11:33:17 CEST] <ubitux> maybe set the option for ffplay btw, so we can try it out
[11:33:30 CEST] <ubitux> i mean, by default
[11:34:12 CEST] <ubitux> "EBC" in patch 6
[11:34:37 CEST] <rcombs> and I intend to implement a simple text file format with lines like `[uuid]:<uri>` so you can provide a list of segments (and optionally UUIDs) instead of scanning the dir
[11:35:36 CEST] <nevcairiel> dont bother, noone is going to use it
[11:35:56 CEST] Action: rcombs pins a badge labeled "no-one" on himself
[11:36:17 CEST] <rcombs> (I have a very specific use-case in mind)
[11:36:17 CEST] <nevcairiel> why do you even use this feature at all is the real question
[11:36:32 CEST] <rcombs> ordered chapters in general?
[11:36:39 CEST] <nevcairiel> yes
[11:36:57 CEST] <nevcairiel> or w ell external segments specifically, i guess
[11:37:08 CEST] <nevcairiel> i used ordered chapters for a few seamless branching blurays
[11:37:13 CEST] <nevcairiel> but always limited to one mkv file
[11:37:22 CEST] <rcombs> I've got a few users bugging me about coalgirls anime encodes and a few users bugging me about 2-in-1 movie encodes
[11:37:34 CEST] <ubitux> i have many samples with external segments
[11:37:43 CEST] <ubitux> "samples" as in "shit i watch"
[11:37:59 CEST] <nevcairiel> stop watching all that illegal anime!
[11:38:00 CEST] <rcombs> til ubitux uses coalgirls
[11:38:04 CEST] <ubitux> :D
[11:39:31 CEST] <ubitux> looking at a random directory, i see at least 10 different teams
[11:39:39 CEST] <ubitux> using this for OP/ED
[11:39:52 CEST] <rcombs> they mostly use it to only encode the OP and ED once
[11:40:00 CEST] <ubitux> yes
[11:40:05 CEST] <rcombs> but there's also the whole concept of "playall" files
[11:40:21 CEST] <rcombs> which makes me think nobody's ever heard of& playlists
[11:40:24 CEST] <rcombs> or `mpv *`
[11:40:49 CEST] <ubitux> > cli
[11:40:52 CEST] <ubitux> > too complicated
[11:41:00 CEST] <rcombs> fine then
[11:41:07 CEST] <rcombs> selecting a bunch of files and double-clicking
[11:41:09 CEST] <nevcairiel> use a m3u playlist then, most players can read those
[11:41:17 CEST] <ubitux> * is not very good btw
[11:41:25 CEST] <ubitux> if you have thousands of episodes, it won't work
[11:41:28 CEST] <rcombs> or would windows open them all separately at the same time
[11:41:30 CEST] <ubitux> but you can mpv a directory so it's fine.
[11:41:37 CEST] <rcombs> ubitux: who puts thousands of episodes in the same dir
[11:41:45 CEST] <ubitux> :-°
[11:41:48 CEST] <nevcairiel> clearly the answer is anime people
[11:41:53 CEST] <ubitux> :D
[11:41:58 CEST] <rcombs> apart from being dumb from an organization perspective that sounds hellish when searching for external segments
[11:42:18 CEST] <nevcairiel> in fact i've had people complain that scanning external segments is too slow
[11:42:24 CEST] <nevcairiel> so i optimized the hell out of it
[11:42:39 CEST] <nevcairiel> down to avoiding one extra system call per file
[11:42:58 CEST] <rcombs> it's really slow in my demuxer right now
[11:43:08 CEST] <rcombs> partly because I'm testing over SMB
[11:43:25 CEST] <rcombs> and partly because I'm calling read_header for each potential file and that's not fast
[11:43:30 CEST] <nevcairiel> i even wrote some simple code to just open the file and read its UID instead of doing a full header parse
[11:43:31 CEST] <nevcairiel> just because
[11:44:28 CEST] <nevcairiel> it probably took me months to get all the kinks worked out, but so far people seem happy
[11:44:36 CEST] <rcombs> I might end up doing the same
[11:44:40 CEST] <nevcairiel> except those idiots that mix different codecs, but i just tell them to go away
[11:45:09 CEST] <rcombs> the hardest thing here is going to be exposing editions
[11:45:19 CEST] <rcombs> because it'll probably need new public APIs
[11:45:22 CEST] <nevcairiel> since its my own local fork, i just added new api functions =p
[11:45:27 CEST] <rcombs> yeah I noticed
[11:45:34 CEST] <mateo`> mov and mxf already handle external segments, why would mkv not looking at the future !
[11:46:52 CEST] <ubitux> rcombs: EVP is not a "new" api
[11:47:06 CEST] <ubitux> you might want to say why you use it
[11:47:12 CEST] <rcombs> ubitux: I'd imagine it's newer than whatever we were using
[11:47:27 CEST] <rcombs> but yeah that's sensible
[11:51:36 CEST] <wm4> rcombs: regarding loading external segments, there should indeed be a way to let the API user do it
[11:51:59 CEST] <wm4> like letting the API user pass a list of URLs to try
[11:52:03 CEST] <nevcairiel> way to bloat up complexity to 9000 wm4 :d
[11:52:21 CEST] <rcombs> nevcairiel: if it's just passing a list it's not that complex
[11:52:26 CEST] <rcombs> and I was planning on doing that anyway
[11:52:49 CEST] <wm4> nevcairiel: well, I like losing control about the way external files are scanned the least
[11:53:00 CEST] <rcombs> wm4: you OK with a file arg instead of taking the list directly as an AVOption with escaping?
[11:53:48 CEST] <wm4> a file with an ad-hoc text format?
[11:54:18 CEST] <rcombs> I was planning on having each line be `[uuid]:<uri>`
[11:54:37 CEST] <nevcairiel> maybe he doesnt know the uuid and just wants to suggest a few files to try
[11:54:38 CEST] <wm4> can't we have an AVDictionary somewhere
[11:55:05 CEST] <rcombs> nevcairiel: that's why it's [uuid] instead of <uuid> :P
[11:55:57 CEST] <rcombs> wm4: can you even pass a dict through AVOptions?
[11:56:07 CEST] <rcombs> or are you saying this should be its own lavf public API
[11:56:13 CEST] <wm4> dunno
[11:56:25 CEST] <wm4> well I think I'm happy enough if you can pass this "fvile" as a string
[11:56:29 CEST] <wm4> *file
[11:56:47 CEST] <wm4> so poass the contents, rather than lavf opening a file
[11:57:07 CEST] <rcombs> OK
[11:57:13 CEST] <rcombs> oh and another thing I need to do before this is done
[11:57:18 CEST] <rcombs> actually cleanup :P
[11:57:21 CEST] <wm4> also, lavf has an open callback somewhere
[11:57:28 CEST] <rcombs> ah yeah
[11:57:30 CEST] <rcombs> I should use that
[11:57:50 CEST] <rcombs> right now I don't free the allocations I make on exit yet
[11:58:07 CEST] <rcombs> so, 'know, another thing for the checklist
[11:58:46 CEST] <rcombs> uh
[11:59:00 CEST] <rcombs> I should probably run the matroska probe on these files before actually opening them
[11:59:37 CEST] <nevcairiel> personally I do that in the same step, open file, check for doctype, if match look for UID, done
[11:59:50 CEST] <rcombs> and also fix the style extension check
[12:02:29 CEST] <rcombs> well 'night
[13:21:20 CEST] <wm4> michaelni_: not calling avformat_find_stream_info() breaks opening some .ts files, but only on ffmpeg, not Libav
[13:23:02 CEST] <michaelni_> skiping avformat_find_stream_info() is not a good idea but how can this be reproduced ?
[13:24:20 CEST] <wm4> why is it not a good idea? I want to open files immediately, not waste time on guessing useless, inaccurate info like bitrate etc.
[13:24:42 CEST] <wm4> yes, for .ts files, reading ahead some makes sense, but still
[13:26:02 CEST] <michaelni_> well, if your application can handle new streams, programs and codec_ids changing midstream, not sure these are all things .. then skiping  avformat_find_stream_info() should be fine i think
[13:26:44 CEST] <wm4> why would codec_ids change midstream
[13:26:46 CEST] <wm4> don'T say flv
[13:26:59 CEST] <michaelni_> codec probing
[13:27:02 CEST] <wm4> flv should be fixed and add the streams only when codec id is known
[13:27:50 CEST] <michaelni_> more "somewhere at the start of a stream but note exactly at the start" than "midstream"
[13:30:17 CEST] <wm4> if codec ids can change midstream, there must be some sort of codec id change side data
[13:30:44 CEST] <nevcairiel> i dont think it necessarily "changes" as such, just goes from none -> proper id
[13:34:28 CEST] <michaelni_> it can change between other codec ids during probing, this should not happen once decoders are opened or probing is done
[13:35:12 CEST] <wm4> pure insanity
[13:54:03 CEST] <michaelni_> IIRC libavformat will not return a packet for a stream before probing for it is done and thus codec_id is fixed
[13:55:02 CEST] <michaelni_> but what do you suggest ?
[13:57:00 CEST] <superware> how can I promote trac.ffmpeg.org/ticket/4893 ? can someone help?
[13:59:08 CEST] <superware> www.ff
[13:59:16 CEST] <superware> sorry
[14:00:15 CEST] <J_Darnley> Sure, here's the solution: --disable-network
[14:02:04 CEST] <superware> great
[14:02:42 CEST] Action: J_Darnley snickers
[14:02:44 CEST] <J_Darnley> sorry
[14:03:35 CEST] <wm4> michaelni_: can the api user know when the codec id is being probed, as opposed to unknown?
[14:03:56 CEST] <nevcairiel> wm4: there is request_probe > 0 in the avstream i think
[14:03:58 CEST] <wm4> michaelni_: what about other parameters (like width/height, extradata, etc.)? when does the API user know that they are known?
[14:03:59 CEST] <nevcairiel> maybe codec context
[14:04:39 CEST] <nevcairiel> wm4: those either appear right with the stream, or not at all - ie. in this case only after decoding
[14:05:07 CEST] <nevcairiel> although extradata might be a bit different
[14:05:10 CEST] <nevcairiel> not sure when that shows up
[14:07:11 CEST] <michaelni_> wm4, request_probe should tell about the probing state
[14:08:24 CEST] <michaelni_> w/h could be set by a parser in which case they should only be set after probing
[14:09:20 CEST] <nevcairiel> if the parser can set it, then usually the decoder can figure it out on its own as well
[14:09:36 CEST] <michaelni_> yes, also spliting a stream into packets can need a parser so it too can happen only after probing
[14:10:14 CEST] <superware> michaelni_: hi, you've successfully helped me a few years ago with a "tough nut" issue :) can you please advice me on trac.ffmpeg.org/ticket/4893 ? what should I do to promote it?
[14:11:02 CEST] <wm4> so it's like find_stream_info is optional, but don't complain if you get stabbed to death by 1000s of nonsense corner cases
[14:12:30 CEST] <michaelni_> not running find_stream_info() is not well tested, maybe we should add some fate tests for that or something
[14:16:47 CEST] <michaelni_> superware, i dont have time ATM to look into this, maybe Thomas Volkert or nicolas are interrested to look into a udp issue but ive not checked what exactly this one is about
[14:19:33 CEST] <michaelni_> but if its a h264 issue then they are the wrong people to ask
[14:19:53 CEST] <nevcairiel> looks like h264 from reading it briefly, the decoder seems to get into an error state and never recover
[14:22:15 CEST] <michaelni_> can this be reproduced via a simple input file (no udp) ? (would be easier to work with a flat file than udp pcap) or is there some difference between the concatenated udp data and whet libavformats udp makes out of it ?
[14:23:00 CEST] <nevcairiel> wonder if one couldnt simply dump the udp stream into a mpegts file for testing
[14:23:07 CEST] <nevcairiel> superware should try that
[14:23:59 CEST] <nevcairiel> perhaps with netcat
[14:24:49 CEST] <michaelni_> its output could be compared to tools/aviocat just to be sure theres no libavformat protocol issue
[14:25:16 CEST] <nevcairiel> oh right, aviocat could also dump it
[14:28:07 CEST] <superware> michaelni_: well, the way I've successfully reproduced it was to omit every other packet (50%) and save that as a pcap for playback.
[14:28:36 CEST] <nevcairiel> it would be easier if you could try to save the resulting stream with aviocat into a file and see if that also fails
[14:28:42 CEST] <nevcairiel> replaying a pcap stream requires effort :p
[14:30:51 CEST] <superware> ok, trying...
[14:31:39 CEST] <nevcairiel> something like "aviocat udp://xxx..xxx.xx file.ts" should hopefully do the job
[14:38:39 CEST] <superware> nevaciriel: where can I get aviocat executable?
[14:39:22 CEST] <J_Darnley> :) I'm reading an HN thread where some guy is asking for suggestions for a platform to test some SIMD code on skylake.  So far 3 people have told him to build his own like a $1000 system is loose change.
[14:39:37 CEST] <J_Darnley> superdump: make tools/aviocat
[14:40:38 CEST] <J_Darnley> sorry that should have been superware ^
[14:42:35 CEST] <Daemon404> J_Darnley, for a company who got a buttload of VC it is
[14:42:41 CEST] <Daemon404> HN has VC-colored glasses
[14:44:46 CEST] <BBB> all startups have VC-colored glasses
[14:44:57 CEST] <Daemon404> and HN is about startups
[14:45:06 CEST] <BBB> HN is startup elite central
[14:45:56 CEST] <BBB> look at me Im YC12"
[14:46:47 CEST] <BBB> reminds me of a woman my wife knew over there with a bumper sticker stanford 04 graduate or something like that
[14:47:00 CEST] <BBB> thats your claim to fame, that you graduated from YC in 12 or Stanford in 04
[14:47:03 CEST] <BBB> ...
[14:47:19 CEST] <J_Darnley> It is a strange place
[14:49:42 CEST] <kierank> J_Darnley: stop reading hn
[14:50:04 CEST] <Daemon404> i enjoy it in a morbid way
[14:50:09 CEST] <Daemon404> like watching a car crash
[14:50:28 CEST] <kierank> https://twitter.com/obencoder/status/655694258758881280
[14:50:43 CEST] <Daemon404> lol
[14:50:48 CEST] <Daemon404> i dont even have to click that
[14:50:50 CEST] <Daemon404> it's the bbc article
[14:50:53 CEST] <Daemon404> abotu 10gige
[14:50:54 CEST] <Daemon404> i bet
[14:51:08 CEST] <nevcairiel> it is
[14:54:13 CEST] <kierank> yeah there's a wall of text about TCP optimisation for web streaminjg
[14:54:16 CEST] <kierank> on hn
[14:54:59 CEST] <JEEB> yea
[14:55:44 CEST] <Daemon404> lol
[15:14:56 CEST] <superware> nevaciriel: aviocat is stuck on avio_read since there's no more packets to read, any idea?
[15:15:30 CEST] <superware> since there's => when there are
[15:15:50 CEST] <nevcairiel> just close it, it should've written the output file anyway
[15:19:05 CEST] <superware> ok
[15:33:44 CEST] <superware> michaelni_: ok, please see http://www.datafilehost.com/d/f097cc19 you will notice the frames stop (by starting PlayCap just after the "bad" point, ffplay is able to play everything that follows, so it must be something in that point)
[15:44:17 CEST] <superware> nevaciriel: can you please verify the producted file indeed causes ffmpeg (and ffplay) to stop outputting frames?
[15:44:56 CEST] <superware> produced*
[16:01:05 CEST] <ubitux> async model, we want to pop a frame; do we want it blocking or not?
[16:01:14 CEST] <wm4> context?
[16:01:41 CEST] <ubitux> imagine a player, rendering thread, poping a frame to be display
[16:02:04 CEST] <ubitux> does it want a fast return to refresh screen or whatever even if it has no new frame
[16:02:07 CEST] <Daemon404> is this a thought experiment
[16:02:13 CEST] <ubitux> or do we want to block until we get the new frame
[16:02:26 CEST] <ubitux> it's about writing an async api
[16:02:39 CEST] <ubitux> we could have both obviously
[16:02:43 CEST] <Mavrik> I've always thought that blocking async APIs make for a cleaner code at the end.
[16:02:44 CEST] <ubitux> depending on an option
[16:03:09 CEST] <ubitux> Mavrik: yeah, but in case of a lag, one is responsive (with drops) and one isn't but will show every frame
[16:03:36 CEST] <wm4> IMO, it should fetch from a thread-safe queue
[16:03:41 CEST] <Mavrik> Good point there.
[16:03:47 CEST] <wm4> the user should be able to get notified if there's a new frame
[16:03:54 CEST] <wm4> this is enough to implement blocking wait too
[16:04:00 CEST] <ubitux> wm4: yes, but let's say the queue is empty, what is the poping frame function supposed to do?
[16:04:04 CEST] <ubitux> wait, or return nothing?
[16:04:42 CEST] <ubitux> the thread poping frames is decoupled from the demux/decode/filter one, its only work is to... get data
[16:04:44 CEST] <wm4> either way would work... if there's a notification mechanism, the other could be implemented ontop of the default semantics
[16:04:56 CEST] <wm4> personally I don't think blocking behavior is all that useful
[16:05:03 CEST] <wm4> because how do you interrupt it?
[16:05:21 CEST] <ubitux> you don't
[16:05:30 CEST] <ubitux> basically i'd say in a player a non-blocking is interesting
[16:05:33 CEST] <ubitux> but in case of a transcode
[16:05:41 CEST] <ubitux> you absolutely want the frame anyway, so you want it blocking
[16:05:49 CEST] <ubitux> so i guess the api should provide both ways?
[16:06:12 CEST] <nevcairiel> could do like most APIs and provide a timeout to the blocking wait, that way you can wait on a condition variable quite efficiently
[16:06:24 CEST] <wm4> timeouts are nonsense
[16:06:44 CEST] <ubitux> that's half assed, not sure in what scheme you would want that
[16:06:53 CEST] <wm4> just provide a sane notification method, so the user can implement his own condition variable based blocking or so
[16:07:01 CEST] <nevcairiel> or if you're waiting on a condition variable anyway, just make it signaled and the function returns
[16:07:32 CEST] <Mavrik> Essentially what you're just trying to do with blocking API is to avoid spinlock waiting.
[16:07:50 CEST] <Mavrik> So if you add a way to be notified on a frame then the main issue with non-blocking API kinda dissapear.
[16:08:06 CEST] <ubitux> the user will have to prevent spinlock himself with cond in the non blocking mode
[16:08:19 CEST] <ubitux> but i'm not sure to see the point of having such complexity
[16:08:29 CEST] <ubitux> when we can deal with the blocking internally instead of letting the user do it
[16:08:52 CEST] <wm4> I don't deny that providing helpers for blocking wait (so a user doesn't have to implement boilerplate) might be useful
[16:08:59 CEST] <nevcairiel> having a totally block-free API makes some decisions a bit weird though .. do I need to feed the decoder more data? do I wait for frames to be done first? etc
[16:09:01 CEST] <wm4> but there need to be some ways to make this not crap
[16:09:21 CEST] <ubitux> ok ok
[16:09:23 CEST] <wm4> for example, it might be better to let this blocking wait return if the decoder needs new _input_
[16:09:49 CEST] <wm4> so maybe decoder_wait(), with return values NEEDS_INPUT and HAS_OUTPUT
[16:10:31 CEST] <wm4> this way no interruption is needed either, because the call will return as soon as the decoder is more or less idle (like with lavc's current API)
[16:11:10 CEST] <wm4> nevcairiel: IMO the decoder should always signal when it wants new input
[16:11:24 CEST] <wm4> you don't want to always feed it input, that'd just lead to excessive buffering
[16:11:48 CEST] <ubitux> well, internally there is an automatically blocking queue
[16:11:55 CEST] <ubitux> (on which as a user you can control the size if you want)
[16:13:08 CEST] <nevcairiel> which means the input push function would eventually block?
[16:13:33 CEST] <ubitux> yes
[16:14:31 CEST] <wm4> IMO mandatory blocking is not good... it just requires the user to juggle around with more threads, and just how do you unblock the threads?
[16:14:37 CEST] <nevcairiel> that needs a function to check if it would block then, so i can decide to process output first before waiting
[16:15:04 CEST] <ubitux> then maybe block with timeout returning EAGAIN if it wasn't able to?
[16:18:58 CEST] <nevcairiel> consider a demuxing driven setup, ie. you always enter the entire process with a new AVPacket to handle. Ideally I want to submit this packet immediately and then process decoder outputs, so in that scenario i wouldn't want to block on submit, but rather on poping frames, until the input queue is (nearly) empty and i need to loop again to get a new packet
[16:19:45 CEST] <nevcairiel> in theory it should never happen in this setup that the input queue overflows th ough
[16:19:48 CEST] <nevcairiel> so it might be ok
[16:20:22 CEST] <nevcairiel> as long as I can  get that information - i need to know if my input queue is (nearly) empty and i need to fetch more data before processing more output frames
[16:20:34 CEST] <ubitux> so you're in a sync model trying to use the async api?
[16:20:45 CEST] <ubitux> by basically pushing and popblock?
[16:21:07 CEST] <nevcairiel> one advantage of that is that it can still decode in the background while i'm doing other things
[16:21:11 CEST] <ubitux> (or maybe not block if there is nothing in output..)
[16:21:38 CEST] <wm4> what nevcairiel said; you can decode in the background, even if your code looks synchronous
[16:21:57 CEST] <nevcairiel> right now the decoder is doing nothing while i'm say filtering the image on the same thread
[16:22:34 CEST] <nevcairiel> with the async API the decodre could be busy on its own async thread while I can still use the "main" thread for filtering and presenting
[16:23:10 CEST] <nevcairiel> I currently emulate that in my own code by simply moving all decoding on a worker thread
[16:23:26 CEST] <nevcairiel> but its a lot of plumbing that this API might be able to simply replace
[16:23:38 CEST] <ubitux> i'm not sure to follow: do you have threads as a user?
[16:23:47 CEST] <ubitux> (in your scenario)
[16:24:01 CEST] <nevcairiel> you could, of course
[16:24:24 CEST] <ubitux> but in the use case you're presenting
[16:24:35 CEST] <nevcairiel> not necessarily
[16:24:47 CEST] <nevcairiel> you could do it all in one thread for simplicity
[16:24:59 CEST] <nevcairiel> while still having the decoder work on its own async worker thread internal to avcodec
[16:25:28 CEST] <ubitux> in the api you have 2 func: one to push a packet, one to pop a frame; which would you see blocking, and what extra api you would like?
[16:26:00 CEST] <nevcairiel> neither does really have to block, as long as there is other informational functions that I can use to block myself
[16:26:07 CEST] <nevcairiel> or both could block if the conditions call for it
[16:26:14 CEST] <nevcairiel> push could block if the internal queue is way too big
[16:26:23 CEST] <wm4> ubitux: you'd have functions for both input and output to 1. query the state, 2. feed/receive data
[16:26:24 CEST] <nevcairiel> (which you should not reach in my model, so non-issue)
[16:26:36 CEST] <ubitux> so you want to deal with the blocking yourself? even in the simplest form, you don't think that can be handled by the api to limit the boilerplate?
[16:26:45 CEST] <nevcairiel> and pop should block if the decoder needs no further interaction right now, ie. no output is ready, and no input is needed
[16:26:49 CEST] <wm4> if the query function returns a positive result, the second operation should never block
[16:26:58 CEST] <wm4> (think of it as poll(), read(), write())
[16:27:29 CEST] <wm4> now, what happens if you feed/receive without querying the state
[16:27:34 CEST] <wm4> blocking would just lead to a deadlock
[16:27:46 CEST] <nevcairiel> depends really
[16:27:50 CEST] <wm4> you can't just block when reading output, because writing input might be required to make progress
[16:28:02 CEST] <nevcairiel> if push just blocks while the input queue is too big, it would eventually unblock as the decoder works through the data
[16:28:29 CEST] <nevcairiel> and if pull can return an error for "need more data", then thats also not a problem
[16:28:32 CEST] <wm4> it could do that, but then it'd unblock with no data to return if the input queue runs out
[16:28:40 CEST] <wm4> and also this loss of buffering would make it inefficient
[16:28:49 CEST] <wm4> (so it's not really useful)
[16:29:25 CEST] <nevcairiel> if you want the full efficiency, you might need to use the poll methods
[16:29:45 CEST] <nevcairiel> or it just asks for more data if the queue input queue is empty, but the decoder still working
[16:30:11 CEST] <nevcairiel> that should hopefully give you enough time to react
[16:30:48 CEST] <wm4> maybe have a decoder_wait() function, which returns NEEDS_INPUT, HAS_OUTPUT, or IDLE
[16:31:05 CEST] <nevcairiel> whats IDLE if not NEEDS_INPUT? :)
[16:31:05 CEST] <wm4> maybe you could have a blocking version too, which wouldn't return IDLE
[16:31:18 CEST] <wm4> IDLE is if it has enough input, but is still waiting for output
[16:31:33 CEST] <nevcairiel> thats not a wait() function then, more like a poll()
[16:31:39 CEST] <nevcairiel> wait sounds like blocking
[16:31:44 CEST] <wm4> <wm4> maybe you could have a blocking version too, which wouldn't return IDLE
[16:31:48 CEST] <wm4> well ok
[16:32:39 CEST] <wm4> ubitux: makes sense?
[16:32:50 CEST] <nevcairiel> i think we lost him
[16:33:15 CEST] <ubitux> i was away 1 minute, i lost track indeed, will read & re-read, and re-ask again if needed
[16:33:17 CEST] <ubitux> give me a moment
[16:34:43 CEST] <ubitux> 16:28 <@nevcairiel> if push just blocks while the input queue is too big, it would eventually unblock as the decoder works through the data // this can not work if you're in a sync model as a user
[16:34:47 CEST] <nevcairiel> personally I would make the pop function have the behavior of the suggested "wait" function (ie. block while decoder is happy, no input needed, no output ready), and on top have a poll function that just returns the three states without blocking for people that want to avoid the block there at all costs
[16:35:03 CEST] <ubitux> (unless you have a query system as suggested)
[16:35:06 CEST] <nevcairiel> ubitux: why not
[16:35:16 CEST] <nevcairiel> it would just pile up in the output queue
[16:37:42 CEST] <wm4> so I guess we agree that whatever you do, there can be 3 states: input needed, output ready, and idle|blocking
[16:37:55 CEST] <ubitux> let's say the internal frame queue has 2 frames, you push a packet which is going to translate to 10 output frames (because sanity), then the api would mandate that you need to pop as much as you can?
[16:38:08 CEST] <ubitux> (before trying to push another packet)
[16:38:13 CEST] <nevcairiel> sure
[16:38:17 CEST] <ubitux> ok
[16:38:25 CEST] <nevcairiel> dont think anything else makes much sense
[16:38:28 CEST] <ubitux> so blocking push and blocking pop
[16:38:41 CEST] <wm4> of course you could always make the queues unbounded, but that'd be silly
[16:38:43 CEST] <nevcairiel> with poll functions to query if it would block
[16:38:47 CEST] <nevcairiel> thats what i would do, anyway
[16:39:26 CEST] <wm4> and this conveniently takes care of both n:m decoding, and async decoding
[16:39:26 CEST] <ubitux> query if what would block? just pushing?
[16:39:35 CEST] <nevcairiel> both
[16:39:46 CEST] <ubitux> the pop query might be innacurate then
[16:39:51 CEST] <nevcairiel> how so
[16:40:19 CEST] <ubitux> well, the frame could not be ready when you query, but soon after
[16:40:26 CEST] <nevcairiel> thats fine
[16:40:28 CEST] <ubitux> even before you try to push another packet
[16:40:52 CEST] <ubitux> but that frame which got just ready might make the frame queue full
[16:41:05 CEST] <nevcairiel> there should be no output frame queue
[16:41:06 CEST] <ubitux> and so you're trying to push a packet even though you can't
[16:41:06 CEST] <wm4> these states are stable, I'd say
[16:41:26 CEST] <nevcairiel> well, no maximum size anyway
[16:41:27 CEST] <ubitux> (or well, we need a packet queue as well i suppose)
[16:41:37 CEST] <ubitux> no maximum size? no no..
[16:41:46 CEST] <nevcairiel> the decoder should always be able to put its decoded frame somewhere
[16:42:14 CEST] <wm4> it's up to the decoder not to accept too much input that it can't output anymore
[16:42:45 CEST] <nevcairiel> if the calling app keeps pushing new data but not cleaning out the output frames, its really its own fault =p
[16:43:05 CEST] <wm4> so you should say feeding input should always succeed?
[16:43:33 CEST] <nevcairiel> not necessarily, there should be an input queue with could have a max size
[16:43:59 CEST] <nevcairiel> but without knowing how many packets make up a frame, setting an output queue size would be hard
[16:44:05 CEST] <nevcairiel> i mean, you could get 10 frames out of 1 packet
[16:44:07 CEST] <nevcairiel> now what
[16:44:39 CEST] <wm4> yeah
[16:44:41 CEST] <wm4> ok
[16:44:59 CEST] <wm4> I mean, the decoder itself can know how many input packets it can take without overflowing the output queue
[16:45:04 CEST] <wm4> in theory
[16:45:22 CEST] <wm4> if there's too much output, it just can stop decoding
[16:45:22 CEST] <nevcairiel> maybe defining an output queue size could be optional
[16:45:41 CEST] <nevcairiel> in a full async model, you might otherwise get a bit of funny business
[16:45:48 CEST] <nevcairiel> ie. one thread reads input and blocks on submit()
[16:45:55 CEST] <nevcairiel> another thread reads output and displays it
[16:46:19 CEST] <nevcairiel> decoder would be working overtime to decode the entire file into memory if its not bounded somewhere
[16:46:51 CEST] <nevcairiel> hard to say really
[16:47:05 CEST] <wm4> yes, that's what I mean by excessive buffering
[16:48:01 CEST] <nevcairiel> i suppose the output queue should have a certain size limit, if you cant process output faster, its no good producing any more anyway
[16:48:31 CEST] <nevcairiel> but then your submit packet function could indeed block forever
[16:48:42 CEST] <nevcairiel> or it needs a return value that says "process output, dude"
[16:50:26 CEST] <nevcairiel> ideally you should be able to use submit and pop without worrying about deadlocking, and only for further optimization or mapping to a sync process use the poll functions
[16:52:04 CEST] <nevcairiel> .. which means you could make submit block indefinitely if output is never getting processed, but then I bet API users would *commonly* make the mistake of not polling first and locking themself up
[16:55:58 CEST] <wm4> nevcairiel: that's why I'm saying there should be a central processing function, which essentially tells the API user what to do (send input, read output, or do wait)
[16:56:57 CEST] <nevcairiel> to summarize: submit() should block if the input queue is full, and the output queue still has room - ie. the input queue will shrink soon without interaction from the user. pop() should block if no output is available, and no input is needed(),  and there should be a poll() function to check the status (need input, has output, working)
[16:57:41 CEST] <nevcairiel> the only question is what "need input" specifically means, input queue not full? input queue empty?
[17:01:23 CEST] <wm4> good question, I would say queue empty
[17:01:44 CEST] <wm4> if a decoder actually needs input, the input queue will quickly become empty again
[17:01:59 CEST] <wm4> and if the user wants extra buffering, he can do that on his own
[17:02:29 CEST] <nevcairiel> input queue needs a packet count function then
[17:03:43 CEST] <nevcairiel> anyway the general concept is  simple, any function should only block if its going to unblock on its own soon without user interaction
[17:04:57 CEST] <wm4> sounds good
[17:17:03 CEST] <martijnb> just so I get the gist, so if the decoder doesn't have output available because it's waiting for more input, you don't want it to block because you want to notify the API user that more input is needed first
[17:17:25 CEST] <martijnb> that's roughly what you just said right
[17:17:31 CEST] <wm4> yes
[17:17:56 CEST] <martijnb> gotcha
[17:18:36 CEST] <nevcairiel> if you do a full async model, this might not be ideal, but you can just usleep(1); and try again hoping that your feeding thread has provided more input now
[17:19:33 CEST] <nevcairiel> or you still get the output-ready callback which you can wait for
[17:19:48 CEST] <wm4> usleep(1? wtf
[17:20:15 CEST] <wm4> use condition variables
[17:20:26 CEST] <nevcairiel> its the lazy approach
[17:20:35 CEST] <ubitux> note that VT and MC async api do NOT have a pop function
[17:20:36 CEST] <nevcairiel> which wont really perform any worse in the end
[17:20:38 CEST] <ubitux> they use a callback system
[17:20:46 CEST] <ubitux> maybe we should map to something like this
[17:20:56 CEST] <ubitux> s/to/do/
[17:21:43 CEST] <ubitux> (which btw raises question to me about the reordering in the callback function in VT; like, how can this even work if the user consume the frame too early...)
[17:21:47 CEST] <wm4> yes I've been arguing that we should have notification callbacks
[17:22:04 CEST] <ubitux> it's not about notification, it's about actually getting the data frame itself
[17:22:10 CEST] <nevcairiel> but its not only notification, its a full blown "have a frame and do something with it" callback
[17:22:18 CEST] <ubitux> mediacodec seems to do it the otherway around for the "pushing" btw
[17:22:32 CEST] <ubitux> like, you are querying a buffer to push
[17:22:32 CEST] <wm4> (what I want is a notification callback would fire whenever input is needed or output is ready)
[17:22:35 CEST] <wm4> +that
[17:22:53 CEST] <ubitux> (and you're not going to get a buffer to fill if the queue is full)
[17:23:51 CEST] <nevcairiel> I think having a output queue and a pop function is fine .. if you want to do all the work in the callback, you can just pop right there and go nuts
[17:23:58 CEST] <nevcairiel> but it makes it easier to use it in a half-sync model
[17:24:27 CEST] <martijnb> hrm
[17:25:14 CEST] <ubitux> nevcairiel: so, can you summarize again (sorry) your suggestion?
[17:25:24 CEST] <ubitux> i'm looking for a model to actually implement
[17:25:29 CEST] <martijnb> I mean, from a modeling standpoint, if you can implement it neatly, it'd probably be nice to just register observers that either wait to hear input is required or more output is ready
[17:25:41 CEST] <ubitux> first for map to other async models, then to put our current sync model in
[17:25:43 CEST] <martijnb> so my 2 cents are with wm4 ;p
[17:27:07 CEST] <martijnb> those are the 2 states the API user wants to act on anyway
[17:28:43 CEST] <ubitux> isn't the notification system adding an annoying threading boilerplate for the user (mandatory? only in async?)
[17:29:50 CEST] <wm4> ubitux: well we'd have the poll/wait functions too
[17:29:54 CEST] <wm4> so, optional
[17:30:31 CEST] <ubitux> can you make a full summary of the api? which functions are available, with what callback, and what is blocking?
[17:33:52 CEST] <nevcairiel> (1) submit, block if input queue full, output queue not full, (2) pop, block if no output ready, still input available, (3) poll(), non-blocking, return if input needed, output ready, or just working, (4) callback when output ready, (5) callback when input queue empty?
[17:35:46 CEST] <wm4> this still needs an explanation how the blocking can be undone by the API user
[17:35:59 CEST] <wm4> do you need to close the codec context on another thread or so?
[17:36:12 CEST] <ubitux> yes, like what to do in case of seek or something
[17:38:49 CEST] <nevcairiel> all these blocks should unblock by themself in  not-too-long
[17:38:59 CEST] <nevcairiel> but i suppose an extra unblock might not be too terrible
[17:39:57 CEST] <ubitux> anyone comfortable with one or more existing async api? like, mediacodec, VT or something else?
[17:40:22 CEST] <wm4> I now know mmal more than I'd like to
[17:40:28 CEST] <ubitux> typically one which is comfortable for the user (prefered to one that allow more)
[17:40:37 CEST] <ubitux> are you satisified with it?
[17:41:00 CEST] <wm4> mmal probably isn't too bad, except the preallocated buffer business
[17:41:33 CEST] <wm4> connecting it with the lavc sync API was hellish
[17:45:23 CEST] <ubitux> mediacodec and videotoolbox provides 2 different API for both model
[17:45:50 CEST] <ubitux> for videotoolbox it makes things slow as shit in sync, but it kinda work
[17:51:08 CEST] <ubitux> https://www.libavg.de/site/projects/libavg/wiki/VideoDecoding
[17:51:13 CEST] <ubitux> maybe some interesting stuff around here
[17:52:38 CEST] <durandal_1707> libavs
[17:53:16 CEST] <mateo`> speaking of mediacodec, i've taken the C route to begin with (so no jni involved for this version)
[17:54:07 CEST] <mateo`> (and i'll get back to the jni version after)
[18:00:21 CEST] <nevcairiel> ubitux: i know the Media Foundation API, which works somewhat similar, call ProcessInput, it can return a status for "process output first", call ProcessOutput, which can return need more input, or otherwise blocks until frame is done
[18:02:01 CEST] <nevcairiel> ..with callbacks on top
[18:02:55 CEST] <ubitux> so having callbacks is recurrent in most (all?) implementations?
[18:07:55 CEST] <nevcairiel> proper async needs callbacks
[21:11:55 CEST] <cone-033> ffmpeg 03James Zern 07master:7b4367d93ea2: vp9_parser: fix endless loop w/0-sized frame
[21:32:47 CEST] <Timothy_Gu> Ah okay that's why: av_realloc returns a 16-byte aligned pointer
[21:36:01 CEST] <Timothy_Gu> now the ultimate question: how can I get an aligned realloc'd memory
[21:36:06 CEST] <jamrial> Timothy_Gu: it should be 32-byte if you have avx enabled at configure time
[21:36:57 CEST] <rcombs> so you don't HAVE_ALIGNED_MALLOC?
[21:37:27 CEST] <rcombs> using posix_memalign or memalign instead?
[21:38:23 CEST] <Timothy_Gu> jamrial: av_malloc is 32-byte aligned, but av_realloc isn't
[21:38:34 CEST] <Timothy_Gu> rcombs: no, at least not on linux
[21:39:00 CEST] <jamrial> it should, assuming either HAVE_ALIGNED_MALLOC or CONFIG_MEMALIGN_HACK are true
[21:39:15 CEST] <Timothy_Gu> on linux both are not
[21:39:46 CEST] <wm4> AFAIK you can't have that
[21:40:06 CEST] <rcombs> is the use of av_realloc() actually necessary?
[21:40:07 CEST] <Timothy_Gu> so should we make CONFIG_MEMALIGN_HACK default on !HAVE_ALIGNED_MALLOC?
[21:40:13 CEST] <wm4> that's also why av_malloc return values can't be used with av_realloc
[21:40:38 CEST] <wm4> realloc doesn't keep the aligned property)
[21:40:40 CEST] <rcombs> seems unusual for it to be used with frame data buffers
[21:40:46 CEST] <Timothy_Gu> rcombs: idk it's used deep in av_new_packet
[21:41:23 CEST] <rcombs> oh, is it packet data?
[21:41:31 CEST] <rcombs> I thought we were talking about AVFrame
[21:41:49 CEST] <Timothy_Gu> oh the packet data is directly passed to an AVFrame in rawvideodec
[21:42:43 CEST] <wm4> AVFrame can be not-aligned for a number of reason
[21:42:46 CEST] <wm4> like cropping
[21:43:50 CEST] <nevcairiel> <Timothy_Gu> now the ultimate question: how can I get an aligned realloc'd memory <-- use windows \o/
[21:44:02 CEST] <nevcairiel> (it has aligned realloc functions)
[21:44:11 CEST] <rcombs> go yell at linus
[21:44:31 CEST] <rcombs> that feels a bit backwards
[21:44:32 CEST] <iive> it's glibc issue
[21:44:35 CEST] <rcombs> usually he's the one doing that
[21:44:44 CEST] <Timothy_Gu> I would if the package management system is better in windows
[21:44:52 CEST] <Timothy_Gu> chocolatey and nuget kind of suck
[21:45:04 CEST] <nevcairiel> so, you are using rawvideo for testing, huh?
[21:45:20 CEST] <nevcairiel> rawvideo in itself is a bit weird, it can result in all sorts of alignment issues because of the packet-data passthrough
[21:45:22 CEST] <Timothy_Gu> I'm actually not
[21:45:55 CEST] <nevcairiel> but but you mentioend rawvideodec
[21:45:58 CEST] <Timothy_Gu> oops actually i am
[21:46:16 CEST] <rcombs> lies
[21:46:18 CEST] <Timothy_Gu> I just used vsynth1 and -stream_loop 10000
[21:47:50 CEST] <Timothy_Gu> anyway, I guess I'll have to keep using movu then :(
[21:48:18 CEST] <rcombs> it's not much slower in cases where data _is_ aligned on modern CPUs, is it?
[21:48:30 CEST] <nevcairiel> dont we have a bunch of things that would just die if an avframe is not properly aligned?
[21:49:05 CEST] <durandal_1707> you mean data pointers?
[21:49:17 CEST] <Timothy_Gu> rcombs: at least not slower on the xeon i'm testing on
[21:49:23 CEST] <iive> btw, since when does windows have alligned (re)alloc ?
[21:49:38 CEST] <rcombs> _aligned_realloc
[21:49:51 CEST] <rcombs> (yay underscores :/)
[21:50:06 CEST] <Timothy_Gu> durandal_1707: basically frame->data[0] & 0x1f != 0
[21:50:23 CEST] <Timothy_Gu> iive: since forever apparently: https://msdn.microsoft.com/en-us/library/y69db7sx(v=vs.71).aspx
[21:50:40 CEST] <Timothy_Gu> "Win 98, Win Me, Win NT, Win 2000, Win XP
[21:51:22 CEST] <durandal_1707> live with it, data may not be aligned
[21:52:13 CEST] <Timothy_Gu> but thing is frame->data[0] & 0xf always == 0
[21:52:22 CEST] <Timothy_Gu> idk if that's a coincidence
[21:52:30 CEST] <nevcairiel> you may just get lucky on your platform
[21:52:47 CEST] <rcombs> could be your libc just mallocs with 16-byte alignment
[21:52:50 CEST] <Timothy_Gu> ok I guess
[21:53:10 CEST] <nevcairiel> and with rawvideo, data[0]+linesize[0] could be totally unaligned
[21:54:05 CEST] <Timothy_Gu> I'm not worrying about linesize as huffyuv encodes by line (i think)
[21:57:03 CEST] <durandal_1707> you are working with that codec?
[21:57:04 CEST] <iive> Timothy_Gu: that's kind of strange, since ffmpeg had to use manual alignment on windows...
[21:57:17 CEST] <nevcairiel> noone cared to look it up
[21:57:23 CEST] <nevcairiel> or silly mingw just didnt offer it
[21:57:26 CEST] <nevcairiel> who knows
[21:57:49 CEST] <iive> yeh, most likely mingw fault.
[21:58:13 CEST] <nevcairiel> internally these functions oftne w ork the same way as the manual alignment hack in ffmpeg however
[21:58:15 CEST] <JEEB> yeah, ye olde mingw used  to either not have that stuff or hide it behind options
[21:58:51 CEST] <JEEB> but yeah, if it has been there since 9x then it makes no sense to limit it, so it probably just didn't have it
[21:59:40 CEST] <Timothy_Gu> durandal_1707: yes
[21:59:52 CEST] <Timothy_Gu> also can someone review https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2015-October/181570.html?
[22:03:55 CEST] <wm4> michaelni_: actually ffmpeg has a similar check like mpg123 in its code:
[22:03:57 CEST] <wm4>                     if((state&SAME_HEADER_MASK) != (s->header&SAME_HEADER_MASK) && s->header)
[22:03:57 CEST] <wm4>                         s->header_count= -3;
[22:04:13 CEST] <wm4> SAME_HEADER_MASK seems to include the bitrate bits
[22:05:13 CEST] <wm4> I have no idea what this specific code does though
[22:05:44 CEST] <wm4> I think it also tries to determine "the" format
[22:18:41 CEST] <michaelni_> hmm, the comment says bitrate is in there but i dont think it is
[22:20:33 CEST] <wm4> hm that seems possible
[22:21:22 CEST] <wm4> is it fine if I simply use SAME_HEADER_MASK?
[22:32:19 CEST] <michaelni_> i think so
[22:33:11 CEST] <cone-033> ffmpeg 03Michael Niedermayer 07master:fb4d350ff309: avcodec/mpegaudio_parser: Update comment to match code
[22:52:29 CEST] <nevcairiel> now the guy who refuses to  benchmark his "optimization" patches complains about lack of information
[22:52:29 CEST] <nevcairiel> he really seems annoying sometimes :p
[00:00:00 CEST] --- Tue Oct 20 2015