[Ffmpeg-devel-irc] ffmpeg-devel.log.20140426

[burek]

[00:34] <cone-743> ffmpeg.git 03Lukasz Marek 07master:b2682db34c54: lavfi/avfilter: fix typos in doxgens
[00:55] <cone-743> ffmpeg.git 03Reynaldo H. Verdejo Pinochet 07master:107f2468c4fc: ffserver: free nacl as needed
[00:55] <cone-743> ffmpeg.git 03Reynaldo H. Verdejo Pinochet 07master:1404e2a389fc: ffserver: free AVStream st before wiping context
[00:55] <cone-743> ffmpeg.git 03Reynaldo H. Verdejo Pinochet 07master:7228bdeebd28: ffserver: fix missing frees at connection setup
[01:40] <cone-743> ffmpeg.git 03Alessandro Ghedini 07master:cdf6eb5a9710: vc1: Do not return an error when skipping b frames
[01:40] <cone-743> ffmpeg.git 03Michael Niedermayer 07master:8b2c3d2ae951: Merge commit 'cdf6eb5a9710566be217a3f17d3d94ac4e4d2662'
[01:56] <cone-743> ffmpeg.git 03Michael Niedermayer 07master:3ba77ea369a0: avcodec/vc1dec: print debug message if a b frame without reference is skiped
[02:26] <cone-743> ffmpeg.git 03Lukasz Marek 07master:b217dc91bfd5: lavfi/avfilter: clarify avfilter_graph_get_filter() doxygen
[04:00] <jamrial> BBB: any idea what might be the reason dst in intra_pred 32x32 functions isn't aligned to 32 bytes?
[04:00] <BBB> uhm... no not really
[04:00] <BBB> is that for chroma or luma?
[04:00] <BBB> it sounds like the picture allocation functions themselves don't align to 32bytes
[04:01] <BBB> (either the pointer - which is unlikely - or the stride - which is actually quite possibly an issue)
[04:05] <BBB> what is pic->line_size[]?
[04:05] <jamrial> Not sure, probably both. All the functions i ported to avx2 will complain about alignment when i try to use movdqa with dst+stride
[04:08] <jamrial> linesize seems to be 384
[04:10] <BBB> and linesize[1]?
[04:12] <jamrial> 192
[04:15] <BBB> and the input pointers allocated are all 32-byte aligned?
[04:16] <BBB> (that's pic->data[0-2])
[04:19] <jamrial> That's what i don't know
[04:19] <BBB> can you print the pointers in hex?
[04:22] <BBB> hm you're right I see the same thing actually
[04:23] <BBB> Allocated picture at pointer 0x7fc4c104c610, 0x7fc4c1069610, 0x7fc4c1071010
[04:23] <BBB> Allocated picture at pointer 0x7fc4c1802410, 0x7fc4c181f410, 0x7fc4c1826e10
[04:24] <BBB> so it seems the input pictures aren't well-allocated
[04:25] <jamrial> Is that some custom output with av_log you added, or something you get with -loglevel?
[04:29] <BBB> custom printfs
[04:29] <BBB> this code in utils.c:
[04:29] <BBB>         // no edge if EDGE EMU or not planar YUV
[04:29] <BBB>         if ((s->flags & CODEC_FLAG_EMU_EDGE) || !is_planar)
[04:29] <BBB>             pic->data[i] = pic->buf[i]->data;
[04:29] <BBB>         else {
[04:29] <BBB>             pic->data[i] = pic->buf[i]->data +
[04:29] <BBB>                 FFALIGN((pic->linesize[i] * EDGE_WIDTH >> v_shift) +
[04:29] <BBB>                         (pixel_size * EDGE_WIDTH >> h_shift), pool->stride_align[i]);
[04:29] <BBB>         }
[04:29] <BBB> is broken
[04:30] <BBB> the input (pic->buf[i]->data) is 32-byte aligned, the output isn't
[04:30] <BBB> I guess pool->stirde_align[] is mis-set
[04:33] <BBB> ah
[04:33] <BBB> /Users/ronaldbultje/Projects/ffmpeg/libavcodec/internal.h:#   define STRIDE_ALIGN 16
[04:33] <BBB> that's the basic problem
[04:33] <BBB> it should be 32
[04:33] <BBB> I'd precede that with:
[04:33] <BBB> #if HAVE_AVX
[04:33] <BBB> #   define STRIDE_ALIGN 32
[04:34] <BBB> let me see if that fixes anything
[04:34] <BBB> yeah all is good then
[04:35] <BBB> http://pastebin.com/skCd46Rp
[04:35] <BBB> it may break other tests in fate, who knows...
[04:35] <BBB> but should be safe
[04:36] <jamrial> Cool, thanks
[04:37] <BBB> np
[04:38] <jamrial> wonder if ubitux's earlier avx2 tests were slow because of this
[04:39] <BBB> well he wasn't really using avx2 as full-width, he was doing half-width loop filter (16px per iteration) and then unpacking to full 32bytes for word-operations only
[04:40] <BBB> you wouldn't expect to gain much from that, I'll see if I can find some time to add mix4 functions for "proper" avx2 loop filter insertion
[04:40] <BBB> that's not exactly easy but it's kind of fun
[04:40] <BBB> mc/idct/intrapred should be easier targets
[04:41] <jamrial> Ah, i see
[04:42] <BBB> vp9 loop filter is kind of ... wierd
[04:42] <BBB> weird*
[04:42] <jamrial> I so far ported five trivial intra_pred functions. I posted a github link last night
[04:42] <jamrial> Great, seems to with with mova now
[04:43] <BBB> ash you duplicated pb_1/3
[04:44] <jamrial> I assumed the originals were 16 bytes, so made instead some 32 bytes there
[04:46] <BBB> yeah they are, it's probably easier to make the existing ones (I think they're in dsputil_mmx.c) 32-bytes
[04:46] <BBB> easier as in more correct
[04:46] <jamrial> Sure
[04:48] <BBB> code looks good
[04:48] <BBB> I guess it's not easy to share code between avx2 and ssse3 right?
[04:49] <jamrial> For some i could add a lot of notcpuflag(avx2) checks and such, but it will look pretty ugly
[04:50] <BBB> I'd say if the code shared is less than ... say 60% or so, let's not worry
[04:50] <jamrial> Now what i need is some kind soul to bench this since i don't have a haswell :P
[04:50] <jamrial> Intel's SDE is magic
[04:50] <BBB> lol
[04:50] <BBB> that sucks
[04:51] <BBB> ok I'm gonna sleep
[04:51] <BBB> bye
[04:51] <jamrial> Night, and thanks again
[08:02] <kurosu_> BBB / jamrial: isn't HAVE_AVX for whether the compiler supports AVX (or is it _INLINE)? In any case, the issue with unconditionnally setting it to 32 is only memory consumption?
[08:07] <jamrial> HAVE_AVX is always true for x86 unless one disables it during configure. _EXTERNAL is disabled if yasm/nasm isn't available
[08:08] <jamrial> there's no check for _INLINE, so it's enabled as long as HAVE_AVX is enabled even if the assembler doesn't support it. but since we don't have any inline avx code it doesn't matter
[08:11] <jamrial> and fate passed here with the change BBB mentioned, so yeah it should be safe
[13:17] <BBB> kurosu_: yeah that's the intention
[13:17] <BBB> kurosu_: clearly if asm/gcc don't do avx, we're not going to have any avx and 32-byte alignment is useless
[13:17] <BBB> kurosu_: if gcc/asm do, then we'll contain avx/2 code - now the question is whether we can execute it, that's a runtime thing
[13:18] <BBB> kurosu_: I don't mind always-enabling it on x86 honestly
[13:18] <BBB> kurosu_: whatever people prefer
[14:10] <nevcairiel> memory allocations already use HAVE_AVX to bump the memory alignment to 32
[14:11] <nevcairiel> seems fitting to use the same here
[14:15] <kurosu_> I didn't check, but ok - I wasn't doubting, I was just trying to educate myself
[14:42] <kierank> great...
[14:43] <kierank> usr/lib/gcc/x86_64-linux-gnu/4.6/../../../../lib/libavfilter.a(avfiltergraph.o): In function `pick_format':
[14:43] <kierank> /media/sf_ffmpeg/libavfilter/avfiltergraph.c:631: undefined reference to `avcodec_find_best_pix_fmt_of_2'
[14:43] <kierank> so avfilter now depends on avcodec
[14:45] <ubitux> this is here since 2012 afaict
[14:46] <kierank> I've never had the problem before
[14:46] <ubitux> i wonder why this is not in lavu though
[14:47] <kierank> it pretty much breaks anything with --disable-everything
[14:47] <ubitux> how do you reproduce?
[14:48] <kierank> ./configure --prefix=/usr --enable-gpl --disable-swscale-alpha --disable-everything --disable-avdevice --disable-filters --enable-avresample --enable-decoder=v210 --enable-filter=smptehdbars --enable-filter=smptebars --enable-filter=drawtext --enable-filter=format --enable-libfreetype
[14:48] <kierank> and then try and link my application
[14:48] <kierank> and it fails
[14:49] <kierank> I don't know why it happened all of a sudden
[14:50] <ubitux> totally unrelated and you probably don't care but i think you can --enable-filters=smptehdbars,smptebars,drawtext,format
[14:55] <kierank> It's not clear what magic incantation I need to get it to link
[14:55] <ubitux> well, pkg-config i guess
[14:56] <kierank> I'm already using that
[14:58] <ubitux> and the pkg-config --libs --static avfilter isn't vomitting enough linker flags?
[14:59] <kierank> dunno
[14:59] <kierank> there's lots of them yes
[14:59] <kierank> gcc -o obecli obecli.o libobe.a -lreadline -L.  -pthread -lavformat -lavcodec -lavresample -lswscale -lavfilter -lfreetype -lz -lrt -lavutil -lm    -lm -lrt -lpthread -lx264 -ltwolame -lmpegts -lstdc++ -ldl -lzvbi -s
[15:00] <kierank> I'm not missing any as far as i can see
[15:02] Action: kierank goes back his old ffmpeg to see if it works
[15:08] <kierank> gah why does it care all of a sudden
[15:09] <kierank> :sigh:
[15:10] <kierank> heh put -lavcodec after -lavfilter and it builds
[15:10] <kierank> wtf
[15:11] <kierank> so the pkg config file is broken basically
[16:56] <michaelni> ubitux, avcodec_find_best_pix_fmt_of_2() should be moved to lavu (with appropriate avcodec less name)
[17:01] <michaelni> ubitux, do you want to move it or should i look into doing that ?
[17:01] <ubitux> not really in the mood for this now sorry
[18:50] <michaelni> ubitux, posted patch
[19:45] <cone-129> ffmpeg.git 03tue46wsdgxfjrt 07master:262ea965e73e: cmdutils: preserve unchanged log flags when setting AV_LOG_SKIP_REPEATED
[19:45] <cone-129> ffmpeg.git 03tue46wsdgxfjrt 07master:669a09fb372f: Add AV_LOG_PRINT_LEVEL flag to include log severity in default log formatting.
[20:02] <wm4> anyone have an idea whether this vp9/webm file is broken? http://a.pomf.se/uylgji.webm
[20:02] <wm4> it works with ffplay, but not vlc or mpv (they show massive blocking in most frames)
[20:03] <wm4> if I use libavformat's demuxer with mpv, it works, same if I enable libavformat video packet parsing in mpv
[20:03] <ubitux> frame 17, 64, 133 (and more later) have no PTS set
[20:03] <wm4> the file was produced by ffmpeg cli apparently, so this could be a case where ffmpeg produces files only ffmpeg can read
[20:05] <wm4> ubitux: to me it looks like every demuxer-level packet has a pts set
[20:05] <wm4> the missing pts probably comes from the parser splitting the packets or so
[20:06] <wm4> in fact, matroska packets can't have no pts
[20:07] <BBB> wm4: was it produced using transmuxing?
[20:07] <BBB> (-vcodec copy)
[20:10] <wm4> asked the user, but no reply yet
[20:16] <wm4> BBB: the user didn't create the file, so it's unknown how it was created
[21:32] <wm4> BBB, ubitux: also the file plays when using libvpx-vp9 as decoder
[21:33] <wm4> without going through extra parsing
[21:34] <nevcairiel> the parser is mandatory for the vp9 decoder, as its required to split the vp9 "super-frames" into individual frames
[21:34] <nevcairiel> libvpx probably handles this internally
[21:35] <wm4> that pretty much goes against the spirit of matroska
[21:36] <nevcairiel> what does matroska have to do with that
[21:37] <wm4> it's the container format
[21:37] <nevcairiel> and why would it care
[21:38] <nevcairiel> the super-frames were intentionally designed to make handling easier for containers, so that every container packet produces a output frame
[21:38] <wm4> and why does the vp9 decoder not eat these frames?
[21:38] <nevcairiel> if they would not exist, you would have frames which never produce any output, messing with timestamps
[21:38] <wm4> *packets
[21:38] <wm4> well, the sample file already has problems with timestamps
[21:39] <nevcairiel> thats probably the parser assigning the packet timestamp to the wrong frame after splitting
[21:40] <nevcairiel> anyway, as far as I understood, it was pulled out of the decoder to allow frame-threading, can't properly frame thread if one packet contains more then one frame
[21:40] <cone-129> ffmpeg.git 03Michael Niedermayer 07master:ebfe154bd522: avcodec/apedec: tmpk==32 is not supported, prevent undefined behavior
[21:40] <cone-129> ffmpeg.git 03Michael Niedermayer 07master:68de2115ca1a: avformat/tee: print errors for each failed bitstream filter
[21:40] <cone-129> ffmpeg.git 03Michael Niedermayer 07master:9b1d7d4fd38f: avformat/oggparsecelt: fix memleak
[21:40] <nevcairiel> in short, want to decode vp9, run the parser first
[21:40] <wm4> then parsing should have been put into libavcodec generic code
[21:44] <nevcairiel> not really, its your responsibility to send the decoder data it understands, if for example some demuxer doesn't properly re-assemble h.264 frames, but sends it one 188 byte TS chunk at a time, you can't expect the decoder to try to fix it for you.
[21:45] <nevcairiel> if you know that your demuxer doesn't do it properly, thats what the parsers are for :)
[21:46] <nevcairiel> (although there may be a bug when remuxing with ffmpeg that it runs the parser over vp9 and breaking the super-frames apart and muxing the result)
[21:48] <nevcairiel> I admit that it may have been nicer for the user if the decoder handled these internally, but I also see the issue that it poses with frame threading
[21:49] <wm4> libavcodec/utils.c could do the parsing
[21:50] <nevcairiel> if someone ever invents a high level API, it  could, but the low level we have right now should not
[21:50] <wm4> anyway, the parser still needs to be fixed to output correct timestamps
[21:51] <wm4> currently using the parser+vpx gives worse results than just vpx...
[21:51] <nevcairiel> worst thing that happens right now is that you end up without a timestamp on the super-frame (as the timestamp was assigned to one of the non-displaying frames), that shouldn't break any sane playback setup
[21:59] <BBB> wm4: so the reason we put it in a parser is the same as the mpeg video decoder (1/2, not 4/h264)
[21:59] <BBB> wm4: it makes design of the decoder easier
[21:59] <BBB> wm4: in this case, the easy thing was multi-threading
[21:59] <wm4> I can understand that
[21:59] <BBB> wm4: I could not design that easily if the parsing was in the decoder
[21:59] <BBB> it sucks, I get that, I really do
[21:59] <BBB> but the tradeoff of not having effective frame-my was worth the suckage
[22:00] <BBB> frame-mt 
[22:00] <wm4> but it's a bit of a problem when the packet format is suddenly changed or constrained (e.g. vpx accepts the "full" packets just fine)
[22:00] <BBB> yeah... we should have files in fate that test that
[22:00] <wm4> so I see the case for doing parsing in the generic libavcodec code (just like libavformat uses the parser)
[22:00] <BBB> so implementors like vlc can easily test that
[22:00] <BBB> it's a layering issue
[22:01] <BBB> parsing in generic libavcodec code has the same issue as doing it in the decoder
[22:01] <BBB> the parsing needs to be done before you call avcodec_decode_videoX()
[22:01] <BBB> so that each packet causes one avcodec_decode_videoX()
[22:01] <BBB> packet being in this case 1 frame
[22:01] <wm4> true, the current API couldn't really handle outputting multiple frames per packet
[22:02] <BBB> sadly...
[22:02] <BBB> I mean, that's a known issue, this isn't new
[22:02] <BBB> like I said, mpeg 1/2 video has this issue
[22:02] <BBB> divx in avi had this issue also if you enabled b frames
[22:02] <BBB> wasn't that called packed b frames
[22:02] <wm4> the other thing is, why does matroska contain "full" packets?
[22:03] <wm4> it should contain one packet per frame - so no parser should be necessary
[22:03] <wm4> and where does the parser get the missing timestamp from anyway?
[22:04] <wm4> so I think the file might have been incorrectly muxed
[22:04] <wm4> though I don't know what the webm spec says about this
[22:05] <BBB> no, vp9 actually wants it this way
[22:05] <BBB> so do you remember packed divx b frames?
[22:05] <BBB> now consider the famous libvpx alt-ref frames
[22:06] <wm4> urgh
[22:06] <BBB> they're invisible frames, not ever output by the decoder, placed like 10-15 frames in the future
[22:06] <BBB> to get a b-frame like benefit in a different way
[22:06] <BBB> so if you make that a frame, you have 16 packets for each 15 frames
[22:06] <BBB> people *hate* this about vp8, you should see trollflamefests no mplayer-devel about this
[22:06] <nevcairiel> vp8/9 really hates timestamp re-ordering, so they come up with alternate ways to be annoying :)
[22:06] <BBB> it's hilarious
[22:06] <BBB> so, they decided to merge the arnr (invisible frame) with the next visible frame
[22:07] <BBB> so it's one-packet-in one-frame-out
[22:07] <BBB> that sucks for decoders, because now packets contain two frames
[22:07] <BBB> I mean, it's not truly 2 frames, you output only one, but you have to decode two sequential frames, which makes frame-my between the two impossible
[22:07] <BBB> that sucks in so many ways
[22:08] <wm4> so, I get that there's a frame that should never be displayed (whether it's a "real" frame or not)
[22:08] <BBB> so I split outside codec layer
[22:08] <BBB> yeah
[22:08] <BBB> it can be a real frame
[22:08] <BBB> then a later frame is just a "take the previous one" reference
[22:08] <BBB> like a b-skip
[22:08] <wm4> still, the decoder seems to output it... or lose the timestamp, or similar
[22:08] <BBB> no the decoder is doing the correct thing, don't worry
[22:08] <BBB> but the decoder doesn't know how to split
[22:09] <wm4> if you run ffprobe -show_frames on the sample frame, you'll see that some frames are missing timestamps
[22:09] <BBB> it expects something else to have done that
[22:09] <nevcairiel> the way the parsers work today is that when they get a timestamp and a packet with multiple frames, they assign the timestamp to the first frame in the packet
[22:09] <nevcairiel> now if the first of these frames is an invisible one, the timestamp is simply lost
[22:09] <BBB> that could be related yes
[22:10] <wm4> so ideally, the parser would return whether a frame has a timestamp or not
[22:11] <wm4> well the parser API is confusing
[22:11] <wm4> I don't even know how it's supposed to return timestamps or other additional info...
[22:11] <nevcairiel> it doesnt return it, you have to get the info from the struct
[22:12] <wm4> oh, I see
[22:12] <wm4> wonderful
[22:12] <nevcairiel> in any case, what the parser does is correct in the generic case, it doesn't know that the frame it just split is never going to be shown and should not have a timestamp assigned
[22:14] <nevcairiel> the entire parser concept is one of the areas which was never revised or cleaned up, and its just not very flexible
[22:28] <BBB> the parsing could certainly use a cleanup
[22:28] <BBB> "this frame is invisible and thus has no timestamp" would be very useful information
[22:29] <BBB> but that's hard mixing-wise, because the muxer has to unparse this again
[22:29] <BBB> (note that we don't do that, so transmuxing is likely broken ATM for vp9)
[22:29] <BBB> I've had that on my list ever since I split it out but I'm just not getting to it :(
[22:32] <cone-129> ffmpeg.git 03Michael Niedermayer 07master:24725f8e096b: avformat/matroskaenc: fix indention level
[22:32] <cone-129> ffmpeg.git 03Michael Niedermayer 07master:66e30a2e65e8: avformat/mpegtsenc: check avformat_new_stream() return
[00:00] --- Sun Apr 27 2014