[Ffmpeg-devel-irc] ffmpeg-devel.log.20140930

[burek]

[00:00] <ubitux> ok so, as we said, the AVFrame contains for audio and video universally recognized forms
[00:00] <ubitux> for subtitle, the equivalent structure is AVSubtitle
[00:00] <akira4> I see.
[00:00] <ubitux> and we decided that the universal representation of subtitles would be... ASS
[00:00] <wm4> and we all hate AVSubtitle!!11
[00:01] <ubitux> wm4: haha
[00:01] <ubitux> akira4: are you familiar with .ass/.ssa, libass or vsfilter?
[00:01] <ubitux> anime subtitles and stuff
[00:01] <akira4> No I'm not.
[00:01] <ubitux> ok
[00:01] <akira4> oh
[00:01] <ubitux> so basically, do you see the karaoke opening in animes?
[00:02] <akira4> Yeah 
[00:02] <ubitux> these are done with ASS/SSA markup
[00:02] <ubitux> so far, it's the most widely used markup for subtitles
[00:03] <akira4> hmm
[00:03] <ubitux> it has a dark history and i won't enter in the details
[00:03] <ubitux> but basically that's the only subtitles markup that has correct rendering engines
[00:03] <wm4> actually, srt is the most common format
[00:03] <ubitux> and that is, vsfilter (original implementation, windows) and libass (linux etc)
[00:03] <akira4> rendering engines?
[00:03] <ubitux> wm4: there is no "srt rendering engine"
[00:04] <ubitux> akira4: yes, translating a markup to the picture itself, the rasterization process
[00:04] <ubitux> basically translating a "<font color=red>foo</font>" into a bitmap with "foo" in red
[00:04] <akira4> cool
[00:05] <ubitux> every unixes now use libass for rendering subtitles
[00:05] <ubitux> and so basically the idea is that the common representation for all subtitles formats is... ass/ssa markup
[00:05] <akira4> okay
[00:06] <ubitux> because it's kind of extremely advanced markup, and it has an engine to actually do the rendering
[00:06] <ubitux> actually, almost every player does this
[00:06] <ubitux> basically, they will convert <font color=red> to the ASS equivalent
[00:06] <ubitux> and send it to libass to get a bitmap
[00:06] <ubitux> and blend it onto the video
[00:07] <akira4> I see.
[00:07] <ubitux> in FFmpeg we try to achieve something similar
[00:07] <ubitux> so in the AVSubtitle structure, we do store the "decoded" markup in ASS
[00:07] <ubitux> there are a lot of very nasty details about this, but i will not talk about it today
[00:07] <ubitux> so anyway
[00:08] <ubitux> in libavcodec you'll find "markup converters" for subtitles
[00:08] <ubitux> if you look at libavcodec/microdvddec.c
[00:08] <ubitux> it will take packets with microdvd markup (so "{c:$ff00000}foo")
[00:09] <ubitux> and convert that string to an ASS markup and store it in AVSubtitle
[00:09] <akira4> just a sec
[00:11] <akira4> is the microdvd_decode_frame part where the conversion is done?
[00:11] <ubitux> (if you want to visualize ASS markup, it looks like this: http://docs.aegisub.org/3.2/ASS_Tags/)
[00:11] <ubitux> akira4: yes
[00:11] <ubitux> akira4: especially the while () block
[00:12] <ubitux> it starts by opening the tags
[00:12] <ubitux> see microdvd_open_tags()
[00:12] <akira4> yep
[00:12] <ubitux> it translates the {c:...} tags into the ass markup {\c&H...}
[00:13] <ubitux> you can have a look at the subrip decoder (badly named srtdec) in libavcodec/srtdec.c
[00:13] <ubitux> but i'm currently making this one sane
[00:14] <ubitux> so you'll see various historical horror
[00:14] <ubitux> i'll explain later what the problem was another time
[00:14] <akira4> haha. Okay
[00:14] <ubitux> but to make thing simple, the timing information was for a long time stored in the data payload
[00:14] <ubitux> and well.....
[00:14] <ubitux> so.
[00:14] <ubitux> these are the main concepts
[00:15] <akira4> usually what is stored in the data payload?
[00:15] <ubitux> the opaque data that only the decoder understands
[00:15] <ubitux> in the case of subtitles, it's the text of the event
[00:15] <ubitux> with the markup
[00:15] <ubitux> the timing information has no place into the data
[00:15] <ubitux> it belongs into the pts and duration fields
[00:15] <akira4> Oh.
[00:15] <akira4> I se
[00:16] <ubitux> it's important because in the ffmpeg chain
[00:16] <ubitux> you can actually play with the timestamps a lot
[00:16] <ubitux> typically you could imagine that in a remuxing you shift the timestamps
[00:16] <ubitux> like, for retiming a .srt
[00:16] <ubitux> you could do that
[00:16] <ubitux> you don't need to understand the markup data for that
[00:16] <akira4> hmm.
[00:16] <ubitux> you just need to demux, alter the AVPacket->pts, and remux it
[00:17] <ubitux> no need for decoding
[00:18] <ubitux> ah, and i forgot the encoding part: for subtitles basically we have a system that parses back the ass markup, with a callback system, so the encoder can print do the inverse operation (ass markup to whatever subtitle markup)
[00:18] <akira4> that's because..decoding would be the part where we take the packet and convert it into a frame and that wouldn't be necessary right?
[00:18] <ubitux> exactly
[00:18] <ubitux> into a AVSubtitle for subtitle, but yes that's the idea
[00:19] <akira4> cool.
[00:19] <ubitux> you don't need to decode the srt to ass (decoder), and convert it back to srt (encoder) before resending the packet to the srt muxer
[00:19] <ubitux> you can just copy opaquely from srt demuxer to srt muxer
[00:19] <akira4> makes sense
[00:19] <ubitux> you can't change the content of the text (because it's not "understandable" except for the decoder), but the AVPacket has the timing informatio that you can just change
[00:20] <ubitux> so, is everything clear enough so far?
[00:20] <akira4> yeah.
[00:20] <ubitux> ok so
[00:20] <ubitux> i don't have much time left for today
[00:20] <ubitux> but i'll give you directions for the qualification task
[00:21] <ubitux> basically what you want to do for now is simply add a demuxer
[00:21] <ubitux> so, you'll need to update libavformat/allformats.c, libavformat/Makefile, and create a file similar to other demuxers
[00:22] <ubitux> libavformat/webvttdec.c might be a good exampl
[00:22] <ubitux> example*
[00:22] <ubitux> and since you don't want to support markup for now
[00:22] <ubitux> you'll make that demuxer output "text" packets
[00:23] <akira4> okay
[00:23] <ubitux> we already have a "text" decoder
[00:23] <akira4> I see.
[00:23] <ubitux> it will "convert" the raw text to ass markup
[00:23] <ubitux> and it actually understands already the '|' as line separator typically
[00:23] <akira4> and about the muxer?
[00:24] <ubitux> don't worry about the muxer
[00:24] <ubitux> you don't really need to write one, no one will use it
[00:24] <akira4> cool.
[00:24] <ubitux> the main user of this are the players
[00:24] <ubitux> and they just want to be able to support all kind of subtitles formats
[00:24] <ubitux> no one is interesting in creating these old broken subtitles formats anymore
[00:24] <akira4> hmm
[00:24] <ubitux> the codec id you are interested in is AV_CODEC_ID_TEXT
[00:25] <ubitux> that's what your demuxer should output
[00:25] <ubitux> if you do that, you should be able to do ffmpeg -i in.stl out.srt
[00:25] <ubitux> (since you have a decoder, and a subrip encoder already exists, it will work just fine a create a proper srt file)
[00:26] <ubitux> if you want examples of other formats outputing AV_CODEC_ID_TEXT texts, you can try: git grep AV_CODEC_ID_TEXT in the libavformat directory
[00:26] <akira4> hmm
[00:27] <ubitux> actually, just look at libavformat/aqtitledec.c
[00:27] <ubitux> anyway
[00:27] <ubitux> after modifying libavformat/allformats.c and libavformat/Makefile
[00:27] <ubitux> don't forget to re-run configure
[00:28] <ubitux> (--cc='ccache cc' is your friend)
[00:28] <akira4> I won't :)
[00:28] <ubitux> i think you have enough information for now to start experimenting
[00:28] <akira4> thank you so much for the help ubitux :)
[00:28] <ubitux> i'll be available in a few hours if you have more questions
[00:29] <ubitux> feel free to ask even if i'm afk, i'll backlog
[00:29] <ubitux> good luck
[00:29] <akira4> cool. thanks :) 
[02:39] <jamrial> anyone know where i can find a stereo truehd/mlp sample longer than 1 minute?
[02:39] <jamrial> or an encoder to make my own
[02:51] <rcombs> not sure on stereo
[02:53] <rcombs> apparently the 'John Lennon Imagine' BD has a TrueHD 2.0 track
[02:56] <jamrial> yeah, DVD-Audio uses mlp as well, stereo, quad or 5.1
[02:56] <jamrial> but i was hoping to find some freely available sample
[05:28] <cone-201> ffmpeg.git 03Michael Niedermayer 07master:61c068308017: avcodec/mpegvideo_enc: Fix leak on bitstream buffer reallocation
[05:35] <pross> gcc asm question: "movw  0(%1), %%eax \n\t"   <-- okay, i get what this is doing
[05:35] <pross> but how do i achieve this:
[05:35] <pross>       "movw  %al(%1), %%eax \n\t"   <-- offset
[05:36] <pross> [^^ should be %%al]
[06:05] <jamrial> pross: the syntax for something like that is "mov (%%eax, %%ebx), %%ecx \n\t"
[06:07] <pross> thank you. i just figured it out (by writing it in yasm and dissasembling it with objdump).
[06:08] Action: pross god bless them at&t engineers. they really are 'special'.
[06:08] <jamrial> heh
[06:10] <rcombs> one of those times when you really wish you had Intel syntax?
[06:13] <pross> i guess i grew up with intel
[06:13] <pross> its my mother tongue
[06:15] <rcombs> ditto
[09:32] <ubitux> i just realized we do actually really support the column thing in ass split
[10:30] <ubitux> there are so much things to do wrt text subtitles...
[10:30] <nevcairiel> not getting bored then!
[10:30] <pross> was gonna say... every day there is a sub story in here
[10:30] <ubitux> :)
[10:31] <ubitux> i really need to fix that decoded state of text subtitles
[10:31] <ubitux> it's really too broken
[10:31] <pross> i will think of you next time i turn on subtitles
[10:32] <ubitux> well, let's first iterate with the pending patchset
[11:15] <ubitux> wm4: so, assuming i'm moving to the muxed ass markup in text subtitles decoder
[11:15] <ubitux> do you think it's ok to have the readorder actually being the packet position?
[11:17] <ubitux> in case of encoding, the encoder will make them continuous again, it just affects the decoding where you would pipe that to libass
[11:23] <ubitux> i wonder how i'll be able to deal with >4GB files though
[11:26] <ubitux> oh, maybe i could actually be nasty and use the dts field for that
[11:27] <ubitux> like, making demuxers store the "id" in AVPacket.dts
[11:27] <av500> what id?
[11:27] <ubitux> readorder
[11:28] <av500> is that not continuous?
[11:28] <ubitux> it should
[11:28] <ubitux> but in the decoder context, that doesn't work
[11:29] <ubitux> the decode will need to output an ASS event, with the readorder set
[11:29] <ubitux> decoder*
[11:29] <ubitux> we can have it in the decoder context and increment it
[11:29] <ubitux> because it will break in case of seeking
[11:30] <ubitux> so the idea is to transmit that information from the demuxer to the decoder throught the packets, in the dts field for instance
[11:30] <ubitux> i was planing to use the AVPacket.pos, but unfortunately it can be 64-bit
[11:30] <ubitux> and the readorder is probably read as 32-bit in many places
[11:31] <ubitux> so what i can do is make sure the demuxers actually set AVPacket.dts to an monotonous incrementing value
[11:31] <cone-34> ffmpeg.git 03Alexander Drozdov 07master:08ccc474b73a: RTMP: fix FD leak in rtmp_open()
[11:31] <ubitux> in the case of standalone text subtitles demuxer that's trivial to do, in the case of matroska for instance, you'll have to read it from the payload
[11:32] <ubitux> does that sound sane or not really?
[11:38] <av500> isnt the readorder part of the payload fields?
[11:38] <av500> like the text itself
[11:38] <ubitux> yes, in the case of ass muxed in mkv
[11:39] <ubitux> but that doesn't work for all the other subtitles formats
[11:39] <ubitux> we have to make sure that this information can be constructed within other containers
[11:39] <ubitux> that might not be the case all the time
[11:46] <benoit-> hi
[11:46] <benoit-> does anyone know if we have any reason to not use pkg-config for x264 configuration?
[11:46] <ubitux> hahahaha
[11:46] <ubitux> benoit-: carl is blocking my patch since months
[11:46] <ubitux> i've been fighting with him for a long time
[11:47] <benoit-> ubitux: see also 4321e507564b2d8a5dfc630f8d6fefba6fdc1e92
[11:47] <ubitux> feel free to raise your voice in the thread 
[11:47] <ubitux> benoit-: i know right
[11:47] <ubitux> benoit-: i sent a patch for this a loong time ago
[11:47] <benoit-> I'll try to find the thread to have his arguments then
[11:48] <benoit-> ubitux: I just fixed it in my tree, and thought it was about impossible it was not already done :)
[11:48] <ubitux> http://ffmpeg.org/pipermail/ffmpeg-devel/2014-August/161120.html
[11:48] <ubitux> and all the following thread
[11:48] <TimNich> michaelni: I have been trying to find my link to the DPP samples, but so far epic fail..
[11:48] <ubitux> benoit-: really it's insane, i should ping him again, but i've lost hope motsly now
[11:49] <benoit-> ubitux: I'll have a look and ping, then
[11:53] <ubitux> benoit-: http://ffmpeg.org/pipermail/ffmpeg-devel/2014-August/161573.html
[11:53] <ubitux> this post is the "main" patch i believe
[12:00] <benoit-> ubitux: why don't we take the issue the other way round, and consider pkg-config as a fallback?
[12:01] <ubitux> because it sucks
[12:01] <ubitux> it means that it will override pkg-config configuration
[12:02] <ubitux> example: if you want to link against the x264 in a custom location instead of the one in the system
[12:02] <ubitux> you can't
[12:02] <ubitux> because it will not fallback since the system one is found
[12:02] <ubitux> totally unexpected behaviour
[12:02] <ubitux> benoit-: it's simple, there is no way to have both the hack and the sane version working properly
[12:02] <benoit-> yep
[12:03] Action: ubitux going afk for a moment
[12:04] Action: benoit- too
[12:46] <ubitux> wm4: i see you do flush on seeking in libass in mpv
[12:46] <ubitux> and it might be necessary to maintain this behaviour
[12:46] <ubitux> if we do make the decoders responsible for setting readorder
[13:20] <cone-34> ffmpeg.git 03Benoit Fouet 07master:1cf4d2e9be17: avcodec/h264_mp4toannexb_bsf: add a case when only SPS/PPS is in the stream.
[13:20] <cone-34> ffmpeg.git 03Benoit Fouet 07master:d5ddcb5f8e7b: avcodec/h264_mp4toannexb_bsf: use the given padding in h264_extradata_to_annexb().
[15:28] <benoit-> hey, I've added support for j2k muxing to our mxf muxer, and I'd like to test it against a "reference" player. Which player should I try?
[15:45] <mateo`> benoit-: thanks for your patch, i'll review it in a bit
[15:46] <wm4> ubitux: I flush only if I have to
[15:47] <mateo`> funny that i had a similar patch for that, and never send it :(
[15:47] <mateo`> https://github.com/mbouron/FFmpeg/commit/0da40621770a5a68d4e014bd86779a4ece945b5e
[15:49] <ubitux> col 8, id 1/3 and 3/3 are different (0x07 instead of 0x01)
[15:50] <ubitux> benoit-: you have 0x0c in caps btw
[15:53] <benoit-> ubitux: 1/3 and 2/3 are different. 2/3 uses generic container
[15:53] <ubitux> i meant in comparison to the patch from mateo` 
[15:53] <benoit-> oh OK
[15:55] <benoit-> This is the version of the jpeg2000 part
[15:55] <benoit-> it's version 7
[15:55] <benoit-> from RP224
[16:10] <mateo`> benoit-: i'm pretty sure those values existed back in the revision 1 day
[16:11] <mateo`> benoit-: sometimes some proprietary player are picky about the ULs and won't recognise because of the version
[16:11] <benoit-> mateo`: no doubt, but our MXF demuxer expects the version to be 7. We could also fix the demuxer so that it does not care about version though...
[16:11] <mateo`> oh right
[16:12] <benoit-> I'm not sure what policy we should follow here... I'm pretty sure we'll find players in pretty much all situations (unable to play with version X or Y)
[16:13] <mateo`> i would say, keep it 7 for now
[16:14] <benoit-> mateo`: I don't have access to other versions, so I'm not 100% sure the ULs would make sense for other versions anyway, so I tend to prefer keeping version 7 anyway :)
[16:15] <mateo`> i don't have access to this kind of stuff anymore :) so yeah keep version 7 as in the specs
[16:36] <Compn> benoit- : i  dont know of any j2k in mxf software to test with :p
[16:38] <benoit-> Compn: I found one on The Internet
[16:39] <benoit-> though it may very well be based on ffmpeg :)
[16:45] <benoit-> mateo`: I compiled bmx tools, and they don't seem to have support for j2k
[17:28] <kierank> benoit-: https://twitter.com/dericed/status/516962642900971520
[17:28] <kierank> was trolling a bit
[17:28] <kierank> but he gave a serious answer
[18:50] <akira4> ubitux, I had a few doubts regarding the qualification task  
[19:18] <ubitux> akira4: yes?
[19:18] <akira4> how do we decide on the structure of the context  for the demuxer?
[19:19] <ubitux> the local context for the demuxer, what you would put in it?
[19:19] <ubitux> you probably just need the FFDemuxSubtitlesQueue for now
[19:21] <akira4> what is the AVclass used for? 
[19:21] <ubitux> the AVClass is only necessary if you have demuxers options
[19:21] <ubitux> you're looking at webvttdec i suppsoe?
[19:22] <ubitux> or maybe aqtiltle?
[19:22] <akira4> both of them actually
[19:22] <ubitux> both of them have an option tab
[19:22] <ubitux> (at the bottom of the file)
[19:22] <ubitux> if you want to have such option, you'll need that class in first position
[19:23] <akira4> so we don't need the options now so its alright not to use the AVclass?
[19:23] <ubitux> yes
[19:23] <akira4> okay
[19:23] <akira4> also
[19:23] <akira4> I saw that the files had a similar structure
[19:23] <ubitux> akira4: look at libavformat/vplayerdec.c
[19:24] <ubitux> it's one of the simplest
[19:24] <akira4> hmm.
[19:24] <ubitux> this one does only have the subtitle queue
[19:25] <akira4> yeah
[19:25] <akira4> in the end of the files
[19:25] <akira4> there is some AVInputOuput
[19:25] <akira4> Format
[19:25] <akira4> how does that work?
[19:25] <ubitux> look at libavformat/allformats.c
[19:26] <ubitux> and REGISTER_DEMUXER() macro
[19:27] <akira4> is it used for registering the demuxer?
[19:27] <wm4> it's how the demuxer implementation is found and called
[19:27] <wm4> so yes
[19:28] <akira4> I see.
[19:28] <wm4> utils.c provides the user-visible  libavformat API (or most of it)
[19:28] <wm4> and utils.c calls the AVInputOuput callbacks
[19:28] <wm4> actually it's AVInputFormat
[19:28] <ubitux> AV{Input,Output}Format yes
[19:29] <akira4> hmm.
[19:29] <akira4> oh yeah, what are the probes in the beginning of the files used for?
[19:30] <ubitux> they're called to check if it's that format or not
[19:30] <ubitux> every probe function of every demuxer will be called on the file
[19:30] <ubitux> until it matches one
[19:30] <ubitux> (to make things simple)
[19:30] <wm4> well, or on a part of the file (like the first 64 bytes)
[19:30] <ubitux> they're is actually a scoring system, if you're not exactly sure
[19:31] <ubitux> yeah right, it's generally just the first bytes
[19:31] <akira4> I see.
[19:31] <akira4> I was going through the .stl file format and the commands in the files start with a $
[19:32] <akira4> I dont really need to touch them in the demuxer right?
[19:32] <ubitux> for now leave them alone
[19:32] <ubitux> you'll actually have to export them as side data in case of that format, but it can be done later
[19:33] <akira4> alright.
[19:36] <ubitux> awesome, stl doesn't have a special header
[19:36] <ubitux> yet another weird probing to do :p
[19:36] <akira4> so then what about the read_header function?
[19:36] <wm4> what does the format look like?
[19:36] <ubitux> wm4: http://documentation.apple.com/en/dvdstudiopro/usermanual/index.html#chapter=19%26section=13%26tasks=true
[19:37] <ubitux> akira4: the read_header is where you will put the whole read of the file actually
[19:37] <ubitux> in the case of the subtitles format, as they are small and often misordered (it's not really a stream like audio or video), you index the whole file
[19:37] <ubitux> that's what the subtitles queue api is for
[19:38] <akira4> Oh. That's interesting.
[19:38] <ubitux> the subtitles queue api does various things for you
[19:38] <ubitux> it sorts the events
[19:38] <ubitux> and it makes easier the seeking
[19:38] <wm4> ubitux: looks terrible
[19:39] <ubitux> wm4: yeah, it's just like all the others actually
[19:39] <ubitux> wm4: a few years ago people were not writing windows managers, but were designing tons of stupid subtitles formats
[19:40] <ubitux> i wonder what's the trend nowadays, it seems the windows manager fashion thing ended somehow
[19:42] <wm4> did it
[19:42] <wm4> maybe nobody feels motivated because they think wayland will kill X
[19:42] <ubitux> did you design a subtitles format?
[19:42] Action: ubitux prepars libavformat/mpvdec.c
[19:42] <wm4> no
[19:43] <wm4> just a edl format
[19:43] <wm4> if ASSv5 actually happens, I participated in its design, I guess
[19:43] <ubitux> you should propose the edl format to ffmpeg 
[19:43] Action: J_Darnley makes a reference to that xkcd comic
[19:43] <wm4> ubitux: but ffmpeg has its own edl format
[19:43] <ubitux> really?
[19:44] <ubitux> .concat? :p
[19:44] <wm4> yes
[19:44] <ubitux> it's more than primitive
[19:44] <ubitux> nicolas wanted to make something more complex
[19:44] <ubitux> but if you can suggest a spec already...
[19:44] <wm4> I don't know why it'd have to be more complex
[19:44] <wm4> the mpv format is also very simple
[19:45] <ubitux> it needs start, end and repeat
[19:45] <ubitux> and for start/end maybe support chapters
[19:45] <ubitux> i guess that's all you would need?
[19:46] <wm4> why would you need repeat
[19:46] <akira4> so then how exactly do we write the probe function for a subtitle format ? (the whole subtitle talk is scaring me now :-/ )
[19:46] <akira4> sorry didnt mean to barge in.
[19:46] <ubitux> michaelni: s/seperator/separator/g
[19:46] <wm4> I'd scan the data for timestamps
[19:47] <ubitux> wm4: read every line, if it starts with a comment, read next line
[19:47] <ubitux> aer
[19:47] <ubitux> akira4 ^
[19:47] <wm4> and yes it's annoying that often enough the probe function duplicates some of the parser code
[19:47] <ubitux> if it starts with a $ read next line
[19:47] <ubitux> if it's an empty line, read next line
[19:47] <ubitux> if it matches the timestamps format line, return probe max
[19:48] <ubitux> if nothing of the above, stop
[19:48] <ubitux> something along these lines, i'd say
[19:49] <akira4> doe that mean that the  probe and read_header function both go through the entire file then? 
[19:49] <ubitux> not really
[19:50] <ubitux> the probe function receive a buffer with the beginning, sometimes it's recalled with a larger buffer (iirc)
[19:50] <ubitux> in your case the probe function will stop reading into that buffer as soon as it doesn't recognize something
[19:50] <ubitux> or as soon as it recognizes something it's sure to be part of a stl file
[19:50] <wm4> the probe function returns a certain score
[19:51] <wm4> which is basically the "confidence" that this format matches
[19:51] <wm4> if the score is over a certain threshold, and no other formats have a higher score, this format is picked
[19:51] <akira4> I see.
[19:52] <wm4> the details what score does what are weird and obscure, though
[19:52] <ubitux> yeah typically if the buffer only contains $foo = bar (if the header is bigger than the probe buffer typically), you can say that it looks like a stl file
[19:52] <ubitux> but you're not sure
[19:52] <ubitux> so you won't return a probe max
[19:52] <ubitux> but it might be :)
[19:52] <ubitux> so not 0 either
[19:53] <akira4> I don't think I follow the header is bigger than the buffer part. 
[19:54] <ubitux> if your .stl file has really a ton of $foo = bar lines at the beginning
[19:54] <ubitux> since the probe callback will be called with a buffer of the beginning of the file, it might not contain any timed line
[19:55] <akira4> hmm.  
[19:57] <akira4> so then the return value won't be the max of the score in this case (if there are a lot of commands in the .stl file ?) 
[19:59] <ubitux> yes
[19:59] <ubitux> an arbitrary value between 0 and the max, maybe max/2 or max/3
[19:59] <ubitux> we might have another macro with such value
[20:01] <akira4> I see.
[20:27] <cone-34> ffmpeg.git 03Clément BSsch 07master:55180b3299c6: Kill timed SRT
[20:27] <cone-34> ffmpeg.git 03Clément BSsch 07master:0eb4a428122b: avcodec/srtdec: use AVBPrint API
[20:31] <ubitux> mmh what's done in libzvbi sounds evil
[20:35] <cone-34> ffmpeg.git 03Michael Niedermayer 07master:cda5d89defaf: avcodec/bitstream_filter: Use av_bitstream_filter_next() instead of direct access in av_bitstream_filter_init()
[20:35] <cone-34> ffmpeg.git 03Jeffrey Wescott 07master:07de0db74b56: avformat/flvenc: When using "-c:d copy", don't require a codec for passing the AMF metadata through from input to output.
[20:52] <wm4> would have been nice if libavutil/eval.h could support strings...
[20:56] <akira4> ubitux,  would you have a look at the probe function that I've written ? here's the link : http://pastebin.com/7JpAkpzc
[21:01] <ubitux> does it work?
[21:02] <ubitux> you can return AVPROBE_SCORE_MAX for now and work on the main code
[21:02] <ubitux> you can do the probing later
[21:02] <ubitux> note that the ff_text thing is just to handle utf-16
[21:04] <akira4> I see. So there is no need for the ff_text_read and all?
[21:04] <ubitux> unless you find an utf-16 sample, it might not be needed
[21:04] <ubitux> of course, you can still use it, just in case
[21:05] <akira4> okay. I referenced the srtdec.c so thought maybe it was necessary
[21:06] <ubitux> you should probably follow the vplayer template
[21:06] <akira4> alright. I'll do that. Thanks.
[21:31] <ubitux> i think i'm almost to the point where i can start the api move
[21:31] <ubitux> but everytime i'm starting it i hit another freaking insanity in the chain
[21:34] <rcombs> Y'know what'd be good? Charset detection.
[21:35] <ubitux> rcombs: then go on crusade with wm4 and Nicolas
[21:35] <ubitux> i don't want to be part of this :p
[21:36] <wm4> charset detection is hard
[21:36] <ubitux> we already detect utf-8 and the 2 utf-16 thanks to wm4
[21:36] <rcombs> it'd basically need to be squashed in the avio layer
[21:36] <wm4> no that'd be even worse
[21:36] <ubitux> -sub_charenc ftw
[21:36] <ubitux> just add a "auto" in there
[21:37] <wm4> ubitux: actually, yes
[21:37] <ubitux> i wonder if iconv could do that...
[21:37] <wm4> as long as the decoder can buffer enough data to make a good guess
[21:37] <wm4> I'd be in favor of this because it doesn't need reopening of the file
[21:37] <rcombs> yeah, that
[21:37] <wm4> ubitux: iconv doesn't do detection
[21:37] <ubitux> [~]- iconv -l |grep -i auto
[21:37] <ubitux> [~]  
[21:37] <ubitux> :(
[21:38] <ubitux> it could make a guess, sometimes... maybe.
[21:38] <rcombs> ubitux confirmed for commie
[21:38] <ubitux> confirmed what?
[21:38] <wm4> there's enca (it's shit), libguess (requires giving a language, plus it's shit), mozilla's chardet (probably not that bad but)
[21:38] <rcombs> -
[21:38] <ubitux> oh :)
[21:38] <rcombs> wm4: universalchardet is really limited
[21:39] <wm4> mozilla's code has been separated into several libs so that's inconsistent too
[21:39] <rcombs> doesn't handle hebrew or arabic at all
[21:39] <rcombs> ironic given the name
[21:39] <wm4> well, detection is hard
[21:39] <wm4> does arabic use legacy charsets?
[21:39] <rcombs> it seems the best thing to do is to use all of them until one works
[21:39] <rcombs> here, lemme toss up a sample
[21:39] <rcombs> (which works with libguess)
[21:40] <wm4> I think best you can do is presenting the user which a choice of likely encodings
[21:41] <rcombs> http://puu.sh/bUkV8/10a4bf2db2.srt
[21:41] <rcombs> wm4: which doesn't work if you're running programmatically
[21:41] <ubitux> so what is the encoding here?
[21:42] <rcombs> but I guess that'd work if `-y` meant "try the first one lol"
[21:42] <rcombs> ubitux: I& forget, lemme check
[21:42] <ubitux> holy shit subrip and ass tags into the same file
[21:42] <rcombs> yup, they exist in the wild
[21:42] <wm4> ubitux: my (your) old srt decoder allows it
[21:42] <wm4> well it didn't
[21:42] <wm4> but now it does
[21:43] <ubitux> really?
[21:43] <ubitux> i did actually add code to escape the { }
[21:43] <wm4> yes, it's semi-common
[21:43] <rcombs> I just know universalchardet returns something entirely wrong and libguess gets it
[21:44] <wm4> ubitux: https://github.com/mpv-player/mpv/blob/master/sub/sd_srt.c#L421
[21:44] <ubitux> this makes me sad wm4 
[21:45] <ubitux> anyway, what if you remove the { } escaping
[21:45] <wm4> good luck trying to explain entire communities how wrong this is
[21:45] <rcombs> uh, CP1256
[21:45] <ubitux> libass will just display if it's recognized, or the text itself if not, no?
[21:45] <rcombs> how did you escape the {}s?
[21:45] <ubitux> with \
[21:46] <rcombs> I'm like 42% sure that doesn't do anything
[21:46] <ubitux> yeah right, i actually added a code in libass for that
[21:46] <rcombs> hah
[21:46] <ubitux> i'm not even sure it was necessary 
[21:46] <rcombs> that's pretty great
[21:46] <wm4> rcombs: vsfilter doesn't understand it
[21:46] <wm4> and in fact there are real life files that are rendered incorrectly because of it
[21:47] <rcombs> WOW
[21:47] <ubitux> honestly, you could drop it
[21:47] <ubitux> assuming it will render known tags and displays the text otherwise
[21:47] <wm4> I tried to get in agreement with xy-vsfilter about how to escape, but it went nowhere
[21:47] <rcombs> nope, unrecognized text in a {} block is a comment
[21:47] <rcombs> and is not displayed
[21:47] <ubitux> ah, right!
[21:48] <ubitux> i guess that's why i did it then
[21:48] <wm4> yeah
[21:48] <wm4> and why I picked this heuristic
[21:48] <ubitux> so people wanting to do stuff like ~o{ lol }o~
[21:48] <ubitux> could actually get them displayed correctly
[21:48] <rcombs> ASSv5 does support \-escaping {}'s, btw
[21:48] <ubitux> i forgot about the comment thing
[21:48] <ubitux> rcombs: you should add it
[21:48] <ubitux> maybe {{ }} or something
[21:48] <rcombs> no, just \{ and \}
[21:49] <ubitux> misread, ok
[21:49] <rcombs> once we have a finished spec, ffmpeg should switch to ASSv5 being the internal format and convert ASS to and from it
[21:49] <ubitux> yes
[21:49] <rcombs> and just assume that ASS tags are valid in SRT because that's how they exist in the wild
[21:50] <ubitux> i'm actually working on making it output ASSv4 for now, and that's already hell
[21:50] <wm4> rcombs: how about creating a renderer first
[21:50] <rcombs> wm4: this is ~future~
[21:50] <ubitux> yes, the ffmpeg srt decoder might need some adjustment
[21:50] <ubitux>         case '{':    /* skip all {\xxx} substrings except for {\an%d}
[21:50] <ubitux>                         and all microdvd like styles such as {Y:xxx} */
[21:50] <ubitux> hahaha
[21:50] <ubitux> i forgot that
[21:50] <ubitux> microdvd tags in srt
[21:50] <rcombs> bahahaha
[21:51] <ubitux> which also starts with {
[21:51] <wm4> wtf
[21:51] <ubitux> :D
[21:51] <rcombs> fuck everything
[21:51] <wm4> I think we can thank vsfilter
[21:51] <wm4> again
[21:51] <wm4> or like I like to call it
[21:51] <wm4> vshitler
[21:51] <ubitux> well vsfilter might not be responsible for that
[21:51] <ubitux> i believe perl script converting from one format to another are responsibles
[21:52] <ubitux> "converting"
[21:52] <ubitux> changing the timestamps thing but not the markup
[21:52] <rcombs> so the future optimal situation would be: SRT parser can parse pretty much any type of tag, internal packet format is ASSv5 (which can escape shit), and we support charenc auto-detection
[21:53] <rcombs> that last one seems the most attainable, by buffering a bunch of text and chucking it at enca, libguess, and universalchardet and seeing what works
[21:53] <wm4> ffmpeg also wants to hardsub demuxed subs
[21:53] <rcombs> (or prompting if it's ambiguous and there's no -y)
[21:53] <ubitux> wm4: it will work smoothless as soon as the decoded form is changed
[21:53] <rcombs> wm4: that needs subtitles in avfilter and ubitux is working on it
[21:53] <ubitux> because it will be in the libavutil api, so public and stuff
[21:54] <ubitux> and so easier to plug into libavfilter
[21:54] <rcombs> ubitux: also avfilter needs a way to provide all attachment streams to a filter
[21:54] <ubitux> yeah
[21:55] <rcombs> I have hacks to do all these things but they're bad and they make me feel bad
[21:55] <ubitux> for that you need... data support in libavfilter
[21:55] <ubitux> nicolas talked about it, and i'll leave him handle this
[21:55] <ubitux> he seems to have a good overview on how sparse stuff like this can be handled in the libavfilter pipeline
[21:56] <rcombs> I wonder what the syntax would be& `[0:t:*]`?
[21:56] <ubitux> probably :P
[21:56] <cone-34> ffmpeg.git 03wm4 07master:c8422f04a325: avformat/aviobuf: fix avio_flush() for read streams
[21:56] <rcombs> ("t" for attachment because "a" is taken? Or are attachment streams similar enough internally to data streams that it'd be `[0:d:*]`?)
[21:57] <wm4> wee
[21:58] <rcombs> it's a good thing we haven't yet seen things like non-UTF-8 subtitles muxed in MKV
[21:58] <rcombs> that'd be a fine mess
[21:59] <ubitux> well
[21:59] <ubitux> we found utf-16 ass files in the wile
[21:59] <ubitux> wild
[21:59] <wm4> but muxed?
[21:59] <ubitux> if you have a muxer not making a difference with utf-8...
[21:59] <ubitux> it might happen!
[21:59] <rcombs> at least UTF-16 is relatively easy to detect
[22:00] <ubitux> matroska mandates utf-8 though
[22:00] <rcombs> >mandates
[22:00] <ubitux> but specs and reality... yeah
[22:00] <wm4> matroska specs especially
[22:00] <rcombs> ASSv5 should mandate you not be a fuckwit
[22:01] <rcombs> see how that goes for us
[22:02] <wm4> probably badly
[22:02] <ubitux> hey btw
[22:02] <rcombs> The FILE MUST be encoded in UTF-8, AND the USER MUST NOT be a FUCKWIT
[22:03] <ubitux> can ASSv5 be proposed as a real RFC?
[22:03] <rcombs> like, too IETF?
[22:04] <ubitux> yes
[22:05] <rcombs> what would that do for us?
[22:06] <ubitux> official standard, could prevent stuff like webvtt or similar to be that much developed
[22:07] <ubitux> i don't know, it has kind of a few benefits
[22:07] <ubitux> maybe more implementations and stuff like that
[22:07] <rcombs> sure, why not
[22:08] <wm4> ubitux: well, I bet that would be rejected, because it's not scriptable and doesn't support fucked up usecases only enterprise web devs could come up with
[22:09] <ubitux> really?
[22:10] <rcombs> that's why we have I-Ds!
[22:32] <akira4> ubitux, I tried running the stldec.c file that I made and apart from the srt timestamps and text I also get the timestamps of the original stl file. 
[22:55] <akira4> also get_pts  function is different from read_ts function right?
[23:11] <ubitux> these are just local function, name it as you wish
[23:14] <hawken> Hi :)
[23:15] <akira4> the fucntions give the duration b/w the timestamps?
[23:15] <akira4> *functions
[23:17] <hawken> Okay so I finally got most of Britz's MVC code cleaned up, and I made a patch and applied it to the old verison of ffmpeg
[23:17] <hawken> Available https://github.com/hawken93/FFmpeg
[23:18] <hawken> I'm going to see if I can make it compile and maybe also work before I try to rebase it to a modern version of ffmpeg
[23:18] <ubitux> akira4: they read the timestamps and set it in the packet yes
[23:20] <akira4> In case of stl, there is a start timestamp and an end timestamp
[23:20] <akira4> so do I return the difference or the start timestamp?
[23:25] <ubitux> akira4: what do you think? :)
[23:25] <akira4> should be the difference according to me 
[23:25] <hawken> ubitux: do you know what Britz can have meant by "uint" - it gives a compilation error. I changed it to uint32_t but I hope I got the right amount of bits. It refers to pps_id, sps_id mostly
[23:25] <ubitux> akira4: then try it
[23:26] <ubitux> you're on the mvc thing?
[23:26] <hawken> It compiles! Wohooooo
[23:26] <hawken> mhm
[23:26] <hawken> let's check how many seconds it takes to crash
[23:27] <ubitux> moving to uint is indeed refering to the uint*_t types instead of unsigned and stuff
[23:27] <hawken> I just assumed 32 bits :P
[23:28] <ubitux> gtg, sorry, 'night ppl
[23:28] <hawken> 'night :)
[23:30] <hawken> AAAnd it crashed right away
[00:00] --- Wed Oct  1 2014