[Ffmpeg-devel-irc] ffmpeg-devel.log.20190129

[burek]

[05:36:03 CET] <cone-274> ffmpeg 03Gyan Doshi 07master:2e2b44baba57: avformat/http: clarify that ffmpeg will attempt to add missing CRLF
[09:40:22 CET] Action: JEEB grmbl grmbls about figuring out which parts of a string in ARIB captions are ruby text
[09:40:34 CET] <JEEB> should read the spec and figure out if libaribb24 already exports that
[09:40:40 CET] <JEEB> (it didn't seem to)
[09:40:59 CET] <JEEB> and then implement it if it doesn't
[09:41:32 CET] <JEEB> because all ruby text seems to be exported in the beginning of the line, which leads to funky results
[10:49:20 CET] <rcombs> so here's an MP3 file that misidentifies as MPEG-PS, but only because of the size of the ID3v2 tag (if I remove it it identifies as MP3) https://puu.sh/CE1qZ/5e09c24034.zip
[10:49:50 CET] <JEEB> so the extension points are not high enough?
[10:49:54 CET] <rcombs> yeah, I guess so
[10:50:11 CET] <rcombs> MP3 gives 51, MPEG gives 52
[10:51:30 CET] <rcombs> well, MP3 gives 12 until the probe buffer expands past the end of the ID3 tag, and then it gives 51
[10:51:36 CET] <rcombs> but at the same time MPEG gives 52
[10:51:47 CET] <rcombs> which are both above the threshold so it picks MPEG
[10:53:33 CET] <nevcairiel> we try to not rely on extension matches much at all
[10:53:48 CET] <nevcairiel> the mpeg-ps probe f unction probably should be more strict somehow
[10:53:50 CET] <JEEB> yes, if possible it shouldn't be used
[11:00:04 CET] <nevcairiel> if the id3 tag fully exhausts the probe buffer, there really isnt anything to be done other then increasing the buffer however
[11:00:17 CET] <JEEB> yea
[11:02:20 CET] <nevcairiel> one could imagine a mechanism that allows a probe function to say something like "this might be a file of mine, but there is a tag block here of X bytes, can you skip that and send me more after?"
[11:23:46 CET] <thardin> isn't there already a way for probe functions to say "more data pls"?
[11:24:46 CET] <thardin> but maybe that doesn't help if the mpeg-ps prober is being too lax
[11:26:48 CET] <rcombs> "more data pls" is just returning a low score, isn't it?
[11:27:23 CET] <rcombs> nevcairiel: the problem isn't really that the tag fully exhausts the buffer
[11:28:04 CET] <rcombs> on the iterations when that's the case, MP3 returns a low score below the threshold, MPEG returns 0, and it tries again
[11:28:55 CET] <rcombs> but then on the final iteration, when the buffer is large enough to get past the ID3 tag, the remaining data past the tag happens to probe 1 higher on MPEG than MP3
[11:30:36 CET] <rcombs> the final probe iteration is on 65536 bytes, of which 54310 are the ID3 tag and the remaining 11226 are actual MP3 frame data
[11:30:59 CET] <rcombs> which ends up probing at 52 in the mpeg routine
[11:31:39 CET] <rcombs> if you strip off the ID3 tag, then the MP3 probe gets to 51 at only 8192 bytes, and at that size the MPEG probe only gives 25
[11:34:39 CET] <thardin> strange
[11:35:36 CET] <nevcairiel> / 1 more than mp3
[11:35:41 CET] <nevcairiel> comment from mpeg.c =p
[11:36:03 CET] <thardin> AVPROBE_SCORE_EXTENSION / 2 + 1; // 1 more than mp3
[11:36:06 CET] <thardin> ye
[11:36:25 CET] <rcombs> it's not actually that case
[11:36:33 CET] <rcombs> it's the "/* PES stream */" one
[11:36:35 CET] <nevcairiel> mpeg-ps is just an annoying format
[11:36:44 CET] <rcombs> that it is
[11:36:51 CET] <nevcairiel> it has basically no sync markers
[11:36:59 CET] <nevcairiel> or reliable magic numbers
[11:37:17 CET] <thardin> this "invalid" business looks suspicious
[11:37:49 CET] <rcombs> vid=9, audio=0, sys=0, invalid=0
[11:38:05 CET] <rcombs> don't ask me why it thinks there are 9 video somethingorothers
[11:38:43 CET] <rcombs> the definition of "invalid" here is pretty narrow
[11:39:25 CET] <nevcairiel> all it looks for is a very narrow bit pattern
[11:39:37 CET] <rcombs> which is also perfectly valid MP3
[11:40:14 CET] <rcombs> maybe MP3 should give a higher score than 51 when it sees lots of perfectly valid MP3 packets (and nothing else)?
[11:40:31 CET] <nevcairiel> it could also give the id3 tag some value
[11:40:36 CET] <nevcairiel> since thats pretty common in mp3 files
[11:41:02 CET] <rcombs> and pretty uncommon in mpeg
[11:42:36 CET] <thardin> the mpegps prober could give a score of zero if it finds an ID3 tag maybe?
[11:42:57 CET] <nevcairiel> i'm sure some crazy person put those together
[11:48:53 CET] <rcombs> this all seems fragile enough that I'm a bit afraid to touch it
[11:49:19 CET] <nevcairiel> the probing of those identifier-free formats is a ll over the place
[11:49:29 CET] <nevcairiel> mpeg-ps and mp3 keep fighting about who is the most inaccurate
[12:15:11 CET] <atomnuker> be glad there isn't an opus raw bitstream container, as any collection of random bits would be a valid opus packet
[12:53:49 CET] <thardin> atomnuker: wouldn't/shouldn't any efficient codec do that?
[12:54:17 CET] <rcombs> not sure anybody's accused MP3 of efficiency
[12:54:48 CET] <rcombs> at least not this decade
[12:56:21 CET] <atomnuker> thardin: usually if a decoder receives a corrupt bitstream it'll error out as it could cause out of bounds/reading past the packet/a definitely irrecoverably corrupt frame
[12:56:42 CET] <atomnuker> if opus gets a weird packet it doesn't care, each choice is always checked against the amount of bits left
[12:57:41 CET] <atomnuker> if there aren't enough bits it falls back to some encoding that would take less bits, and even then if the packet ends, it'll just think the encoder didn't have anything to signal afterwards so its zero
[12:58:10 CET] <thardin> yeah, that's pretty much how I would make it
[12:58:25 CET] <thardin> else you're wasting bits
[12:59:04 CET] <atomnuker> well, maybe not, suppose you're sending e.g. golomb values to specify a codebook (which you usually look up in an array of such)
[12:59:48 CET] <atomnuker> golomb would be efficient and not waste any bits, but at the same time a corrupt packet could easily specify a huge nonexistant codebook
[13:00:04 CET] <thardin> you have to handle that anyway, no?
[13:00:05 CET] <atomnuker> so rather than continue you'd be better off erroring out
[13:00:58 CET] <atomnuker> its just that the opus bitstream doesn't have any places where a packet would be obviously corrupt
[13:01:43 CET] <atomnuker> okay, there is one, if the packet size is larger than 1275 bytes, but that would be a demuxer/user problem
[18:00:51 CET] <atomnuker> erm
[18:01:01 CET] <atomnuker> why is there a copy of fft_template in libavutil?
[18:01:17 CET] <atomnuker> as well as fft_init_table.c?
[18:01:24 CET] <atomnuker> they're not hooked up to anything
[18:01:29 CET] <atomnuker> they're not even compiled
[18:03:49 CET] <atomnuker> nevermind, my dir's extremely dirty and I can't bring myself to delete anything
[18:43:16 CET] <Compn> lol atomnuker. the problem with being a dev :D
[00:00:00 CET] --- Wed Jan 30 2019