[Ffmpeg-devel-irc] ffmpeg-devel.log.20190829

[burek]

[10:56:01 CEST] <JEEB> > write a 30min+ TTML packet with a default subtitle time base
[10:56:11 CEST] <JEEB> > pkt->duration > INT_MAX
[10:56:30 CEST] <JEEB> `Application provided duration: 2264762930 is invalid`
[10:56:31 CEST] <JEEB> vOv
[10:56:48 CEST] <JEEB> probably gets fixed if I force time base to be 1/1000
[10:57:14 CEST] <nevcairiel> why one subtitle f or 30 minutes, slow readers? :D
[10:57:24 CEST] <JEEB> nevcairiel: the spec says one document for the whole fragment
[10:57:33 CEST] <nevcairiel> the spec is dumb
[10:57:39 CEST] <JEEB> yes, yes it is
[10:57:48 CEST] <JEEB> I've felt that in my bones for many, many times
[10:57:54 CEST] <nevcairiel> heh
[10:58:13 CEST] <JEEB> because now I have to keep a list of AVPackets for each track that's getting squashed packets
[10:58:35 CEST] <JEEB> and then when fragment is generated or footer written you generate the final packet
[10:58:56 CEST] <JEEB> that does lead to fun things like fragmentation based on time with only that track in the mux not working (unsurprisingly)
[10:59:35 CEST] <JEEB> until I add that logic to also check the queue
[10:59:45 CEST] <JEEB> for tracks that have the flag of squashing packets
[11:00:25 CEST] <JEEB> TTML in mp4 </3
[11:03:21 CEST] <JEEB> nevcairiel: also there's two profiles seemingly. an older "unofficial" one that uses tag dfxp, where the TTML document's times are to be interpreted from the start of the sample (so fragment start + sample pts). and then you have stpp which is the official spec which says that the times of the documents should be on the "global time line", so unrelated to the actual sample's timestamp
[11:03:46 CEST] <nevcairiel> opposing interpretation, fun.
[11:04:20 CEST] <JEEB> at least they use differing codec_tags
[11:05:02 CEST] <nevcairiel> assuming everyone is sane and sticks to those :D
[11:06:00 CEST] <JEEB> yea I've seen exoplayer's comments which seemed to note something about offsetting and thus I have this fright that they might be interpreting stpp like dfxp
[11:06:22 CEST] <JEEB> I haven't tested yet since this crapola is mostly utilized to feed live ingests
[11:07:35 CEST] <JEEB> but yea, the squashing logic is fun to test
[11:17:18 CEST] <JEEB> ok, by dropping time base to 1:1000 that worked
[11:17:55 CEST] <JEEB> will have to figure out if there's a better way to do that duration check, although for such subtitles I might as well set time base to 1:1000 by default
[11:18:11 CEST] <JEEB> since you don't get any additional fractions in the documents
[11:27:33 CEST] <J_Darnley> Does anyone here use draw_horiz_band and h264?
[11:27:51 CEST] <J_Darnley> I know the feature flag is commented in master
[11:31:35 CEST] <J_Darnley> First, I'll get the api test to use it
[11:31:44 CEST] <JEEB> :)
[11:31:59 CEST] <JEEB> (also no, haven't used band drawing with H.264)
[11:32:14 CEST] <J_Darnley> I put it in vc2 once
[11:32:20 CEST] <J_Darnley> but that was easy
[11:32:38 CEST] <J_Darnley> everything in order, all on the same thread
[11:34:38 CEST] <JEEB> for a moment I thought I'd be able to dump the contents of an AVPacket easily with ffprobe :)
[11:34:47 CEST] <durandal_1707> do not touch draw horiz band, you will kill small kittens
[11:34:57 CEST] <JEEB> although I guess... I could maybe -c copy -f raw
[11:35:26 CEST] <J_Darnley> durandal_1707: too late
[11:35:34 CEST] <durandal_1707> i dump stuff from packets all the time
[11:35:34 CEST] <J_Darnley> I need the practice
[11:35:47 CEST] <J_Darnley> its been a year since I last killed one
[11:36:54 CEST] <JEEB> durandal_1707: what's the muxer you usually use for it? rawvideo?
[11:37:14 CEST] <durandal_1707> yes
[11:37:28 CEST] <JEEB> yes, that worked \o/
[11:38:40 CEST] <durandal_1707> add draw horiz band to png
[12:46:23 CEST] <J_Darnley> do I need to stick a framer in here?
[12:46:55 CEST] <kierank> J_Darnley: framer?
[12:47:07 CEST] <kierank> 10:34:46 <"durandal_1707> do not touch draw horiz band, you will kill small kittens
[12:47:08 CEST] <kierank> nonsense
[12:47:43 CEST] <J_Darnley> actually, I don't think this sample is what I want to use
[12:48:28 CEST] <kierank> J_Darnley: I would say get it working single threaded first
[12:48:31 CEST] <kierank> them sort threads out
[12:48:47 CEST] <J_Darnley> does the test use threads?
[12:49:04 CEST] <J_Darnley> "ctx->thread_count = 1"
[12:49:45 CEST] <J_Darnley> I think this is the weird assembled file josh put together for a different test
[12:49:55 CEST] <kierank> https://github.com/FFmpeg/FFmpeg/blob/master/tests/api/api-band-test.c
[12:49:56 CEST] <kierank> not this one
[12:50:28 CEST] <kierank> the chunks one is this one https://github.com/FFmpeg/FFmpeg/blob/master/tests/api/api-h264-slice-test.cc
[12:50:30 CEST] <kierank> -c
[12:51:24 CEST] <J_Darnley> okay, i need to copy some stuff from the first to the second
[12:51:52 CEST] <kierank> hmm yeah lots of combinations
[13:06:58 CEST] <J_Darnley> why does the draw_horiz_band test negate frame dimentions twice?
[13:07:14 CEST] <J_Darnley> -((-ctx->width) >> pix_fmt_desc->log2_chroma_w)
[13:07:57 CEST] <J_Darnley> I thought we hated shifting negative numbers these days.
[13:08:15 CEST] <J_Darnley> is it for rounding?
[13:09:03 CEST] <durandal_1707> hackers remain hackers? or they grow up?
[13:09:32 CEST] <J_Darnley> A student of one kind or another wrote it
[13:10:03 CEST] <durandal_1707> draw horiz band is evil, kierank does not care :)
[13:10:30 CEST] <J_Darnley> like all broadcast people he has a low latency fetish
[13:10:52 CEST] <durandal_1707> modern codecs use macroblocks not bands
[13:11:14 CEST] <J_Darnley> a block level callback sounds great for performance
[13:16:24 CEST] <kierank> it's good for cache
[13:16:32 CEST] <kierank> you can map the data to the format you need whilst it's still in cache
[13:18:30 CEST] <J_Darnley> wtf
[13:18:38 CEST] <J_Darnley> what is active_thread_type
[13:18:54 CEST] <J_Darnley> 0 is no threads?
[13:19:23 CEST] <BBB> J_Darnley: double neg rounds up
[13:19:45 CEST] <BBB> it's the same as (ctx->width + (1 << log2_chroma_w) - 1) >> log2_chroma_w
[13:19:47 CEST] <BBB> but shorter
[13:19:48 CEST] <J_Darnley> I did wonder
[13:21:03 CEST] <J_Darnley> whatever, I'll just change that check
[13:24:01 CEST] <J_Darnley> dammit make!  why won't you rebuild this?
[13:25:47 CEST] <J_Darnley> make: *** No rule to make target 'tests/api/api-h264-slice-test'.  Stop.
[13:29:24 CEST] <J_Darnley> APITESTPROGS-$(call DEMDEC, H264, H264) += api-h264-slice
[13:29:32 CEST] <J_Darnley> somtimes I really hate make magic
[13:37:07 CEST] <J_Darnley> but that program in in the variable
[13:37:21 CEST] <J_Darnley> so make knows to build it for the test
[13:37:48 CEST] <J_Darnley> so why don't you know it by filename?
[13:44:58 CEST] <J_Darnley> the test reports error concealment if I use 1 thread
[13:46:07 CEST] <J_Darnley> but somehow it ends up with the same frame hash
[13:52:51 CEST] <kierank> J_Darnley: yes we saw this last time
[14:08:55 CEST] <JEEB> heh, all this timestamp handling makes me think of how much weird things I will have to handle :D in my case I'm at least getting packets with dts/pts that is monotonically rising and not AV_NOPTS_VALUE
[14:36:15 CEST] <Lynne_> durandal_1707: I just can't find any motivation to submit patches when they'll be bikeshed for useless asserts and my opinions would get ignored
[14:38:29 CEST] <Lynne_> or worse, be accused of being rude for brining up a valid but ignored point, then be accused for being aggressive for brining up the twice ignored valid point, then get thrown under a bus for attacking people because I can't seem to get any replies to my original point
[14:38:51 CEST] <Lynne_> and even better when later on people think "hmm, that point was valid, why wasn't this brought up earlier?"
[14:41:55 CEST] <durandal_1707> Lynne_: i will apply your patches
[14:43:22 CEST] <durandal_1707> just post then on ml
[15:41:29 CEST] <J_Darnley> what is return code -22?
[15:42:23 CEST] <J_Darnley> EINVAL
[15:42:35 CEST] <J_Darnley> because this has no pixel format
[15:53:11 CEST] <J_Darnley> whoa, no way
[15:53:18 CEST] <J_Darnley> I don't eblieve this works
[16:03:10 CEST] <J_Darnley> no it doesn't work, frames differ
[16:06:15 CEST] <J_Darnley> works in single thread
[16:06:22 CEST] <J_Darnley> but 2 or more doesn't
[16:07:00 CEST] <J_Darnley> Are some of these calls for different frames?
[16:10:32 CEST] <J_Darnley> i have no idea
[16:10:43 CEST] <J_Darnley> the AVFrame pointer is the same
[16:10:56 CEST] <J_Darnley> but this tets only allocates 1
[16:16:38 CEST] <kierank> J_Darnley: you're doing chunks, sliced threads and draw_horiz?
[16:16:43 CEST] <kierank> and your sample is x264 slice threads encoded?
[16:17:00 CEST] <J_Darnley> the sample is the nal file
[16:17:14 CEST] <kierank> ok
[16:17:38 CEST] <J_Darnley> yes to the other
[16:18:04 CEST] <kierank> not sure if the nal file sample is progressive or interlaced
[16:18:17 CEST] <kierank> could be a progressive vs interlaced problem
[16:18:27 CEST] <J_Darnley> maybe
[16:18:39 CEST] <J_Darnley> why does that work on 1 thread then?
[16:19:05 CEST] <kierank> dunno
[16:22:40 CEST] <kierank> J_Darnley: are you reordering the slices
[16:22:46 CEST] <kierank> maybe try dumping the output
[16:22:50 CEST] <kierank> see what's different
[16:23:04 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:e0c973e5bea9: avformat/mpsubdec: Remove floating point usage
[16:23:04 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:37bc8e3249c8: avcodec/cavsdec: Limit the number of access units per packet to 2
[16:23:04 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:dea2591d4fbc: avcodec/vb: Check input packet size to be large enough to contain flags
[16:23:04 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:7e9aecc9f358: avcodec/tta: Fix integer overflow in prediction
[16:23:04 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:738ff94f7c38: tools/target_dec_fuzzer: adjust pixel threshold for SANM, as it allows coding gigantic images on tiny input
[16:23:05 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:96efaa9a1a22: tools/target_dec_fuzzer: Adjust GDV pixel threshold down by a factor of 2
[16:23:05 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:ba823394f65a: tools/target_dec_fuzzer: Adjust maxpixels for indeo4
[16:23:06 CEST] <cone-764> ffmpeg 03Michael Niedermayer 07master:15a65c13e184: avcodec/ivi: Allocate bufs later
[17:16:04 CEST] <JEEB> hah, got myself debugging capability by using the segment muxer and csv output + making it utilize the rawvideo muxer for segments by using the extension .rgb xD
[17:17:48 CEST] <J_Darnley> impressive hackery
[17:18:24 CEST] <J_Darnley> kierank (and anyone else following): the differing frames have obvious errors
[17:18:29 CEST] <JEEB> `-map 0:2 -c copy -f segment -segment_format data -segment_list debug_segments/index.csv -break_non_keyframes 1 -segment_time 0.01 'debug_segments/%04d.rgb'`
[17:18:40 CEST] <J_Darnley> the first frame has a large grey gap
[17:18:47 CEST] <JEEB> 0:2 being an XML based sample format that I'm not going to write a decoder for at least now
[17:19:24 CEST] <JEEB> so now I get an index out of the csv, and then every document goes into its own "rgb" file :)
[17:19:36 CEST] <JEEB> also yes the segment_format says data but it seems like it's being 100% ignored
[17:19:52 CEST] <JEEB> the extensions trumps all
[17:20:24 CEST] <J_Darnley> 3rd frame has a green gap at the right edge
[17:21:37 CEST] <kierank> J_Darnley: is your draw_horiz_band thread safe
[17:21:54 CEST] <kierank> i.e could two threads call fwrite at the same time
[17:22:00 CEST] <kierank> and cause that
[17:22:03 CEST] <J_Darnley> I think memcpy is fine
[17:22:15 CEST] <J_Darnley> if I don't overlap regions
[17:23:15 CEST] <kierank> what allocates the main frame buffer?
[17:23:25 CEST] <J_Darnley> main() I think
[17:23:56 CEST] <kierank> I'm not sure you can do that
[17:23:59 CEST] <kierank> because of the frame slice boundary
[17:24:07 CEST] <kierank> |frame1_slice|frame2_slice|
[17:24:14 CEST] <kierank> you could get a callback from frame2_slice first
[17:24:20 CEST] <kierank> then frame1_slice
[17:24:23 CEST] <kierank> or do you handle that?
[17:24:47 CEST] <J_Darnley> what?
[17:25:18 CEST] <kierank> if each thread is working on a slice
[17:25:21 CEST] <J_Darnley> I copy from AVFrame into a flat buffer based on the arguments given to the callback.
[17:25:29 CEST] <kierank> J_Darnley: can you pastebin your code
[17:28:07 CEST] <J_Darnley> ah no, I am allocating wring, I think
[17:30:36 CEST] <J_Darnley> and it allegedly asserts on a line that has no assert when I run in gdb
[17:30:58 CEST] <J_Darnley> time for --disable-optimizations
[17:46:32 CEST] <J_Darnley> kierank: https://pastebin.com/8WyQhX4D
[17:47:48 CEST] <J_Darnley> save into tests/api/api-h264-slice-test.c if you want to diff it with git
[17:49:34 CEST] <kierank> I think one frame might work with that code
[17:49:43 CEST] <kierank> But it doesn't handle the boundaries afaict
[17:51:37 CEST] <kierank> Also not really sure how the test works tbh because I think it assumes every nal is a slice
[17:51:50 CEST] <kierank> That's the fault of the original test I think
[17:52:39 CEST] <kierank> Will look when back at pc with a diff
[17:53:19 CEST] <cone-764> ffmpeg 03Aman Gupta 07master:59e651c05244: configure: fix --enable-omx compile on raspberry pi
[17:58:42 CEST] <kierank> J_Darnley: do you have an easy link to the .nal sample
[17:58:46 CEST] <kierank> I don't want to rsync the whole lot
[17:59:21 CEST] <J_Darnley> just a mo
[17:59:38 CEST] <kierank> I wonder if he removed the sei
[17:59:49 CEST] <kierank> and maybe merged the sps/pps with first slice
[17:59:53 CEST] <J_Darnley> https://0x0.st/s6ap.nal
[18:01:06 CEST] <kierank> J_Darnley: seems not
[18:01:09 CEST] <kierank> but maybe it works anyway
[18:01:49 CEST] <J_Darnley> it is between 33 and 50% of frames that come out identical
[18:02:02 CEST] <J_Darnley> Does valgrind do threads?
[18:02:03 CEST] <kierank> bugger, no AUDs
[18:02:06 CEST] <kierank> no it doesn't
[18:02:11 CEST] <J_Darnley> tsan then
[18:02:22 CEST] <kierank> J_Darnley: the problem is when you have, say 4 threads and 2 are working on frame n and two are working on frame n+1
[18:02:32 CEST] <kierank> if the two that are working on frame n+1 return first I think it breaks the code
[18:02:53 CEST] <kierank> J_Darnley: does avframe return pts
[18:03:08 CEST] <kierank> or some frame number identifier, then you could identify which frame was which
[18:03:39 CEST] <J_Darnley> AVFrame in the callback?  I can check
[18:04:01 CEST] <kierank> yeah
[18:04:07 CEST] <kierank> I think it has an incrementing counter or something
[18:05:10 CEST] <J_Darnley> pts and dts look like INT64_MIN on the first callback
[18:06:04 CEST] <J_Darnley> there's 2 picture numbers which I will watch
[18:06:54 CEST] <kierank> J_Darnley: maybe write a new file for each frame
[18:07:00 CEST] <kierank> so you know the fwrite calls don't collide
[18:07:07 CEST] <kierank> or new buffer for every frame
[18:07:09 CEST] <kierank> and then write them all at the end
[18:07:13 CEST] <J_Darnley> How can that?  they're in the main thread
[18:07:22 CEST] <kierank> sorry not fwrite
[18:07:49 CEST] <kierank> actually wait a second
[18:08:07 CEST] <J_Darnley> c
[18:08:12 CEST] <kierank> so  while (ret >= 0) {
[18:08:28 CEST] <kierank> that will ring true but other frames may have been completed and memcpyied to your buffer by then
[18:08:59 CEST] <kierank> other slices I mean
[18:09:22 CEST] <kierank> printf might tell you but not guaranteed to be ordered I guess
[18:09:49 CEST] <J_Darnley> yes
[18:09:59 CEST] <J_Darnley> you did say I needed to get single thread working forst
[18:10:01 CEST] <J_Darnley> it does
[18:10:07 CEST] <J_Darnley> now I neeed to handle order
[18:10:19 CEST] <kierank> I would just allocate n buffers
[18:10:23 CEST] <kierank> for the time being
[18:10:29 CEST] <kierank> just to see if it's an ordering problem
[18:10:36 CEST] <kierank> and use the frame counter in avframe to pick the right one
[18:10:44 CEST] <kierank> then write them all at the end of the file
[18:10:55 CEST] <J_Darnley> coded_picture_number incements
[18:11:00 CEST] <kierank> that'll do
[18:11:03 CEST] <J_Darnley> so I'll use that for the test
[18:11:08 CEST] <kierank> no bframes so should be fine
[18:13:01 CEST] <J_Darnley> q
[18:32:20 CEST] <cone-764> ffmpeg 03Paul B Mahol 07master:0067da587a08: avcodec/msrle: add a flush() callback
[18:38:35 CEST] <J_Darnley> oh dammit
[18:38:37 CEST] <J_Darnley> src/libavfilter/vsrc_testsrc.c: In function color_gradient:
[18:38:37 CEST] <J_Darnley> src/libavfilter/vsrc_testsrc.c:718:1: error: control reaches end of non-void function [-Werror=return-type]
[18:40:10 CEST] <J_Darnley> and another one!
[18:42:33 CEST] <J_Darnley> two more
[18:54:35 CEST] <J_Darnley> wow tsan is slooo~ooow
[18:57:03 CEST] <J_Darnley> and this seems to be even worse
[18:57:13 CEST] <J_Darnley> fewer frames are identical
[18:57:39 CEST] <J_Darnley> and I do have at least one data race
[18:59:39 CEST] <J_Darnley> memcpy in the callback
[19:00:56 CEST] <J_Darnley> Maybe I need a new AVFrame for each call to decode
[19:05:49 CEST] <J_Darnley> i get data races with the stock test anyway
[19:08:38 CEST] <J_Darnley> it seems as though fate knows
[19:08:51 CEST] <J_Darnley> but nobody looks
[19:09:45 CEST] <J_Darnley> oh, h264 is filled with them it seems
[19:12:12 CEST] <J_Darnley> hm, I want a whitelist, not a blacklist
[19:14:17 CEST] Action: J_Darnley goes to look for the ref counting patch for the fuzzer
[19:20:58 CEST] <kierank> J_Darnley: yeah we had this discussion a few years ago, tsan is full of false positives
[20:51:28 CEST] <durandal_1707> gonna apply imm5 decoder soon
[21:07:57 CEST] <cone-764> ffmpeg 03Paul B Mahol 07master:7c0b3ba7ddaa: avcodec: add IMM5 decoder
[22:59:54 CEST] <kierank> durandal_1707: what is imm5
[22:59:59 CEST] <kierank> is that thing with hevc and h264
[23:00:09 CEST] <durandal_1707> yes
[23:02:22 CEST] <j-b> nice
[23:03:13 CEST] <durandal_1707> not really, piece of cctv crap
[23:03:55 CEST] <durandal_1707> should be forbiden and punished by law to make such codecs
[23:06:30 CEST] <jamrial> can't even call it a codec. it's like a wrapper
[23:07:44 CEST] <durandal_1707> thera are already less and more complicated codecs wrappers
[23:53:20 CEST] <jamrial> ubitux: your coverage client hasn't run since early last year. any chance to resurrect it?
[00:00:00 CEST] --- Fri Aug 30 2019