[Ffmpeg-devel-irc] ffmpeg-devel.log.20180519

[burek]

[00:07:21 CEST] <durandal_1707> michaelni: have you applied your probe patchset?
[00:17:36 CEST] <jkqxz> tmm1:  Intending to finish at some point.  There is some cleanup to do, and I'm still not sure that I like the overall structure of it.
[00:55:55 CEST] <tmm1> jkqxz: cool
[00:56:04 CEST] <tmm1> anyone know what h264 SEI type 128 is?
[00:58:52 CEST] <jkqxz> An error?
[01:02:31 CEST] <tmm1> hmm yea, i wonder if its a bug in the parser
[01:03:20 CEST] <tmm1> seeing "unknown SEI type 128" on https://tmm1.s3.amazonaws.com/vts.mpg
[01:10:54 CEST] <jkqxz> Coming from what?  I only see both timing types and and udr with captions.
[01:20:12 CEST] <tmm1> printed from h264_sei.c, using `ffmpeg -loglevel debug -i vts.mpg -map v -c copy -f null /dev/null`
[01:21:17 CEST] <tmm1> i see the same when using h264_analyze (timing, udr only)
[01:30:24 CEST] <jkqxz> Trailing zeroes on the NAL unit confuse the SEI code in the parser.
[01:30:35 CEST] <jkqxz> It looks for more stuff after the end, that 128 is the rbsp trailing bits.
[01:33:13 CEST] <tmm1> ah ok
[01:40:08 CEST] <tmm1> i guess ff_h264_sei_decode should take the nal's size_bits as an argument and use that to ignore the trailer
[01:41:38 CEST] <nevcairiel> it takes a getbitcontext, that should probably just be initialized with the right amount of bits
[01:42:48 CEST] <jkqxz> Yeah.
[01:43:53 CEST] <jamrial> h264_parser doesn't use ff_h2645_packet_split, which does make sure the nal is valid
[01:44:14 CEST] <jamrial> assuming this call to ff_h264_sei_decode comes from there and not from h264dec
[01:45:28 CEST] <tmm1> yep
[01:45:31 CEST] <tmm1> creating the gb using size_bits fixed it
[01:50:27 CEST] <tmm1> sent patch to the list
[02:14:26 CEST] <tmm1> looks like that's breaking something else
[02:20:16 CEST] <jamrial> tmm1: ff_h2645_extract_rbsp() does not set nal.set_bits
[02:20:30 CEST] <jamrial> err, nal.size_bits
[02:26:20 CEST] <tmm1> oh, well that would explain it
[02:35:09 CEST] <jamrial> one solution is setting that there instead of only in ff_h2645_packet_split()
[02:35:28 CEST] <jamrial> another is to port h264_parser to use the latter
[02:38:59 CEST] <hanna> Simply linking to a libavcodec.so built with opencl enabled does all sorts of weird stuff when loading the process: http://0x0.st/seJ6.txt
[02:39:04 CEST] <hanna> Why is this the case and can it be removed?
[02:39:14 CEST] <hanna> When libavcodec is built without opencl, linking to libavcodec.so does nothing out of the ordinary
[02:40:23 CEST] <hanna> Ah, seems like that happens because of linking to libOpenCL.so, so not an FFmpeg bug
[02:44:52 CEST] <jkqxz> I think you want to link with the ICD loader, not a vendor ICD.  The vendor ICDs may do crazy shit.
[02:45:16 CEST] <jkqxz> Which ICD is that?
[03:50:02 CEST] <jamrial> tmm1: https://pastebin.com/H2TKGDK4 this seems to fix the invalid say error
[03:52:08 CEST] <jamrial> s/say/sei
[03:54:20 CEST] <jamrial> still, h264_parser should use nal.size_bits
[03:58:04 CEST] <tmm1> ah interesting
[03:58:18 CEST] <tmm1> yea, i guess its probably worth doing both
[04:04:32 CEST] <cone-189> ffmpeg 03Aman Gupta 07master:2b2f2f65f38c: avformat: add fields to AVProgram/AVStream for PMT change tracking
[04:04:32 CEST] <cone-189> ffmpeg 03Aman Gupta 07master:24579bf53747: avformat/mpegts: keep track of PMT details in AVProgram/AVStream
[04:04:32 CEST] <cone-189> ffmpeg 03Aman Gupta 07master:16b4f97b721d: avformat/mpegts: add merge_pmt_versions option
[04:18:38 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:13d83899df3c: avcodec/videotoolbox: improve logging of decoder errors
[04:18:39 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:ef9478d26463: avcodec/videotoolbox: fix kVTCouldNotFindVideoDecoderErr trying to decode HEVC on iOS
[04:18:40 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:e40922c16c3e: avcodec/videotoolbox: cleanups
[04:18:41 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:e8caf67f56c6: avcodec/videotoolbox: split h264/hevc callbacks
[04:18:42 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:76716518a817: avcodec/hevc: remove videotoolbox hack
[04:18:43 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:2884575d97cb: avcodec/videotoolbox: fix decoding of some HEVC videos
[04:18:44 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:84bf631018e9: avcodec/mediacodecdec: clarify delay_flush specific code
[04:18:45 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:3054e53ddcfa: avcodec/mediacodecdec: use AV_TIME_BASE_Q
[04:18:46 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:33042d632d12: avcodec/mediacodecdec: refactor pts handling
[04:18:47 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:6f55a36be9a2: avcodec/mediacodec_wrapper: add helper to fetch SDK_INT
[04:18:48 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:df2c811b7ce3: avcodec/mediacodecdec: restructure mediacodec_receive_frame
[04:18:49 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:db5631e4087f: avcodec/mediacodecdec: wait on first frame after input buffers are full
[04:18:50 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:bb6a34f237ec: avcodec/mediacodecdec: add workaround for buggy amlogic mpeg2 decoder
[04:18:51 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:97aea63340d1: avformat/mpegts: skip non-PMT tids earlier
[04:18:52 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:ef28571efe23: avformat/mpegts: use MAX_SECTION_SIZE instead of hardcoded value
[04:18:53 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:7ad163c258f2: avformat/mpegts: clean up whitespace
[04:18:54 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:c343eabfb77c: avformat/mpegts: parse sections with multiple tables
[04:18:55 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:852f78443a9d: avformat/mpegts: reindent after last change
[04:18:56 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:d1845e7f1a39: avformat/mpegts: initialize section_buf to fix valgrind test failure
[04:18:57 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:8336a6627045: avformat/mpegts: fix incorrect indentation
[04:18:58 CEST] <cone-189> ffmpeg 03Aman Gupta 07release/4.0:da399903c76d: ffprobe: fix SEGV when new streams are added
[07:20:06 CEST] <Mindless`> I found a bug and Trac registration is broken.  ffprobe produces wrong XML; "data_hash" attribute is sometimes in the text -- https://pastebin.mozilla.org/9085828
[07:25:04 CEST] <Mindless`> Nevermind -- I tried a few more times and it finally didn't give me a gateway error.
[11:31:41 CEST] <durandal_1707> atomnuker: instead of doing stylistic changes to source code you should really finish atrac9 and fft/dct :-/
[13:45:18 CEST] <JEEB> -33
[14:30:24 CEST] <durandal_1707> atomnuker: which algo is best for audio noise reduction ?
[14:56:20 CEST] <cone-096> ffmpeg 03Martin Vignali 07master:644130bcaa22: avdevice/sdl2output : fix setting window_size
[14:56:20 CEST] <cone-096> ffmpeg 03Martin Vignali 07master:411f7141a3c3: avdevice/sdl2 : add option to define if the window quit action is available
[15:39:58 CEST] <gagandeep> kierank: i am looking at get_bits.h, i need to obtain values from bitstream, so i am using get_bits
[15:40:19 CEST] <kierank> Ok
[15:40:25 CEST] <kierank> What values
[15:40:41 CEST] <gagandeep> but get_bits is not running, do i need to initialize the api
[15:41:11 CEST] <gagandeep> can you give me pointers to using the api
[15:41:16 CEST] <kierank> Yes but what do you need exactly
[15:41:37 CEST] <kierank> What tag are you looking at
[15:42:27 CEST] <gagandeep> i have got the tag, and now the table is in the bitstream buffer, since size hasn't been provided for the table i need to use values directly from the bitstream
[15:42:45 CEST] <gagandeep> table = peak value table
[15:43:08 CEST] <kierank> gagandeep: can you link me in cineform decoder
[15:44:20 CEST] <gagandeep> kierank: https://github.com/gopro/cineform-sdk/blob/master/Codec/decoder.c#L23658
[15:45:07 CEST] <gagandeep> i have all the tags and other stuff, they are using base to hold the pointer for table in stream for later use but can i also do that
[15:47:12 CEST] <kierank> what does peak value table do
[15:47:52 CEST] <gagandeep> there are some values in subband that are overwritten by this table which combined with difference coding produces the desired result
[15:48:00 CEST] <gagandeep> only that part is left for interlaced
[15:48:22 CEST] <gagandeep> if subband value > level, then it is replaced by peak table value
[15:48:26 CEST] <kierank> interesting
[15:49:24 CEST] <kierank> I don't see in the code where you need to get bits
[15:49:31 CEST] <kierank> I think I did all the bitstream work
[15:50:33 CEST] <gagandeep> that code is in the dequantisation routine somewhere else in decoder.c
[15:50:39 CEST] <kierank> looks like there's a table in the binary
[15:50:44 CEST] <kierank> and it gives an x,y position
[15:52:11 CEST] <gagandeep> i will see what i can do with bitstream then
[15:52:24 CEST] <kierank> you shouldn't need bitstream code to do this
[15:53:20 CEST] <kierank> I can see why you think that
[15:53:27 CEST] <kierank> but you don't actually need get_bits code for this
[15:53:46 CEST] <kierank> basically it seems like there is a lookup table in the file itself
[15:54:26 CEST] <kierank> for each subband
[15:55:53 CEST] <kierank> gagandeep: do you want me to explain what the tag code does?
[15:56:23 CEST] <gagandeep> sure why not
[15:57:15 CEST] <kierank> do you see how it creates a lookup table in the frame data itself?
[15:57:40 CEST] <kierank> and then for each coefficient it checks the value in the lookup table
[15:57:42 CEST] <kierank> https://github.com/gopro/cineform-sdk/blob/master/Codec/decoder.c#L19676
[15:59:25 CEST] <gagandeep> yeah, i also said that it replace values from the table
[15:59:28 CEST] <gagandeep> so
[16:00:02 CEST] <kierank> they way they do run/level is a bit odd
[16:00:25 CEST] <kierank> it's super hand optimised
[16:00:36 CEST] <gagandeep> yeah
[16:00:54 CEST] <gagandeep> hand optimised without documentation
[16:01:03 CEST] <gagandeep> super tough to see
[16:01:14 CEST] <kierank> yeah
[16:02:23 CEST] <gagandeep> this is what i am aslo doing just need to get the peak value using get_bits
[16:02:27 CEST] <gagandeep> is my approach
[16:02:58 CEST] <kierank> not sure about that
[16:03:18 CEST] <gagandeep> or can i also get the pointer to the table like they did during updating the header
[16:03:19 CEST] <kierank> their FSM is equivalent to our get_vlc ode
[16:03:26 CEST] <gagandeep> yeah
[16:03:49 CEST] <kierank> but they seem to have some kind of special subband type where they zero everything out
[16:04:11 CEST] <kierank> and then start reading vlcs but then runs they don't seem to care about
[16:04:28 CEST] <gagandeep> no, that subband is the same one,
[16:04:35 CEST] <gagandeep> they just use fsm to fill it
[16:04:52 CEST] <gagandeep> they dequantize the fsm before hand and decompand before hand
[16:05:04 CEST] <kierank> but where do they handle the runs?
[16:07:19 CEST] <gagandeep> by runs you me entropy code?
[16:07:25 CEST] <gagandeep> *mean
[16:07:58 CEST] <kierank> yes, it uses run level coding
[16:08:08 CEST] <kierank> so it says I have a level of 52 or whatever, some number
[16:08:12 CEST] <kierank> repeat this for n runs
[16:08:22 CEST] <kierank> but I don't see where they handle the run part in the code
[16:10:04 CEST] <gagandeep> kierank: search 'dequantfsm' i think there is something of that function with fsm
[16:10:14 CEST] <gagandeep> in the same file
[16:10:29 CEST] <kierank> oh so they do something like: level1, 0, 0, 0, level2, 0, 0 ,0 , 0
[16:10:34 CEST] <kierank> then in dequant they fill the holes
[16:10:55 CEST] <gagandeep> yeah initialize whole subband with zero and skip
[16:11:03 CEST] <gagandeep> that was in the peaks function
[16:11:12 CEST] <kierank> ok I understand half of it now
[16:11:16 CEST] <kierank> but I don't understand rowptr[1]
[16:11:19 CEST] <kierank> what's that?
[16:12:09 CEST] <gagandeep> they are using only 4 bits out of 8bits of stream to refference to fsm and other 4bits to refference for the next index
[16:13:01 CEST] <gagandeep> rowptr[0] and rowptr[1] are just two elements in band next to each other
[16:14:53 CEST] <gagandeep> also they are first running the code leaving out the last 1000 or so elements and then finally complete it by writing the same code again
[16:15:00 CEST] <gagandeep> a mess really
[16:16:48 CEST] <kierank> gagandeep: so the peak detection only happens on the first level out of the run as far as I can tell?
[16:17:55 CEST] <gagandeep> it can theoretically happen on others as well, but i have the sample with level 1 peak detection currently
[16:18:25 CEST] <kierank> I still don't see where they handle the runs
[16:18:48 CEST] <kierank> durandal_1707: cineformsdk is messy code
[16:20:35 CEST] <gagandeep> kierank: the threaded part was handling it above
[16:20:49 CEST] <gagandeep> i can only say it must happen in 'dequantfsm' function
[16:20:52 CEST] <gagandeep> check that one
[16:21:54 CEST] <gagandeep> kierank:https://github.com/gopro/cineform-sdk/blob/master/Codec/decoder.c#L20352
[16:28:16 CEST] <kierank> I still don't understand why they don't do peak detection on rowptr[1]
[16:31:22 CEST] <gagandeep> maybe they don't need to, they are just doing peak detection to reduce the very large values which result after transform
[16:31:35 CEST] <gagandeep> and since they are doing difference coding later
[16:31:42 CEST] <kierank> ah unless rowptr[1] is the run
[16:31:50 CEST] <kierank> and rowptr[0] is level
[16:33:30 CEST] <gagandeep> no i dont think so
[16:33:40 CEST] <gagandeep> they are using rowptr for difference coding later
[16:34:15 CEST] <kierank> I guess then you can just integrate peak detection into existing loop
[16:35:35 CEST] <gagandeep> https://github.com/gopro/cineform-sdk/blob/master/Codec/decoder.c#L20493
[16:35:42 CEST] <gagandeep> yeah, i have done that
[16:36:54 CEST] <kierank> I understand what they are doing (working with a compressed buffer in a sense)
[16:36:57 CEST] <kierank> but it's bloody confusing
[16:37:25 CEST] <gagandeep> kierank: https://imgur.com/a/iLriA6N
[16:37:32 CEST] <gagandeep> this is the troublesome sample
[16:37:44 CEST] <kierank> hmm looks close
[16:37:55 CEST] <gagandeep> the white rows are due to difference coding which kept the value persisting
[16:38:21 CEST] <kierank> ah it also has difference coding?
[16:38:55 CEST] <gagandeep> when decompanding is not done, difference coding is done
[16:39:15 CEST] <gagandeep> really a mess, it was written somewhere in codeset structs
[16:41:38 CEST] <gagandeep> kierank: i will study the bitstream part and hopefully i should be done with this by tomorrow
[16:41:45 CEST] <gagandeep> afk
[16:41:51 CEST] <kierank> ok
[16:45:58 CEST] <JEEB> so there's a question about getting the QP data from an AVFrame (side data currently I think). our APchanges' most recent thing was "Add AV_FRAME_DATA_QP_TABLE_PROPERTIES and AV_FRAME_DATA_QP_TABLE_DATA.", where the documentation instructs to use "av_frame_get_qp_table", which in turn is already deprecated
[16:46:06 CEST] <JEEB> can someone remind me what I tell a user in this case?
[16:46:22 CEST] <JEEB> http://git.videolan.org/?p=ffmpeg.git;a=blob;f=doc/APIchanges#l60
[16:46:46 CEST] <JEEB> https://ffmpeg.org/doxygen/trunk/group__lavu__frame.html#ggae01fa7e427274293aacdf2adc17076bca880fdfc059a05584749d695cc54e4717
[16:47:33 CEST] <JEEB> so the side data structure is internal, and the docs say to go for the function. the function is already by itself deprecated :D
[17:04:24 CEST] <atomnuker> durandal_1707: I'm glad at least someone other than me noticed the changes
[17:51:39 CEST] <kierank> "only zero values are run length coded."
[17:51:41 CEST] <kierank> explains a lot
[18:07:57 CEST] <jamrial> tmm1: your latest mpegts change has a memory leak
[18:38:35 CEST] <durandal_1707> atomnuker: you havent answered my other question, aren't you the audio expert?
[18:52:51 CEST] <atomnuker> durandal_1707: right, I forgot, for voice its the neural network thing in libopus, for anything else, I'd try what audacity is doing (capturing a sample and using it as a signature)
[19:15:31 CEST] <kierank> durandal_1707: do you understand cineform fsm?
[19:20:56 CEST] <durandal_1707> kierank: nope
[19:21:09 CEST] <kierank> https://github.com/gopro/cineform-sdk/blob/master/Codec/decoder.c#L19699
[19:21:19 CEST] <kierank> I don't understand why it only does peak detection on one coefficient
[19:24:36 CEST] <durandal_1707> atomnuker: spectral subtraction is interactive and bad quality
[19:47:16 CEST] <durandal_1707> ubitux: you probably never heard of av_always_inline?
[19:52:09 CEST] <atomnuker> durandal_1707: maybe audacity does something different with the noise signature? last time I used it it was okay
[19:53:51 CEST] <ubitux> durandal_1707: i did, why?
[19:55:17 CEST] <ubitux> it just makes sure the inline is honored with non-optimized flags, iirc
[19:55:44 CEST] <ubitux> (but the compiler is still free to ignore it afaik)
[19:56:21 CEST] <ubitux> always being "please try to do it with -O0 too", never "i order you to inline the function"
[20:00:19 CEST] <iive> ubitux, huh?
[20:00:30 CEST] <ubitux> i'm wrong?
[20:06:13 CEST] <iive> if you find a case where always inline does not inline, you can report it as bug.
[20:09:56 CEST] <nevcairiel> inline is just a hint, not a commandment, thats what forceinline is for
[20:23:05 CEST] <durandal_1707> why vc1 patches are still not reviewed?
[20:23:27 CEST] <wm4> because
[20:23:30 CEST] <JEEB> I think at least one of them got comments on creating artifacts on some sample
[20:24:17 CEST] <tmm1> jamrial: did a test fail?
[20:24:42 CEST] <jamrial> yes, fate-mpegts-probe-pmt-merge
[20:24:53 CEST] <jamrial> under valgrind
[20:25:37 CEST] <tmm1> thanks i will check it out
[20:26:31 CEST] <durandal_1707> why mbedtls is not applied?
[20:26:49 CEST] <tmm1> can i run valgrind tests locally somehow?
[20:28:46 CEST] <JEEB> yea
[20:29:10 CEST] <JEEB> http://fate.ffmpeg.org/report.cgi?time=20180519085700&slot=x86_64-archlinux-gcc-valgrind-no-undef
[20:29:23 CEST] <JEEB> --target-exec='valgrind --error-exitcode=1 --malloc-fill=0xa2 --leak-check=full --gen-suppressions=all --suppressions=/home/fate/ffmpeg/tests/fate-valgrind.supp --undef-value-errors=no'
[20:29:27 CEST] <JEEB> is the key
[20:29:42 CEST] <JEEB> it tells the runners to prepend that to the FATE processes
[20:30:18 CEST] <JEEB> if you look at the stderr it will actually also list the command used
[20:30:27 CEST] <JEEB> ffprobe -show_entries stream=codec_name -print_format default -bitexact -v 0 -merge_pmt_versions 1 -i /home/fate/fate-suite/mpegts/pmtchange.ts
[20:30:36 CEST] <JEEB> and that run under valgrind
[20:33:04 CEST] <tmm1> gotcha thanks
[20:37:05 CEST] <jamrial> wow, ticket 7219, what a crappy memcpy implementation
[20:38:43 CEST] <BtbN> why does it even break?
[20:38:51 CEST] <jamrial> badly made macro
[20:38:51 CEST] <BtbN> Does it not properly put () around its arguments
[20:38:55 CEST] <jamrial> not wrapping arguments
[20:39:15 CEST] <BtbN> Well, blame OSX I'd say
[20:39:23 CEST] <jamrial> so the second argument being full of commas makes it trip
[20:43:28 CEST] <jamrial> we could i guess change these memcpy calls, assuming apple is not going to bother fixing this "secure" header in 10.11
[20:44:33 CEST] <jamrial> michaelni: ^ you added these recently. what do you think?
[20:45:52 CEST] <durandal_1707> atomnuker: i generate noise with anoisesrc and mix it with music, audacity ouput sounds bad
[20:52:26 CEST] <durandal_1707> atomnuker: what do you think about non-local means denoising for audio ?
[21:25:31 CEST] <JEEB> jamrial: I thought 10.11 was going out of support soon'ish?
[21:25:32 CEST] <atomnuker> durandal_1707: someone wrote a paper, you could certainly give it a try
[21:25:57 CEST] <atomnuker> ftp://ftp.math.ucla.edu/pub/camreport/cam08-56.pdf
[21:28:49 CEST] <atomnuker> hmm, I wonder if using an s-transform for spectral subtraction would be better than just an fft-based one
[21:31:01 CEST] <durandal_1707> atomnuker: what is s-transform?
[21:44:02 CEST] <atomnuker> its a pseudo frequency/time one, its used in seismology to filter out noise
[23:55:35 CEST] <jdarnley> Does anyone have any tips about "de-obfuscating" media files in video games?
[23:55:43 CEST] <jdarnley> I want to be able to decode the ogg files shipped with this one, Ys 8.
[23:56:34 CEST] <durandal_1707> jdarnley: tried vgmstream?
[23:56:46 CEST] <jdarnley> No.  I am not familiar with that.
[23:58:03 CEST] <jdarnley> Holy shit!  I remember Halley's Comet Software.
[00:00:00 CEST] --- Sun May 20 2018