[Ffmpeg-devel-irc] ffmpeg-devel.log.20180313

[burek]

[00:18:27 CET] <atomnuker> jamrial: (1 << (m + 32)) and (t*qf + qf) need to be done with 64 bits, but the variables need to be only 32 bits
[00:18:39 CET] <atomnuker> ULL promotes to 64 bits on windows, right?
[00:25:16 CET] <jkqxz> UINT64_C()
[00:27:17 CET] <atomnuker> seriously? but loads of code uses ULL
[00:28:21 CET] <jkqxz> It probably works, but if you want a definitely-64-bit value then the those are better.
[00:29:56 CET] <jamrial>  /mingw64/x86_64-w64-mingw32/include/stdint.h:#define UINT64_C(val) val##ULL
[00:30:42 CET] <jamrial> no idea what it expands to on msvc, but i know that ULL is supported there, at least
[00:31:36 CET] <jkqxz> It also has the same type as uint64_t, so you don't have problems with same-sized format conversion specifiers.
[00:31:45 CET] <jkqxz> (Not that that's relevant here.)
[00:36:29 CET] <atomnuker> patch sent
[00:45:56 CET] <atomnuker> jkqxz: you mean vaExportSurfaceHandle() is in a libva release which is only a few weeks old? but I've been using it for ages, and it has been in the codebase since September
[00:47:44 CET] <jkqxz> Because I added it then and expected it to go in an earlier release.
[00:48:15 CET] <jkqxz> I assume you built libva+i965 master (because that was what was needed to make it work)?
[00:49:47 CET] <atomnuker> no, standard debian unstable
[02:13:58 CET] <cone-867> ffmpeg 03Yusuke Nakamura 07master:af7e953a5956: h264_mp4toannexb_bsf: Fix start code size of parameter sets.
[02:13:58 CET] <cone-867> ffmpeg 03Philipp M. Scholl 07master:040b28aeccce: avformat/pcm: decrease delay when reading PCM streams.
[02:30:01 CET] <atomnuker> jamrial: did you test to see if the patch fixes the warnings?
[02:31:33 CET] <tmm1> atomnuker: can you review/merge https://patchwork.ffmpeg.org/patch/7949/
[02:33:13 CET] <atomnuker> yeah, did patchwork screw up spaces or something?
[02:33:23 CET] <atomnuker> (3 * (s->end_mb_y - s->start_mb_y) * s->mb_width) >> field_pic); looks out of place, and so does the rest
[02:33:58 CET] <tmm1> that's how i wrote it, to try to fit within 80ch
[02:34:01 CET] <tmm1> feel free to change
[02:37:15 CET] <atomnuker> grr
[02:39:56 CET] <jamrial> atomnuker: it does
[02:42:25 CET] <atomnuker> tmm1: I'm not sure if some of those ATOMIC_INITs need to be stores, could you submit a new version which properly aligns arguments on the next line with the opening brackets of a function? because its super ugly like this
[02:42:36 CET] <atomnuker> michaelni's the person who needs to review this I think
[02:42:40 CET] <rcombs> seems muxing opus into dash crashes
[02:43:28 CET] <tmm1> atomnuker: if i align any of those lines they go over 80chars
[02:43:57 CET] <atomnuker> let them, its not a hard limit, its a suggestion, thankfully
[02:44:52 CET] <atomnuker> as long as they don't stick out that much and that often its fine
[02:45:03 CET] <tmm1> ok. there's ATOMIC_VAR_INIT() and atomic_init(), is there a store variant too?
[02:47:06 CET] <atomnuker> yes, its called atomic_store()
[02:47:46 CET] <atomnuker> the reason why I'm unsure whether ATOMIC_INIT should be used is because most of these functions are not init functions
[02:48:33 CET] <atomnuker> and I think that macro just silences compiler warnings rather that to make sure there's no races when reading/writing to it
[02:49:05 CET] <cone-867> ffmpeg 03Rostislav Pehlivanov 07master:688060fbb723: vc2enc: properly promote operations to 64 bits
[02:49:13 CET] <atomnuker> jamrial: thanks, pushed
[02:49:58 CET] <tmm1> ok sent and cc'd michael
[03:22:11 CET] <atomnuker> does anyone know how to add a dependency on a .h file in the build system for a particular component which needs to be generated?
[03:22:39 CET] <atomnuker> can't really use what opencl uses since it compiles the resulting .c files
[04:33:54 CET] <gagandeep> guys can you tell me how is pixel format handled in ffmpeg
[04:34:25 CET] <gagandeep> and how avframe displays
[05:39:39 CET] <gagandeep_> guys what is chroma offset in YUV
[05:56:25 CET] <wm4> gagandeep_: sub pixel offset of how chroma pixels are aligned to luma pixels
[06:06:11 CET] <gagandeep_> i think the code in the cineform code is using to shift values of chroma pixel, not reallign
[06:06:15 CET] <gagandeep_> though i can
[06:06:27 CET] <gagandeep_> *can't get the tag for that in codec.h
[06:07:32 CET] <gagandeep_> i have implemented the wavelet transform specific to interlaced frames in cineform but there is a single white pixel always at the intersection of different colors
[06:09:42 CET] <gagandeep_> wm4: https://imgur.com/a/uvN5i what do you think could be the error here
[06:14:52 CET] <wm4> dunno, I was talking about AVChromaLocation
[06:15:15 CET] <gagandeep_> ohh
[07:04:35 CET] <rcombs> jkqxz: scale of 1 to 10, how hard would vf_tonemap be to do GPU-side
[07:34:58 CET] <JEEB> rcombs: i mean, mpv already does linearization and tone map on the gpu :D (in glsl/hlsl)
[07:35:01 CET] <rcombs> gah d6d605eb05c3ca32f591016c345eb2ad9e81c554 fucking hell
[07:35:14 CET] <rcombs> "guess nobody's ever going to need this again"
[07:35:19 CET] <rcombs> guess what, I need that again
[07:36:55 CET] <rcombs> also how does this not regress remuxing MPEGTS AAC to MP4
[07:37:37 CET] <rcombs> >The bitstream filter generated extradata is no longer available during write_header(), and as such not usable with non seekable output. The FATE tests are updated to reflect this.
[07:37:39 CET] <rcombs> what the fuck?
[07:38:38 CET] <rcombs> the fuuuuuuuuuuck?
[07:39:23 CET] <rcombs> when is jamrial online, I need a fuckin' word
[07:39:41 CET] <rcombs> it looks like he broke the existing tests and covered it up by changing them to test something different and not broken
[07:41:34 CET] <rcombs> oh, no, it does still work, just, the tests don't actually test what they're supposed to anymore
[07:42:01 CET] <rcombs> and now it works via new-extradata side-data, which means the muxer has to seek back to rewrite the header
[07:42:10 CET] <rcombs> which means segment.c stuff is broken
[07:44:15 CET] <rcombs> oh no the test was for segment.c
[07:44:29 CET] <rcombs> yeah he fuckin' broke a test, saw it was broken, and changed the input so it worked
[07:45:47 CET] Action: rcombs borrows some ffmpeg rage from wm4
[07:47:48 CET] <wm4> feel free, I have enough
[07:48:14 CET] <wm4> seemed like a nice simplification though (if it had not caused regression or w/e)
[07:48:42 CET] <rcombs> I mean, I've got nothing against moving stuff to packet extradata
[07:49:01 CET] <rcombs> but yanking the entire delay-header mechanism without checking the fucking case it was created for
[07:49:03 CET] <rcombs> what the fuck
[07:49:09 CET] <rcombs> "well _I_ don't need this"
[07:49:35 CET] <rcombs> suppose it's my fault for not keeping track of every patch on the mailing list and complaining about bullshit
[07:51:51 CET] <rcombs> so, I see two ways to deal with this
[07:52:49 CET] <rcombs> either revert d6d605eb05c3ca32f591016c345eb2ad9e81c554 and restructure the relevant code to accept the new-extradata stuff in check_bitstream somehow
[07:53:07 CET] <rcombs> which would probably mean I'd need to add a check_bitstream2
[07:53:28 CET] <rcombs> or apply my old patch that makes segment.c output seekable
[08:07:33 CET] <rcombs> oh no actually this generally breaks writing AAC to matroska with live=1
[08:07:39 CET] <rcombs> nice fuckin' job
[08:21:57 CET] <atomnuker> rcombs: it'll be pretty easy to port a number of filters to glsl
[08:22:10 CET] <atomnuker> once I finish my vulkan patchset
[10:16:26 CET] <gagandeep_> kierank: i believe the fields were to be aligned in the opposite order than the one i had coded so i have corrected that
[10:17:11 CET] <gagandeep_> kierank: https://imgur.com/a/uvN5i this is the new interlaced i am getting and below is the testcfhd (cineform sdk) output
[10:18:12 CET] <gagandeep_> kierank: https://pastebin.com/qVpYySFY this is the new git diff
[10:36:57 CET] <durandal_1707> gagandeep_: much better, perhaps you need to reverse something
[10:48:38 CET] <gagandeep_> durandal_1707: thanks, i am now looking at some chroma offset values used in cineform sdk but i am unable to figure out how to extract that info form sample header
[10:58:55 CET] <durandal_1707> gagandeep_: can't you encode gradient or something to check what is an issue?
[10:59:51 CET] <durandal_1707> because it looks like also Y plane have issue
[11:00:52 CET] <durandal_1707> why interlaced path uses different subband?
[11:03:50 CET] <gagandeep__> which one for horizontal filter or vertical/
[11:04:28 CET] <gagandeep__> horizontal is a 2,6 wavelet transform the one originally kierank had written while the vertical is temporal
[11:05:33 CET] <gagandeep__> i have used the existing buffers and rearranged them so naming convention is not followed at the moment as i am testing it
[11:10:28 CET] <gagandeep__> durandal_1707: for encoding i need adobe premiere right, and it is not free
[11:18:25 CET] <jdarnley> I know I've asked this before but what program do you people use to view differences between two binary files?
[11:19:19 CET] <jdarnley> I think bxd was recommended before which I was using but after some update on my system it just prints garbage into my terminal.
[11:19:40 CET] <jdarnley> I tried recompiling but that didn't fix it so I am now looking for a new one.
[11:27:02 CET] <durandal_1707> jdarnley: fix your system
[11:47:21 CET] <jdarnley> I would investigate but I have work to do.
[11:48:10 CET] <jdarnley> I found vbindiff which actually has a package on Arch so I've just used that.
[13:42:45 CET] <kierank> jdarnley: hxd
[14:40:43 CET] <gagandeep__> kierank: https://imgur.com/a/uvN5i this is the interlaced image output today and below is the cineform sdk output
[14:41:08 CET] <gagandeep__> kierank: https://pastebin.com/qVpYySFY the diff text
[14:41:54 CET] <gagandeep__> i have not looked at clipping for bits and chroma offset, can you look into this git diff
[14:42:16 CET] <kierank> gagandeep__: in the evening yes
[14:43:22 CET] <kierank> gagandeep__: looks good though
[14:43:56 CET] <gagandeep__> so 2-3 hrs later
[14:44:41 CET] <gagandeep__> i am looking at the function used to process the last level and it has 30000 lines of code
[14:45:00 CET] <gagandeep__> a lot of code needs to be skipped because of threaded
[14:45:52 CET] <gagandeep__> but the threaded actually runs on the computer and when i tried disabling it, the make command gave error
[14:46:10 CET] <kierank> LOL 30000 lines of ocde
[14:48:34 CET] <gagandeep__> yeah, just the function for the last level
[14:48:39 CET] <gagandeep__> o.O
[14:49:23 CET] <gagandeep__> though i have totally confirmed that the interlaced's last level is done using horizontal-temporal transform by looking at the tag values
[14:50:20 CET] <kierank> I would check you haven't broken other samples
[14:50:53 CET] <gagandeep__> broken other samples, meaning?
[14:51:34 CET] <gagandeep__> ohh, for the threaded ?
[14:53:51 CET] <gagandeep__> kierank: by the way the dequantization that you have performed should be final, right?
[14:54:15 CET] <kierank> gagandeep__: explain?
[14:54:26 CET] <wm4> michaelni, jamrial: so what about those qscale patches, ok, or should I resend them with side data containing pointers (to reduce the number of side datas to 1), or something else?
[14:54:53 CET] <gagandeep__> i mean right now for me looking at both the quantization and prescaling is a bit too much for me
[14:55:11 CET] <kierank> ok
[14:55:19 CET] <kierank> I think mine is right but it was reverse engineered remember
[14:55:22 CET] <jamrial> wm4: no, don't. doing things like calling make_writable() on it would fuck things up, as it replaces the free() function
[14:55:57 CET] <jamrial> so pointers in side data can't be done, at least without big changes
[14:55:58 CET] <gagandeep__> k, first i will check that last function used by testcfhd to produce the correct output
[14:56:21 CET] <gagandeep__> kierank: what are codebook?
[14:56:49 CET] <kierank> gagandeep__: you shouldn't need to do anything about those but they map Variable Length Codes to transform coefficients
[14:57:11 CET] <gagandeep__> thanks, i was getting dizzy looking at all the numbers of vlc
[14:57:13 CET] <wm4> good point about make_writable
[14:57:16 CET] <jamrial> and by "it" i mean the side data avbufferref containing the pointer
[14:58:14 CET] <nevcairiel> buffers probably need to remain flat structures, its not designed with complex data in mind
[14:58:15 CET] <kierank> gagandeep__: your patch needs some cosmetic work
[14:58:53 CET] <jamrial> tbh make_writable() is not really meant to be used for buffers with custom free() functions and such created with av_buffer_create()
[14:58:59 CET] <jamrial> since that stuff is not propagated
[14:59:23 CET] <wm4> we could totally introduce a copy callback that's called in this situation
[15:00:09 CET] <wm4> to be fair there are other situations where make_writable makes no sense, like for hw devices and frames
[15:02:18 CET] <gagandeep> kierank: got disconnected, did you write anything back
[15:02:43 CET] <kierank> 1:58:14 PM <"kierank> gagandeep__: your patch needs some cosmetic work
[15:03:39 CET] <gagandeep> i will do that, i was saying before that the white pixels are still a problem
[15:04:01 CET] <gagandeep> rest assured whatever i submit i will try to do so by following the guide
[15:05:03 CET] <gagandeep> k, kierank will be logging of now, been tracing the testcfhd file for 4 hrs now
[15:05:15 CET] <gagandeep> need rest :)
[15:05:19 CET] <gagandeep> goodbye
[15:05:23 CET] <kierank> bye, well done
[15:05:50 CET] <kierank> oh bleh
[15:05:55 CET] <kierank> i got confused with your images
[15:05:58 CET] <kierank> thought it was a before/after
[15:06:03 CET] <kierank> ok still problems then
[19:39:16 CET] <rcombs> jamrial: what was the motivation behind d6d605eb05c3ca32f591016c345eb2ad9e81c554 and f63c3516577d605e51cf16358cbdfa0bc97565d8?
[19:40:17 CET] <rcombs> jamrial: it seems like the latter broke a test-case, and just modifies the input to mask the breakage
[19:40:28 CET] <jamrial> f63c351657 fixed the bsf to not violate the api, as stated in the commit message
[19:41:09 CET] <jamrial> d6d605eb05 removed the delaying code that was added to mux.c to workaround that bsf violating the api
[19:42:44 CET] <rcombs> by API violation, you mean the BSF updating par_out?
[19:43:18 CET] <jamrial> yeah, that can only happen during init(), not while filtering a packet
[19:43:39 CET] <rcombs> so, fine, but the delaying wasn't to account for that
[19:43:53 CET] <rcombs> it was to account for how _muxers_ can need packet data before they write the header
[19:43:59 CET] <rcombs> it wasn't strictly BSF-specific, even
[19:44:42 CET] <rcombs> like, if you want that to use the new-extradata side-data instead of propagating from par-out, that's fine
[19:44:46 CET] <jamrial> the "autobsf" option, used to enable the functionality removed in d6d605eb05 literally says "add needed bsfs automatically (delays header until each stream's first packet is written)"
[19:45:12 CET] <rcombs> yeah, it happened to always involve bsfs at the time
[19:45:35 CET] <rcombs> but it's also applicable to e.g. mov, which needs to read data from the first packet in AC3 and EAC3
[19:46:09 CET] <rcombs> (which is _usually_ not a problem, since MOV headers are usually written at the end, but in some cases they're written early)
[19:46:12 CET] <jamrial> for that use case, there's avformat_init_muxer() that can be called before avformat_write_header()
[19:46:33 CET] <jamrial> you added the former function, even
[19:46:47 CET] <rcombs> yeah, but that doesn't get packet data
[19:47:11 CET] <jamrial> the caller (like ffmpeg.c) does
[19:47:20 CET] <rcombs> the point is that check_bitstream will be called for each stream before write_header will be called, so the muxer can handle any data from the bitstream it needs
[19:47:35 CET] <jamrial> init_muxer(), decode some packets, write_header(), write_packet()
[19:47:59 CET] <rcombs> that doesn't pass the packet data _to the muxer_, though
[19:48:18 CET] <rcombs> for example: https://gist.github.com/a3b1a9f879a7a40b230eb8676bda660f
[19:48:59 CET] <rcombs> or the case that the test you modified originally tested (remuxing ADTS into non-seekable MKV, e.g. segment.c)
[19:52:10 CET] <rcombs> if you've got a different solution for those types of issues, I'm all ears, but currently the best thing I can think of is to (1) revert d6d605eb05c3ca32f591016c345eb2ad9e81c554, (2) add code to mux.c to accept the new-extradata side-data and update stream->codecpar->extradata
[19:54:00 CET] <rcombs> perhaps only if it happens before write_header() is called? Or if you'd argue that updating codecpar->extradata after init() should only happen within an individual muxer, I could add an internal function to do that which I could call from check_bitstream (and I could re-call check_bitstream with bsf output)
[19:54:46 CET] <rcombs> am I missing something?
[19:55:29 CET] <nevcairiel> Muxer par is technically user controlled and changing it from within avformat would be invalid
[19:55:50 CET] <rcombs> nevcairiel: matroskaenc.c updates it
[19:55:52 CET] <jamrial> no, i'm trying to understand what you're trying to do with the ac3 gist above
[19:56:05 CET] <jamrial> the mov muxer has logic to write a new extradata that shows up long after the header was written
[19:56:31 CET] <rcombs> jamrial: doesn't help if you've already cut the header segment
[19:56:38 CET] <rcombs> or if the output is otherwise non-seekable
[19:56:43 CET] <rcombs> same with matroskaenc
[19:57:51 CET] <rcombs> is matroskaenc non-compliant, then?
[19:58:03 CET] <jamrial> matroskaenc.c changing the muxer par can probably be fixed
[19:58:43 CET] <rcombs> could store it in an internal struct instead
[19:58:47 CET] <rcombs> like movenc does
[19:58:55 CET] <nevcairiel> 99% of cases it probably doesn't matter anyway
[19:59:12 CET] <nevcairiel> But it's strictly invalid
[19:59:20 CET] <jamrial> rcombs: it uses a tmp codecpar for the flac extradata case
[19:59:27 CET] <rcombs> personally I'd be comfortable updating the docs to say that either caller or lavf can change it between packets
[19:59:30 CET] <jamrial> not sure why it doesn't for aac
[19:59:42 CET] <rcombs> since afaik that wouldn't break anything
[19:59:48 CET] <rcombs> jamrial: well you wrote it :P
[19:59:59 CET] <jamrial> i know, that's why i'm wondering why :p
[20:00:26 CET] <rcombs> so do you get what the point is now?
[20:01:56 CET] <jamrial> non seekable output that can't go back to rewrite the header?
[20:02:29 CET] <rcombs> yeah
[20:02:50 CET] <rcombs> in particular segment.c
[20:03:05 CET] <rcombs> which reminds me, somehow this works in dashenc (for eac3, anyway) and I don't understand how
[20:04:19 CET] <rcombs> if you don't write the packet data in the header for AC3, QuickTime maps the channels wrong
[20:04:29 CET] <rcombs> if you don't write it for EAC3, QuickTime doesn't play the track at all
[20:04:43 CET] <rcombs> no idea why, since it's all completely redundant, but ¯\_(Ä)_/¯ Apple
[20:11:07 CET] <jamrial> i'd rather not have the delayed code back in the tree, seeing how ffmpeg.c even had code written around such behavior (weird and probably wrong checks changing the output codecpar based on packet data)
[20:12:00 CET] <jamrial> but i don't know of an alternative
[20:12:45 CET] <rcombs> what was ffmpeg.c doing
[20:13:09 CET] <rcombs> I was intending for it to be completely transparent to the consumer
[20:13:10 CET] <jamrial> still does. some weird crap with field_order
[20:14:48 CET] <rcombs> oh uh
[20:14:49 CET] <rcombs> interesting
[20:15:05 CET] <rcombs> that's wacky and imo should be in mux.c
[20:15:25 CET] <rcombs> seems like another reasonable reason to delay, though
[20:15:44 CET] <rcombs> if you need to know interlacedness during write_header and that's derived from packet data
[20:16:24 CET] <rcombs> but it should either be in mux.c (if we decide that lavf updating codecpar is acceptable) or in the muxer's check_bitstream (if we decide it's not)
[20:17:37 CET] <jamrial> the latter sounds better
[20:22:12 CET] <rcombs> alright, so, to sum up: revert removal of delay, resend packets to check_bitstream post-bsfs, add check_bitstream code to movenc and matroskaenc for the relevant data
[20:23:20 CET] <rcombs> and matroskaenc moves the AAC extradata from codecpar to something internal
[20:23:31 CET] <jamrial> if there's no other way...
[20:23:44 CET] <jamrial> that one is unrelated and should be simple, yes
[20:24:09 CET] <rcombs> well I haven't got any other good ideas, at least
[20:24:50 CET] <rcombs> like, dashenc and segment.c could do this internally, but it'd be much more complex and duplicate a bunch of code, and wouldn't help non-segmented streaming output
[20:27:16 CET] <cone-194> ffmpeg 03Martin Vignali 07master:f869e54d228d: avcodec/hap : move parse_section_header to hap.c in order to be use by new bsf filter
[20:27:17 CET] <cone-194> ffmpeg 03Martin Vignali 07master:7b0cb2e032d3: avcodec/hapqa_extract_bsf : add new bsf filter
[20:28:23 CET] <cone-194> ffmpeg 03Martin Vignali 07master:414c0e27a981: fate/hap : move decoding test to a separate file
[20:28:24 CET] <cone-194> ffmpeg 03Martin Vignali 07master:f56ee7403384: fate/hap : add test for hap encoding
[20:32:48 CET] <cone-194> ffmpeg 03Aman Gupta 07master:3172b31223e1: avcodec/mediacodecdec: fix immediate EAGAIN with buffered packet
[22:22:13 CET] <rcombs> nevcairiel: jamrial: what would you think of saying that muxpar can be updated by lavf after avformat_write_header, for implementation simplicity?
[22:22:24 CET] <rcombs> consumers shouldn't be doing anything with it after that anyway
[22:23:13 CET] <rcombs> I'm not super attached to the concept, but it'd make things easier and reduce per-muxer work
[23:05:03 CET] <cone-194> ffmpeg 03James Almer 07master:2b6699f024ae: avcodec/hapqa_extract: move the reference in the bsf internal buffer
[00:00:00 CET] --- Wed Mar 14 2018