[Ffmpeg-devel-irc] ffmpeg.log.20170920

[burek]

[00:20:33 CEST] <DHE> kevinn_: see the doc/examples directory
[00:21:26 CEST] <kevinn_> I see it, thank you!
[02:55:14 CEST] <kevinn_> DHE: hey are you familiar with this example: http://git.videolan.org/?p=ffmpeg.git;a=blob;f=doc/examples/decode_video.c;hb=HEAD
[02:57:32 CEST] <DHE> kevinn_: got a specific question?
[02:58:05 CEST] <kevinn_> I just had a question as to where my actual picture data is... Is it in frame->data[0] on line 78?
[02:59:02 CEST] <DHE> it depends on the format. for planar data, frame->data[0] will have all the data in frame->linesize[0] strides
[02:59:47 CEST] <kevinn_> I think the data I am passing is what is called packed data, so will frame->data[0] also be packed?
[03:00:13 CEST] <kevinn_> or am I confusing that term with something else
[03:01:00 CEST] <DHE> the opposite of planar means that each compontent (Y,U,V or R,G,B) will be in frame->data[0], data[1], and data[2].
[03:02:01 CEST] <kevinn_> Ah! then I am using planar! okay cool, thank you so much for the help!
[03:02:12 CEST] <DHE> or do I have them backwards... :/
[03:02:24 CEST] <DHE> I don't do this very much so I might have my terms mixed up slightly
[03:03:49 CEST] <kevinn_> okay well let me test and see if that is the case
[03:04:36 CEST] <SavinaRoja> I'm tying to use the ffmpeg static builds for linux on linux server, but I keep getting an error each time I use ffmpeg on output "Error initializing output stream 0:0"
[03:09:39 CEST] <SavinaRoja> https://pastebin.com/Y8J8UEjy is a minimal reconstruction
[03:11:08 CEST] <DHE> no error reported from x264... that's unexpected
[03:11:27 CEST] <DHE> as an experiment, before the output filename put -crf 25
[03:11:46 CEST] <DHE> since you didn't specify any parameters
[03:12:40 CEST] <SavinaRoja> https://pastebin.com/TujcuthM
[03:13:34 CEST] <DHE> not that..
[03:14:01 CEST] <SavinaRoja> DHE: the original script, which runs on my local machine, has a lot more specification
[03:24:54 CEST] <SavinaRoja> DHE: was there some other information I should provide?
[03:25:35 CEST] <DHE> nothing comes to mind. someone else might be able to provide more insight
[03:27:56 CEST] <SavinaRoja> I may just need to have their admin actually update the system... local code runs as expected
[03:31:13 CEST] <SavinaRoja> but something else might be wrong, because this same error happens with the system installed version
[03:42:26 CEST] <kevinn_> DHE: so I checked frame->data[0] byte for byte and but the data seems totally incorrect...
[03:42:38 CEST] <kevinn_> and it doesn't even have the full framebuffer size
[03:42:46 CEST] <kevinn_> it segfaults part way through
[03:44:40 CEST] <DHE> kevinn_: did you check the linesize? see the save function from the sample application
[03:46:24 CEST] <kevinn_> I ran x264 on a 1376x768 buffer with each pixel being 32 bits. The line size says it is 1376 but only reaches index 391151 (color wise).
[03:46:38 CEST] <kevinn_> save function... do you mean decode?
[03:47:03 CEST] <kevinn_> or do you mean pgm_save
[03:47:04 CEST] <kevinn_> ?
[03:47:17 CEST] <DHE> yeah the save function
[03:48:10 CEST] <kevinn_> okay let me make it look identical to that
[03:49:08 CEST] <DHE> hmm... 768 * 1376 should be good, but it wasn't
[03:51:18 CEST] <kevinn_> wait a minute, this doesn't make sense: fwrite(buf + i * wrap, 1, xsize, f);. buf is a char array, so each element is 1 byte long. xsize is 1376. This function should copy line by line, but that can't be possible because each of my colors is 32 bits. so xsize should really be 1376 * 4
[03:51:21 CEST] <kevinn_> right?
[03:52:47 CEST] <DHE> then it's probably in 3 planes. it'll be either red or luma in frame[0]
[03:52:55 CEST] <DHE> check if the other frames have data
[03:54:32 CEST] <kevinn_> okay the other frames do have data...
[03:54:42 CEST] <kevinn_> is the alpha channel just chopped off?
[03:55:25 CEST] <DHE> h264 doesn't do transparency
[03:55:59 CEST] <kevinn_> right, okay that's fine
[03:56:27 CEST] <kevinn_> but how can I tell if these points are yuv or rgb?
[03:57:21 CEST] <kevinn_> the numbers are jumping around so much, I fear it might by yuv...
[03:58:00 CEST] <DHE> AVFrame has a `format` field which will be one of AV_PIX_FMT_*  such as YUV420P
[04:01:30 CEST] <kevinn_> ahh, if I set it to AV_PIX_FMT_BGR32 will it be one buffer?
[04:01:44 CEST] <kevinn_> like packed into frame[0]?
[04:01:48 CEST] <DHE> well you get AVFrames from the decoder, so it's the decoder that provides them
[04:02:54 CEST] <kevinn_> sorry, I don't get what you mean
[04:04:27 CEST] <DHE> the decoder provides frames in the format it wants. for h264 that's almost always YUV
[04:05:38 CEST] <kevinn_> hmm, okay so youre saying I can't change the format?
[04:07:48 CEST] <SavinaRoja> DHE: very very odd, I can do h265 output
[04:13:25 CEST] <kevinn_> DHE: can I not change the format? I added this bit of code: frame->format = AV_PIX_FMT_BGR32; and it didn't have an effect on the output
[04:39:25 CEST] <kevinn_> Does anyone know how to change the decode format for libavcodec?
[04:42:56 CEST] <jeffW_> Hi, I'm trying to build ffplay.exe by follow this guide from the wiki:   https://trac.ffmpeg.org/wiki/CompilationGuide/MSVC      I am using MSYS2, and have  performed      pacman -S make mingw-w64-x86_64-gcc mingw-w64-x86_64-sdl2 mingw-w64-x86_64-diffutils mingw-w64-x86_64-pkg-config
[04:43:12 CEST] <jeffW_> I have opened: "VS2015 x64 Native Tools Command Prompt" and run  vcvarsall.bat x86_amd64     then in  MSYS2/Mingw-w64  I  have run:   export PATH="/c/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64":$PATH
[04:43:23 CEST] <jeffW_> Then I checked this by typing      which link      produced:    /c/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/link
[04:43:55 CEST] <jeffW_> I then performed:    ./configure --prefix=/c/ffmpeg-output-shared --disable-network --disable-postproc --enable-gpl --enable-version3 --enable-asm --disable-encoders --arch=x86_64 --disable-ffserver --disable-ffmpeg --disable-ffprobe --disable-doc --enable-shared --disable-static --disable-bzlib --disable-libopenjpeg --disable-iconv --disable-zlib --pkg-config=pkg-config --toolchain=msvc
[04:44:04 CEST] <jeffW_> This gave me the error:   cl is unable to create an executable file.  If cl is a cross-compiler, use the --enable-cross-compile option.  Only do this if you know what cross compiling means.  C compiler test failed.
[04:44:21 CEST] <jeffW_> As it says in the wiki, if you get this error, it could be caused by using the wrong link.exe (But, as I explained, I already verified that is not the issue), or, "This can also indicate that you are mixing 64bit and 32bit versions of cl.exe and link.exe."
[04:44:31 CEST] <jeffW_> Since I verified via   which link   and   which cl   that msys2/mingw-w64 is getting them from  /c/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/link    I don't see how that could be possible.
[04:44:43 CEST] <jeffW_> Can anyone help me figure out where I am going wrong here?
[05:29:40 CEST] <kepstin> kevinn_: the video format returned by the decoder is based on the video stream itself and the decoder implementation. If you want something different, then use libswscale to convert it (but note that the conversion can cause some loss in precision)
[05:30:29 CEST] <kevinn_> kepstin: the video stream itself is in that format, for some reason the decoder is splitting the bits up
[05:31:17 CEST] <kepstin> how was the video made?
[05:31:34 CEST] <kevinn_> the video was made using x264
[05:31:37 CEST] <kepstin> if the decoder's giving you YUV, then the stream is in YUV
[05:31:57 CEST] <kevinn_> ahh, actually that was a bit inaccurate when I stated that
[05:32:06 CEST] <kevinn_> It actually turned out to be rgb
[05:32:10 CEST] <kepstin> note that when using x264 with the ffmpeg wrapper, you have to use the 'libx264rgb' encoder if you want an RGB stream.
[05:32:43 CEST] <kevinn_> but it is still splitting it up into three buffers which I don't want
[05:33:14 CEST] <kevinn_> anyway to make it output into one buffer? something like this? AV_PIX_FMT_BGRA
[05:45:39 CEST] <kevinn_> kepstin: if you're still around is there anyway you can help me have avcodec_receive_frame output in a different format? such as AV_PIX_FMT_BGRA?
[05:46:07 CEST] <kevinn_> I implemented get_format to see what formats are supported and the only one it printed was 82
[05:46:38 CEST] <kepstin> kevinn_: use libswscale to convert it to the format you want. if it's rgb with the same depth it's a lossless conversion.
[05:47:32 CEST] <kevinn_> but is there no way to have libav output in the right format? what purpose does AV_PIX_FMT_BGRA serve if it can't be used?
[05:49:11 CEST] <kepstin> some encoders/decoders use that rather than a planar rgb format, and you can convert to/from it
[05:49:58 CEST] <kepstin> the formats supported depend on what the video stream contains + how the person who implemented the decoder decided to handle it
[05:50:43 CEST] <kepstin> with the ffmpeg h264 decoder, since the internal way it represents rgb is planar, that's what you get out.
[05:51:12 CEST] <kepstin> adding support for other formats would mean converting in the decoder, and that would be silly since that's what libswscale is for
[05:51:45 CEST] <kevinn_> okay, I see, thank you so much for your help, I will use libswscale.
[05:52:21 CEST] <kevinn_> Is libswscale quick enough where if I do it to every frame it will still look as if it is streaming?
[05:52:31 CEST] <kevinn_> or will I likely have to multithread?
[05:52:44 CEST] <kepstin> not if you're running it on a 100mhz pentium with a 4k stream
[05:52:52 CEST] <kepstin> need more info
[05:53:16 CEST] <kepstin> (on any reasonably modern computer, converting from planar to packed rgb should be ridiculously fast)
[05:53:22 CEST] <kevinn_> 2k stream on an arm processor
[05:53:40 CEST] <kevinn_> rpi3
[05:53:54 CEST] <kepstin> hmm, why do people keep trying to do video stuff on rpi :(
[05:54:21 CEST] <kepstin> it'll probably work fine tho for decoding and converting to bgra.
[05:54:30 CEST] <kevinn_> you don't think it'll be fast enough?
[05:54:41 CEST] <kevinn_> okay :)
[05:55:01 CEST] <kepstin> most of the people who run into issues on rpi are trying to encode, which is rather more cpu intensive.
[05:55:46 CEST] <kepstin> but really, you're just gonna have to try and see for yourself :/
[06:03:32 CEST] <jeffW_> Could anyone help me figure out how to get FFmpeg to compile with MSYS2 ? I think I've tried everything, and the best I've gotten is exe's that crash when I try to use them.
[07:28:52 CEST] <pankaj_> I'm trying to compress video but not getting success for mp4 files
[07:31:09 CEST] <pankaj_> https://pastebin.com/nhgyxeAK
[07:31:19 CEST] <pankaj_> command line
[08:29:02 CEST] <dongs> does ffmpeg do MMT streams
[08:29:44 CEST] <dongs> https://en.wikipedia.org/wiki/MPEG_media_transport i.e. thsi
[08:31:45 CEST] <dystopia_> it can probably decode it
[08:32:01 CEST] <dongs> its a container, not encoding format
[08:32:40 CEST] <dystopia_> then i dunno
[08:33:38 CEST] <dongs> http://lists.ffmpeg.org/pipermail/ffmpeg-devel-irc/2015-October/003151.html
[08:33:41 CEST] <dongs> some discussion from 2015
[08:34:41 CEST] <dongs> http://gstreamer-devel.966125.n4.nabble.com/MPEG-MMTP-Element-td4684325.html
[08:34:42 CEST] <dongs> RIP
[08:36:17 CEST] <dongs> i've got an entire satellite doing ~500mhz worth of shit with MMTP, i wouldnt mind looking at some of it
[08:36:55 CEST] <dystopia_> which sat out of curiosity?
[08:37:17 CEST] <dongs> 132E so probly of no interest to you
[08:37:18 CEST] <dystopia_> i can receive 5w to 28.2e
[08:37:24 CEST] <dystopia_> ahh
[08:37:26 CEST] <dongs> yea, other side of the world :D
[08:38:21 CEST] <dongs> hmm i see 188byte packets in there with what looks like TS
[08:38:25 CEST] <dongs> but alignment is inconsistent
[08:38:45 CEST] <dystopia_> do you have transedit?
[08:38:59 CEST] <dystopia_> i believe there is a free version
[08:39:15 CEST] <dongs> yeah i have transedit/dvbiewer license. that isnt gonna help tho, this is lower level than that
[08:39:15 CEST] <dystopia_> very good at analyzing transponders
[08:41:23 CEST] <dongs> ya ffmpeg tries to sync to ts adn fails, but it definitely looks like mpeg2ts inside some weird container on top of it
[08:41:53 CEST] <dystopia_> does ts-doctor tell you anything?
[08:42:02 CEST] <dystopia_> has a free trial also if you have no license
[08:48:14 CEST] <dongs> anyaw,y im almsot certain its mmtp with mpegts as its payload
[08:48:27 CEST] <dongs> because there's definitely ts packets inside but t hey're intermixed with garbage
[08:48:40 CEST] <dongs> maybe ill write some simple thing to go through and pull out everything with TS sync byte every 188bytes
[08:48:43 CEST] <dongs> and see if that works
[08:54:28 CEST] <dongs> that was easy
[08:59:33 CEST] <dystopia_> it worked?
[08:59:58 CEST] <dongs> it worked to pull about half the packets out and at least it looks like a transport stream etc, but too many cc errors, maybe packets are out of order or something in the mmt shit
[09:00:12 CEST] <dongs> or it could be several transport streams muxed into a single thing (which is hte point of mmt i guess)
[09:03:33 CEST] <dongs> got a bug where i forget to write last packet after sync loss
[09:03:38 CEST] <dongs> maybe that'll fix it.
[09:13:59 CEST] <dongs> damn
[09:14:04 CEST] <dongs> the garbage nbetween ts bytes is variable too
[09:14:20 CEST] <dongs> and mmtp doesnt seem to have any sync header type shit so i dont actually know if its mmtp stuff or not
[09:32:40 CEST] <dongs> oh well giving up and moving my shit to the actual satellite i wanetd to be at. looks like opensource isnt gonna deliver anything useful this time either
[10:36:36 CEST] <chuckleplant> Hi, I'm decoding an incoming RTSP stream. Everything works fine but I sometimes see some stutter. This is because I'm not using presentation timestamp or framerate at all. If I display frames as they come, they are ordered and fine... however, now that I want to use the PTS, I see that from the RTSP stream they are not in strict order
[10:38:46 CEST] <chuckleplant> Here's a plot of the PTS of my AVFrames after I decode them: https://imgur.com/a/gNlhI
[10:39:03 CEST] <chuckleplant> Should I be sorting frames as I queue them in my display buffer?
[10:39:36 CEST] <BtbN> decoders should give you frames in display order
[10:40:22 CEST] <chuckleplant> They are in display order (their PTS is not) .. but I'm not sure how to display them for their appropriate duration
[10:40:45 CEST] <BtbN> that makes no sense
[10:41:12 CEST] <BtbN> either they are in display order, so their PTS are monotonically increasing, or they are not
[10:41:57 CEST] <chuckleplant> I say they're in display order, because I see no problem when visualizing the image, however, their PTS plot is the one I linked above
[10:42:57 CEST] <BtbN> that plot makes no sense. In no location of a decoding chain
[10:48:17 CEST] <chuckleplant> I sure made a mistake somewhere.. let me rephrase the question and forget about the PTS for a sec. Assuming a proper decoding pipeline. What is the proper way to display frames at the correct framerate? I already have a buffer of decoded AVFrames
[10:49:01 CEST] <BtbN> You display them at the time their pts says, relative to from where you started playing
[10:51:25 CEST] <JEEB> the PTS are just a way for you to synchronize on the time line (taking into account that different streams can and will have their own time_base)
[10:52:03 CEST] <chuckleplant> Ok I see now
[10:52:32 CEST] <chuckleplant> In my decoding pipeline the PTS is not set. When I get a frame with avcodec_decode_video2, the PTS value is always the same
[10:52:48 CEST] <JEEB> for example audio packets might be on a time base of 1/4800, and then video on 1001/24000 for example
[10:53:07 CEST] <chuckleplant> I must say this is a live RTSP H264 stream
[10:53:09 CEST] <JEEB> and then the PTS should be the thing to synchronize between those
[10:53:29 CEST] <JEEB> yea, it might be garbage in garbage out if the input doesn't have timestamps
[10:53:43 CEST] <JEEB> I recommend looking at it with ffprobe's -show_frames for a while
[10:53:52 CEST] <JEEB> if you want to parse it as json there's -of json
[10:54:27 CEST] <JEEB> there's a way to limit ffprobe to a certain amount so you can stop it after X seconds without ctrl+C
[10:54:36 CEST] <JEEB> (just don't ask me the parameter :P)
[10:54:55 CEST] <chuckleplant> JEEB, the incoming stream does have PTS set in all the NAL units
[10:55:24 CEST] <chuckleplant> Maybe I should be setting the AVPacket PTS somewhere before decoding?
[10:55:27 CEST] <chuckleplant> hmmm
[10:58:24 CEST] <chuckleplant> JEEB, thanks! Not completely clear on the time_base.. Las I checked my time_base was  0/2, does that make sense?
[11:10:25 CEST] <chuckleplant> So I now set the appropriate AVPacket->pts from the incoming nal units. However, the PTS after decoding is still garbage
[11:10:39 CEST] <chuckleplant> The decoder just does not set it
[11:10:42 CEST] <chuckleplant> does this make sense?
[11:43:21 CEST] <BtbN> chuckleplant, if your timebase is 0, that's not too surprising.
[11:43:40 CEST] <BtbN> You also shouldn't need to parse any NAL units. The demuxer should give you valid timestamps already.
[11:46:32 CEST] <JEEB> there's a h264 demuxer for raw annex B (and the rtsp protocol should pretty much stick you with it)
[11:46:47 CEST] <JEEB> so that you can just feed the "rtsp://.." url to lavf
[11:46:56 CEST] <JEEB> and you get AVPackets from it
[11:50:58 CEST] <chuckleplant> JEEB, I thought I only needed a demuxer when I did not have a raw H264 stream. WHat I'm getting in my decoder are nal units with a presentation timestamp set. Should I feed that to a demuxer instead of filling the AVPackets manually?
[11:51:58 CEST] <JEEB> I would test that out, although if your input is RTSP you can just feed that to libavformat
[11:52:05 CEST] <JEEB> so that you don't have to add the H.264 stuff
[11:52:16 CEST] <JEEB> it's just rtsp and the rtsp protocol picks whatever demuxer you need
[11:54:30 CEST] <chuckleplant> Assuming I can't use libavformat, what must be set to AVPacket for the decoder to provide a PTS in the decoded AVFrames?
[11:54:47 CEST] <chuckleplant> Due to the nature of my application and codebase I can't use libavformat, just libavcodec
[11:58:12 CEST] <JEEB> ok, it seems like the parsing stuff is in the H.264 parser and the "demuxer" is just a probe function :)
[11:59:42 CEST] <JEEB> although it does limited stuff with DTS/PTS
[12:00:48 CEST] <JEEB> chuckleplant: basically you need to set valid DTS at the very least, might need the PTS as well although for that you'd have to parse the bit stream more (the decoder does the reordering for you, but the values you get for PTS etc might be funky
[12:00:53 CEST] <JEEB> so start with DTS
[12:00:55 CEST] <JEEB> then PTS
[12:01:45 CEST] <chuckleplant> JEEB, thanks. My h264 parser only provides a PTS for each Nal unit. How can I determine the DTS from that?
[12:11:47 CEST] <BtbN> Doing the re-ordering magic manually is a mess
[12:12:28 CEST] <BtbN> I don't think you strictly needs dts though, if your packets come in in the right order
[12:14:01 CEST] <Wodjin> Hello everyone, i'm using ffmpeg to try and add text to a video, the text has a background box containing it and I'm wondering if there is any way to fade the text and the box both in and out? more info on my issue here: https://video.stackexchange.com/questions/22379/fade-text-with-background-with-ffmpeg
[12:15:38 CEST] <BtbN> you'll probably have to come up with a forumla to do the fading based on frame count or timestamps
[12:15:49 CEST] <BtbN> and then somehow put that into the overlay/blend filters
[12:16:59 CEST] <chuckleplant> BtbN, I don't want to re-order, just to know the PTS for rendering. I know my decoded frames are in the right order (because they render properly).  It's just that there's a time stutter on the display side, decoding is fine... besides the PTS thing.
[12:17:24 CEST] <BtbN> If you want to deduce the input dts, you will have to re-order the pts.
[12:18:33 CEST] <Wodjin> hmm are you talking about something like fontcolor_expr? I've used that to fade the text itself but from what I've seen on other ppl's posts it looks like there is no alpha channel on the container box for the drawtext function
[12:18:50 CEST] <Wodjin> also, thanks for the quick feedback BtbN!
[12:20:22 CEST] <chuckleplant> BtbN, you mentioned I may not need dts. Assuming packets are ordered, how would you proceed? (thanks for your patience)
[12:20:43 CEST] <BtbN> chuckleplant, just decode them in the right order, and pass through the pts. The decoder should re-order them
[12:20:57 CEST] <BtbN> you need a valid and correct timebase though. if it's 0, the timestamps will be 0 or nonsensical
[12:21:26 CEST] <BtbN> Wodjin, the blend filter has an all_expr argument, where you can stick in a formula
[12:21:33 CEST] <BtbN> https://ffmpeg.org/ffmpeg-filters.html#blend_002c-tblend
[12:21:40 CEST] <BtbN> There are some examples for fading
[12:23:10 CEST] <chuckleplant> I see, then maybe it's about the time_base. So far, no matter what I do, my decoder always sets (or doesn't) the same PTS : -9223372036854775808   , which is nonsensical indeed
[12:23:45 CEST] <BtbN> If you are parsing the raw h264 yourself, the timebase is in the SPS
[12:24:05 CEST] <chuckleplant> I see, thanks!
[12:30:46 CEST] <chuckleplant> BtbN, I'm testing with fprobe, is time_base in api, equivalent to timecode?
[12:31:00 CEST] <BtbN> what?
[12:31:10 CEST] <chuckleplant> BtbN, please forget that last line
[12:31:20 CEST] <BtbN> The time_base is the increment the timestamps represent
[12:31:53 CEST] <BtbN> so if the timebase is 1/30, and the timestamp increments from 1 to 2, it means 1/30 of a second
[12:32:54 CEST] <chuckleplant> BtbN, so ffprobe does print a time_base of 1/90000,  but my application does not.. :(   Do I need to instantiate a AVStream? Or should the CodecCtx be getting this information too?
[12:34:25 CEST] <BtbN> I thought you're only using avcodec?
[12:34:36 CEST] <chuckleplant> Yes, I'm testing the incoming stream with ffprobe
[12:35:09 CEST] <BtbN> avcodec will not give you the timebase.
[12:35:13 CEST] <BtbN> it expects it as an input
[12:35:21 CEST] <BtbN> So you will have to peel it out of the raw bitstream yourself
[12:35:30 CEST] <chuckleplant> I see
[12:35:32 CEST] <BtbN> which is why using more then just avcodec is probably a good idea
[12:54:10 CEST] <Wodjin> BtbN, sorry for the delayed response, can i apply a blend filter to the drawtext filter only? to make only the result of the drawtext fade in and out? Sorry I never used this filter so I'm slightly lost :)
[12:54:33 CEST] <BtbN> you'll have to build a complex filter chain, yes
[12:55:33 CEST] <Wodjin> allright, thanks for the pointer, i'll dig deeper into it
[13:18:37 CEST] <chuckleplant> BtbN, i've been reading on sps and pps.. is there a method in ffmpeg to parse the sps and get back the time_base info? I can use libavformat, just not the whole h264 parser. Can't find in ffmpeg code where time_base is set either
[13:26:46 CEST] <BtbN> chuckleplant, it will be somewhere in the h264 parsing code in avcodec
[13:27:02 CEST] <BtbN> but likely private API to use individually
[14:24:47 CEST] <echelon> hi
[14:25:08 CEST] <echelon> i'm recording from v4l2 webcam source
[14:25:26 CEST] <echelon> but would like to be able to do playback while it's recording
[14:25:35 CEST] <echelon> i have to start from the beginning every time
[14:25:48 CEST] <echelon> i'm using an mkv container, is there something else i could do?
[15:23:58 CEST] <c_14> echelon: -live 1
[15:24:27 CEST] <echelon> c_14: to ffmpeg?
[15:24:31 CEST] <c_14> yes
[15:24:35 CEST] <c_14> as an output option
[15:24:52 CEST] <echelon> what does that do to file structure
[15:25:11 CEST] <c_14> It just tells the matroska muxer not to write the CRC elements
[15:25:25 CEST] <echelon> cool, thanks! :)
[15:31:20 CEST] <advin> what is libavcodec/videotoolbox.h file is?
[15:38:23 CEST] <DHE> it's a header file. what more do you need?
[15:40:20 CEST] <paradoxspiral> Hey, I'm encoding a vid to x265 10bit. htop shows that ffmpeg uses between 35-40% of my cpu (r7 1700), io should be fine. Is it expected that it doesn't use my cpu's full potential, if so, why?
[15:41:04 CEST] <c_14> how big is the video?
[15:41:20 CEST] <c_14> (resolution)
[15:41:30 CEST] <DHE> when the number of CPU cores is high multi-threading becomes less effective and CPU utilization doesn't reach 100%
[15:41:45 CEST] <advin> Thanks, I know it's headerfile, but what I wonder is the purpose/use of that header file.
[15:41:59 CEST] <paradoxspiral> 1280x720, c_14
[15:42:32 CEST] <paradoxspiral> 4 hours long. i left it on over the night and it got 1h done
[15:43:43 CEST] <advin> h265 require more cpu time than h264.
[15:44:13 CEST] <paradoxspiral> tbh I'd expect at least 60% usage, iirc AMD always advertize their great encoding potential
[15:46:06 CEST] <DHE> that's more a statement of x265 than on AMD. the software isn't making full use of the hardware
[15:46:19 CEST] <DHE> believe me, I've run a ryzen at 100%
[15:46:36 CEST] <paradoxspiral> yeah, i was wondering
[15:46:45 CEST] <paradoxspiral> whether it was because of ffmpeg or x265
[15:46:54 CEST] <paradoxspiral> does that even make sense? maybe not
[15:48:13 CEST] <DHE> while ffmpeg (the CLI itself) could use some multi-threading improvements, I think x265 is the larger bottleneck if you're running at 1/8 realtime speed and not using the CPU very well
[15:50:00 CEST] <paradoxspiral> I guess nothing I can do then >>
[16:03:31 CEST] <alexpigment> hey guys, is there any way to build ffmpeg such that x264 uses a max number of threads, but is otherwise "auto"?
[16:04:17 CEST] <c_14> I'm pretty sure x264 doesn't report the number of threads it uses internally to ffmpeg
[16:04:22 CEST] <c_14> (though it might, not sure about the api)
[16:04:30 CEST] <c_14> would require patching ffmpeg (and/or x264) though
[16:04:57 CEST] <c_14> paradoxspiral: if you knew beforehand you could split the source file into x sections, encode those with different processes and then stitch them together afterwards
[16:06:05 CEST] <paradoxspiral> c_14: ah, that'd work I think. ty
[16:06:27 CEST] <alexpigment> c_14: so the -threads command can't be limited in some mathematical way?
[16:06:40 CEST] <c_14> well it can
[16:06:43 CEST] <c_14> if you use your shell
[16:08:23 CEST] <c_14> well
[16:08:33 CEST] <c_14> you can always set -threads to the max number
[16:08:42 CEST] <c_14> but then you'll always have that many threads
[16:11:43 CEST] <alexpigment> is there any downside to specifying 16 threads, then running it on a dual core non-hyperthreaded system?
[16:12:02 CEST] <alexpigment> in other words, is it possible to overload a system by defining more threads than x264 would automatically allocate by itself?
[16:12:25 CEST] <c_14> yeah, you get threading overhead
[16:12:37 CEST] <alexpigment> yeah, that's my thought
[16:13:24 CEST] <alexpigment> i just wish you could define something like "-threads (0,<17)"
[16:14:03 CEST] <alexpigment> I suppose I could edit the x264 source code to do this somehow
[16:15:28 CEST] <JEEB> that's something that you could in the libavcodec level
[16:15:42 CEST] <JEEB> -threads_max etc
[16:16:12 CEST] <JEEB> libx264 just wants a number or auto from you which you then adjust in libx264.c
[16:16:24 CEST] <JEEB> no need to touch libx264 itself unless there's some specific reason :)
[16:17:48 CEST] <alexpigment> JEEB: if the numeric value of auto is 0, do you think it would still be possible to limit to 16 at the libavcodec level?
[16:18:11 CEST] <alexpigment> because it would have to limit to what auto turns out to be on the given system, which I suspect is actually in libx264
[16:18:27 CEST] <alexpigment> (in reality, it's 1.5x the number of logical cores)
[16:20:28 CEST] <JEEB> alexpigment: yes, you just make another avcodec option for libx264 which is 'threads_max' or so
[16:20:42 CEST] <JEEB> although... if you want to use auto
[16:20:51 CEST] <alexpigment> yeah, i'm wanting to use auto
[16:21:12 CEST] <alexpigment> to prevent using 16 threads on a system with 2 or 4 logical cores
[16:21:13 CEST] <JEEB> ok, then you might need to poke libx264's developers first, then the libavcodec wrapper
[16:21:24 CEST] <alexpigment> yeah that's what I'm thinking
[16:21:25 CEST] <JEEB> wait how the fuck would it go so high :P
[16:21:35 CEST] <alexpigment> well i need to limit it to 16
[16:21:37 CEST] <JEEB> since it's 3*<virtual cores> / 2
[16:21:42 CEST] <JEEB> IIRC
[16:21:45 CEST] <alexpigment> right
[16:22:13 CEST] <alexpigment> i'm just thinking that it would be hard to limit it in libavcodec when I'm using auto because it would have to know what the numeric value of auto is
[16:23:02 CEST] <JEEB> according to my calc the thread number for four virtual (as I'm not sure what "logical" is) cores is (4*3) / 2 => 6
[16:23:07 CEST] <alexpigment> yes
[16:23:09 CEST] <alexpigment> that is correct
[16:23:15 CEST] <JEEB> which is what auto would go for
[16:23:18 CEST] <alexpigment> right
[16:23:27 CEST] <alexpigment> but what if you have a system with 32 logical cores
[16:23:36 CEST] <JEEB> right, yes. now that example makes sense :P
[16:23:36 CEST] <alexpigment> then you get 48 threads and 4K encoding fails
[16:23:54 CEST] <JEEB> I'm an asshole in that sense
[16:24:21 CEST] <JEEB> but yea, adding another parameter into the libx264 API shouldn't be too hard :P
[16:24:30 CEST] <JEEB> poke #x264dev if the core devs of x264 are interested in that
[16:24:34 CEST] <alexpigment> so i'm wanting to keep auto up to a point, but go no further than 16, because there's a warning explicitly saying that anything above 16 is recommended
[16:24:45 CEST] <alexpigment> will do. thanks JEEB
[16:25:01 CEST] <JEEB> I could see that as useful for myself as well I guess
[16:25:12 CEST] <JEEB> although currently I'm just defining thread counts manually :P
[16:29:56 CEST] <alexpigment> JEEB: yeah, for a scenario where you're only using one system, it's not a huge problem per se. although on a 16-core/32-logical system, you would have to know to not set it to auto, which is kinda bad from a user perspective
[16:30:10 CEST] <alexpigment> and of course, we're about to get a hell of a lot more of those on the market
[16:30:55 CEST] <JEEB> alexpigment: well you usually have a transcoding system and that's where I so far have handled limiting threads
[16:31:09 CEST] <JEEB> looking at the already running jobs and the core count etc
[16:31:33 CEST] <alexpigment> Sure, if you've got a transcoding system set up and you know what you're doing, it's not a problem
[16:31:52 CEST] <alexpigment> I think that's a pretty niche situation though to be honest
[16:32:16 CEST] <JEEB> I'd hope most places providing such services have some sort of workload control system :D
[16:33:21 CEST] <alexpigment> Yeah, I'm just thinking about a dude with ffmpeg trying to encode a single video file
[16:33:34 CEST] <alexpigment> Doesn't know what threads are, so he doesn't specify them
[16:33:46 CEST] <alexpigment> Just using the bare minimum to get his file from point A to B
[16:33:52 CEST] <JEEB> but yea, as I noted it makes sense to have a 'max_threads' or so param
[16:34:06 CEST] <alexpigment> Yes, and it makes sense for that to be automatically set to 16
[16:34:07 CEST] <JEEB> what the default for it would be is up to debate
[16:34:37 CEST] <alexpigment> well, i'm specifically referring to this message: "[libx264 @ 000000000054acc0] Application has requested 48 threads. Using a thread count greater than 16 is not recommended."
[16:35:37 CEST] <JEEB> yea, that warning also I don't remember if it depends on the resolution (vertical one) or not
[16:35:40 CEST] <JEEB> if it doesn't then yes
[16:35:53 CEST] <JEEB> otherwise you might want to add calculation or so
[16:35:58 CEST] <alexpigment> right
[16:36:34 CEST] <alexpigment> ok, well I'll be looking into this more. thanks for the suggestions and I'll see what I can come up with by digging around the source code
[16:37:28 CEST] <bjorn__> Hey all! Im working on this: https://trac.ffmpeg.org/ticket/4443
[16:37:28 CEST] <JEEB> oh
[16:38:03 CEST] <bjorn__> Ive got it working so that I can generate a transparent gif just fine  but only a single frame.
[16:38:53 CEST] <bjorn__> Multiple frames cause problems with the gif encodings oprtimizations, so I cant successfully create an animated gif with transparencies.
[16:39:09 CEST] <c_14> bjorn__: you probably want #ffmpeg-devel for devel-related talk
[16:39:22 CEST] <bjorn__> Oops, you are right  I
[16:39:26 CEST] <bjorn__> m in the wrong room.
[16:39:33 CEST] <bjorn__> Thanks!
[16:50:41 CEST] <alexpigment> JEEB: so regarding the pthreads.c file, I've found the section you're talking about
[16:51:15 CEST] <alexpigment> so basically the warning triggers if threads is greater than max auto threads
[16:52:01 CEST] <alexpigment> is it possible to edit that line such that it actually sets threads to max auto threads if threads is greater?
[16:52:32 CEST] <alexpigment> i'm usually don't write actual code myself, so I'm a bit green on how to do that
[17:02:07 CEST] <JEEB> alexpigment: with a quick look the check was arbitrary, so actually the warning should be shut up with libx264
[17:02:25 CEST] <JEEB> since libx264 itself limits auto threads by default
[17:02:33 CEST] <alexpigment> hmmm
[17:03:01 CEST] <alexpigment> and max auto threads should be 16, right
[17:03:01 CEST] <alexpigment> ?
[17:03:05 CEST] <JEEB> no
[17:03:42 CEST] <alexpigment> then what is it?
[17:03:46 CEST] <JEEB> you'd have to read the code to check what the limits are within x264
[17:03:56 CEST] <alexpigment> can you point me to the particular file?
[17:04:25 CEST] <JEEB> i'm on a phone amd I'm just repearing what bugmaster said on #x264dev
[17:04:29 CEST] <JEEB> so no
[17:04:39 CEST] <alexpigment> k
[17:18:27 CEST] <BytesBaconBB> Has anyone done any encoding with hardware GPU acceleration? How's the quality vs software x264 encoding?
[17:19:36 CEST] <Johnjay> With that segment filter is there a way to segment a video/audio clip into chunks of length N
[17:19:40 CEST] <Johnjay> except for the last one?
[17:19:51 CEST] <Johnjay> e.g. if it's hour and 45 minutes and you split it into 30 min chunks
[17:20:04 CEST] <Johnjay> It segments it to file1, file2, file3 where file3 is 45 minutes
[17:21:11 CEST] <c_14> if you hardcode segment_times
[17:21:54 CEST] <Johnjay> er wait I can give it a list of times?
[17:21:58 CEST] <Johnjay> i forgot the syntax lol
[17:22:23 CEST] <c_14> >Specify a list of split points. times contains a list of comma separated duration specifications, in increasing order. See also the segment_time option.
[17:23:03 CEST] <Johnjay> I guess I'm thinking of splitting into chunks of at least length N, as opposed to exactly length N
[17:23:30 CEST] <Johnjay> after I tried splitting a file that was like 1 hr 18 min into two 30 min files and one 18 min one
[17:23:32 CEST] <c_14> Well, that's easy
[17:23:36 CEST] <c_14> Just don't split it into chunks
[17:23:42 CEST] <c_14> They'll be "at least" the length
[17:24:09 CEST] <c_14> But no, I know of no solution besides calculating the times yourself
[18:16:16 CEST] <JEEB> BytesBaconBB: hw encoding is never used because it gives better quality but because low CPU usage is required for some reason or another
[18:16:51 CEST] <JEEB> it can give more speed (but for example x264 superfast is pretty darn fast), but generally high compression ratio use cases are not what those things are used for
[18:17:04 CEST] <JEEB> low latency/realtime is what they're generally made for and other requirements are minimal
[18:21:39 CEST] <DHE> or where you can't spare much CPU at all (eg: gaming on a weak CPU)
[18:21:59 CEST] <DHE> opencl offload on x264 is very much hit-and-miss. sometimes it's good, sometimes it's not.
[18:22:12 CEST] <DHE> and by 'not' I very much mean a throughput penalty for using your GPU
[18:22:44 CEST] <JEEB> and that feature is very minimal anyways
[18:22:58 CEST] <JEEB> it's just one of the lookaheads moved to an opencl device with a lowres mode
[18:23:19 CEST] <DHE> lowres? oh well screw that then
[18:25:12 CEST] <JEEB> yea, to make the upload of data faster it uses lowres
[18:25:34 CEST] <JEEB> anyways, it's a PoC by the multicoreware people to get them PR
[18:25:36 CEST] <JEEB> basically
[18:25:41 CEST] <JEEB> the good part is that it at least kind of works
[18:25:44 CEST] <JEEB> :D
[18:43:49 CEST] <DHE> curious if you could get "better" results using hardware decoding and keep it in the GPU that way...
[18:44:08 CEST] <JEEB> well most of the encoding process then needs the images
[18:44:26 CEST] <JEEB> as I said, only one of the lookaheads was being done on the opencl device
[18:44:27 CEST] <JEEB> :D
[18:44:27 CEST] <BtbN> you cannot pass GPU frames to x264
[18:44:39 CEST] <JEEB> it can take in NV12 at least
[18:44:44 CEST] <JEEB> but not textures, yes
[19:42:22 CEST] <C129> Is it possible to preserve subtitle stream timing when cutting up a view via ffmpeg and the -ss parameter?
[20:12:16 CEST] <Martchus> C129: Don't know. As a workaround you could specify the input file twice (and use map to not get all tracks twice).
[20:27:06 CEST] <alexpigment> JEEB: per our earlier conversation, I found the code you were talking about in x264/encoder/encoder.c
[20:27:22 CEST] <JEEB> k
[20:27:52 CEST] <alexpigment> after it defined the max threads, i just put a line that said if max threads are greater than 16, set them to 16
[20:27:55 CEST] <alexpigment> which works for me
[20:27:57 CEST] <alexpigment> thanks for your help
[20:28:56 CEST] <alexpigment> i also did the same for max sliced threads, but I wasn't sure if it should be half the number of threads or not
[20:29:09 CEST] <alexpigment> 16 seems fine unless you think there's a reason to limit to 8
[20:31:44 CEST] <JEEB> alexpigment: umm, the question is whether or not the 16 threads limit is actually needed for x264
[20:31:53 CEST] <JEEB> because as we learned the message came from VERY GENERIC CODE
[20:31:58 CEST] <JEEB> in libavcodec
[20:32:10 CEST] <alexpigment> yes, but I've got a case where I know it fails on a 16-core system
[20:32:12 CEST] <JEEB> I know libavcodec's decoders are often limited
[20:32:13 CEST] <alexpigment> because of the auto logic
[20:32:23 CEST] <JEEB> was that an out of memory case or otherwise?
[20:32:33 CEST] <JEEB> if it was OOM then I don't think that's a threading problem
[20:32:38 CEST] <alexpigment> I can't say for sure, but it said "conversion failed"
[20:32:54 CEST] <alexpigment> when I set the threads lower below a certain threshold, it worked as expected
[20:33:09 CEST] <JEEB> E_MORE_INFO_NEEDED
[20:33:13 CEST] <alexpigment> 48 failed, 32 failed, 24 worked
[20:33:23 CEST] <JEEB> I've had 48 threads work JustFine with auto
[20:33:37 CEST] <alexpigment> there's no more information needed. I'm just saying that I had a failure case that was easily reproducible, and I've fixed it by not allowing the auto logic to make dumb decisions
[20:33:46 CEST] <alexpigment> JEEB: you're running a 64-bit copy of FFMPEG presumably
[20:33:48 CEST] <JEEB> and if x264 *itself* already limits auto threads
[20:33:52 CEST] <JEEB> alexpigment: thus I asked if it was OOM
[20:33:58 CEST] <JEEB> in which case it is not really a threading problem
[20:34:00 CEST] <alexpigment> what is OOM?
[20:34:04 CEST] <JEEB> Out Of Memory
[20:34:11 CEST] <alexpigment> right
[20:34:17 CEST] <alexpigment> how would I know?
[20:34:22 CEST] <alexpigment> it just says conversion failed
[20:34:33 CEST] <JEEB> somewhere higher it should say, -v debug might be needed
[20:34:33 CEST] <alexpigment> ffmpeg doesn't crash or anything
[20:34:37 CEST] <alexpigment> k
[20:34:41 CEST] <alexpigment> i'll try that
[20:34:44 CEST] <JEEB> yes, since people are surprisingly good at catching allocation failures sometimes :P
[20:35:13 CEST] <JEEB> I've also had a subtitle renderer not crash after it tried to render a typo'd vector of a huge length in a 32bit process :P
[20:35:25 CEST] <JEEB> just by catching the allocation failure and trying to handle it somehow
[20:35:36 CEST] <JEEB> of course with initialization stuff in that case you just say "nope"
[20:36:01 CEST] <JEEB> anyways, there really is no reason to use 32bit for transcoding since you have so much to gain from 64bit  (more registers, more SIMD etc)
[20:36:25 CEST] <JEEB> and if you need to use some input module that is crappy enough to be limited to 32bit, then you run just input in its own 32bit process
[20:36:32 CEST] <alexpigment> error: "malloc of size 43687840 failed"
[20:36:32 CEST] <JEEB> and pipe that into 64bit
[20:36:39 CEST] <JEEB> alexpigment: welcome to allocation failures
[20:36:42 CEST] <JEEB> so yes, not a threading problem at all
[20:36:50 CEST] <alexpigment> JEEB: yeah, it's just about limiting size and not having two versions
[20:36:56 CEST] <alexpigment> JEEB: how do you figure?
[20:37:14 CEST] <JEEB> more threads might need more buffers, but that doesn't mean the threading is broken
[20:37:25 CEST] <alexpigment> no one claimed it's broken
[20:37:47 CEST] <alexpigment> basically for this 32-bit FFMPEG, if you run a 4K transcode on a 16-core processor, it fails
[20:37:49 CEST] <JEEB> basically, it wants to allocate its buffers and the amount of buffers that is needed is not enough in 32bit application space
[20:37:53 CEST] <alexpigment> that's the start and end of the problem
[20:37:57 CEST] <JEEB> it has NOTHING to do with threading
[20:38:06 CEST] <alexpigment> more threads = more memory
[20:38:17 CEST] <JEEB> not threads, the buffers related to them. but yes
[20:38:31 CEST] <alexpigment> then i'm not sure what your point in
[20:38:32 CEST] <alexpigment> *is
[20:38:58 CEST] <JEEB> but in this case I am deeply against adding any sort of limitations if things aren't actually broken, but rather you just hit a thing because your input is large enough that 32bit process space ends for you
[20:39:20 CEST] <JEEB> as this is different from "over X threads is bad for quality" kind of things
[20:39:44 CEST] <alexpigment> oh well that's separate
[20:39:50 CEST] <alexpigment> there are several places in the code that say to limit it
[20:40:14 CEST] <JEEB> yes, as bugmaster said there already are some limits in place for thread counts that would have negative effects
[20:40:14 CEST] <alexpigment> including in encoder.c
[20:40:42 CEST] <alexpigment> I'm not entirely sure about that, but I'll defer to your judgement
[20:41:05 CEST] <JEEB> well the guy said so and the message you were seeing HAD NOTHING TO DO WITH X264
[20:41:14 CEST] <JEEB> which is why my comment was that it should be hidden from libx264
[20:41:22 CEST] <JEEB> (somehow, it's generic code I bet so meh)
[20:41:23 CEST] <alexpigment> It says to avoid using too many threads because it affects VBV. It's not 100% clear to me that the code below that achieves that goal
[20:41:44 CEST] <JEEB> I haven't looked at the code, just throwing generic statements around so far
[20:41:45 CEST] <JEEB> anyways
[20:41:50 CEST] <JEEB> my point was...
[20:42:14 CEST] <JEEB> in any case, you are basically bringing the problem onto yourself in this case. limit threads according to input resolution if you must support 32bit binaries
[20:42:34 CEST] <JEEB> although that is always fuzzy because you can't be sure at which point exactly it will OOM
[20:42:35 CEST] <alexpigment> JEEB: you're making a lot of assumptions about my role in this :)
[20:42:55 CEST] <alexpigment> At any rate, 16 seems safe
[20:43:11 CEST] <JEEB> because depending on input decoding threads, filters, other stuff the amount of buffering outside of just x264 is done
[20:43:18 CEST] <JEEB> also input demuxing buffers blah blah
[20:43:22 CEST] <alexpigment> This will basically just keep blowing up as Threadripper gains popularity and Intel's new 10+ core chips come out
[20:43:28 CEST] <JEEB> so just accept that 32bit is shit
[20:43:32 CEST] <JEEB> and limit threads
[20:43:39 CEST] <alexpigment> JEEB: already done :)
[20:43:54 CEST] <JEEB> you don't need to patch x264 for it, since it's not x264's fault
[20:44:00 CEST] <JEEB> just set less threads, and done
[20:44:14 CEST] <alexpigment> JEEB: setting less threads means that you're often setting MORE threads when you don't need to
[20:44:22 CEST] <alexpigment> this is a thread cap that still preserves the auto thread logic
[20:44:54 CEST] <alexpigment> so on a 2-core processor, it uses 3 threads. on a 4 core processor, it uses 6. on a 16 core processor, it caps to 16
[20:44:56 CEST] <alexpigment> etc
[20:45:14 CEST] <alexpigment> much better than being hamfisted about it and forcing it to always be 16
[20:45:27 CEST] <JEEB> well, I would have per-profile thread counts :P
[20:45:33 CEST] <JEEB> but whatever
[20:45:48 CEST] <JEEB> but I do kind of agree it might make sense to have a max_threads parameter in x264
[20:45:49 CEST] <alexpigment> for your use case, I agree that it's much much simpler to do that
[20:46:02 CEST] <JEEB> so that in your crappy 32bit piece of shit you can set that
[20:46:08 CEST] <alexpigment> haha
[20:46:24 CEST] <JEEB> instead of patching x264 and most likely leaving it there for 64bit as well
[20:46:27 CEST] <JEEB> :P
[20:46:45 CEST] <alexpigment> It's not like I'm submitting a patch for this. This is a hack
[20:46:48 CEST] <JEEB> also it is absolutely mind-boggling how people are OK with the slower performance of 64bit still
[20:46:57 CEST] <alexpigment> 32-bit, you mean?
[20:47:00 CEST] <JEEB> yes
[20:47:19 CEST] <alexpigment> it's really just about avoiding having two binaries or including both and dealing with an extra 30MB of data
[20:47:28 CEST] <alexpigment> seems trivial to some, but still
[20:47:28 CEST] <JEEB> I'd say that's miniscule
[20:47:34 CEST] <JEEB> I mean, at this point of time
[20:47:38 CEST] <alexpigment> in practice, I agree with you 100%
[20:47:58 CEST] <JEEB> the only people on 28,800bps modems I've seen were All Nippon Airways' LA office
[20:48:02 CEST] <JEEB> don't ask why I was there >_>
[20:48:20 CEST] <JEEB> and yes, on 28,800bps modem you request a USB stick or a CD with the app
[20:48:23 CEST] <alexpigment> Japan has 28.8k still?
[20:48:26 CEST] <JEEB> no, that was in LA
[20:48:30 CEST] <JEEB> at LAX
[20:48:30 CEST] <alexpigment> oh right
[20:48:34 CEST] <alexpigment> sorry
[20:48:35 CEST] <alexpigment> :)
[20:48:59 CEST] <alexpigment> also that's weird that anywhere in america still has 28.8k
[20:49:00 CEST] <JEEB> the worst I've had in an apartment in Japan in ~2005 was 3 megabits or so?
[20:49:26 CEST] <alexpigment> I had 28.8k up until 2000 or so because the phone lines sucked where I lived
[20:49:32 CEST] <JEEB> well, it was an office so there Muxt Have Been a Reason
[20:49:36 CEST] <JEEB> *Must
[20:49:41 CEST] <alexpigment> And then we beta tested cable before it was released to the rest of the city, so that was nice ;)
[20:49:47 CEST] <JEEB> probably too expensive or corporate bullshit or whatever
[20:50:38 CEST] <alexpigment> I have no clue what decisions would lead someone to keep dial-up, but oh well
[20:50:47 CEST] <JEEB> but yes, in other words your issue wasn't threading but rather OOM due to more threads requiring more buffers
[20:51:08 CEST] <JEEB> and yes, more threads with VBV can be a problem but not sure at which point you start hitting it
[20:51:15 CEST] <JEEB> I would guess there should be docs on that somewhere
[20:51:29 CEST] <JEEB> also as usual probably dependant on specific parameters
[20:52:28 CEST] <alexpigment> JEEB: yeah, I was pretty sure I mentioned that I thought it was memory related from the get-go, but either way, no worries
[20:52:55 CEST] <alexpigment> (I said that in the x264dev channel)
[21:04:56 CEST] <devinheitmueller> alexpigment: there are plenty of places in the US where broadband simply isnt available (i.e. rural areas).  Its dial-up or nothing.
[21:05:16 CEST] <devinheitmueller> Hence for many people, its not a decision to choose slow speeds - they have no alternaitve.
[21:05:23 CEST] <alexpigment> devinheitmueller: I was really referring to major cities, but I get your point
[21:05:29 CEST] <JEEB> just for the reference, this was about LAX :D
[21:05:43 CEST] <alexpigment> On the other hand, dialup is usually not used anyway in those cases. It's cellular or satellite
[21:05:55 CEST] <devinheitmueller> Yeah, most cities are ok.  Its the rural areas that are the issue.
[21:06:51 CEST] <alexpigment> Not that I'd personally want to be on cellular or satellite for my main source of internet, but I'd still prefer that over dialup
[21:07:15 CEST] <devinheitmueller> Anybody from #ffmpeg planning on attending the VDD conference this weekend?
[21:08:35 CEST] <JEEB> absolutely no-one 8)
[21:08:45 CEST] <JEEB> I am totally not trying to finish a presentation >_>
[21:09:09 CEST] <alexpigment> JEEB: that's not what I heard. I heard that you were *totally trying to finish a presentation* right now
[21:09:15 CEST] <alexpigment> ;)
[21:09:23 CEST] <JEEB> nîn
[21:12:14 CEST] <fsphil> quick question, trying x11grab via the api and just getting "Protocol not found". Code is basically just https://pastebin.com/raw/hb8zqhX8
[21:12:32 CEST] <fsphil> I'm sure i've done something stupid but I can't see it
[21:13:08 CEST] <fsphil> (that pastebin is missing "fmt = av_find_input_format("x11grab");")
[21:18:45 CEST] <fsphil> ah n/m, I missed init'ing libavdevice
[21:33:57 CEST] <frankyboy_> hello :) i want to apply  boxblur filter on video 2 times, but in different prefiod of time. but i a, failing in syntax https://pastebin.com/QEQ1Tap8 where i am worng
[21:36:16 CEST] <frankyboy_> wrong*
[22:10:17 CEST] <dingwat> Anyone have experience with encoding an 8K video for youtube upload with ffmpeg? x264 doesn't support 8K and I'm not sure which codec I should try instead
[22:10:59 CEST] <BtbN> Why wouldn't x264 support 8k?
[22:11:49 CEST] <BtbN> You will need _tons_ of RAM though. No matter what encoder you use.
[22:12:01 CEST] <dingwat> As I understand it, x264 maximum resolution is 4096x2304
[22:12:14 CEST] <dingwat> BtbN: yeah, that's my other concern, but I'll tackle that when I get there
[22:12:29 CEST] <dingwat> I guess I'll try HEVC and see what happens if I try to upload to youtube
[22:12:37 CEST] <dingwat> Not sure if I have enough RAM, but meh
[22:13:42 CEST] <dingwat> Uh oh my memory is going down the drain
[22:14:39 CEST] <JEEB> dingwat: x264 is an encoder
[22:14:52 CEST] <JEEB> it has a maximum resolution of probably the maximum values of int
[22:15:01 CEST] <JEEB> (and possibly some other funky stuff)
[22:15:08 CEST] <BtbN> x264, since 2017-06-14, should support h264 levels up to 6.2, and so up to 8,192×4,320 at 120
[22:15:10 CEST] <JEEB> most definitely not some arbitrary profile limit like 4096x2304 :)
[22:15:20 CEST] <JEEB> that's just levels though
[22:15:25 CEST] <BtbN> http://git.videolan.org/?p=x264.git;a=commit;h=6f8aa71ce797be01fd2ebe53c072a6696ea19828
[22:15:27 CEST] <JEEB> it would happily still write you 5.1 before
[22:15:37 CEST] <JEEB> but actually encode something incompatible with that in reality :P
[22:15:40 CEST] <BtbN> 5.1 is limited to 4,096×2,160 at 60 though
[22:15:50 CEST] <JEEB> yes, but that doesn't stop x264 from encoding it
[22:15:53 CEST] <dingwat> JEEB: ah ok, I thought I was limited by H.264 5.2
[22:16:09 CEST] <BtbN> you should definitely grab a very recent x264 still
[22:16:12 CEST] <JEEB> of course
[22:16:14 CEST] <BtbN> so it does not violate the spec
[22:16:21 CEST] <alexpigment> BtbN: i thought 5.1 was limited to @30 at the resolution
[22:16:27 CEST] <alexpigment> (checking wikipedia now)
[22:16:36 CEST] <JEEB> not that youtube will most likely care since it just feeds the stream into libavcodec
[22:16:45 CEST] <JEEB> and libavcodec should be able to decode just fine
[22:16:49 CEST] <dingwat> I'm definitely not able to encode h.264 with ffmpeg, it OOMs immediately. I thought the 5.2 limit was the reason, but maybe it's different
[22:16:49 CEST] <JEEB> not caring about the level
[22:17:02 CEST] <JEEB> dingwat: 32bit?
[22:17:15 CEST] <dingwat> JEEB: 64bit windows
[22:17:24 CEST] <JEEB> yes, but the ffmpeg.c binary you're using :P
[22:17:32 CEST] <JEEB> if it OOMs it's 32bit most likely
[22:17:36 CEST] <JEEB> get a 64bit one
[22:18:08 CEST] <alexpigment> JEEB: well, it can happen at 64-bit too if you don't have enough physical memory, right?
[22:18:11 CEST] <dingwat> I'm able to (slowly) encode x265 (0.2 fps) right now
[22:18:33 CEST] <JEEB> dingwat: x265 will be both slower and worse bang for the buck most likely
[22:18:40 CEST] <alexpigment> dingwat: try x264 and specify -threads 2
[22:18:42 CEST] <alexpigment> (as a test)
[22:18:47 CEST] <alexpigment> see if you run out of memory
[22:18:57 CEST] <JEEB> also I think youtube actually doesn't let you push HEVC into it?
[22:19:03 CEST] <JEEB> or have they changed that
[22:19:05 CEST] <alexpigment> you can of course bump that up more, but it's just to test
[22:19:11 CEST] <dingwat> Yeah, I'm not sure if they support HEVC yet
[22:19:20 CEST] <alexpigment> hard to believe youtube would turn away HEVC in 2017
[22:19:21 CEST] <JEEB> alexpigment: before that just make the idiot use 64bit FFmpeg
[22:19:33 CEST] <dingwat> OK, so there shouldn't be any reason I can't do x264 with 8K? I will try to get that working then
[22:19:36 CEST] <JEEB> there's no reason to use 32bit, losing the speed-ups
[22:19:37 CEST] <alexpigment> JEEB: where did you see he was using 32-bit?
[22:19:40 CEST] <BtbN> YouTube uses ffmpeg 0.10 or something. No HEVC support
[22:19:47 CEST] <JEEB> if he ran out of memory it's 32bit
[22:19:52 CEST] <JEEB> 64bit process will not fall the malloc()
[22:19:58 CEST] <JEEB> it will just swap your shit if you go too far
[22:20:02 CEST] <BtbN> unless you actually are out of memory
[22:20:17 CEST] <BtbN> 8k transcoding can easily munch 10GB+, if not 20GB+ depending on ref frame count
[22:20:18 CEST] <alexpigment> JEEB: fair point. I asked above if it was possible with insufficient ram, but if it goes to swap, then that makes sense
[22:20:19 CEST] <JEEB> well in 64bit that is rather unlikely, unless you also run out of swap or something
[22:20:31 CEST] <JEEB> also considering the goddamn speedups
[22:20:39 CEST] <JEEB> 32bit just makes my head hurt
[22:21:14 CEST] <alexpigment> JEEB: before you get back into a 32-bit rage, let's not assume he's using 32-bit without further proof
[22:21:19 CEST] <alexpigment> ;)
[22:21:25 CEST] <dingwat> This machine only has 8GB RAM, plus I'm running Vivado and a whole boatload of Chrome tabs. Let me clean some stuff up and try again
[22:21:28 CEST] <JEEB> he OOM'd and noted his *windows* was 64bit
[22:21:34 CEST] <JEEB> dingwat: that doesn't cause OOM
[22:21:46 CEST] <JEEB> I mean, you have to really work hard to get OOM with 64bit
[22:22:14 CEST] <JEEB> dingwat: just grab a 64bit one from https://ffmpeg.zeranoe.com/builds/ :V
[22:22:15 CEST] <alexpigment> For what it's worth, I did a test on a 64-bit copy of FFMPEG earlier and it crashed Premiere :)
[22:22:24 CEST] <alexpigment> the system took a nosedive for sure, but ffmpeg kept running
[22:22:31 CEST] <dingwat> Sorry, it's hard to reply timely because my whole OS is really laggy right now. Let me check which build I have and give you the exact error message from x264
[22:22:33 CEST] <JEEB> yes, OOM is really hard
[22:22:48 CEST] <JEEB> as someone who has debugged OSS software suddenly eating 60+ gigs
[22:23:02 CEST] <JEEB> the thing just didn't stop until it caused the system to pop
[22:23:31 CEST] <alexpigment> i cringe at the thought of what it means for a system to "pop"
[22:23:41 CEST] <JEEB> and it only got killed by oom-killer because swap was run out of
[22:23:52 CEST] <JEEB> and when you run out of swap the system starts finally killing things
[22:24:57 CEST] <alexpigment> dingwat: anyway, like I suggested earlier, try doing this with -threads 2 in x264
[22:25:00 CEST] <alexpigment> or even -threads 1
[22:25:13 CEST] <JEEB> alexpigment: also regarding youtube and HEVC - they just want to stay the hell away from the licensing mess
[22:25:16 CEST] <alexpigment> each additional thread will eat up more memory, and significantly so at higher ersolutions
[22:25:18 CEST] <JEEB> and I don't blame them
[22:25:37 CEST] <alexpigment> JEEB: I suppose that's fair. I just don't see how they can fight that codec for input formats
[22:26:21 CEST] <JEEB> if you upload 4:2:2, 10bit that's usually prores or something else
[22:26:31 CEST] <furq> i'm pretty sure youtube supports hevc now
[22:26:33 CEST] <JEEB> I don't think many editors or whatever output HEVC for HDR etc
[22:26:33 CEST] <alexpigment> yeah, prores is 422 10 by default
[22:26:38 CEST] <JEEB> furq: hmm
[22:26:43 CEST] <JEEB> could be possible
[22:26:50 CEST] <furq> i've not tried it but i've heard people reporting that it works
[22:26:53 CEST] <alexpigment> JEEB: the big ones all do. Premiere definitely does
[22:27:09 CEST] <JEEB> well, then google caved in regarding HEVC
[22:27:14 CEST] <JEEB> and my info was old :)
[22:27:22 CEST] <furq> it's a pretty recent change
[22:27:24 CEST] <alexpigment> once the camera manufacturers started using it, i'm sure they had to
[22:27:34 CEST] <JEEB> I've only seen a single camera so far
[22:27:36 CEST] <alexpigment> as well as, you know, Blu-ray UHD
[22:27:39 CEST] <furq> someone's claiming that hevc is "now free for sites like youtube" but this is a reddit post so it's probably bullshit
[22:27:54 CEST] <furq> also i doubt youtube really care about supporting people uploading bdremuxes
[22:27:55 CEST] <JEEB> yea, they did improve the licensing around the late last year I think
[22:28:20 CEST] <JEEB> and yes, "it's on BD" doesn't really matter since the person uploading to youtube most likely will upload the master he has on hand
[22:28:23 CEST] <furq> the new iphones record in hevc don't they
[22:28:24 CEST] <alexpigment> furq: yeah, i was kinda saying that as a secondary concern. really just implying that it's a legit standard at this point
[22:28:25 CEST] <JEEB> *it's on UHD BD
[22:28:33 CEST] <furq> if they're out yet
[22:28:35 CEST] <JEEB> it has always been a legit standard
[22:28:40 CEST] <JEEB> the licensing mess just made it eww
[22:28:48 CEST] <JEEB> UHD BD crowd was the only one that really took it in
[22:29:00 CEST] <furq> a lot of euro broadcasters are using it for hd aren't they
[22:29:04 CEST] <furq> i know cz uses it for dvb-t2
[22:29:11 CEST] <JEEB> test broadcasts I've seen
[22:29:17 CEST] <JEEB> not actual real broadcasts
[22:29:40 CEST] <alexpigment> but yes, Apple adopted it for iOS
[22:29:52 CEST] <JEEB> yea, after disabling it in the OS for quite a while
[22:29:53 CEST] <alexpigment> so that's a huge thing to try and fight :)
[22:29:55 CEST] <JEEB> now they went HEVC :)
[22:30:02 CEST] <JEEB> yea, that's probably the biggest single thing
[22:30:07 CEST] <JEEB> with the brainfart that is HEIF
[22:30:10 CEST] <furq> ;_;
[22:30:14 CEST] <alexpigment> it's an image format
[22:30:17 CEST] <alexpigment> that's based on HEVC
[22:30:17 CEST] <JEEB> https://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/215003.html
[22:30:31 CEST] <JEEB> just read the comment and it's like wutlol.jpg
[22:31:06 CEST] <alexpigment> i was thinking it was HEVC based... according to that thread it's just JPEG
[22:31:12 CEST] <furq> TL;DR this format is bad, but at least we're finally moving past JPEG.
[22:31:17 CEST] <furq> alexpigment: it can be multiple codecs
[22:31:20 CEST] <furq> the headline one is hevc
[22:31:26 CEST] <alexpigment> ahh
[22:31:29 CEST] <alexpigment> wiki: "The HEIF specification also defines the means of storing High Efficiency Video Codec (HEVC)-encoded intra images and HEVC-encoded image sequences in which inter prediction is applied in a constrained manner.
[22:31:29 CEST] <alexpigment> "
[22:31:46 CEST] <furq> just as webp was getting settled in, now this
[22:31:53 CEST] <alexpigment> I want to say the big advantage is the storing of multiple images
[22:32:03 CEST] <furq> just like everyone's favourite image format, tiff
[22:32:09 CEST] <alexpigment> So that you can do multiple shots and let the user choose the best one without deleting the others
[22:32:14 CEST] <JEEB> I recommend you read the rant/description on that post :P
[22:32:15 CEST] <alexpigment> furq: lol
[22:32:23 CEST] <furq> i feel like you can already do that by having multiple files
[22:32:51 CEST] <JEEB> and yea, HEIF is just abusal of ISOBMFF
[22:32:59 CEST] <JEEB> (colloquially called mp4)
[22:33:00 CEST] <furq> the first time that's happened
[22:34:04 CEST] <Mavrik> ^^
[22:34:21 CEST] <alexpigment> the rant is pretty funny, but I'll be the first to admit I understood only half of what he said :)
[22:34:45 CEST] <dingbat> Welp, my computer ded
[22:35:02 CEST] <furq> i love to store more than 2^16 images in one file
[22:35:03 CEST] <alexpigment> 8K vs Computer = 8K wins?
[22:35:32 CEST] <Mavrik> " Hopefully that never happens, because it would be a pain to work around."
[22:35:39 CEST] <dingbat> Pretty much. I'm gonna set up networking and then try ffmpeg again on my Linux machine, which has 32GB
[22:35:44 CEST] <furq> For some reason my phone running iOS 11 doesn't actually export HEIFs; it just gives me JPEGs with a .HEIC extension.
[22:35:55 CEST] <furq> i assume he's running it on an old iphone that doesn't have the encoder
[22:36:03 CEST] <dingbat> What ffmpeg build should I use on 64bit Ubuntu 16.04?
[22:36:13 CEST] <furq> dingbat: https://www.johnvansickle.com/ffmpeg/
[22:36:29 CEST] <furq> i should probably add that link to the bot
[22:37:28 CEST] <dingbat> 3.3.4 or git build?
[22:37:47 CEST] <furq> either's probably fine
[22:37:58 CEST] <furq> they both use the same libx264
[22:40:41 CEST] <dingbat> Ok. Now I just need to restart windows and get a share mounted. Hopefully the overhead from networked disk doesn't slow it too much
[22:58:32 CEST] <dingbat> Computer finally restarted...
[23:12:51 CEST] <dingwat> Ok. Computer is working again and I've got the source images mounted in linux. Now to try again
[23:16:03 CEST] <dingwat> Welp, it doesn't immediately OOM. Using x264 and it's crawling along (0.5fps), but it seems like it's working
[23:17:00 CEST] <alexpigment> dingwat: what's your memory usage?
[23:17:55 CEST] <dingwat> So far, about 9.2GiB. This machine has 32GB and 8 cores. It looks like it's only using about 4ish cores
[23:18:11 CEST] <alexpigment> seems reasonable
[23:18:47 CEST] <alexpigment> there may be a bottleneck thread that's basically limiting what it can do in parallel
[23:19:07 CEST] <alexpigment> (i've never tried 8K personally, so I don't know how parallelized it can be)
[23:19:35 CEST] <dingwat> Yeah, it also might be limited by network
[23:19:41 CEST] <BytesBaconBB> Thanks JEEB for the info about hardware encoding. I guess I'll just let it run on the CPU and look at faster rigs for the future.
[23:23:29 CEST] <dingwat> Hmm. It might be bottlenecked with network or maybe even the spinny disk on the far end. I'm seeing up to 400Mbps over the network. My linux machine has an SSD, but the Windows machine has a spinny disk
[23:47:28 CEST] <dingwat> Oof. It's been ~30mins and I'm 15% of the way through....
[23:49:11 CEST] <JEEB> that's fast
[23:49:20 CEST] <JEEB> I can tell you stories about the reference encoder for HEVC
[23:49:29 CEST] <JEEB> with a frame going past after a minute and a half
[23:50:01 CEST] <dingwat> Yeah this machine is pretty beefy. I have a lot of spare CPU and only intermittent network usage, so I'm not sure what's holding it back
[23:50:29 CEST] <JEEB> what's the input?
[23:50:43 CEST] <dingwat> JEEB: png frames
[23:51:02 CEST] <JEEB> ok, so you most likely are having colorspace conversion after decode at least
[23:51:14 CEST] <dingwat> Yeah, probably
[23:51:18 CEST] <JEEB> and PNG decoding is not threaded (albeit I'm not sure how quick it is)
[23:51:44 CEST] <JEEB> I would basically start checking which part of the chain is bottlenecking you after you finish that one thing :P
[00:00:00 CEST] --- Thu Sep 21 2017