[Ffmpeg-devel-irc] ffmpeg.log.20190916

[burek]

[06:10:30 CEST] <bpleat> Windows 10 x64, ffmpeg v4.2 64-bit static.
[06:11:33 CEST] <bpleat> (I've tried longer functions, but I can't even get the basic to work... feel like a DFU here)
[09:26:53 CEST] <petecouture> I have an HLS live stream process which is concating source mp4s together and using a long filter complex string, transcodes them across a dynamic number of bitrate profiles. However if the source media has different profiles, Im getting HLS playback generation issues where some of the source videos dont encode to HLS. The segments for them dont get formed and the player throws an unlabeled error. Once the playhe
[09:26:54 CEST] <petecouture> reached the end of these videos it seems to pick up again just fine when it loops back to the first media. Duplicating source content shows seemless playback between videos.
[09:27:14 CEST] <petecouture> I know in these cases you need to scale and reset the aspect ratio which Im doing.
[09:27:37 CEST] <petecouture> But playback still throws errors.
[09:28:02 CEST] <petecouture> Here is the script https://pastebin.com/FP5ygvGE
[09:29:52 CEST] <petecouture> Usually in these issues its a timecode sync between media files, but isnt the concat filter supposed to reset that?
[09:50:00 CEST] <petecouture>  Guess what Im running into is known. Looks like theres an issue with the concat demuxer being able to handle videos with different parameters.
[11:11:02 CEST] <Radiator> Hi, I have a lot of memory leaks for some reasons I only do a loop on av_read_frame and avcodec_send_packet with a few AVFrames that I make sure I free everytime. I even call av_frame_unref to be sure but in 20s I reach 1GB of memory allocations... Is ffmpeg and its libraries are memory proof ?
[11:14:55 CEST] <pink_mist> if you use them correctly, they won't leak memory
[11:16:54 CEST] <Radiator> Ok, so that's for sure my fault ? Just to be sure, I must use av_frame_alloc and once done with it, av_frame_free, right ?
[11:17:39 CEST] <pink_mist> don't ask me. I have no idea about the API, I just know that ffmpeg is written well enough to not leak memory if used correctly
[11:18:05 CEST] <Radiator> Ahah, that's fine, thanks :)
[11:31:00 CEST] <snooky> moin
[11:32:39 CEST] <cehoyos> Radiator: When you use the term "leak", this implies that you ran valgrind (or an equivalent tool) that shows you exactly where the "leak" happened
[11:33:46 CEST] <JEEB> --leak-check=full --track-origins=yes with valgrind are my go-to things
[11:34:10 CEST] <BtbN> You usually don't have to call av_frame_alloc
[11:34:36 CEST] <BtbN> But depends on what you're doing obviously.
[11:34:51 CEST] <Radiator> cehoyos: Sadly, I work on windows and cannot use valgrind nor gdb
[11:35:10 CEST] <BtbN> keep in mind that video frames are rather large
[11:35:25 CEST] <cehoyos> Equvalent tools exist.
[11:35:30 CEST] <BtbN> so if you are transcoding 1080p or something, 1GB memory use is not unheard of. As long as it does not keep infinitely growing, there's no leak.
[11:35:34 CEST] <Radiator> BtbN: In my case I do, I do hwaccel so I have to alloc them prior to do the transfer form the hardware to cpu
[11:35:41 CEST] <cehoyos> My point was of course that you most likely do not have a memory leak...
[11:35:53 CEST] <Radiator> @Bt
[11:36:48 CEST] <Radiator> Oops, sorry. BtbN: My issue here is that it keep rising to the top. In 40seconds I hit the 2GB and so on. It doesn't stabilize
[11:37:13 CEST] <BtbN> You are not freeing something you allocate then
[11:38:45 CEST] <Radiator> Exactly and I wondered if it would come from my manipulations or the libs. I commented most of my code to highlight the biggest sections and it comes from the decoding. And I always finish by freeing my pointers and all so I don't know
[11:39:36 CEST] <BtbN> It's pretty unlikely that there is a major leak like that in ffmpeg.
[11:39:47 CEST] <Radiator> Yeah I bet
[11:39:53 CEST] <BtbN> You are most definitely doing something wrong.
[11:40:33 CEST] <Radiator> Oh, I don't free the packet, that might be it !
[11:49:59 CEST] <JEEB> one can use gdb just fine, but I still haven't found a proper valgrind alternative on windows :P
[12:10:10 CEST] <Radiator> Well I found the origin. It was the call to av_frame_get_buffer that raped my RAM. Never trust code sample on stack overflow when you don't know what it do.
[13:25:23 CEST] <beniamino> I have some code that is using av_buffersink_get_frame(s->buffersink_ctx, ctx->filt_frame) to get data. It works fine, but now I need to receive frames that have a fixed size. So I replaced it with ret = av_buffersink_get_samples(s->buffersink_ctx, ctx->filt_frame, 1152). I had expected this to be a simple replacement but now the call fails with Cannot allocate memory. Can anyone suggest why this is happening?
[13:26:06 CEST] <JEEB> Radiator: you have some examples on doc/examples. then you can also check the command line applications' usage under fftools/
[13:47:35 CEST] <karanveersingh> #ffmpeg , Need help in running 4k video encoding
[13:49:02 CEST] <karanveersingh> I have a system with 2 socket , 88 core processor with 8GB of RAM , I have to run Transcoding of 4k .mkv video files and do a live streaming using rtmp
[13:49:47 CEST] <JEEB> you might want to check how fast your setup first of all decodes your inputs
[13:49:59 CEST] <JEEB> something like `ffmpeg -v verbose -i INPUT -f null -`
[13:50:13 CEST] <JEEB> then keep adding what you need and checking how the frame rate handles itself
[13:52:21 CEST] <karanveersingh> Below is the command I am using , the problem is that after 2 simoultanious video streams , the fps start dropping and speed become less
[13:52:23 CEST] <karanveersingh> sudo ffmpeg -re -i Stranger09.mkv -c:v libx264 -b:v 50M -preset ultrafast -tune zerolatency -b:a 128k -s 4096x2160 -bufsize 5M -x264opts keyint=500 -g 60 -pix_fmt yuv420p -f flv rtmp://194.167.137.11/live-test/Strange09_4k
[13:53:25 CEST] <cehoyos> (Don't use sudo for encoding with ffmpeg) Try without network output to make sure the network is not the bottleneck
[13:54:02 CEST] <JEEB> also check how many threads libx264 (which you seem to be using) uses by default
[13:54:07 CEST] <cehoyos> And test how many inputs your drive support concurrently
[13:54:09 CEST] <JEEB> and use -threads:v:X to limit
[13:54:36 CEST] <cehoyos> Also, I would expect that slower x264 presets will still run fast enough on your systems
[13:56:01 CEST] <karanveersingh> so as of network bottleneck , nginx+rtmp resides on the same server , so same server is transcoder and streaming
[14:08:03 CEST] <karanveersingh> i have 88 core processor , so is it with libx264 it is cores x 1.5 for frame threads ?
[14:08:51 CEST] <karanveersingh> I can see lot many threads when running top -H -p <pid>
[14:15:46 CEST] <JEEB> you should see it with at least -v verbose when it starts
[14:15:59 CEST] <JEEB> there's x264 logging in there that notes the exact amount of threads it uses by default
[14:16:10 CEST] <JEEB> but yes, I think it does virtual_CPUs * 3 / 2 by default
[14:16:35 CEST] <DHE> there's 2 (?) separate thread pools it uses for encoding.
[14:16:56 CEST] <JEEB> that too
[14:17:05 CEST] <JEEB> it has the lookahead and the primary stuff
[14:17:14 CEST] <JEEB> lookahead one is generally smaller
[14:18:38 CEST] <karanveersingh> As an intel TLC ssd specification , one drive supports 12 streams of 1080p video with H264 encoding with 60 fps
[14:20:28 CEST] <DHE> well that's a bullshit number. what you want is sustained read or write megabytes per second and IOPS for set queue depths
[14:22:07 CEST] <DHE> as an aside, is there any reason you can't pre-transcode the video ahead of streaming it out to the network if you already have it on disk?
[14:22:44 CEST] <karanveersingh> ffmpeg running 36 threads / H264 , so I have to limit my threads  ?
[14:23:30 CEST] <karanveersingh> VOD is next task but right now live streaming is needed
[14:25:27 CEST] <karanveersingh> Without sudo as mentioned earlier , the fps and bit rate is dropping more
[14:26:08 CEST] <karanveersingh> Slow preset futher makes the things worse
[14:26:50 CEST] <karanveersingh> ignore the sudo one
[14:29:48 CEST] <kepstin> if you want to run multiple encodes on the system, i'd recommend limiting the threads per ffmpeg so  the number of threads used by all the encoders is d the number of cpu threads
[14:31:09 CEST] <karanveersingh> Thanks , thats i am trying right now , threads 2
[14:31:35 CEST] <kepstin> 2 threads is probably not going to be enough for realtime 4k encoding :)
[14:33:37 CEST] <karanveersingh> yea 8)
[14:33:47 CEST] <karanveersingh> can you recommend a good number
[14:37:48 CEST] <kepstin> i don't have your system, so no. You need to test to see how many threads you need for your application.
[14:40:33 CEST] <karanveersingh> Running multiple streams now , 8 threads works fine with single 4k stream
[14:55:51 CEST] <karanveersingh> its not coming up good , 2 concurrent live streams runs fine with many combinations but as soon as I start 3 video , fps drops , speed goes slow
[15:01:42 CEST] <karanveersingh> one Q , is H264 good for 4k .mkv file with Hevc (x265)
[15:04:16 CEST] <DHE> I'd recommend some NUMA awareness in your transcoding. but it means one job can only use half the CPUs because it sticks to one socket
[15:05:33 CEST] <another> i think x264 doesn't scale *that* well
[15:05:44 CEST] <furq> it doesn't
[15:05:50 CEST] <another> do even 1 socket should be fine
[15:05:52 CEST] <furq> you get big diminishing returns around 32 threads iirc
[15:12:12 CEST] <karanveersingh> Below is the command i am using , do point out where it can be corrected
[15:12:14 CEST] <karanveersingh> sudo ffmpeg -re -i Stranger09.mkv -c:v libx264 -b:v 50M -preset ultrafast -tune zerolatency -b:a 128k -s 4096x2160 -bufsize 5M -x264opts keyint=500 -g 60 -pix_fmt yuv420p -f flv rtmp://194.167.137.11/live-test/Strange09_4k
[15:16:20 CEST] <another> don't use sudo
[15:16:21 CEST] <DHE> so just from a consistency standpoint, you have keyint=500 but also -g 60 which is the same parameter
[15:16:32 CEST] <karanveersingh> yea , not using sudo
[15:16:38 CEST] <DHE> yet you keep pasting it
[15:16:59 CEST] <karanveersingh> just realized that
[15:17:26 CEST] <another> remove -tune zerolatency
[15:18:36 CEST] <another> this is definitely not the place to use it
[15:21:41 CEST] <another> also: add -c:a aac
[15:21:56 CEST] <another> just saw that flv defaults to mp3
[15:25:24 CEST] <furq> karanveersingh: zerolatency disables frame threading so that's probably why the performance is bad
[15:28:36 CEST] <karanveersingh> testing it right away , thanks
[15:28:55 CEST] <DHE> and with a CPU like that, I'd guess you could get away with -preset veryfast
[15:29:07 CEST] <DHE> maybe. but if you're experimenting, it'd be worth trying
[15:43:33 CEST] <karanveersingh> Thanks , will try this out
[15:44:05 CEST] <karanveersingh> calling a day after this , will keep trying
[15:44:30 CEST] <karanveersingh> thanks all guys for sharing knowledge
[19:12:23 CEST] <auri_> Is there a way to make the dash muxer skip to a specific segment number? I can seek input through normal means, but starting segment number always remains at 0
[19:55:13 CEST] <Jasper_> Does anyone know who owns https://ffmpeg.zeranoe.com ? The build here is marked as static, but is actually a shared build. https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-latest-win64-static.zip
[20:01:50 CEST] <cards> Would anyone here be inclined to take up maintaining SWF (vector flash) to MPEG conversion?  People from the site NewGrounds started a project 2 years ago but it's not actively maintained, and could use a bit of polishing.  https://github.com/Herschel/Swivel
[20:01:53 CEST] <cehoyos> Jasper_: Do you mean the binary does not work?
[20:02:44 CEST] <cehoyos> cards: We already work on a project that involves some work, why don't you clone the repository and start working on it?
[20:02:47 CEST] <cards> Swivel has a few bugs when dealing with SWF animations that automatically loop at the end, or have "click to play" screens at the beginning, especially.
[20:02:48 CEST] <Jasper_> cehoyos: it relies on DLLs. There are existing tools out there that download that zip file, extract only ffmpeg.exe, and expect it to work :/
[20:03:00 CEST] <cehoyos> Again: Does the binary not work?
[20:03:05 CEST] <cards> cehoyos: why don't I what?
[20:03:36 CEST] <Jasper_> cehoyos: not without the corresponding DLL files, which should not be the case for a static build.
[20:03:48 CEST] <cehoyos> cards: How do you know that https://github.com/Herschel/Swivel could use a bit of polishing?
[20:03:56 CEST] <cards> are you one of those "why don't you do it if you're such a master programmer, which you never claimed to be, but you dared to speak in here" insufferable people?
[20:04:04 CEST] <cehoyos> You did claim it;-)
[20:04:13 CEST] <cards> pretty sure i didn't
[20:04:33 CEST] <cehoyos> In this case: Sorry, I am not a native speaker, this is how I interpreted it.
[20:05:03 CEST] <cehoyos> jasper_: See https://ffmpeg.zeranoe.com/forum/
[20:05:21 CEST] <cehoyos> Note that I am surprised that the binaries do not work (but bugs are of course possible)
[20:05:46 CEST] <Jasper_> Future static builds are fine. It's just the one marked "latest" (which is not latest)
[20:06:26 CEST] <cehoyos> I don't see "latest" on the Download page...
[20:06:56 CEST] <Jasper_> You can see it from the open directory here, which is linked to by the front page: https://ffmpeg.zeranoe.com/builds/win64/static/
[20:07:27 CEST] <Jasper_> I was referred to this channel by the footer of the site, so I imagined the maintainer might be in here
[20:07:33 CEST] <cehoyos> https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20190916-1db6e47-win64-static.zip Did you try this one?
[20:07:45 CEST] <cards> Jasper_: which exact version is bad
[20:07:48 CEST] <Jasper_> That one also looks to be broken
[20:08:16 CEST] <cehoyos> See https://ffmpeg.zeranoe.com/contact/ for the actual contact (The FFmpeg project does not offer binaries, we do not own zeranoe.com)
[20:08:18 CEST] <Jasper_> ffmpeg-20190916-1db6e47-win64-static.zip has a filesize of 25MB, compared to the rest of them which have filesizes of 63MB.
[20:08:21 CEST] <cards> it's the official download site for windows builds, but i never had a problem with the static download.  you can tell it's a static download because the files are YUGE
[20:08:25 CEST] <cehoyos> Yes, I saw this
[20:08:55 CEST] <cehoyos> Note that there is nothing "official" about Zeranoe, we offer a line (as we do for other people who are not more "official")
[20:09:01 CEST] <cehoyos> s/line/link
[20:09:05 CEST] <cards> hmm
[20:09:53 CEST] <cards> Jasper_: you appear to be correct.  the win64 20190916-1db6e47 build for Static and Shared are the same filesize, give-or-take 100kb
[20:10:00 CEST] <Jasper_> Yeah, but since it was linked, I wondered if the maintainer was lurking in here
[20:10:09 CEST] <cards> Static should be 3x the filesize
[20:10:45 CEST] <cards> i'd just use the 4.2.1 for now then
[20:11:12 CEST] <cehoyos> Note that release builds contain more bugs and less features than current git head and are therefore unsupported here and on the bug tracker;-)
[20:11:40 CEST] <cards> still, nightly builds should be built right
[20:11:52 CEST] <Jasper_> Yep. There are just some tools like https://github.com/ShareX/ShareX/blob/master/ShareX.MediaLib/FFmpegDownloader.cs#L35 which are hardcoded to the latest URL
[20:12:03 CEST] <cehoyos> https://ffmpeg.zeranoe.com/builds/win64/static/ffmpeg-20190914-8efc9fc-win64-static.zip This seems the last working build afaict
[20:12:23 CEST] <cehoyos> Then contact Zeranoe
[20:13:23 CEST] <Jasper_> Yeah, I was trying to find their contact information -- seems that form is the best anyone can find. Cheers.
[20:13:41 CEST] <cards> "Kyle Schwarz" <zeranoe at gmail.com>
[20:13:54 CEST] <cards> it's on the bottom of that download page
[20:13:54 CEST] <Jasper_> Thanks!
[20:14:30 CEST] <cards> well actually, his email is on the github linked from his dl page
[20:17:48 CEST] <cards> cehoyos: I would say that his builds are as "official" as official can get in an open source project.  99.7% of the users of ffmpeg are using his builds.
[20:18:03 CEST] <cards> not a mere courtesy link
[20:19:08 CEST] <cehoyos> It is exactly that: A courtesy link
[20:20:07 CEST] <cards> you lose the right to call something a mere unofficial courtesy link when it becomes the defacto standard though
[20:20:40 CEST] <cehoyos> From a quick look, Kyle Schwarz has no commit rights, he rarely comments on any FFmpeg mailing list, I don't think he was ever at one of the developer meetings
[20:20:42 CEST] <beniamino> (I posted this earlier but am hoping some people in different time zones might be on now). I have some code that uses libav to transcode music into mp3 format. The code currently uses libmp3lame but I want to use libshine for speed on a raspberry pi. Unlike libmp3lame, libshine requires a fixed frame size. The original code fetches music from a sink using av_buffersink_get_frame(&). I had hoped to replace this with av_buffersink_get_samples(&, frame_size)
[20:20:43 CEST] <beniamino> I can pass the resulting frames to libshine. However, when I do this, av_buffersink_get_samples failes with error -12 Cannot allocate memory (though av_buffersink_get_frame works fine). Can anyone suggest why this might happen?
[20:20:46 CEST] <cards> "do you use windows? the most popular operating system in the world?  click here!"
[20:21:30 CEST] <cehoyos> The same is true for the other people who offer FFmpeg binaries (vlc binaries, by comparison, are offered by the videolan developers themselves)
[20:21:46 CEST] <cards> cehoyos: there you go again believing that the only way someone can be involved in a project is to commit code changes
[20:21:59 CEST] <cehoyos> This has nothing to do with MS Windows, the same is true for the macos and linux static binaries that are also offered by kind supporters
[20:22:38 CEST] <cards> there are a lot of non-coders in the world who contribute to projects.  such as site hosts, Q&A, bug finders, app integrators, and end-users.
[20:22:50 CEST] <cards> those people are more important than the coders themselves
[20:22:53 CEST] <cehoyos> No, there are many are many other ways to be involved: Posting on the user mailing list, being active on irc, coming to developer meetings, run the infrastructure of the project etc.
[20:25:05 CEST] <cehoyos> But neither Kyle nor this guy - https://evermeet.cx/ffmpeg/ - who even mentions "I'm not in any way affiliated with ffmpeg.org"
[20:25:26 CEST] <cehoyos> are part of the FFmpeg project
[20:25:45 CEST] <cards> ffmpeg.org is not "the project"
[20:26:03 CEST] <cards> it's an entity that is party to the project
[20:26:26 CEST] <durandal_1707> everyone is part of FFmpeg
[20:26:41 CEST] <cehoyos> Just to make sure: From all we (the FFmpeg developers) know, Kyle (and his counterparts) could be people who secretely plan to attack you, we simply cannot know
[20:26:50 CEST] <cehoyos> (Personally, I trust them completely)
[20:28:09 CEST] <cards> cehoyos: i would hope that [you] (ffmpeg developers and project managers) have a little more "know" before linking to and establishing the most popular download site for ffmpeg builds known
[20:28:26 CEST] <cehoyos> Sadly, we do not...
[20:28:42 CEST] <cehoyos> (That's what I tried to explain)
[20:28:45 CEST] <cards> you speak for the organization?
[20:29:05 CEST] <cehoyos> No, there is no "organization" and nobody can speak for FFmpeg
[20:29:09 CEST] <durandal_1707> they are unofficial builds
[20:29:14 CEST] <furq> they're builds some guy makes that a lot of people use
[20:29:52 CEST] <furq> it's a courtesy link on the site because otherwise 99% of lines in this channel would be "hi how do i run a .tar.bz2 on windows 10"
[20:30:02 CEST] <cehoyos> LOL, yes
[20:30:15 CEST] <cards> anyone can automate windows builds if they want to
[20:30:19 CEST] <furq> they sure can
[20:30:22 CEST] <furq> lots of people do
[20:30:31 CEST] <cards> you don't have to rely on some random Nigerian Prince to do it for you
[20:30:59 CEST] <cards> there are github plugins that perform this function
[20:31:00 CEST] <cehoyos> Please consider doing it (I have a suspicion you get a lot of traffic that you can use)
[20:31:32 CEST] <cehoyos> The FFmpeg project only provides source code, no binaries
[20:31:48 CEST] <cards> that's not something you want to brag about
[20:31:59 CEST] <cehoyos> You misunderstand...
[20:36:02 CEST] <beniamino> I cant see from the docs any reason why av_buffersink_get_samples(&, frame_size) would fail when av_buffersink_get_frame(&) works. Yet just making that one change makes the code crash reliably
[20:37:08 CEST] <durandal_1707> beniamino: av_buffersink_get_samples may return AVERROR(EAGAIN) , meaning it needs more data?
[20:43:09 CEST] <beniamino> durandal_1707: The code handes AVERROR(EAGAIN) by just waiting for more data. But av_buffersink_get_samples fails with a different error code. Its -12 which I think is AVERROR(ENOMEM)
[20:44:08 CEST] <durandal_1707> beniamino: and how you setup filter graph?
[20:44:56 CEST] <beniamino> durandal_1707: Im not sure how to answer that question :-) but the code is here: https://github.com/ejurgensen/forked-daapd/blob/master/src/transcode.c
[20:46:09 CEST] <durandal_1707> seen this? http://git.videolan.org/?p=ffmpeg.git;a=blob;f=doc/examples/filtering_audio.c
[20:48:06 CEST] <durandal_1707> that code first check frame than error code, while it should be other way around
[20:55:13 CEST] <beniamino> interesting. can you clarify what exactly do you mean by check frame? lines 264 onward of your link seem to have a similar logic to lines 599 onward of mine
[20:55:38 CEST] <durandal_1707> frame is nott NULL
[21:08:14 CEST] <beniamino> i switched them and it didnt make a difference. but i will look more carefully at the logic of that bit
[00:00:01 CEST] --- Tue Sep 17 2019