[Ffmpeg-devel-irc] ffmpeg-devel.log.20190223

[burek]

[08:00:49 CET] <orlcp440> Where can I learn about the software architecture of ffmpeg libraries? I know they have the concept of muxer/demuxer, encoder/decoder, parsers, protocols, filters etc. I would like to learn about the whole architecture so that I can contribute to the library. For example, I would like to know which module is responsible for keeping the streaming rate when sending RTP packets over IP. I am using PCM audio and sending it throug
[13:23:34 CET] <cone-322> ffmpeg 03Martin Vignali 07master:6cc8cfe30c09: avcodec/proresenc_aw : simplify frame flags
[13:23:34 CET] <cone-322> ffmpeg 03Martin Vignali 07master:2016f9c3bc23: avcodec/proresenc_aw : move picture encoding after frame header write
[13:23:34 CET] <cone-322> ffmpeg 03Martin Vignali 07master:8a4a952e45e8: avcodec/proresenc_aw : move scantable to prores context
[13:23:34 CET] <cone-322> ffmpeg 03Martin Vignali 07master:e7cbbb7374c6: avcodec/proresenc_aw : merge funcs subimage_with_fill and subimage_alpha_with_fill
[13:23:34 CET] <cone-322> ffmpeg 03Martin Vignali 07master:132ed206d6b8: avcodec/proresenc_aw : add interlace encoding
[13:23:35 CET] <cone-322> ffmpeg 03Martin Vignali 07master:37e4c226c06c: avcodec/proresenc_aw : indent after prev commit
[17:01:43 CET] <durandal_1707> hurray, today fixed 2 parsing bugs
[17:06:27 CET] <durandal_1707> only small overread from ASPX remains
[17:09:06 CET] <JEEB> najs
[21:28:45 CET] <orlcp440> Where can I find the software architecture for ffmpeg libraries? I am trying to do development on ffmpeg and not just use its libraries. One question I have is what module is responsible to send packets at a fix bitrate into RTP or is the RTP encoder/muxer supposed to do that? I see packets going out to the Ethernet network but they are going out way too fast and not at the rate I want
[21:29:38 CET] <JEEB> libavformat has protocols and muxers
[21:30:34 CET] <JEEB> (and demuxers)
[21:30:46 CET] <orlcp440> What I am doing is converting an audio into a .mkv matroska and saving it into a file with the bitrate, sampling rate and all that good info. However, I am saving raw PCM samples of the music in the .mkv file and no video. Then opening it and sending the audio to RTP
[21:30:47 CET] <JEEB> and libavcodec has the decoders and encoders
[21:32:37 CET] <orlcp440> @JEEB: Cool, is there any documentation online that explains all that? I am interested in the RTP protocol and muxing part. I had figured out a lot by a good tutorial online but I cannot find where is the module that takes care that the data is being sent to the network at a proper bitrate
[21:33:50 CET] <JEEB> rtpenc packets streams within "RTP" encapsulation, and rtpproto is the RTP protocol implementation that then passes the stuff out (or takes stuff in)
[21:33:59 CET] <JEEB> both within libavformat
[21:34:34 CET] <orlcp440> The best document/explanation of the architecture is by ffmpeg I have found is this one https://github.com/leandromoreira/ffmpeg-libav-tutorial but it does not talk about protocols nor parsers
[21:35:10 CET] <JEEB> rtpproto uses the UDP protocol within itself it seems
[21:35:27 CET] <orlcp440> @JEEB ok so far I am following and knew that part
[21:37:53 CET] <orlcp440> @JEEB: In this model you are describing do you think whatever module is passing the data to rtpenc should do it at a certain rate or should the rtpproto read some dictionary entry and learn how fast and at what rate it should send its data to the next module which in this case is UDP?
[21:38:34 CET] <JEEB> the protocol should be the one sending stuff out at some rate
[21:38:50 CET] <JEEB> the UDP protocol already has such a feature, so most likely  the RTP protocol just isn't passing those options
[21:40:08 CET] <JEEB> see the AVOptions struct in udp.c
[21:40:26 CET] <JEEB> specifically the bitrate option
[21:40:49 CET] <orlcp440> So, then somehow the module that is sending the data to rtpenc should have a mechanism to tell it at what rate it wants that data to be sent to UDP so that rtpenc can pass along that information to rtpproto right?
[21:41:08 CET] <JEEB> you don't have to take it that far
[21:41:38 CET] <JEEB> just make sure RTP protocol has the way to receive an AVOption, and passes it to the UDP protocol it utilizes inside
[21:41:56 CET] <JEEB> since the RTP muxer shouldn't care about speed at all
[21:42:12 CET] <orlcp440> ohh cool, so the UDP protocol module has a bitrate info and it is the one responsible to send data at a certain rate
[21:43:00 CET] <JEEB> yes, the udp protocol already has options for that. most likely the  RTP protocol doesn't, and it doesn't pass the stuff to the UDP protocol underneath
[21:43:01 CET] <orlcp440> You are a life saver @JEEB, I am going to check that out right now
[21:44:21 CET] <JEEB> do note that if the RTP protocol does not utilize all of UDP protocol underneath, then you are kind of in trouble and will have to make the logic required be shared between the UDP and RTP protocols
[21:44:47 CET] <JEEB> because the last thing you - or anyone else wants is to have the same code copypasted in both modules :P
[21:46:01 CET] <JEEB> I would guess ffurl_open_whitelist does handle quite a bit of that inside tho
[21:46:50 CET] <JEEB> yup, that opens one or more UDP protocols underneath
[21:47:11 CET] <JEEB> see rtp_open after it does the option parsing
[21:49:21 CET] <orlcp440> @JEEB: Hmm.... ok let me check that out
[21:49:33 CET] <JEEB> and in rtp_write it then calls ffurl_write
[21:49:45 CET] <JEEB> so yes, it looks like it just utilizes UDP protocol underneath
[21:49:59 CET] <JEEB> -> you should just add the relevant AVOptions to RTP and pass them down to UDP
[21:53:38 CET] <orlcp440> Ok I have to go back and see how that option gets passed by the user. I guess it would be in a dictionary, and then follow it through all the modules to make sure it gets into UDP
[21:55:15 CET] <JEEB> basically add the values for bitrate and burst_bits that are in UDPContext to RTPContext, and add those options to the AVOption list of rtpproto
[21:55:27 CET] <JEEB> that way RTP will be taking in those options, which the user can provide
[21:55:41 CET] <JEEB> since it's just adding something from one protocol to another, it shouldn't be too hard
[21:55:45 CET] <orlcp440> A lot of modules use RTP to send their media over UDP so I am sure all that mechanism is already in place. Maybe the user is not adding that bitrate entry into its dictionary
[21:56:06 CET] <JEEB> just look at the AVOption list in rtpproto :P
[21:56:12 CET] <JEEB> it doesn't have those
[21:56:34 CET] <JEEB> it has some things like pkt_size or connect that are passed through to UDP
[21:56:51 CET] <JEEB> it's just that the ones setting the speed control haven't been added
[21:56:53 CET] <orlcp440> really? so how other things do it? 
[21:57:00 CET] <JEEB> wat
[21:57:26 CET] <JEEB> anything that uses the RTP protocol in FFmpeg cannot pass the options to the underlying UDP protocol
[21:57:34 CET] <JEEB> because they are... NOT THERE
[21:57:49 CET] <JEEB> you can just look at the AVOption list in rtpproto.c
[21:58:09 CET] <JEEB> it's at the top of the file together with the RTPContext which contains the variables into which the options are parsed
[21:58:10 CET] <orlcp440> @JEEB: so how are people currently doing it? It seems like a basic requirement for streaming
[21:58:38 CET] <JEEB> currently as far as I know in *FFmpeg* people just do it with raw UDP
[21:58:44 CET] <JEEB> RTP doesn't seem to have rate control
[21:58:52 CET] <JEEB> or they do rate control outside of FFmpeg
[21:59:04 CET] <JEEB> or they patch whatever I'm mentioning locally
[21:59:12 CET] <JEEB> (Because they're lazy to post the patch upstream)
[21:59:45 CET] <orlcp440> @JEEB: lol that is hard to believe for me but I guess you are right. I cannot see any other explanation
[22:00:20 CET] <JEEB> I'm a masochist that does user support on #ffmpeg and trust me 90%+ of all people just use the raw UDP protocol without RTP :P
[22:00:25 CET] <JEEB> just plain MPEG-TS over UDP
[22:00:29 CET] <orlcp440> @JEEB: I guess it will be my responsibility to do just that now
[22:01:12 CET] <JEEB> which is also most likely why the bitrate and burst_bits AVOptions are in the UDP protocol (and haven't been added to the RTP protocol)
[22:01:44 CET] <orlcp440> @JEEB: Wow, that is crazy. Well, you can tell that I am new to this whole world of digital audio/video and these libraries because I am so surprise to hear what you are telling me
[22:04:12 CET] <JEEB> lolwat, do I see correctly that the open function doesn't pass any options to ffurl_open_whitelist
[22:04:20 CET] <orlcp440> When I finished this project i am working on I will definitely send the patch upstream 
[22:04:57 CET] <JEEB> ok, so it just passes the options through building a URL
[22:05:03 CET] <JEEB> well, I guess that was the older way?
[22:05:19 CET] <JEEB> nowadays you just fill an AVDictionary with the options that were set and you want to pass on
[22:05:41 CET] <orlcp440> @JEEB: Yes, dictionaries are the way to go now
[22:05:56 CET] <JEEB> orlcp440: if you have means to test quickly I can whip you up a patch
[22:06:01 CET] <JEEB> since this seems simple enough
[22:07:01 CET] <orlcp440> Yeah, but I am not at work today plus this change is a very small change as part of a whole lot of things that I am introducing
[22:08:14 CET] <orlcp440> So, it is not worth it right now for me since I want to test everything very well
[22:15:50 CET] <kierank> orlcp440: honestly anyone who wants to do proper "realtime a/v" doesn't use ffmpeg
[22:15:58 CET] <kierank> realtime as in things like rtp where you don't have 10 second like buffers
[22:15:59 CET] <kierank> like http does
[22:18:45 CET] <JEEB> the UDP protocol does seem to work surprisingly OK, with the bitrate and burst_bits options folk were getting surprisingly burst-less stuff
[22:18:59 CET] <JEEB> and the MPEG-TS muxer did surprise me at how it did indeed create a muxrate-specific mux
[22:19:14 CET] <JEEB> but yes, the defaults are for quite some buffering in various places
[22:19:39 CET] <JEEB> which is why my replies are rather inconvenient for people who ask about low latency setups on #ffmpeg, since I've not drilled into all the possible options in various things
[22:19:44 CET] <JEEB> (to enable low latency)
[22:20:58 CET] <kierank> the "real" way to do it is spin on a high priority thread
[22:21:03 CET] <kierank> you can get quite nice precision with that
[22:21:22 CET] <JEEB> yea, the UDP protocol does spin up a new thread and the whole feature is gated behind PTHREAD ifdefs
[22:22:08 CET] <kierank> the receive, right?
[22:22:10 CET] <kierank> not the send
[22:22:16 CET] <JEEB> send has that too
[22:22:26 CET] <JEEB> otherwise bitrate/burst_bits really wouldn't work that well
[22:22:37 CET] <kierank> ah
[22:22:38 CET] <kierank> interesting
[22:22:55 CET] <JEEB> yea, #if HAVE_PTHREAD_CANCEL
[22:22:56 CET] <JEEB> in there
[22:23:20 CET] <JEEB> and the write()s just keep feeding into the fifo
[22:23:24 CET] <JEEB> which the thread then handles
[22:29:22 CET] <orlcp440> @kierank: So what do people use for proper "realtime a/v" if not ffmpeg?
[22:29:30 CET] <JEEB> upipe is one thing
[22:29:35 CET] <JEEB> (which I do like various parts of)
[22:29:45 CET] <kierank> orlcp440: vlc
[22:31:24 CET] <JEEB> the RTCP thing just for the client<->server comms, right?
[22:32:35 CET] <orlcp440> @kierank: libvlc? Really? I did research on the web and found that most people/projects use ffmpeg and that is why I chose this library instead. By the way, if you are in this channel then you must find ffmpeg useful too
[22:33:05 CET] <JEEB> decoders
[22:38:22 CET] <atomnuker> files too, just not mpegts
[22:39:00 CET] <JEEB> yea but since he was talking RTP I was limiting to whatever was mostly going over UDP if looking at #ffmpeg
[22:39:03 CET] <durandal_1707> mpegts is file
[22:48:38 CET] <JEEB> orlcp440: https://github.com/jeeb/ffmpeg/commit/84c3261dd3cff17eb32aea06391bb6646311c6c1
[22:48:46 CET] <JEEB> dunno why I did it, but I /think/ that should be it
[22:53:00 CET] <JEEB> (I would rather use some sort of loop for the UDP protocol options since it's just the value and option name that differ in those two av_dict_set_ints - but I was lazy :P
[22:54:18 CET] <cone-350> ffmpeg 03Tristan Matthews 07master:00f54c15f45b: rtpenc_chain: forward strict_std_compliance flags to rtp muxer
[22:54:18 CET] <cone-350> ffmpeg 03Michael Niedermayer 07master:61523683c5a9: avcodec/jvdec: Check available input space before decode8x8()
[22:54:18 CET] <cone-350> ffmpeg 03Michael Niedermayer 07master:177b40890c6d: avcodec/zmbv: obtain frame later
[22:54:18 CET] <cone-350> ffmpeg 03Michael Niedermayer 07master:3f68948cb303: avcodec/pnm: Avoid structure pointer dereferences in inner loop in pnm_get()
[22:54:18 CET] <cone-350> ffmpeg 03Michael Niedermayer 07master:fc32e08941ea: avcodec/mlpdec: Insuffient typo
[22:54:42 CET] <orlcp440> @JEEB: jeez you are fast
[22:57:03 CET] <orlcp440> @JEEB: so, for realtime a/v you use upipe or libvlc?
[22:57:26 CET] <JEEB> I have been looking into upipe but currently my flow is FFmpeg-based.
[22:57:45 CET] <JEEB> and this is regarding I/O and (de)muxing stuff
[22:58:04 CET] <JEEB> you will in any case be using FFmpeg for a lot of your decoding
[22:58:45 CET] <JEEB> for encoding you can either use FFmpeg's encoders, FFmpeg's encoder wrappers (such as the one for libx264), or the other encoder libraries themselves (such as libx264 itself)
[23:06:38 CET] <orlcp440> JEEB: I see, yeah because I read somewhere that even libvlc uses ffmpeg's libraries underneath for certain things
[23:06:54 CET] <JEEB> of course it does
[23:06:57 CET] <JEEB> decoding
[23:07:32 CET] <JEEB> VLC uses its own protocol readers and writers (f.ex. it uses lib555 to do RTSP) and demuxers/muxers where possible
[23:07:42 CET] <JEEB> but then for many a format it just uses FFmpeg for decoding
[23:08:10 CET] <orlcp440> @JEEB: Interesting
[23:11:42 CET] <durandal_1707> avoid ffmpeg at all costs, use paid solutions
[23:58:04 CET] <durandal_1707> doom9 finally hacked, just needs time to get completely offline
[23:59:42 CET] <JEEB> "finally", it has had crap put onto it every now and then
[00:00:00 CET] --- Sun Feb 24 2019