[FFmpeg-devel] [RFC] Switching ffmpeg.c to a threaded architecture

[Paul B Mahol]

On Tue, Apr 5, 2022 at 11:06 PM Soft Works <softworkz at hotmail.com> wrote:

>
>
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Anton Khirnov
> > Sent: Tuesday, April 5, 2022 9:46 PM
> > To: FFmpeg development discussions and patches <ffmpeg-
> > devel at ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [RFC] Switching ffmpeg.c to a threaded
> > architecture
> >
> > Quoting Michael Niedermayer (2022-04-05 21:15:42)
> > > On Mon, Apr 04, 2022 at 01:29:48PM +0200, Anton Khirnov wrote:
> > > > Hi,
> > > > this WIP patchset is the first major part of my ongoing work to
> > change
> > > > ffmpeg.c architecture such that every
> > > > - demuxer
> > > > - decoder
> > > > - filtergraph
> > > > - encoder
> > > > - muxer
> > > > lives in its own thread. The advantages of doing this, beyond
> > increased
> > > > throughput, would be enforced separation between these components,
> > > > making the code more local and easier to reason about.
> > > >
> > > > This set implements threading for muxers. My tentative plan is to
> > > > continue with encoders and then filters. The patches still need
> > some
> > > > polishing, especially the last one. Two FATE tests do not yet
> > pass, this
> > > > will be fixed in later iterations.
> > > >
> > > > Meanwhile, comments on the overall approach are especially
> > welcome.
> > >
> > > I agree that cleanup/modularization to make the code easier to
> > > understand is a good idea!
> > > Didnt really look at the patchset yet.
> > > I assume these changes have no real disadvantage ?
> >
> > Playing the devil's advocate, I can think of the following:
> > 1) ffmpeg.c will hard-depend on threads
> > 2) execution flow will become non-deterministic
> > 3) overall resource usage will likely go up due to inter-thread
> >    synchronization and overhead related to new objects
> > 4) large-scale code changes always carry a higher risk of regressions
> >
> > re 1): should not be a problem for any serious system
> > re 2): I spent a lot of effort to ensure the _output_ remains
> >        deterministic (it actually becomes more predictable for some
> >        cases)
> > re 3): I expect the impact to be small and negligible, respectively,
> > but
> >        would have to be measured once the conversion is complete
> > re 4): the only way to avoid this completely would be to stop
> >        development
> >
> > Overall, I believe the advantages far outweigh the potential
> > negatives.
>
> Hi,
>
> do I understand it right that there won't be a single-thread
> operation mode that replicates/corresponds the current behavior?
>
> Not that I wouldn't welcome the performance improvements, but one
> concern I have is debugging filtergraph operations. This is already
> a pretty tedious task in itself, because many relevant decisions
> are made in sub-sub-sub-sub-sub-functions, spread over many places.
> When adding an additional - not even deterministic - part to the
> game, it won't make things easier. It could even create situations
> where it could no longer be possible to replicate an error in a
> debugger - in case the existence of a debugger would cause a variance
> within the constraints of the non-determinism range.
>
>
Can you elaborate more?, otherwise this is PEBKAC.


> From another point of view, this is a change, so fundamental like
> ffmpeg(.c) hasn't seen in a long time.
> I would at least suppose that this could cause issues at many ends,
> and from experience, there may be additional ends where it's rather
> unexpected to  have effects.
>
> In that context, I think that doing a change of such a wide scope
> in an irreversible way like this, would impose quite a burden on
> many other developers, because sooner or later, other developers
> will run into situations where something is no longer working like
> before and you'll regularly wonder whether this might be a consequence
> of ffmpeg.c threading change or caused by other changes.
> But then, you won't be able anymore to bisect on that suspicion,
> because the threading change can't be reverted and (as long as it's
> not shortly after the change) there might have been too many other
> changes to easily port them back to a state before the threading
> change.
>
> I wonder whether this couldn't be done in a way that the current
> behavior can be preserved and activated by option?
>
> Wouldn't it be possible to follow an approach like this:
>
> - Assuming the code would be fine and it would mark the desired
>   end result
> - Put it aside and start over from the current HEAD
> - Iteratively morph the code current code in a (possibly) long
>   sequence of refactoring operations where every single one
>   (and hence in sum) are semantically neutral - until the code
>   is turned more and more into what has already been developed
> - eventually, only few differences will be left, and these can
>   be made switchable by an option - as a result, both - old and
>   new operation modes would be available.
>
> I don't know whether there's a name to this approach, probably
> there is, yet I never cared. Way more important is that I always
> had good results following this methodology.
> The funny thing about it is, that when you have a reliable tooling
> for refactoring, you can even stop thinking (well, sort of..)
> while transforming the code. Also, when you can't imagine how
> the end result would look like or wonder whether it would
> work out at all, it's fun to watch the morphing (if you're
> doing it no-brain-wise)
>
> softworkz
>
>
>
>
>
>
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>