[FFmpeg-devel] [RFC] 5 year plan & Inovation

[Michael Niedermayer]

On Thu, Apr 18, 2024 at 04:02:07PM +0200, Niklas Haas wrote:
> On Wed, 17 Apr 2024 15:58:32 +0200 Michael Niedermayer <michael at niedermayer.cc> wrote:
> > Hi all
> > 
> > The pace of inovation in FFmpeg has been slowing down.
> > Most work is concentarted nowadays on code refactoring, and adding
> > support for new codecs and formats.
> > 
> > Should we
> > * make a list of longer term goals
> > * vote on them
> > * and then together work towards implementing them
> > ?
> > 
> > (The idea here is to increase the success of larger efforts
> >  than adding codecs and refactoring code)
> > It would then also not be possible for individuals to object
> > to a previously agreed goal.
> > And it would add ideas for which we can try to get funding/grants for
> > 
> > (larger scale changes need consensus first that we as a whole want
> >  them before we would be able to ask for funding/grants for them)
> > 
> > Some ideas and why they would help FFmpeg:
> > 
> > * Switch to a plugin architecture
> >     (Increase the number of developers willing to contribute and reduce
> >      friction as the team and community grows)
> 
> This would almost surely hurt productivity

i dont agree, ill elaborae below


> as it will require exposing,
> versioning, documenting and maintaining a vast number of internal APIs

yes to some extend that is needed. It can be more or less depending
on how things are implemented


> which we are currently at the liberty of modifying freely.

we are modifying these APIs for decades. That modification of APIs
their documentation and all code using them costs time.

More so we have issues with patch-management. And i claim this is
not the mailing list but a lack of time and in some cases a lack of
interrest in many areas.

A plugin system moves this patch-management to people who actually
care, that is the authors of the codecs and (de)muxers.

Our productivity as is, is not good, many patches are ignored.
The people caring about these patches are their Authors and yet they
are powerless as they must sometimes wait many months for reviews
Besides that, developers are leaving for various reasons and they
are forced to setup full forks not being able to maintain their
code in any other way.
IMO A plugin system would improve productivity as everyone could work
on their own terms.
No week or month long delays and weekly pinging patches
No risk about patches being rejected or ignored
No need to read every other discussion on the ML. One can just
simply work on their own plugin looking just at the API documentation
...



> 
> > * ffchat
> >     (expand into realtime chat / zoom) this would
> >     bring in more users and developers, and we basically have almost
> >     all parts for it already but some people where against it
> 
> This seems like a user application on top of FFmpeg, not something that
> should be part of FFmpeg core. Can you explain what modifications in
> FFmpeg would be necessary for something like this?

ffmpeg, ffplay, ffprobe are also user applications.


> 
> > * client side / in browser support
> >     (expand towards webapps, webpages using ffmpeg client side in the browser)
> >     bring in more users and developers, and it will be costly for us
> >     if we let others take this area as its important and significant
> 
> I don't understand this point - don't all major browsers already vendor
> FFmpeg for decoding?

FFmpeg does more than decoding.


> 
> > * AI / neural network filters and codecs
> >     The future seems to be AI based. Future Filters and Codecs will use
> >     neural networks. FFmpeg can be at the forefront, developing these
> 
> We already have TensorFlow support, no? (vf_sr etc.) What more is
> needed?

more of that AND more convenience

lets pick a comparission
to run fate
you write "make fate"
if you want to do it with the samples its
make fate-rsync ; make fate

if you want to use vf_sr, its reading the docs, looking at some scripts reading their docs
and i presume selecting a training set ? creating a model ? ....

how many people do that ?

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240418/0cb4396b/attachment.sig>