[FFmpeg-devel] I've written a filter in Rust

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Michael Niedermayer
> Sent: Freitag, 21. Februar 2025 14:22
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] I've written a filter in Rust
> 
> Hi
> 
> On Fri, Feb 21, 2025 at 10:01:56AM +0100, Tomas Härdin wrote:
> > tor 2025-02-20 klockan 23:49 +0100 skrev Michael Niedermayer:
> > > Hi
> > >
> > > On Thu, Feb 20, 2025 at 02:06:47PM +0100, Leandro Santiago wrote:
> > > > [insert meme here]
> > > [...]
> > > > I also recorded a video showing the filter in action [7].
> > > [...
> > > > [7] https://youtu.be/U_y4-NnaINg
> > >
> > > cool, it doesnt detect everyone though
> > >
> > > also i think this shows how useful a plugin framework would be for
> > > ffmpeg
> > >
> > > with plugins everyone could use,test and contribute to this today.
> > > without plugins, this needs to be merged in ffmpeg git master. (which
> > > will take some time i suspect)
> >
> > Have we not gone over and rejected plugins many times? I recall points
> 
> no
> there was no formal and no public informal vote that i remember.
> 
> ive raised the issue with plugins many times. Because it would
> allow people and myself to contribute more complex features and
> end the stagnation of FFmpeg.

Yup, that's exactly one of the reasons why other projects like GStreamer
have gained that much popularity. It might not have happened when
ffmpeg would have been more open and extensible in the first place.

> Think of AI filters (like text to image, automated language translation of
> subtiltes,
> image to metadata, audio to subtitle, upscaling, and infinite more ideas)
> Neural network based image, video and audio compression
> thats both local neural networks and also using some server based APIs

I had similar things in mind already. The graphicsubs2text filter kind of uses
AI already, but it was only possible because it does it through a standard 
lib (tesseract). But as soon as it comes to anything which requires using 
an API that is vendor-specific, that's something which nobody will want 
to have in ffmpeg, and it might also change rapidly in a way that it's already
outdated when its containing ffmpeg version gets packaged by a distro.


> In general innovation needs the innovators to be in charge.
> Plugins arent the only way to achieve that of course.

Totally agree. Saying that we have no solution for these things is a path 
to becoming Irrelevant in the future.


What do you people think about the feasibility of a compile-time plugin
mechanism?
Given that on most platforms (except Windows), you can't easily mix 
binaries from different origins (i.e. built differently), it would be much
easier when plugin binaries could be compiled as part of building ffmpeg
rather than having to set up and maintain separate builds for each plugin.

sw