[FFmpeg-devel] [RFC]] swscale modernization proposal

[Michael Niedermayer]

On Sun, Jun 23, 2024 at 12:24:57AM +0200, Niklas Haas wrote:
> On Sat, 22 Jun 2024 21:52:42 +0200 Michael Niedermayer <michael at niedermayer.cc> wrote:
> > On Sat, Jun 22, 2024 at 05:10:28PM +0200, Niklas Haas wrote:
> > > On Sat, 22 Jun 2024 15:23:22 +0100 Andrew Sayers <ffmpeg-devel at pileofstuff.org> wrote:
> > > > On Sat, Jun 22, 2024 at 03:13:34PM +0200, Niklas Haas wrote:
> > > > [...]
> > > > > 
> > > > > ## Comments / feedback?
> > > > > 
> > > > > Does the above approach seem reasonable? How do people feel about introducing
> > > > > a new API vs. trying to hammer the existing API into the shape I want it to be?
> > > > > 
> > > > > I've attached an example of what <avscale.h> could end up looking like. If
> > > > > there is broad agreement on this design, I will move on to an implementation.
> > > > 
> > > > API users seem to have difficulty with this type of big change[[1],
> > > > and doing the interface before the implementation means there's less
> > > > reason for developers to switch while you're still looking for feedback.
> > > > 
> > > > What's the plan to bring them along?
> > > 
> > > Since SwsContext is entirely internal, we can continue providing the
> > > current API on top of whatever internal abstractions we arrive at. As
> > > a trivial example, sws_scale() can construct a temporary AVFrame based
> > > on the provided information, and simply pass that to avscale_frame(). So
> > > I don't think legacy API users are at risk, or pressure to switch,
> > > unless they want access to *new* functionality.
> > > 
> > > For that, the harder step is moving from sws_scale() to
> > > sws_scale_frame(). This is something API users can *already* do. By
> > > contrast, moving from sws_scale_frame() to avscale_frame() should
> > > hopefully be simple, since it just requires making sure the AVFrame is
> > > correctly tagged. Usually, the flow is in the opposite direction - users
> > > receive a correctly tagged AVFrame and manually forward this information
> > > to the SwsContext. So, most of the time, moving to a fully AVFrame-based
> > > API will result in deleting code, rather than adding it.
> > > 
> > > If we wanted to maximize the transition comfort, we should consider
> > > re-using the sws_scale_frame() entrypoint directly. The main reason I am
> > > hesitant to do this is because sws_scale_frame() is currently tied to
> > > the stateful configuration of SwsContext, and mostly ignores the AVFrame
> > > metadata. While changing that is possible, it possibly presents a bigger
> > > API break than simply introducing a new function.
> > 
> > I agree we should keep using the same swscale.h header. It matches the library
> > name thats installed (thats also what the user expects and what (s)he is used to),
> > and its what users #include today.
> > Also its not a audio? scaler so the A is confusing.
> > 
> > And sws_scale_frame() should be used obviously if thats as you say does
> > "maximize the transition comfort"
> > 
> > Maybe simply adding an option for the library user to set the behavior
> > (favour AVFrame properties vs initial properties)
> > And then eventually deprecate and phase out the initial ones
> > 
> > The big advantage here is that we capture all users, noone stays on the old
> > API. And the transition is also simpler, if its just a flag to flip for someone
> > to try the new fully stateless system.
> 
> This could definitely work. We could then also eventually flip the
> condition to where the new behavior becomes the default, and you need to
> set a flag to *disable* it.
> 
> And eventually deprecate sws_init_context(), sws_setCoefficients() etc.
> altogether and just have sws_alloc_context() + sws_scale_frame() be the
> preferred front-ends.
> 
> I expect the actual amount of work to be similar; rather than taking
> SwsContext and pulling everything high-level out into AVScaleContext, we
> start with SwsContext and pull everything low-level out into separate
> sub-contexts (e.g. one SwsScaleContext for each individual scaling
> step).

Yes, i prefer this

thanks alot

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

"Nothing to hide" only works if the folks in power share the values of
you and everyone you know entirely and always will -- Tom Scott

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240623/ebddc695/attachment.sig>