[FFmpeg-devel] SWS cleanup / SPI Funding Suggestion

[James Almer]

On 10/13/2023 7:34 PM, Niklas Haas wrote:
> On Fri, 13 Oct 2023 21:19:34 +0200 Michael Niedermayer <michael at niedermayer.cc> wrote:
>> Hi everyone
>>
>> I propose using 15k$ from SPI for funding sws cleanup work.
>> this is substantially less than what people belive this needs (see IRC logs from yesterday or so)
>> So it really is more a small price for a good deed and not proper payment.
>> This of course is only available to competent developers. (exact rules or how thats determined
>> would still need to be decided unless its a clear case)
>> Also the exact outcome and goal would need to be discussed by the community and whoever
>> does the work.
>> But some goals would probably be to make sws
>> * pleasent to work with
>> * similar speed or faster
>> * proper multithreading
>> * proper full colorspace convertion not ignoring gamma, primaries, ...
>> * clean / understandable modular design (maybe everything can be a "Filter" inside sws
>>    that get build into a chain)
>>
>> Proper payment (50k$ maybe) would be too much in relation to what SPI has ATM (150k$)
>>
>> Above all, this is just my oppinion, the actual SPI funding also would need to
>> be approved by the community. This can happen after a specific volunteer comes forth
>> or before, whichever way the community prefers.
>>
>> thx
> 
> My gut instinct is that the correct path forwards is to first create a
> new API which can internally dispatch to swscale, libplacebo or zimg
> based on what's compiled/available/preferred (e.g. using GPU filters for
> hwframes, CPU filters for swframes, zimg (or vf_colorspace's primitives)
> for colorspace conversions ...).
> 
> Much of the pain points of my own recent experience with swscale
> revolves around libswscale's very low level API, complete lack of
> understanding of most AVFrame metadata, and relatively complex

Anton wrote and pushed an AVFrame based API. It can surely be 
improved/extended to use AVFrame metadata.

> configuration requirements. (vf_scale's comments even acknowledge that
> libswscale should handle this stuff internally, so users just need to
> point an AVFrame at both ends and let it do the right thing
> automatically)
> 
> The second pain point of extending libswscale itself is that the
> high-level configuration and low-level "scaling" primitives are deeply
> entangled, reconfigured in various places, and hard to touch without
> fear of breaking edge cases.
> 
> A new API would solve both problems. It would allow us to write new,
> AVFrame-based, high-level "business logic", which could continue to call
> into the low-level swscale implementation details and kernels for the
> time being, and also tap into GPU filters (a la libplacebo) where
> possible.
> 
> As time moves on we could replace those underlying primitives by cleaned
> up rewrites where necessary, until the dependency on libswscale itself
> is null.
> 
> Simultaneously, vf_scale can be rewritten on top of this API, which
> would allow e.g. auto-scale filters to start doing scaling on GPU when
> conversion between two hwformats is needed.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".