[FFmpeg-devel] [PATCH v2 1/4] avcodec/aom_film_grain: add AOM film grain synthesis

[Niklas Haas]

On Mon, 11 Mar 2024 22:32:01 +0000 Mark Thompson <sw at jkqxz.net> wrote:
> This is not correct.  Along with scalable cases, the multiple param sets are to support applying film grain at the display resolution after scaling, providing a better result than upscaling the grain applied at the decode resolution.
> 
> For example, you could have a scalable stream with operating points of 1920x1080, 1280x720 and 640x360.  The AFGS1 metadata associated with the stream would then have film grain parameters for those three resolutions, plus perhaps 2560x1440 and 3840x2160.
> 
> In the ideal case you then pick the operating point for decode based on your available bandwidth and decode capabilities, and the resolution for film grain application based on the display.  The decode happens without any film grain, the clean video is upscaled, and then the film grain is applied immediately before display.
> 
> A current conforming AV1 implementation which only supports applying film grain as part of the decode process can do so and produce a conforming result, but the quality may not be as good as the ideal case because the presence of noise will affect the upscale quality and also the grain itself will be scaled in a way which may not look correct.
> 
> I'm not sure what the best way to expose this is.  For a player application an option to select the intended display resolution and then export an AV1 film grain side data as it is now is sufficient, but that doesn't really work in an application like ffmpeg where the target resolution isn't directly known.

One way to get there would be to attach multiple AVFilmGrainAOMParams
structs as side data to the frame, plus adding the extra metadata for
the resolution (and colorimetry etc.) to that struct.

May be a bit awkward to deal with on the user side, though.

> (Also note that a transcode can carry the AFGS1 message from the source to the output without ever touching it, as long as the target resolution satisfies the requirement on the coded resolution being available in the param sets.  It seems desirable to support this possibility.)
> 
> Thanks,
> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".