[FFmpeg-devel] [PATCH] avcodec/cuviddec: Add handling HDR10+ sidedata on cuviddec.

[Carlos Ruiz]

> It's just that the cuviddec decoder is more of a relic, from before the
> native hwaccel existed.
> The only reason it's not straight up deprecated is that it's sometimes
> nice to have a "second opinion" when issues crop up, and there are a few
> specific features like hardware-deinterlacing that can't be exposed via
> the native hwaccel.
>
> But I'd normally not like to expand it even further and add complex and
> large features to it. Whenever possible, the native nvdec hwaccel should
> be used.

As someone who recently switched from *_cuvid to * + hwaccel=cuda I have to
agree
that the performance (timing-wise at least) has been very comparable for
decoding.

The only comment I'd like to make though, and this might be a bit of
unpopular opinion
based on some other threads I read, but there are huge advantages of hwaccel
accelerating not just decoding but also resizing (and I guess optionally
cropping).
I know that ideally a decoder should only decode, but think about a common
usecase
in the AI world we live in: you get a bunch of simultaneous 4k (or 1080p)
incoming
rtsp streams and you want to decode the video and pass it through some ML
model,
e.g. in TensorRT (to stick with the Nvidia example). The native hevc codec
doesn't
support resizing, so you decode video at full 4k on the gpu, which means
allocating
something like 5-10 surfaces at 3840x2160 which becomes 250MB of GPU memory,
and then you have immediately take all of those frames, pass them through a
filterchain,
scale them down to e.g. 640x360, and waste CUDA cores instead of leveraging
the
dedicated video downsizing inside the NVDEC chip. Now do that for 50 camera
streams
and you'll quickly run out of GPU memory with a GPU utilization under 10%
haha.

This is exactly why I submitted a patch yesterday that would allow using
the hevc
codec with nvdec hwaccel, while resizing on the gpu like hevc_cuviddec
does, and
the memory (and GPU) consuption goes waaay down (e.g. 6MB of GPU VRAM
instead of 250MB per camera). I know this is a different discussion but
thought
it was appropriate to share because deprecating cuviddec or rejecting my
patch
would leave part of the community out.


On Fri, Sep 20, 2024 at 12:48 PM Timo Rothenpieler <timo at rothenpieler.org>
wrote:

> On 20/09/2024 04:08, 김윤주 wrote:
> > Does native decoder refer to hevc (hevcdec.c)?
> > I tried using hevc and in environments with low CPU performance,
> hevc_cuvid
> > was much faster.
> > So, I used hevc_cuvid for decoding but encountered an issue where HDR10+
> > sidedata did not exist.
> > That's why I wrote this patch.
>
> You did turn on hwaccel, right?
> I don't see why the native decoder would be much slower at parsing then
> nvidias parsers.
>
> > I thought that the original content's sidedata should be preserved even
> > after decoding regardless of which decoder is used.
> > And I thought hevc_cuvid was the only way to get Nvidia hwaccel support.
> > If I'm mistaken about anything, please let me know.
>
> It's just that the cuviddec decoder is more of a relic, from before the
> native hwaccel existed.
> The only reason it's not straight up deprecated is that it's sometimes
> nice to have a "second opinion" when issues crop up, and there are a few
> specific features like hardware-deinterlacing that can't be exposed via
> the native hwaccel.
>
> But I'd normally not like to expand it even further and add complex and
> large features to it. Whenever possible, the native nvdec hwaccel should
> be used.
>
> > Also, is it correct to respond to your comments like this?
> > It seems quite different from the format you sent.
>
> Top-posting isn't exactly liked here, though I don't really have a
> strong opinion on it.
>
> > Apologies, as I'm still relatively new to the FFmpeg community and have a
> > lot to learn.
> > Any additional guidance would be greatly appreciated.
> >
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Timo
> > Rothenpieler
> > Sent: Friday, September 20, 2024 4:48 AM
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/cuviddec: Add handling HDR10+
> > sidedata on cuviddec.
> >
> > On 19.09.2024 06:43, yoonjoo wrote:
> >> Implemented decoding of NAL units and handling HDR10+ sidedata by
> >> referring to hevcdec.
> >
> > Why? Can't you just use the native decoder with nvdec hwaccel?
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://protect2.fireeye.com/v1/url?k=c11e9bb6-9e85a2a9-c11f10f9-
> > 000babdfecba-c4d76825032a0b52&q=1&e=342e88a0-295e-4dc1-a57f-
> >
> 8f5aab974009&u=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org
> > with subject "unsubscribe".
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>