[FFmpeg-devel] Let us review and collebrate on vvc native decoder.

[Nuo Mi]

Hi Ronald,
Thank you for the detailed comment. You read code really quickly.

On Sat, Jan 14, 2023 at 10:28 PM Ronald S. Bultje <rsbultje at gmail.com>
wrote:

> Hi,
>
> On Sat, Jan 14, 2023 at 8:32 AM Nuo Mi <nuomi2021 at gmail.com> wrote:
>
> > On Sat, Jan 14, 2023 at 9:13 PM Nuo Mi <nuomi2021 at gmail.com> wrote:
> > > I am happy to send out the first draft of vvc decoder.
> > > It's not ready for upstream yet, but it's a good base ground for review
> > > and future improvement.
> > >
> > > It has the following features:
> > > * C only
> > > + Fast. On a 4 cores laptop, it can get 30~35+ fps for 10bits 1080P.
> > > + Support traditional features I, P B frames, 8/10 bits, chroma
> 400,420,
> > > 422, and 444,
> > > + Support VVC new tools like MIP, CCLM, AFFINE, GPM, DMVR, PROF, BDOF,
> > > LMCS, ALF
> > > -  Not support RPR, PCM, IBC, PALETTE, and other minor features yet.
> > >
> > > Please help review the attached patch and provide your valuable
> comment.
> >
>
> The custom threading framework (Executor, Task) - can you elaborate on this
> further? If this is meant to be used outside (since it lacks VVC prefix),
> it should be properly namespaced (FF, AV). It seems like you're trying to
> combine row/col/frame threading from vvc_thread.c. These (frame, col/row)
> are fairly common in FFmpeg and there already is existing API for that
> (although you can't combine them; ask kurosu who IIRC has a private branch
> doing this for hevc :-) ). This probably needs some further discussion
> depending on what the rest of the community wants. Right now, this adds VVC
> private functionality to enable combination of "slice" (row/col) and
> "frame" threading, which other codecs wouldn't have. This is probably not a
> desirable end state.
>

Yes. It can be used by other codecs or even filters too. The task
allocation, and the task readiness check
delegate to the caller. so It gets more flexible from the caller's or
executor's view.
When I read the hevc code it was a little surprise to me since it can't
combine frame and slice threading.
I guess it will impact the fps, but I do not have time to provide it.
I will check with kurosu offline, It's better to have a unified solution
for hevc and vvc. I can follow him if the api can do frame + slice/CTU
multi-thread.


> Some functions contain av_assert calls, e.g. chroma_mc_bi or
> get_luma_weight. Are these not yet implemented or impossible because of
> bitstream coding constraints? If former, please use
> avpriv_report_missing_feature() so the user is informed of the incomplete
> decoding. If the latter, please add a comment and remove #if 0 code. For
> other cases of missing features (e.g. PCM), don't silently return 0, but
> use avpriv_report_missing_feature() so the user is informed.
>

Yes. weight prediction and  PCM are missing features, it's listed at
https://github.com/ffvvc/FFmpeg/issues.
 I or some kindly contributor will implement it before we upstream it.
Thank you for your suggestion, if we can't fix it before upstream, I will
use avpriv_report_missing_feature()


> How come vvcdsp has only 8/10 bits/component code but vvcpred has 8/9/10/12
> bits/component code?
>

I will remove it.


> Not a lot of comments other than references to the spec...
>
Before I send out the patch, it's like walking in a dark tunnel. I try my
best to get to another side.
Not have so much time to polish things. I will add more comments and do
some react to make the code more readable.


> Is any of the code (e.g. tools such as sao dsp) shareable with HEVC? This
> would reduce the total implementation requirement for arch-specific code.
>

SAO can be fully reused, most inter predict code can be reused, and some
parts of intra-predict and deblocking can be reused..
I will create a full list at the github repo.

>
> Ronald
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>