[FFmpeg-devel] [PATCH v0] Implement promeg decoder.

[Michael Niedermayer]

Hi Devin

On Fri, Jan 17, 2025 at 12:54:10PM -0500, Devin Heitmueller wrote:
> Hello all,
> 
> I've got some experience in this particular area, so figured it might
> be helpful to offer another voice.
> 
> Kieran makes a number of excellent points, and I largely agree with
> his discussion of the problem space.

> It's an intrinsically difficult
> problem.

sure


> The data arrives on multiple sockets, leading to all sorts
> of opportunities for timing behavior and reordering issues.

how does this matter?
Either i have received a error free packet or i have not
Either i have a set of equations taht allows some kind of optimal
unique solution allowing recovery of a missing packet or i dont.

When its time to output a packet either it can be satisfied because
its determined by the avaialable packets or its not.

Where is the complexity here ?

At this level everything in multimedia is complex or nothing is.
Its more a matter of viewpoint than the exact problem at hand.


> And, in
> my experience, I certainly agree with Kieran's assertion that the
> quality of implementations vary wildly across both open source and
> commercial products (e.g. hardware encoders and transcoders) that
> attempt to implement the standard.

of course. Thats true always everywhere, isnt it?


> Not to mention the
> interoperability issues when receiving streams from all these
> different products.

And FFmpeg has always been the most compatible when it comes to our
demuxers and decoders.


> 
> Building an implementation where you are confident that it does it
> "right" generally means having a large corpus of test vectors that
> exercise edge cases both in delivery timing and packet loss, assessing
> the result to determine which packets should have been recoverable
> versus what was actually recovered.

Of course


> This is considerably beyond the
> sorts of tests ffmpeg typically does in their FATE suite (which AFAIK
> doesn't attempt to validate libavformat protocols at all).

And that is really bad. FATE should cover libavformat protocols
We need some instrumentation/connextion thing that allows us to
inject testdata into them without a real network on the other side
in a clean simple and platform independant way.


[...]
> Marking it as experimental seems like a reasonable ask, given that
> it's brand new code.  That would both allow it to be evaluated/tested
> by those who care about it, and there's a base onto which to submit
> fixes/improvements.  And, like with anything else, if it's determined
> that the implementation works very poorly and/or it gets abandoned, it
> can always be dropped from the tree.

yes

And we seem to agree 95% on all this.

The part i do not agree with and iam not convinced about is that this
cannot be done in a clean and fully working and heuristics free
way in the real world.

You sure can write an implementation that uses hundreads of heuristics
to build a short binary coding tree. for example Shannon–Fano codes
could be a start point. But that doesnt mean there isnt an optimal
and clean solution (which there is and its huffman trees)

Or for a more complex problem you can look at early MPEG encoders
how they accumulated heuristics to decide to use intra or inter MBs
or make other decissions.
This did not preclude the use of Rate Distortion to make these
decissions in later encoders with later than also psycho-vissual
metrics.
These are no longer heuristics, these are optimal solutions under
a given Rate Distortion model. This is also no nonsense
its "modern" encoders in the real world. (modern as in last few decades)

So the claim that it "requires heuristics" and cannot be done cleanly
touches me the wrong way. That claim I disagree with

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

You can kill me, but you cannot change the truth.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20250119/32a6e33f/attachment.sig>