[MPlayer-dev-eng] [PATCH] reduce flashing on resize with VDPAU

[Uoti Urpala]

On Tue, 2009-03-24 at 11:16 -0700, Stephen Warren wrote:
> Uoti Urpala wrote:
> > > Vdpau is missing a cancel queued surface, so it is not easy to remove an
> > > already queued surface from the presentation queue - if we wanted to
> > > drop an earlier frame.
> > 
> > This also sounds like a problem for UI functionality. If we use the same
> > window for UI elements, does queuing multiple frames in the presentation
> > queue mean we've committed to a significant period of time with zero UI
> > responsiveness?
> 
> The functionality isn't missing; it doesn't make sense to have.
> 
> In any system where you queue surfaces for display, the length of the queue
> Is going to be your OSD/UI latency. There's always a trade-off:
> 
> a) Very low (or zero) queue depth: Immediate (or fast) UI responsiveness,
> but crappy video jitter due to the possibility of your app being scheduled
> out at inopportune times, causing video to be delivered late.

This doesn't match what I see in practical MPlayer use. The timing
accuracy on current kernels is a lot better than one vsync interval, and
timing inaccuracies are not a cause of noticeable jitter.

> b) Higher queue depth: Slower UI responsiveness (but we're typically only
> talking a small handful of VSYNCs) but no video jitter, since the surfaces
> are being dequeued/displayed in HW.

Maybe this is helpful on a significantly loaded system, but I doubt the
difference is noticeable on an otherwise idle one.

> In either case (a) or (b) above, you'd never cancel display of a surface
> in the queue; this would cause video stream drops, jerks in any animation
> in the OSD/UI, etc.

Only if you canceled it so close to the display time that a replacement
could not be queued in time.

I don't consider the ability to drop queued surfaces to be essential,
but I can think of a couple more reasons why it could be useful:
- It can be simpler to create a long queue "just in case", but modify it
later if you see you do have time or an UI event happens. Long latency
may be rare enough that cutting the queue shorter is appropriate if you
know you have an UI event. If latency results from the actions of the
program itself (you don't have a separate thread for the video output,
but occasionally execute tasks that can take some time in the same
thread) then you may need to queue say 150 ms of frames for the
worst-case duration of a call to other functionality, but know that you
can safely modify the queue beyond 20 ms while executing the video
output code.
- Decoding and queuing frames in batches could be usable to avoid
processor wakeups and saving the battery on laptops, but that'd require
queuing multiple frames and so ruin UI latency.

As for the specific case discussed in this thread, video with frame rate
higher than the display vsync rate, could the video driver not add a
mode where it automatically drops queued frames if it knows they can not
be shown before it will be time for the _next_ queued frame already? The
driver should have the information needed to determine that easily.

> Do note however, that the rate at which surfaces are pushed through the
> presentation queue is 100% decoupled from the frame-rate of the video clip,
> if any, being played. There's absolutely nothing stopping the app from
> compositing a single decoded video picture into N different output surfaces,
> so that the UI can run at e.g. 60Hz whilst the video runs at even 10Hz. I've
> worked on a number of applications that work exactly this way. If using this
> technique, UI latency is always precisely a function of VSYNC rate and queue
> depth, and is orthogonal to video frame rate.

Does this constant-framerate mode give any benefit except in the case
where you want to play constant-FPS UI animations? Otherwise specifying
the queue depth in milliseconds instead of constant number of frames and
queuing additional frames when UI changes seems superior.