[MPlayer-G2-dev] Re: slices in g2

[D Richard Felker III]

OK, here's the proposal. Arpi says the two types of slices are:
>   when you call next filter's draw_slice after each slice rendering
>   to next vf's buffer completed, and the other type is when you have
>   own small buffer where one slice overwrites the previous one)

So let's start there. First, a refresher on the vp (video pipeline)
layer and buffer types for mp_images...

Video pipeline consists of a collection of decoders, filters, output
devices, and encoders (vd/vf/vo/ve), collectively referred to as
nodes. All can either serve as a source of images, or a destination,
and some (filters) can serve as both. The nodes of the pipeline are
connected by links. A link maintains the negotiated parameters for
passing images between nodes: image format, dimensions, aspect,
strides, etc., and manages the buffers that will pass between its
endpoints. Most filters have exactly one input and one output link,
but the possibility is allowed for a filter to have multiple inputs or
multiple outputs.

Again, the link serves as the broker for images that will pass over
it. Image structures (still called mp_image_t for now) belong to the
link they originate with, and cannot be passed around to other parts
of the pipeline. However, via EXPORT and DIRECT type images, the same
buffers can be passed on in either direction in the chain.

Buffer types for images are as follows:

AUTO(matic): allocated and managed by the vp layer. Automatically
uses the negotiated strides for the link. No owner.

DIRECT(rendering): buffer pointers are filled in by the destination
node's get_buffer function. The destination node becomes the owner,
and will be informed via release_buffer when the reference count
reaches zero.

EXPORT: buffer pointers are filled in by the source node after
requesting the image. Keep in mind that they need not point to a
codec-internal buffer. They might point to the buffers from an
AUTO-type image earlier in the pipeline, or (in some very fancy
multiple-vo setups :) a DIRECT-type buffer from another branch of the
pipeline. The source node which obtains the EXPORT-type buffer becomes
its owner and will be informed when it's released.

[Note that both EXPORT and DIRECT buffers are very useful for avoiding
excess copying between filters. EXPORT should not be thought of as a
backwards-compatibility type for old codecs, because it can do a lot
more than just that!]

INDIRECT: the image has no buffer pointers. Instead, it must be drawn
into via slices. The image structure, which is owned by the
destination node, exists only to carry meta-information about the
frame (pts, etc.). It should also carry in its private data area some
way for the destination node to identify which hidden buffer it
corresponds to, if there is more than one such buffer.

DUMMY: used for dropped frames. No buffers whatsoever, only pts and
perhaps some other metainformation.



OK, now for the new stuff:

If you recall, we were considering Arpi's point that there are two
types of slices. For now (but not in the final design) we'll call them
smart slices (notification as parts are completed) and dumb slices
(source buffer for slices becomes invalid after use).

First, dumb slices. I'd like to propose that drawing via dumb slices
can _always_ be done into DIRECT or INDIRECT type buffers.

The first thing that should come to mind here is that, if a node
supports INDIRECT buffers, it _must_ accept whatever slices it's given
and it _must_ be able to function without any additional source
context. This means that INDIRECT buffers are appropriate for vo
drivers to use, and for filters that just operate locally on pixels
(for instance, equalizer, format/colorspace converter, non-filtered
field splitter). However, filters that can't just operate locally
should not provide INDIRECT buffers, and should instead use the smart
slices method described below.

Note that for the DIRECT case, the vp layer just copies the region and
then notifies the destination node, so in effect you get smart slices
for free!

Now, smart slices! :)

After obtaining an AUTO or EXPORT image, a source node which supports
slices should register with the link layer that it wishes to do smart
slices. This gives the destination node an opportunity to report
whether it supports slices, and to request any buffers it needs from
the next node in the pipeline, etc.

If the destination node accepts the request for slices, the source
node is _required_ to call commit_slice [exactly?] once for each
region of the image.







Now for some API fun...
Functions provided by the vp link layer:

vp_attach_slices: takes an image (AUTO or EXPORT type) and attempts to
initiate smart slice rendering through it.

vp_commit_slice: takes a slice-capable image as a source and notifies
the destination node that the specified slice has been completed.

vp_draw_slice: takes a source from arbitrary pointers and draws the
specified slice to a DIRECT or INDIRECT image, passed as its
destination.



Functions implemented by vp nodes:

attach_slices: Called to request that the destination node prepare for
smart slice rendering from the image passed as an argument. The
destination node should store any references it needs to keep in the
appropriate private data area of the image structure.

commit_slice: The destination node's commit_slice is called by the vp
layer when the source node has finished rendering a slice.

draw_slice: Similar to commit_slice, but only called when rendering
into an INDIRECT buffer or if the destination node does not implement
commit_slice.

get_buffer: Used to obtain DIRECT and INDIRECT buffers from a node.

release_buffer: Called when the reference count on a DIRECT, INDIRECT,
or EXPORT image reaches zero.

detach_slices: Called when the reference count on an image used for
smart slices reaches zero.





Source node implementation:

A source should _never_ request an INDIRECT buffer unless there is a
performance benefit, e.g. reusing a small buffer that will remain in
the processor's cache, or unless it is wrapping an external codec or
filter that only supports dumb style slices.

Instead, sources should use vp_attach_slices when possible. If the
destination node supports INDIRECT buffers but not attach_slices, then
the vp link layer will emulate attach_slices by obtaining an INDIRECT
buffer (i.e. smart slices can be emulated with dumb slices).




Destination node implementation:

In general, a destination node should not support both INDIRECT
buffers and attach_slices; to do so is redundant. Supporting both may
be useful in (very rare) cases where the filter can enjoy better
performance by using smart slices instead of dumb slices.

Again, destination nodes should not support INDIRECT buffers unless
they only perform pixel-localized filterring, stride arithmetic, or
other tasks that do not require the context of nearby pixels.

The destination node should check the buffer type when receiving
commit_slice or draw_slice calls. Often different types call for
different reactions.







OK, I think that's about it for now. Keep in mind that I at least
roughly know what I'm talking about here, since I'm writing the code
as I write the specs. (To make sure what I say is possible in code. :)
Sometime soon I'll post some code, but if I did that now I'd be
drinking cola 'til after the newyear... :))

Rich