[FFmpeg-devel] [PATCH 16/23] avcodec/hevc/refs: export Stereo 3D side data

[James Almer]

On 9/17/2024 8:53 AM, Anton Khirnov wrote:
> Quoting James Almer (2024-09-15 00:12:52)
>> (is there a reason you wanted the view_id side data and this one
>> applied to the frame before get_buffer()?),
> 
> So that the caller knows what view is get_buffer() called for.
> 
>> As this is now being called before ff_progress_frame_get_buffer() it
>> became a no-op and any stereo 3d side data in the input packet will be
>> appended to the frame, resulting in something like:
>>
>>> [Parsed_showinfo_0 @ 00000281481551c0]   side data - View ID: view id: 0
>>> [Parsed_showinfo_0 @ 00000281481551c0]   side data - Stereo 3D: type - frame alternate, view - right, primary_eye - none
>>> [Parsed_showinfo_0 @ 00000281481551c0]   side data - Spherical Mapping: rectilinear
>>> [Parsed_showinfo_0 @ 00000281481551c0]   side data - Stereo 3D: type - unspecified, view - packed, primary_eye - none, baseline: 19240, horizontal_disparity_adjustment: 0.0200, horizontal_field_of_view: 63.400
>>
>> We don't really want to lose the information that's coded in the
>> container but not in the bitstream (Which happened in the previous
>> version of the patch too), so we should instead amend the container
>> level side data with the bitstream information.
>>
>> Something like:
> 
> That results in this information NOT being available to caller in
> get_buffer().
> 
> The least bad solution I see is have ff_decode_frame_props() merge
> container information into existing side data, if set.

We need to ensure two things:

- Frame side data as output by lavc should contain the stereo3d 
information from the decoder alongside any from avctx. This should be in 
a single frame side data entry.
- Output stream side data as passed to lavf by the CLI should contain 
information that applies to the entire stream, which includes all the 
views packets may contain. This obviously also applies to what is passed 
to the encoder beforehand.

The former can be done, like you said above, having 
ff_decode_frame_props() merge the side data. The latter however would 
require changes to the CLI so the output encoder is not blindly 
initialized with the side data from the first frame that comes out of 
lavfi, which for mv-hevc always reports one specific view (and frame 
alternate type).
The CLI would need to use the knowledge of what the user requested to 
the decoder (one view? all views?) to update the global side data that's 
passed to the encoder and ultimately makes it to the muxer.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240917/2bf46cb6/attachment.sig>