[FFmpeg-devel] [PATCH 0/5] Apple Spatial Metadata

[Derek Buitenhuis]

So I am a bit hesitant to send this, because while some of the boxes are documented
officially by Apple, some of what are created by the Vision Pro / iPhones are not,
because the official PDF is outdated.

It's a call we have to make whether or not we want to push this, or hold out faith
Apple will eventually update their document.

I reverse engienered the rest of the boxes based on the what the AVFoundation APIs
expose for spatial files. See each commit message.

Example file: https://www.dropbox.com/scl/fi/g4b0q6c0e0xb9kwxcsvaa/IMG_1722.MOV?rlkey=az1qik442xut3jwq73jvcau9n&dl=0
Example stream from Apple: https://developer.apple.com/streaming/examples/

I can provide more if needed.

Anyone with an iPhone 15+ or Vision Pro can make some files, too, of course.

Some other notes:
  * I did not implement the 'must' box, as described in [1], since its only purpose
    seems to be to fail input files. I can add this as information-only if people want.
    I have not found any samples with this box, including files made by Apple.
  * Right now I am checking if the side data values are zero, as this is the best I could
    come up with, but comments welcome.
  * I like the idea that the 'dadj' box sits inside a 'cmfy' box, it seems very wholesome.

[1] https://developer.apple.com/av-foundation/HEVC-Stereo-Video-Profile.pdf

Derek Buitenhuis (5):
  avutil/spherical: Add more spherical types
  avutil/stereo3d: Fill out stereo info provided by Vision Pro files
  fftools/ffprobe: Print more Stereo 3D info from side data
  avformat/mov: Add support for exporting Video Extension Usage info
  avformat/mov: Add support for reading and exporting horizontal field
    of view

 fftools/ffprobe.c     |   8 ++
 libavformat/mov.c     | 308 ++++++++++++++++++++++++++++++++++++++++++
 libavutil/spherical.c |   3 +
 libavutil/spherical.h |  16 +++
 libavutil/stereo3d.c  |  52 +++++++
 libavutil/stereo3d.h  |  80 +++++++++++
 libavutil/version.h   |   2 +-
 7 files changed, 468 insertions(+), 1 deletion(-)

-- 
2.43.0