[FFmpeg-devel] Add protocol for Android content providers

[Matthieu Bouron]

Le jeu. 15 févr. 2024, 9:46 AM, Zhao Zhili <quinkblack at foxmail.com> a
écrit :

>
> > 在 2024年2月15日，下午3:57，Matthieu Bouron <matthieu.bouron at gmail.com> 写道：
> >
> > On Thu, Feb 15, 2024 at 12:13:59PM +0800, Zhao Zhili wrote:
> >>
> >>
> >>>> On Feb 14, 2024, at 06:50, Matthieu Bouron <matthieu.bouron at gmail.com>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> On Android, content providers are used for accessing files through
> shared
> >>> mechanisms. One typical case would be an app willing to open a video
> from
> >>> Google Photos, gallery apps, TikTok, Instagram or some other providers.
> >>> A content URI looks something like "content://authority/path/id", see:
> >>> https://developer.android.com/reference/android/content/ContentUris
> >>>
> https://developer.android.com/guide/topics/providers/content-provider-basics
> >>>
> >>> It can currently be somehow managed through clumsy means such as using
> a "fd:"
> >>> filename and crafting a special AVOption, which also has the drawback
> of
> >>> requiring the third party to carry around opened file descriptors
> (with the
> >>> multiple opened file limitations implied). Custom AVIOContexts are
> also an
> >>
> >> File descriptor is a general abstraction layer, it target more
> platforms than
> >> Android specific content provider. Android provided getFd() API since
> API
> >> level 12, I guess that’s the default method to deal with content
> provider in
> >> native code. It’s a few lines of code to get native fd in Java, but
> dozens of code
> >> in C with JNI, which is what this patchset done.
> >>
> >> For multiple opened file limitations issue, they can close the file
> descriptor after
> >> open. It’s unlikely to reach the limit in normal case without leak.
> >>
> >> I’m OK to provide this android_content_protocol helper if user requests.
> >
> > I've been doing this kind of work for 3/4 users (including myself) at
> this
> > point and have to do it another time, this is what motivated me to
> propose
> > this patchset.
> >
> >>
> >>> option. Both options will have to deal with the JNI though and end
> users will
> >>> have to re-implement the same exact thing.
> >>
> >> User still need to deal with JNI with the new android_content_protocol,
> more or
> >> less, it’s unavoidable.
> >
> > The advantage I see of using this protocol is that the user only need to
> > call av_jni_set_jvm() + av_jni_set_android_app_ctx() at the start of the
> > application and FFmpeg will handle the content-uri transparently. This is
> > especially helpful if the Android application rely on multiple libraries
> > that in turn rely on FFmpeg to read medias.
>
> The url still need to be passed from Java to C via JNI, it’s not much
> different compared to pass fd.
>

It's not that much different I agree. But let's say you have a rendering
engine (in C) where you need to pass hundreds of media (from the user) to
render a scene, each media is used at different time during the rendering.
And Ffmpeg is not a direct dependency and can be called from different
libraries/places used by the rendering engine. Calling
av_jni_set_android_app_ctx() and you're done, you can pass the content URI
to the engine (passing fd at this stage is not an option imho). You still
need to convert the uri from java string to c before calling the c code,
but it's a direct translation which is typically part of a binding.



> >
> >>
> >>>
> >>> This patchset addresses this by adding a content provider protocol,
> which has
> >>> an API fairly similar to fopen. Android 11 appears to provide something
> >>> transparent within fopen(), but FFmpeg doesn't use it in the file
> protocol, and
> >>> Android < 11 are still widely used.
> >>>
> >>> The first part move the JNI infrastructure from avcodec to avutil (it
> remains
> >>> internally shared, there is little user implication),
> >>
> >> OK. JNI infrastructure should belong to avutil at the first place, so
> hwcontext_mediacodec
> >> and so on can use it. Unfortunately for those new avpriv_.
> >
> > What do you mean by "Unfortunately" ? Would you like to make the JNI API
> > public ?
>
> I think it’s our target to reduce the number of avpriv API, not increase
> it. Does duplicate the compile unit work in this case so we don’t need to
> export the symbols?
>

Directly including ffjni.c from libavformat/file.c works. We still need to
pass the application context though (could be added to avcodec/jni.h)


> >
> > [...]
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscri
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>