[FFmpeg-devel] working with file descriptors on Android

[Alex Cohn]

On Sat, Jul 25, 2020 at 9:37 PM Matthieu Bouron <matthieu.bouron at gmail.com>
wrote:
>
> On Wed, Jul 22, 2020 at 2:38 PM Alex Cohn <alexcohn at netvision.net.il>
wrote:
>
> > Usually, we employ the `pipe:` protocol to deal with the numerical
> > file descriptors, and this answers all our needs. On Android, there is
> > a different use case in which numerical file descriptors appear, and
> > this is not covered well with `pipe:` protocol.
> >
> > This happens when a file is opened in scoped storage
> > (https://developer.android.com/training/data-storage#scoped-storage).
> > Currently, there is an exception that still allows `stdio.h` - based
> > access to the media files
> > (
> >
https://developer.android.com/preview/privacy/storage#media-files-raw-paths
> > ),
> > but the document says that it may be slow (we still cannot have true
> > evidence since Android 11 is not released yet), and anyways the clean
> > way to access media files on what is known as 'external storage' is
> > through a document picker Intent
> > (
> >
https://developer.android.com/reference/android/content/Intent#ACTION_OPEN_DOCUMENT
> > and
> >
https://developer.android.com/reference/android/content/Intent#ACTION_CREATE_DOCUMENT
> > ).
> >
> > The Intent produces a `content://` URI from which a DocumentProvider
> > can produce an integer file descriptor. This descriptor can be passed
> > to ffmpeg via pipe: protocol, and almost works, except for a few
> > glitches.
> >
> >  1. This fd must be closed after use. Pipe is not closeable.
> >
> >  2. This fd is seekable, and therefore can be used to work with `.mp4`
> > or some other file formats that don't work through pipe protocol.
> >
> >  3. We can find the actual file name extension for this fd, to
> > facilitate `av_guess_format()` both for input and for output.
> >
> > I have recently prepared two approaches to face this issue. One is an
> > easy patch for the `file:` protocol that recognizes the `/proc/self/`
> > prefix and uses the number as fd. This relies heavily on Java (or
> > Kotlin) processing of the results of document picker. The other way
> > adds a `content://` protocol and does all heavy lifting (calling
> > system Java API through JNI) itself.
> >
>
> Hi,
>
> I already submitted a patch for the second approach in 2015-2016
> (while submitting JNI support + MediaCodec). This approach was
> rejected because the project did not want to have jni support in
> libavutils but instead have it scoped to libavcodec with the decoder.
>
> Original thread:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2015-October/180593.html
> The patch:
> https://www.mail-archive.com/ffmpeg-devel@ffmpeg.org/msg29451.html
>
> I use this patch in production at work (which contains a hack to import
jni
> support to libavformat):
>
https://github.com/Stupeflix/FFmpeg/commit/6d06b14939cb05e8a377d2ba14ed689b361f0303
>
> I don't know if the project has changed his mind regarding the java/jni
> stuff but I'd also like to upstream this feature (I think content uri
> support
> is a nice and useful feature for the Android platform).
>
> Best regards,
> Matthieu
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".

Thanks for sharing, Matthieu. Such discussions are poorly discoverable. I
have learned a lot from the thred starting with
https://www.mail-archive.com/ffmpeg-devel@ffmpeg.org/msg29498.html.
Recently, I pledged to have JNI_GetCreatedJavaVMs() properly documented and
exposed (https://github.com/android/ndk/issues/1320). It would be great if
you add your insights to that thread.

I agree that adaptation of the content Uri scheme should begin with
defining a policy of how JNI be contained in ffmpeg.

If this is still out of scope, the alternatives are:
 - to patch the `file:` protocol, and let it understand that
"/proc/self/69" should not be fopen()-ed but rather parsed to extract and
use the number as FileContext->fd.
 - to create an `fd:` protocol (which will be equivalent to `pipe:` but
with close() and seek())
 - to extend the `pipe_options`, introducing `closeable` and `seekable`
(off by default).

IMO, easy access to the actual file name is also important, to keep
compatibility with av_guess_format(). Here is how I do it with `file:`
protocol:

________________________________________________________________

public static String getCommandParameter(Context context, Uri uri) {

    String displayName = "unknown";
    Cursor cursor = context.getContentResolver().query(uri, null,
null, null, null);
    try {
        if (cursor != null && cursor.moveToFirst()) {
            displayName =
cursor.getString(cursor.getColumnIndex(DocumentsContract.Document.COLUMN_DISPLAY_NAME));
        }
    } catch (Throwable ex) {
        Log.e(TAG, "failed to get column", ex);
    } finally {
        if (cursor != null)
            cursor.close();
    }

    int fd = -1;
    try {
        ParcelFileDescriptor parcelFileDescriptor =
context.getContentResolver().openFileDescriptor(uri, "r");
        fd = parcelFileDescriptor.detachFd();
    } catch (Throwable e) {
        Log.e(TAG, "obtaining ParcelFileDescriptor for " + uri, e);
    }

    return "file:/proc/self/" + fd + "/" + displayName;

}
________________________________________________________________


Thanks,
Alex Cohn