[FFmpeg-devel] Add Mediacodec audio decoders support

[Cosmin Stejerean]

> On Jul 16, 2024, at 6:04 PM, Zhao Zhili <quinkblack at foxmail.com> wrote:
> 
> 
> 
>> On Jul 16, 2024, at 23:48, Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel at ffmpeg.org> wrote:
>> 
>> 
>> 
>>> On Jul 16, 2024, at 4:58 PM, Zhao Zhili <quinkblack at foxmail.com> wrote:
>>> 
>>> 
>>> 
>>>> On Jul 16, 2024, at 21:20, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>>>> 
>>>> On Wed, Jul 10, 2024 at 6:31 PM Matthieu Bouron
>>>> <matthieu.bouron at gmail.com> wrote:
>>>>> 
>>>>> On Wed, Jul 10, 2024 at 4:04 PM Zhao Zhili <quinkblack at foxmail.com> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Jun 12, 2024, at 21:42, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> This patchset adds Mediacodec audio decoders support. Currently, only AAC, AMR,
>>>>>>> MP3, FLAC, VORBIS and OPUS are supported.
>>>>>>> 
>>>>>>> This is mainly useful to avoid shipping Android builds of FFmpeg that are
>>>>>>> subjects to licensing/patents (due to AAC and AMR).
>>>>>> 
>>>>>> I’m not keen on put OS audio decoder/encoder wrapper into FFmpeg. They don’t bring
>>>>>> new features, they don’t improve performance. I know these type of wrapper exist in current
>>>>>> project, but I’m not sure if it’s a good idea to add more.
>>>>> 
>>>>> I agree that on the technical side it doesn't bring new features nor
>>>>> performance improvements. It's all about avoiding licensing/patents
>>>>> issues with AAC and AMR here.
>>>>> In this specific case we already have the wrapper infrastructure, the
>>>>> audio part only needs small adjustments to work.
>>>>> Moreover, if that helps, I can reduce the scope of the patch to AAC
>>>>> and AMR only and get rid of mp3/flac/vorbis/opus support. What do you
>>>>> think ?
>>>> 
>>>> Ping.
>>>> 
>>>> IMHO, this benefits users wanting to ship an Android app that relies
>>>> on FFmpeg upstream in countries that are subject to AAC/AMR licensing.
>>>> While I agree that it's not great from a purely technical pov since we
>>>> already have better native decoders, it will allow the use of FFmpeg
>>>> in such situation without the need to use or create another FFmpeg
>>>> fork dedicated to Android. Plus, as I said above, we already have the
>>>> wrapper and the additional code to make it work for audio is
>>>> relatively small and scoped. Restricting the wrapper to AAC/AMR seems
>>>> like a good compromise to me.
>>> 
>>> Sounds reasonable, but I’d like to get more opinions. cc TC.
>> 
>> To add another data point, the platform decoders might also be more secure due to sandboxing. I believe as of Android Q the software decoders provided by MediaCodec have been moved to run within a constrained sandbox.
> 
> I don’t think what Android says “secure” apply to use. The sandbox means not to
> break the OS and leak information from one APP to another APP, because mediacodec
> service is shared by all APP. An APP with FFmpeg inside and use software decoder/encoder
> don’t have such issues since the APP don’t share decoders/encoders with other APP.
> 

It's still an issue if a vulnerability in a software decoder gives an attacker access to the same app's data, even if per app sandboxing keeps a compromised application from accessing data from other apps. 

For example consider a messaging app. If you receive a message containing media that exploits a vulnerability in a software decoder and that gives the attacker access to all of your contacts and messages within the app, that's definitely a security issue. 
The app itself might also have other permissions to system wide resources, etc. 


- Cosmin