[FFmpeg-devel] [PATCH] added support for hardware assist H264 video encoding for the Raspberry Pi

Aman Gupta ffmpeg at tmm1.net
Wed Jun 22 19:20:04 CEST 2016


This patch should fix the assertion failure you see:
https://github.com/FFmpeg/FFmpeg/commit/1087f0dc172a9adf779e41bf2dc82639fb04ebd4

Aman
On Sat, Jun 18, 2016 at 8:43 AM Amancio Hasty <ahasty at gmail.com> wrote:

>
> > On Jun 16, 2016, at 11:16 AM, Aman Gupta <ffmpeg at tmm1.net> wrote:
> >
> > The patchset that was merged into libav is now available in ffmpeg as
> well:
> > https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/omx.c
> >
> > You can compile ffmpeg from the master branch
> > with --enable-omx --enable-omx-rpi
> >
> > Aman
> >
> > On Thu, Jun 16, 2016 at 2:16 AM, Amancio Hasty <ahasty at gmail.com> wrote:
> >
> >>
> >>> On May 9, 2016, at 7:55 AM, Amancio Hasty <ahasty at gmail.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> So what is the next step?
> >>>
> >>> If you want testers I suggest posting to ccrisan’s motionpie mailing
> >> list.
> >>> Cheers
> >>> Amancio
> >>>
> >>>
> >>>> On Mar 31, 2016, at 7:27 PM, Amancio Hasty <ahasty at gmail.com> wrote:
> >>>>
> >>>> I am not a lawyer…
> >>>>
> >>>>
> >>>> I updated the patch.  vc264.c now has a the copyright notice embedded
> in
> >>>> a volatile global so if a binary is compiled against vc264.o , the
> >> copyright notice
> >>>> can be displayed by:
> >>>> strings ffmpeg | grep -i copyright
> >>>>
> >>>> LICENSE.md has been updated to include Broadcom’s copyright notice.
> >>>>
> >>>> A distribution of a  binary that includes vc264.o should include
> >> LICENSE.md and if
> >>>> that is missing,  the copyright notice can be displayed via the shell
> >>>> command ‘strings’ .
> >>>>
> >>>> Amancio
> >>>> <c-0001-added-support-for-hardware-assist-H264-video-encodin.patch>
> >>>>> On Mar 22, 2016, at 12:12 PM, Lou Logan <lou at lrcd.com> wrote:
> >>>>>
> >>>>> On Mon, 21 Mar 2016 20:07:01 -0700, Amancio Hasty wrote:
> >>>>>
> >>>>>> From 874a72eec2a78f4935fea091003e534b5f8d5413 Mon Sep 17 00:00:00
> 2001
> >>>>>> From: Amancio Hasty <ahasty at gmail.com>
> >>>>>> Date: Mon, 21 Mar 2016 18:56:05 -0700
> >>>>>> Subject: [PATCH] added support for hardware assist H264  video
> >> encoding for
> >>>>>> the Raspberry Pi
> >>>>>>
> >>>>>> ---
> >>>>>> configure              |  12 ++
> >>>>>> libavcodec/Makefile    |   1 +
> >>>>>> libavcodec/allcodecs.c |   2 +
> >>>>>> libavcodec/vc264.c     | 387
> >> +++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>>> 4 files changed, 402 insertions(+)
> >>>>>> create mode 100644 libavcodec/vc264.c
> >>>>>>
> >>>>> [...]
> >>>>>> diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
> >>>>>> index 2a25d66..3c7bd9b 100644
> >>>>>> --- a/libavcodec/allcodecs.c
> >>>>>> +++ b/libavcodec/allcodecs.c
> >>>>>> @@ -74,6 +74,7 @@ void avcodec_register_all(void)
> >>>>>>  initialized = 1;
> >>>>>>
> >>>>>
> >>>>> Nit: Whitespace on the line above should be removed.
> >>>>>
> >>>>> [...]
> >>>>>> --- /dev/null
> >>>>>> +++ b/libavcodec/vc264.c
> >>>>>> @@ -0,0 +1,387 @@
> >>>>>> +/*  H.264 hardware assist video encoding code taken from
> >>>>>> + * raspberry's os :
> >>>>>> + *   /opt/vc/src/hello_pi/hello_encode/encode.c
> >>>>>> + */
> >>>>>> +
> >>>>>> +/*
> >>>>>> +Copyright (c) 2012, Broadcom Europe Ltd
> >>>>>> +Copyright (c) 2012, Kalle Vahlman <zuh at iki>
> >>>>>> +                    Tuomas Kulve <tuomas at kulve.fi>
> >>>>>> +All rights reserved.
> >>>>>> +
> >>>>>> +Redistribution and use in source and binary forms, with or without
> >>>>>> +modification, are permitted provided that the following conditions
> >> are met:
> >>>>>> +* Redistributions of source code must retain the above copyright
> >>>>>> +      notice, this list of conditions and the following disclaimer.
> >>>>>> +      * Redistributions in binary form must reproduce the above
> >> copyright
> >>>>>> +      notice, this list of conditions and the following disclaimer
> >> in the
> >>>>>> +      documentation and/or other materials provided with the
> >> distribution.
> >>>>>> +      * Neither the name of the copyright holder nor the
> >>>>>> +      names of its contributors may be used to endorse or promote
> >> products
> >>>>>> +      derived from this software without specific prior written
> >> permission.
> >>>>>> +
> >>>>>> +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> >> "AS IS" AND
> >>>>>> +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
> >> THE IMPLIED
> >>>>>> +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> >> ARE
> >>>>>> +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
> >> BE LIABLE FOR ANY
> >>>>>> +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> >> DAMAGES
> >>>>>> +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> >> SERVICES;
> >>>>>> +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> >> CAUSED AND
> >>>>>> +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> >> OR TORT
> >>>>>> +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
> >> USE OF THIS
> >>>>>> +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> >>>>>
> >>>>> I wonder if any of the above legalese is compatible. Granted, I see a
> >>>>> similar paragraph in "libavformat/aadec.c".
> >>>>>
> >>>>>> + * ffmpeg driver for hardware assist video H.264 encoding using
> >> Broadcom's GPU
> >>>>>> + * Copyright (C) 2016 Amancio Hasty ahasty at gmail.com
> >>>>>> + *
> >>>>>> + *
> >>>>>> + * This file is part of FFmpeg.
> >>>>>> + *
> >>>>>> + * FFmpeg is free software; you can redistribute it and/or
> >>>>>> + * modify it under the terms of the GNU Lesser General Public
> >>>>>> + * License as published by the Free Software Foundation; either
> >>>>>> + * version 2.1 of the License, or (at your option) any later
> version.
> >>>>>> + *
> >>>>>> + * FFmpeg is distributed in the hope that it will be useful,
> >>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> GNU
> >>>>>> + * Lesser General Public License for more details.
> >>>>>> + *
> >>>>>> + * You should have received a copy of the GNU Lesser General Public
> >>>>>> + * License along with FFmpeg; if not, write to the Free Software
> >>>>>> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> >> 02110-1301 USA
> >>>>>> + *
> >>>>>> + */
> >>>>>> +
> >>>>>> +
> >>>>>> +/**
> >>>>>> + * @ file vc264.c
> >>>>>> + * Broadcom bm2865's Visual Core hardware assist h264 using
> >>>>>> +   openMax interface to the GPU.
> >>>>>> +
> >>>>>> +*/
> >>>>>> +
> >>>>>> +#include <stdio.h>
> >>>>>> +#include <stdlib.h>
> >>>>>> +#include <string.h>
> >>>>>> +#define OMX_SKIP64BIT
> >>>>>> +#include "bcm_host.h"
> >>>>>> +#include "ilclient.h"
> >>>>>> +#include "avcodec.h"
> >>>>>> +#include "internal.h"
> >>>>>> +
> >>>>>> +typedef struct VC264Context {
> >>>>>> +  OMX_VIDEO_PARAM_PORTFORMATTYPE format;
> >>>>>> +  OMX_PARAM_PORTDEFINITIONTYPE def;
> >>>>>> +  COMPONENT_T *video_encode;
> >>>>>> +  COMPONENT_T *list[5];
> >>>>>> +  OMX_BUFFERHEADERTYPE *buf;
> >>>>>> +  OMX_BUFFERHEADERTYPE *out;
> >>>>>> +  ILCLIENT_T *client;
> >>>>>> +  OMX_VIDEO_PARAM_BITRATETYPE bitrateType;
> >>>>>> +  int  width;
> >>>>>> +  int height;
> >>>>>> +  int bit_rate;
> >>>>>> +} VC264Context;
> >>>>>> +
> >>>>>> +
> >>>>>> +static int vc264_init(AVCodecContext *avctx) {
> >>>>>> +
> >>>>>> +
> >>>>>> +
> >>>>>> +   OMX_ERRORTYPE r;
> >>>>>> +   int error;
> >>>>>> +
> >>>>>> +
> >>>>>> +
> >>>>>> +   VC264Context *vc = avctx->priv_data;
> >>>>>> +
> >>>>>> +   vc->width = avctx->width;
> >>>>>> +   vc->height = avctx->height;
> >>>>>> +   vc->bit_rate = avctx->bit_rate;
> >>>>>> +   printf("vc264: bit rate %d \n", avctx->bit_rate);
> >>>>>> +#if FF_API_CODED_FRAME
> >>>>>> +FF_DISABLE_DEPRECATION_WARNINGS
> >>>>>> +
> >>>>>> +   avctx->coded_frame = av_frame_alloc();
> >>>>>> +   avctx->coded_frame->pict_type = AV_PICTURE_TYPE_I;
> >>>>>> +FF_ENABLE_DEPRECATION_WARNINGS
> >>>>>> +#endif
> >>>>>> +
> >>>>>> +
> >>>>>> +   memset(&vc->list, 0, sizeof(vc->list));
> >>>>>> +   bcm_host_init();
> >>>>>> +   if ((vc->client = ilclient_init()) == NULL) {
> >>>>>> +      return -3;
> >>>>>> +   }
> >>>>>> +   error = OMX_Init();
> >>>>>> +
> >>>>>> +   if (error  != OMX_ErrorNone) {
> >>>>>> +      ilclient_destroy(vc->client);
> >>>>>> +      av_log(avctx,AV_LOG_ERROR,"in vc264_init OMX_Init failed ");
> >>>>>> +     return -4;
> >>>>>> +    }
> >>>>>> +
> >>>>>> +   // create video_encode
> >>>>>> +   r = ilclient_create_component(vc->client, &vc->video_encode,
> >> (char *) "video_encode",
> >>>>>> +                           ILCLIENT_DISABLE_ALL_PORTS |
> >>>>>> +                           ILCLIENT_ENABLE_INPUT_BUFFERS |
> >>>>>> +                           ILCLIENT_ENABLE_OUTPUT_BUFFERS);
> >>>>>
> >>>>> Tabs should be converted to spaces. There are many instances of tabs
> >>>>> being used in this patch.
> >>>>>
> >>>>> Others will have to provide a more technical review (not to mention
> >>>>> possible additions docs, Changelog, MAINTAINERS, and
> >>>>> libavcodec/version.h).
> >>>>> _______________________________________________
> >>>>> ffmpeg-devel mailing list
> >>>>> ffmpeg-devel at ffmpeg.org
> >>>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>>>
> >>>
> >>
> >> Any thoughts on how we may proceed forward?
> >>
> >> Thank You,
> >> Amancio
> >>
> >>
> >> _______________________________________________
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel at ffmpeg.org
> >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel <
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
>
>
> This is what I getting over here with RPI3:
>
> ./ffmpeg_g -loglevel debug -f v4l2 -input_format yuv420p -framerate 25
> -video_size 640x480 -i /dev/video0 output.h264
> ffmpeg version N-80363-g403a53c Copyright (c) 2000-2016 the FFmpeg
> developers
>   built with gcc 4.9.2 (Raspbian 4.9.2-10)
>   configuration: --enable-gpl --enable-omx --enable-omx-rpi
> --enable-nonfree --extra-libs=-ldl --extra-cflags=' -pipe '
>   libavutil      55. 24.100 / 55. 24.100
>   libavcodec     57. 46.100 / 57. 46.100
>   libavformat    57. 38.100 / 57. 38.100
>   libavdevice    57.  0.101 / 57.  0.101
>   libavfilter     6. 46.101 /  6. 46.101
>   libswscale      4.  1.100 /  4.  1.100
>   libswresample   2.  1.100 /  2.  1.100
>   libpostproc    54.  0.100 / 54.  0.100
> Splitting the commandline.
> Reading option '-loglevel' ... matched as option 'loglevel' (set logging
> level) with argument 'debug'.
> Reading option '-f' ... matched as option 'f' (force format) with argument
> 'v4l2'.
> Reading option '-input_format' ... matched as AVOption 'input_format' with
> argument 'yuv420p'.
> Reading option '-framerate' ... matched as AVOption 'framerate' with
> argument '25'.
> Reading option '-video_size' ... matched as AVOption 'video_size' with
> argument '640x480'.
> Reading option '-i' ... matched as input file with argument '/dev/video0'.
> Reading option 'output.h264' ... matched as output file.
> Finished splitting the commandline.
> Parsing a group of options: global .
> Applying option loglevel (set logging level) with argument debug.
> Successfully parsed a group of options.
> Parsing a group of options: input file /dev/video0.
> Applying option f (force format) with argument v4l2.
> Successfully parsed a group of options.
> Opening an input file: /dev/video0.
> [video4linux2,v4l2 @ 0x2189260] fd:3 capabilities:85200005
> [video4linux2,v4l2 @ 0x2189260] Current input_channel: 0, input_name:
> Camera 0, input_std: 0
> [video4linux2,v4l2 @ 0x2189260] Setting time per frame to 1/25
> [video4linux2,v4l2 @ 0x2189260] All info found
> Input #0, video4linux2,v4l2, from '/dev/video0':
>   Duration: N/A, start: -140423796.316413, bitrate: 92160 kb/s
>     Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (I420 /
> 0x30323449), yuv420p, 640x480, 0/1, 92160 kb/s, 25 fps, 25 tbr, 1000k tbn,
> 1000k tbc
> Successfully opened the file.
> Parsing a group of options: output file output.h264.
> Successfully parsed a group of options.
> Opening an output file: output.h264.
> [file @ 0x218c930] Setting default whitelist 'file,crypto'
> Successfully opened the file.
> detected 4 logical cores
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'video_size' to value
> '640x480'
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pix_fmt' to value '0'
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'time_base' to value
> '1/1000000'
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'pixel_aspect' to
> value '0/1'
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'sws_param' to value
> 'flags=2'
> [graph 0 input from stream 0:0 @ 0x2195270] Setting 'frame_rate' to value
> '25/1'
> [graph 0 input from stream 0:0 @ 0x2195270] w:640 h:480 pixfmt:yuv420p
> tb:1/1000000 fr:25/1 sar:0/1 sws_param:flags=2
> [format @ 0x2195530] compat: called with args=[yuv420p]
> [format @ 0x2195530] Setting 'pix_fmts' to value 'yuv420p'
> [AVFilterGraph @ 0x2194b40] query_formats: 4 queried, 3 merged, 0 already
> done, 0 delayed
> [h264_omx @ 0x218c2c0] Using OMX.broadcom.video_encode
> [h264_omx @ 0x218c2c0] OMX state changed to 2
> [h264_omx @ 0x218c2c0] OMX state changed to 3
> [h264_omx @ 0x218c2c0] OMX port 201 settings changed
> [h264 @ 0x218b010] Using AVStream.codec to pass codec parameters to muxers
> is deprecated, use AVStream.codecpar instead.
> Output #0, h264, to 'output.h264':
>   Metadata:
>     encoder         : Lavf57.38.100
>     Stream #0:0, 0, 1/25: Video: h264 (h264_omx), 1 reference frame,
> yuv420p, 640x480, 0/1, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
>     Metadata:
>       encoder         : Lavc57.46.100 h264_omx
> Stream mapping:
>   Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_omx))
> Press [q] to stop, [?] for help
> cur_dts is invalid (this is harmless if it occurs once at the start per
> stream)
> [rawvideo @ 0x218ac30] PACKET SIZE: 460800, STRIDE: 960
> Clipping frame in rate conversion by 0.000008
> Assertion ret <= 0 failed at libavcodec/utils.c:1956
> Aborted
>
> How did you test h264 encoding?
>
> Thanks!
> Amancio
>
>
>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>


More information about the ffmpeg-devel mailing list