[FFmpeg-devel] [PATCH] added support for hardware assist H264 video encoding for the Raspberry Pi
Amancio Hasty
ahasty at gmail.com
Sat Jun 25 18:10:56 CEST 2016
On June 22, 2016 at 12:12:02 PM, Amancio Hasty (ahasty at gmail.com) wrote:
On Jun 22, 2016, at 10:20 AM, Aman Gupta <ffmpeg at tmm1.net> wrote:
This patch should fix the assertion failure you see:
https://github.com/FFmpeg/FFmpeg/commit/1087f0dc172a9adf779e41bf2dc82639fb04ebd4
Aman
On Sat, Jun 18, 2016 at 8:43 AM Amancio Hasty <ahasty at gmail.com> wrote:
On Jun 16, 2016, at 11:16 AM, Aman Gupta <ffmpeg at tmm1.net> wrote:
The patchset that was merged into libav is now available in ffmpeg as
well:
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/omx.c
You can compile ffmpeg from the master branch
with --enable-omx --enable-omx-rpi
Aman
On Thu, Jun 16, 2016 at 2:16 AM, Amancio Hasty <ahasty at gmail.com> wrote:
On May 9, 2016, at 7:55 AM, Amancio Hasty <ahasty at gmail.com> wrote:
Hi,
So what is the next step?
If you want testers I suggest posting to ccrisan’s motionpie mailing
list.
Cheers
Amancio
On Mar 31, 2016, at 7:27 PM, Amancio Hasty <ahasty at gmail.com> wrote:
I am not a lawyer…
I updated the patch. vc264.c now has a the copyright notice embedded
in
a volatile global so if a binary is compiled against vc264.o , the
copyright notice
can be displayed by:
strings ffmpeg | grep -i copyright
LICENSE.md has been updated to include Broadcom’s copyright notice.
A distribution of a binary that includes vc264.o should include
LICENSE.md and if
that is missing, the copyright notice can be displayed via the shell
command ‘strings’ .
Amancio
<c-0001-added-support-for-hardware-assist-H264-video-encodin.patch>
On Mar 22, 2016, at 12:12 PM, Lou Logan <lou at lrcd.com> wrote:
On Mon, 21 Mar 2016 20:07:01 -0700, Amancio Hasty wrote:
>From 874a72eec2a78f4935fea091003e534b5f8d5413 Mon Sep 17 00:00:00
2001
From: Amancio Hasty <ahasty at gmail.com>
Date: Mon, 21 Mar 2016 18:56:05 -0700
Subject: [PATCH] added support for hardware assist H264 video
encoding for
the Raspberry Pi
---
configure | 12 ++
libavcodec/Makefile | 1 +
libavcodec/allcodecs.c | 2 +
libavcodec/vc264.c | 387
+++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 402 insertions(+)
create mode 100644 libavcodec/vc264.c
[...]
diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c
index 2a25d66..3c7bd9b 100644
--- a/libavcodec/allcodecs.c
+++ b/libavcodec/allcodecs.c
@@ -74,6 +74,7 @@ void avcodec_register_all(void)
initialized = 1;
Nit: Whitespace on the line above should be removed.
[...]
--- /dev/null
+++ b/libavcodec/vc264.c
@@ -0,0 +1,387 @@
+/* H.264 hardware assist video encoding code taken from
+ * raspberry's os :
+ * /opt/vc/src/hello_pi/hello_encode/encode.c
+ */
+
+/*
+Copyright (c) 2012, Broadcom Europe Ltd
+Copyright (c) 2012, Kalle Vahlman <zuh at iki>
+ Tuomas Kulve <tuomas at kulve.fi>
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
are met:
+* Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ * Redistributions in binary form must reproduce the above
copyright
+ notice, this list of conditions and the following disclaimer
in the
+ documentation and/or other materials provided with the
distribution.
+ * Neither the name of the copyright holder nor the
+ names of its contributors may be used to endorse or promote
products
+ derived from this software without specific prior written
permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
I wonder if any of the above legalese is compatible. Granted, I see a
similar paragraph in "libavformat/aadec.c".
+ * ffmpeg driver for hardware assist video H.264 encoding using
Broadcom's GPU
+ * Copyright (C) 2016 Amancio Hasty ahasty at gmail.com
+ *
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later
version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
02110-1301 USA
+ *
+ */
+
+
+/**
+ * @ file vc264.c
+ * Broadcom bm2865's Visual Core hardware assist h264 using
+ openMax interface to the GPU.
+
+*/
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#define OMX_SKIP64BIT
+#include "bcm_host.h"
+#include "ilclient.h"
+#include "avcodec.h"
+#include "internal.h"
+
+typedef struct VC264Context {
+ OMX_VIDEO_PARAM_PORTFORMATTYPE format;
+ OMX_PARAM_PORTDEFINITIONTYPE def;
+ COMPONENT_T *video_encode;
+ COMPONENT_T *list[5];
+ OMX_BUFFERHEADERTYPE *buf;
+ OMX_BUFFERHEADERTYPE *out;
+ ILCLIENT_T *client;
+ OMX_VIDEO_PARAM_BITRATETYPE bitrateType;
+ int width;
+ int height;
+ int bit_rate;
+} VC264Context;
+
+
+static int vc264_init(AVCodecContext *avctx) {
+
+
+
+ OMX_ERRORTYPE r;
+ int error;
+
+
+
+ VC264Context *vc = avctx->priv_data;
+
+ vc->width = avctx->width;
+ vc->height = avctx->height;
+ vc->bit_rate = avctx->bit_rate;
+ printf("vc264: bit rate %d \n", avctx->bit_rate);
+#if FF_API_CODED_FRAME
+FF_DISABLE_DEPRECATION_WARNINGS
+
+ avctx->coded_frame = av_frame_alloc();
+ avctx->coded_frame->pict_type = AV_PICTURE_TYPE_I;
+FF_ENABLE_DEPRECATION_WARNINGS
+#endif
+
+
+ memset(&vc->list, 0, sizeof(vc->list));
+ bcm_host_init();
+ if ((vc->client = ilclient_init()) == NULL) {
+ return -3;
+ }
+ error = OMX_Init();
+
+ if (error != OMX_ErrorNone) {
+ ilclient_destroy(vc->client);
+ av_log(avctx,AV_LOG_ERROR,"in vc264_init OMX_Init failed ");
+ return -4;
+ }
+
+ // create video_encode
+ r = ilclient_create_component(vc->client, &vc->video_encode,
(char *) "video_encode",
+ ILCLIENT_DISABLE_ALL_PORTS |
+ ILCLIENT_ENABLE_INPUT_BUFFERS |
+ ILCLIENT_ENABLE_OUTPUT_BUFFERS);
Tabs should be converted to spaces. There are many instances of tabs
being used in this patch.
Others will have to provide a more technical review (not to mention
possible additions docs, Changelog, MAINTAINERS, and
libavcodec/version.h).
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Any thoughts on how we may proceed forward?
Thank You,
Amancio
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel <
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
This is what I getting over here with RPI3:
./ffmpeg_g -loglevel debug -f v4l2 -input_format yuv420p -framerate 25
-video_size 640x480 -i /dev/video0 output.h264
ffmpeg version N-80363-g403a53c Copyright (c) 2000-2016 the FFmpeg
developers
built with gcc 4.9.2 (Raspbian 4.9.2-10)
configuration: --enable-gpl --enable-omx --enable-omx-rpi
--enable-nonfree --extra-libs=-ldl --extra-cflags=' -pipe '
libavutil 55. 24.100 / 55. 24.100
libavcodec 57. 46.100 / 57. 46.100
libavformat 57. 38.100 / 57. 38.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 46.101 / 6. 46.101
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging
level) with argument 'debug'.
Reading option '-f' ... matched as option 'f' (force format) with argument
'v4l2'.
Reading option '-input_format' ... matched as AVOption 'input_format' with
argument 'yuv420p'.
Reading option '-framerate' ... matched as AVOption 'framerate' with
argument '25'.
Reading option '-video_size' ... matched as AVOption 'video_size' with
argument '640x480'.
Reading option '-i' ... matched as input file with argument '/dev/video0'.
Reading option 'output.h264' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file /dev/video0.
Applying option f (force format) with argument v4l2.
Successfully parsed a group of options.
Opening an input file: /dev/video0.
[video4linux2,v4l2 @ 0x2189260] fd:3 capabilities:85200005
[video4linux2,v4l2 @ 0x2189260] Current input_channel: 0, input_name:
Camera 0, input_std: 0
[video4linux2,v4l2 @ 0x2189260] Setting time per frame to 1/25
[video4linux2,v4l2 @ 0x2189260] All info found
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: -140423796.316413, bitrate: 92160 kb/s
Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (I420 /
0x30323449), yuv420p, 640x480, 0/1, 92160 kb/s, 25 fps, 25 tbr, 1000k tbn,
1000k tbc
Successfully opened the file.
Parsing a group of options: output file output.h264.
Successfully parsed a group of options.
Opening an output file: output.h264.
[file @ 0x218c930] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 4 logical cores
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'video_size' to value
'640x480'
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'time_base' to value
'1/1000000'
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'pixel_aspect' to
value '0/1'
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'sws_param' to value
'flags=2'
[graph 0 input from stream 0:0 @ 0x2195270] Setting 'frame_rate' to value
'25/1'
[graph 0 input from stream 0:0 @ 0x2195270] w:640 h:480 pixfmt:yuv420p
tb:1/1000000 fr:25/1 sar:0/1 sws_param:flags=2
[format @ 0x2195530] compat: called with args=[yuv420p]
[format @ 0x2195530] Setting 'pix_fmts' to value 'yuv420p'
[AVFilterGraph @ 0x2194b40] query_formats: 4 queried, 3 merged, 0 already
done, 0 delayed
[h264_omx @ 0x218c2c0] Using OMX.broadcom.video_encode
[h264_omx @ 0x218c2c0] OMX state changed to 2
[h264_omx @ 0x218c2c0] OMX state changed to 3
[h264_omx @ 0x218c2c0] OMX port 201 settings changed
[h264 @ 0x218b010] Using AVStream.codec to pass codec parameters to muxers
is deprecated, use AVStream.codecpar instead.
Output #0, h264, to 'output.h264':
Metadata:
encoder : Lavf57.38.100
Stream #0:0, 0, 1/25: Video: h264 (h264_omx), 1 reference frame,
yuv420p, 640x480, 0/1, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc57.46.100 h264_omx
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per
stream)
[rawvideo @ 0x218ac30] PACKET SIZE: 460800, STRIDE: 960
Clipping frame in rate conversion by 0.000008
Assertion ret <= 0 failed at libavcodec/utils.c:1956
Aborted
How did you test h264 encoding?
Thanks!
Amancio
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Hello Aman,
Thanks for the patch. I noticed that in libav/utils.c:
ret = avctx->codec->encode2(avctx, avpkt, frame, got_packet_ptr);
1325
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1325>
if (!ret) {
1326
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1326>
if (*got_packet_ptr) {
1327
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1327>
if (!(avctx->codec->capabilities & AV_CODEC_CAP_DELAY)) {
1328
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1328>
if (avpkt->pts == AV_NOPTS_VALUE)
1329
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1329>
avpkt->pts = frame->pts;
1330
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1330>
if (!avpkt->duration)
1331
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1331>
avpkt->duration = ff_samples_to_time_base(avctx,
1332
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1332>
frame->nb_samples);
1333
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1333>
}
1334
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1334>
avpkt->dts = avpkt->pts;
1335
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1335>
} else {
1336
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1336>
avpkt->size = 0;
1337
<https://git.libav.org/?p=libav.git;a=blob;f=libavcodec/utils.c;h=bc1beee4626d339717d328c605b60bd564580447;hb=HEAD#l1337>
}
Which is different than ffmpeg’s libavcodec/utils.c ….
Last but not least I get a bunch or warnings in ffmpeg when I run it
with your patch.
./ffmpeg_g -f v4l2 -input_format yuv420p -framerate 25 -video_size 640x480
-i /dev/video0 a.h264
DTS 140462612887087, next:1760000 st:0 invalid dropping PTS
140462612887087, next:1760000 invalid dropping st:0
….
Thanks again!
Amancio
With respect to your patch this was posted :
https://git.libav.org/?p=libav.git;a=commit;h=0c9c4004ed57de210b4d83c7b39bbfb00b86b9af
It just deletes setting “ret”. I tried it over here and it seems to work.
Thanks!
Amancio
More information about the ffmpeg-devel
mailing list