[FFmpeg-devel] [PATCH v2 1/1] avformat: Add IPFS protocol support.
Mark Gaiser
markg85 at gmail.com
Wed Feb 2 16:23:34 EET 2022
On Wed, Feb 2, 2022 at 2:29 PM Michael Niedermayer <michael at niedermayer.cc>
wrote:
> On Tue, Feb 01, 2022 at 10:58:30PM +0100, Mark Gaiser wrote:
> > This patch adds support for:
> > - ffplay ipfs://<cid>
> > - ffplay ipns://<cid>
> >
> > IPFS data can be played from so called "ipfs gateways".
> > A gateway is essentially a webserver that gives access to the
> > distributed IPFS network.
> >
> > This protocol support (ipfs and ipns) therefore translates
> > ipfs:// and ipns:// to a http:// url. This resulting url is
> > then handled by the http protocol. It could also be https
> > depending on the gateway provided.
> >
> > To use this protocol, a gateway must be provided.
> > If you do nothing it will try to find it in your
> > $HOME/.ipfs/gateway file. The ways to set it manually are:
> > 1. Define a -gateway <url> to the gateway.
> > 2. Define $IPFS_GATEWAY with the full http link to the gateway.
> > 3. Define $IPFS_PATH and point it to the IPFS data path.
> > 4. Have IPFS running in your local user folder (under $HOME/.ipfs).
> >
> > Signed-off-by: Mark Gaiser <markg85 at gmail.com>
> > ---
> > configure | 2 +
> > doc/protocols.texi | 30 +++++
> > libavformat/Makefile | 2 +
> > libavformat/ipfsgateway.c | 267 ++++++++++++++++++++++++++++++++++++++
> > libavformat/protocols.c | 2 +
> > 5 files changed, 303 insertions(+)
> > create mode 100644 libavformat/ipfsgateway.c
> >
> > diff --git a/configure b/configure
> > index 5b19a35f59..6ff09e7974 100755
> > --- a/configure
> > +++ b/configure
> > @@ -3585,6 +3585,8 @@ udp_protocol_select="network"
> > udplite_protocol_select="network"
> > unix_protocol_deps="sys_un_h"
> > unix_protocol_select="network"
> > +ipfs_protocol_select="https_protocol"
> > +ipns_protocol_select="https_protocol"
> >
> > # external library protocols
> > libamqp_protocol_deps="librabbitmq"
> > diff --git a/doc/protocols.texi b/doc/protocols.texi
> > index d207df0b52..7c9c0a4808 100644
> > --- a/doc/protocols.texi
> > +++ b/doc/protocols.texi
> > @@ -2025,5 +2025,35 @@ decoding errors.
> >
> > @end table
> >
> > + at section ipfs
> > +
> > +InterPlanetary File System (IPFS) protocol support. One can access
> files stored
> > +on the IPFS network through so called gateways. Those are http(s)
> endpoints.
> > +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to
> be send
> > +to such a gateway. Users can (and should) host their own node which
> means this
> > +protocol will use your local machine gateway to access files on the
> IPFS network.
> > +
> > +If a user doesn't have a node of their own then the public gateway
> dweb.link is
> > +used by default.
> > +
> > +You can use this protocol in 2 ways. Using IPFS:
> > + at example
> > +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > + at end example
> > +
> > +Or the IPNS protocol (IPNS is mutable IPFS):
> > + at example
> > +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> > + at end example
> > +
> > +You can also change the gateway to be used:
> > +
> > + at table @option
> > +
> > + at item gateway
> > +Defines the gateway to use. When nothing is provided the protocol will
> first try
> > +your local gateway. If that fails dweb.link will be used.
> > +
> > + at end table
> >
> > @c man end PROTOCOLS
> > diff --git a/libavformat/Makefile b/libavformat/Makefile
> > index 3dc6a479cc..4edce8420f 100644
> > --- a/libavformat/Makefile
> > +++ b/libavformat/Makefile
> > @@ -656,6 +656,8 @@ OBJS-$(CONFIG_SRTP_PROTOCOL) +=
> srtpproto.o srtp.o
> > OBJS-$(CONFIG_SUBFILE_PROTOCOL) += subfile.o
> > OBJS-$(CONFIG_TEE_PROTOCOL) += teeproto.o tee_common.o
> > OBJS-$(CONFIG_TCP_PROTOCOL) += tcp.o
> > +OBJS-$(CONFIG_IPFS_PROTOCOL) += ipfsgateway.o
> > +OBJS-$(CONFIG_IPNS_PROTOCOL) += ipfsgateway.o
> > TLS-OBJS-$(CONFIG_GNUTLS) += tls_gnutls.o
> > TLS-OBJS-$(CONFIG_LIBTLS) += tls_libtls.o
> > TLS-OBJS-$(CONFIG_MBEDTLS) += tls_mbedtls.o
> > diff --git a/libavformat/ipfsgateway.c b/libavformat/ipfsgateway.c
> > new file mode 100644
> > index 0000000000..2b52fc9392
> > --- /dev/null
> > +++ b/libavformat/ipfsgateway.c
> > @@ -0,0 +1,267 @@
> > +/*
> > + * IPFS and IPNS protocol support through IPFS Gateway.
> > + * Copyright (c) 2022 Mark Gaiser
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA
> > + */
> > +
> > +#include "libavutil/avassert.h"
> > +#include "libavutil/avstring.h"
> > +#include "libavutil/internal.h"
> > +#include "libavutil/opt.h"
> > +#include "libavutil/tree.h"
> > +#include "avformat.h"
> > +#include <fcntl.h>
> > +#if HAVE_IO_H
> > +#include <io.h>
> > +#endif
> > +#if HAVE_UNISTD_H
> > +#include <unistd.h>
> > +#endif
> > +#include <sys/stat.h>
> > +#include <stdlib.h>
> > +#include "os_support.h"
> > +#include "url.h"
> > +
>
> > +typedef struct Context {
>
> i also agree that the name is a bit too unspecific
>
Done. changed to IPFSGatewayContext
>
>
> > + AVClass *class;
> > + URLContext *inner;
> > + char *gateway;
> > +} Context;
> > +
> > +// A best-effort way to find the IPFS gateway.
> > +// Only the most appropiate gateway is set. It's not actually requested
> (http call) to prevent
> > +// a potential slowdown in startup. A potential timeout is handled by
> the HTTP protocol.
> > +//
> > +// When done and the return value is 1, a potential IPFS Gateway is set
> in the gateway
> > +// variable in the inner URLContext.
>
> > +static int ff_populate_ipfs_gateway(URLContext *h)
>
> ff_ prefix seems not needed
>
Done. Removed the prefix from both functions.
For the record, I thought it was mandatory to prefix internal functions
with "ff_"?
>
>
> [...]
>
> > + FILE *gateway_file;
> > + char gateway_file_data[1000];
>
> [...]
>
> > + if (!gateway_file) {
> > + av_log(h, AV_LOG_ERROR, "Unable to open the IPFS gateway file
> (full uri: %s).\n", ipfs_gateway_file);
>
> > + return -1;
>
> more specific error codes would be better and more consistent to existing
> code
>
Added above the function as:
// Return codes can be:
// 1 : A potential gateway is found and set in c->gateway
// -1: The IPFS data folder could not be found
// -2: The gateway file could not be found
// -3: The gateway file is found but empty
// -9: Unhandled error
>
>
> > + }
> > +
>
> > + fscanf(gateway_file, "%[^\n]", gateway_file_data);
>
> how large is the array ?
> how large is the data written into the array ?
> what is the relation between the 2 ?
>
Yeah... I'm going to need more information on that one.
What do you mean?
Is this wrong?
>
> [...]
> > + int ret;
> > +
> > + ret = ffurl_read(c->inner, buf, size);
> > +
> > + return ret;
>
> if thats all that is needed then
>
> return ffurl_read(c->inner, buf, size);
>
> is simpler
>
Ahh nice, fixed for ipfs_read, ipfs_seek and ipfs_close!
>
>
> thx
>
> [...]
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> If you drop bombs on a foreign country and kill a hundred thousand
> innocent people, expect your government to call the consequence
> "unprovoked inhuman terrorist attacks" and use it to justify dropping
> more bombs and killing more people. The technology changed, the idea is
> old.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
More information about the ffmpeg-devel
mailing list