[FFmpeg-devel] [PATCH] lavf/dashenc: Add support for per-stream container type selection.
Jeyapal, Karthick
kjeyapal at akamai.com
Mon Nov 12 14:12:19 EET 2018
On 11/12/18 5:20 PM, Andrey Semashev wrote:
> On 11/12/18 8:20 AM, Jeyapal, Karthick wrote:
>>
>> On 11/8/18 10:27 PM, Andrey Semashev wrote:
>>> This commit restores the ability to create DASH streams with codecs
>>> that require different containers that was lost after commit
>>> 2efdbf7367989cf9d296c25fa3d2aff8d6e25fdd. It extends the dash_segment_type
>>> option syntax to allow to specify segment container types for individual
>>> streams, in addition to the default container type that is applied to
>>> all streams. The extended syntax is backward compatible with the previous
>>> syntax.
>> Thanks for sending the patch. I understand your requirement completely.
>> But I feel that this option for mapping streams with container format is little confusing. Also, the relevant code is relatively big, and thus difficult to maintain in future.
>> I have a middle ground suggestion. If your goal is to achieve the earlier behavior broken commits, then I propose the following.
>> Option "dash_segment_type" could take one more option "auto" (instead of mp4 or webm).
>> When "auto" is chosen, the muxer could choose webm format for VP8, VP9, vorbis, opus streams and mp4 format for all other streams.
>> In this method the previous behavior of dashenc muxer could be restored with little addition to the overall code. Also it's usage will be simpler and easier to understand.
>
> This solution might be ok for just restoring the previous capability, but I think the ability for selecting the container format by the user is still more useful. For example, Opus can be muxed in both mp4 (although, with experimental flag) and webm, and it may make sense to some users to select mp4. (In my case, though, I wanted webm, hence the patch.)
In that case they could select "dash_segment_type" as "mp4", in which case all streams including opus will be muxed in mp4 format. I don't see a use-case where somebody wants opus in mp4 and would want other streams in webm format.
>
> Besides the parser, it doesn't add much code, and if I can improve the patch, please let me know how. If you absolutely don't want this functionality, that's ok, I'll keep this patch for my local builds and submit an "auto" option patch instead.
I am not absolutely against this patch. But I am just trying to find if there is a use-case for such an advanced option. If there is a practical requirement for such a use-case, then I agree that we should review and push this patch for sure.
But if the requirement is just theoretical at this point, then I would prefer the "auto" option patch. Maybe we could revisit this advanced options patch when a real requirement comes up.
>
>>> ---
>>> doc/muxers.texi | 8 ++-
>>> libavformat/dashenc.c | 161 +++++++++++++++++++++++++++++++++++-------
>>> 2 files changed, 140 insertions(+), 29 deletions(-)
>>>
>>> diff --git a/doc/muxers.texi b/doc/muxers.texi
>>> index 62f4091e31..4418b38c76 100644
>>> --- a/doc/muxers.texi
>>> +++ b/doc/muxers.texi
>>> @@ -289,8 +289,12 @@ Set container format (mp4/webm) options using a @code{:} separated list of
>>> key=value parameters. Values containing @code{:} special characters must be
>>> escaped.
>>> - at item dash_segment_type @var{dash_segment_type}
>>> -Possible values:
>>> + at item -dash_segment_type @var{dash_segment_type}
>>> +Sets the container type for dash segment files. Syntax is "<type> <type>:a,b,c <type>:d,e" where <type> is
>>> +the container type and a, b, c, d and e are the indices of the mapped streams. When no indices are specified,
>>> +the container type is set for all streams.
>>> +
>>> +Possible container type values:
>>> @item mp4
>>> If this flag is set, the dash segment files will be in in ISOBMFF format. This is the default format.
>>> diff --git a/libavformat/dashenc.c b/libavformat/dashenc.c
>>> index f8b3d106d5..626dc76413 100644
>>> --- a/libavformat/dashenc.c
>>> +++ b/libavformat/dashenc.c
>>> @@ -84,6 +84,8 @@ typedef struct OutputStream {
>>> int64_t first_pts, start_pts, max_pts;
>>> int64_t last_dts, last_pts;
>>> int bit_rate;
>>> + SegmentType segment_type;
>>> + const char *format_name;
>>> char codec_str[100];
>>> int written_len;
>>> @@ -131,8 +133,7 @@ typedef struct DASHContext {
>>> int64_t timeout;
>>> int index_correction;
>>> char *format_options_str;
>>> - SegmentType segment_type;
>>> - const char *format_name;
>>> + const char *segment_types_str;
>>> } DASHContext;
>>> static struct codec_string {
>>> @@ -188,14 +189,6 @@ static void dashenc_io_close(AVFormatContext *s, AVIOContext **pb, char *filenam
>>> }
>>> }
>>> -static const char *get_format_str(SegmentType segment_type) {
>>> - int i;
>>> - for (i = 0; i < SEGMENT_TYPE_NB; i++)
>>> - if (formats[i].segment_type == segment_type)
>>> - return formats[i].str;
>>> - return NULL;
>>> -}
>>> -
>>> static int check_file_extension(const char *filename, const char *extension) {
>>> char *dot;
>>> if (!filename || !extension)
>>> @@ -375,6 +368,8 @@ static void dash_free(AVFormatContext *s)
>>> c->nb_as = 0;
>>> }
>>> + av_freep(&c->segment_types_str);
>>> +
>>> if (!c->streams)
>>> return;
>>> for (i = 0; i < s->nb_streams; i++) {
>>> @@ -621,13 +616,13 @@ static int write_adaptation_set(AVFormatContext *s, AVIOContext *out, int as_ind
>>> if (as->media_type == AVMEDIA_TYPE_VIDEO) {
>>> AVStream *st = s->streams[i];
>>> avio_printf(out, "\t\t\t<Representation id=\"%d\" mimeType=\"video/%s\" codecs=\"%s\"%s width=\"%d\" height=\"%d\"",
>>> - i, c->format_name, os->codec_str, bandwidth_str, s->streams[i]->codecpar->width, s->streams[i]->codecpar->height);
>>> + i, os->format_name, os->codec_str, bandwidth_str, s->streams[i]->codecpar->width, s->streams[i]->codecpar->height);
>>> if (st->avg_frame_rate.num)
>>> avio_printf(out, " frameRate=\"%d/%d\"", st->avg_frame_rate.num, st->avg_frame_rate.den);
>>> avio_printf(out, ">\n");
>>> } else {
>>> avio_printf(out, "\t\t\t<Representation id=\"%d\" mimeType=\"audio/%s\" codecs=\"%s\"%s audioSamplingRate=\"%d\">\n",
>>> - i, c->format_name, os->codec_str, bandwidth_str, s->streams[i]->codecpar->sample_rate);
>>> + i, os->format_name, os->codec_str, bandwidth_str, s->streams[i]->codecpar->sample_rate);
>>> avio_printf(out, "\t\t\t\t<AudioChannelConfiguration schemeIdUri=\"urn:mpeg:dash:23003:3:audio_channel_configuration:2011\" value=\"%d\" />\n",
>>> s->streams[i]->codecpar->channels);
>>> }
>>> @@ -773,6 +768,120 @@ end:
>>> return 0;
>>> }
>>> +static inline void set_all_segment_types(AVFormatContext *s, int format_idx)
>>> +{
>>> + DASHContext *c = s->priv_data;
>>> + for (int i = 0; i < s->nb_streams; ++i) {
>>> + OutputStream *os = &c->streams[i];
>>> + os->segment_type = formats[format_idx].segment_type;
>>> + os->format_name = formats[format_idx].str;
>>> + }
>>> +}
>>> +
>>> +static int parse_segment_types(AVFormatContext *s)
>>> +{
>>> + DASHContext *c = s->priv_data;
>>> + const char *p = c->segment_types_str;
>>> + enum { type_expected, type_parsed, parsing_streams } state;
>>> + int i, format_idx;
>>> +
>>> + // Set the default container type in case if some streams are not mentioned in the string
>>> + set_all_segment_types(s, 0);
>>> +
>>> + if (!p)
>>> + return 0;
>>> +
>>> + // Parse per-stream container types: mp4 webm:0,1,2 and so on
>>> + state = type_expected;
>>> + format_idx = 0;
>>> + while (*p) {
>>> + if (*p == ' ') {
>>> + if (state == type_parsed)
>>> + set_all_segment_types(s, format_idx);
>>> +
>>> + state = type_expected;
>>> + ++p;
>>> + continue;
>>> + }
>>> +
>>> + if (state == type_expected) {
>>> + for (i = 0; i < SEGMENT_TYPE_NB; i++) {
>>> + if (av_strstart(p, formats[i].str, &p)) {
>>> + state = type_parsed;
>>> + format_idx = i;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + if (state != type_parsed) {
>>> + av_log(s, AV_LOG_ERROR, "DASH segment type not recognized at position %d of \"%s\"\n", (int)(p - c->segment_types_str + 1), c->segment_types_str);
>>> + return AVERROR_MUXER_NOT_FOUND;
>>> + }
>>> +
>>> + continue;
>>> + }
>>> +
>>> + if (state == type_parsed) {
>>> + if (*p != ':') {
>>> + av_log(s, AV_LOG_ERROR, "Unexpected character at position %d of \"%s\", a colon is expected\n", (int)(p - c->segment_types_str + 1), c->segment_types_str);
>>> + return AVERROR(EINVAL);
>>> + }
>>> +
>>> + state = parsing_streams;
>>> + ++p;
>>> + continue;
>>> + }
>>> +
>>> + if (state == parsing_streams) {
>>> + while (*p && *p != ' ') {
>>> + if (*p == ',') {
>>> + ++p;
>>> + } else if (*p == 'a' || *p == 'v') {
>>> + // Select all streams of the given type
>>> + enum AVMediaType type = (*p == 'v') ? AVMEDIA_TYPE_VIDEO : AVMEDIA_TYPE_AUDIO;
>>> +
>>> + for (i = 0; i < s->nb_streams; ++i) {
>>> + if (s->streams[i]->codecpar->codec_type == type) {
>>> + OutputStream *os = &c->streams[i];
>>> + os->segment_type = formats[format_idx].segment_type;
>>> + os->format_name = formats[format_idx].str;
>>> + }
>>> + }
>>> +
>>> + ++p;
>>> + } else {
>>> + // Select a stream by index
>>> + OutputStream *os;
>>> + const char* end_p = p;
>>> + i = strtol(p, (char**)&end_p, 10);
>>> + if (p == end_p) {
>>> + av_log(s, AV_LOG_ERROR, "Failed to parse stream index at position %d of \"%s\"\n", (int)(p - c->segment_types_str + 1), c->segment_types_str);
>>> + return AVERROR(EINVAL);
>>> + }
>>> + if (i < 0 || i >= s->nb_streams) {
>>> + av_log(s, AV_LOG_ERROR, "Selected stream %d not found!\n", i);
>>> + return AVERROR(EINVAL);
>>> + }
>>> +
>>> + os = &c->streams[i];
>>> + os->segment_type = formats[format_idx].segment_type;
>>> + os->format_name = formats[format_idx].str;
>>> +
>>> + p = end_p;
>>> + }
>>> + }
>>> +
>>> + state = type_expected;
>>> + }
>>> + }
>>> +
>>> + // Complete segment type setup if no streams are specified after the container type
>>> + if (state == type_parsed)
>>> + set_all_segment_types(s, format_idx);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> static int write_manifest(AVFormatContext *s, int final)
>>> {
>>> DASHContext *c = s->priv_data;
>>> @@ -992,6 +1101,9 @@ static int dash_init(AVFormatContext *s)
>>> if ((ret = parse_adaptation_sets(s)) < 0)
>>> return ret;
>>> + if ((ret = parse_segment_types(s)) < 0)
>>> + return ret;
>>> +
>>> for (i = 0; i < s->nb_streams; i++) {
>>> OutputStream *os = &c->streams[i];
>>> AdaptationSet *as = &c->as[os->as_idx - 1];
>>> @@ -1017,13 +1129,10 @@ static int dash_init(AVFormatContext *s)
>>> if (!ctx)
>>> return AVERROR(ENOMEM);
>>> - c->format_name = get_format_str(c->segment_type);
>>> - if (!c->format_name)
>>> - return AVERROR_MUXER_NOT_FOUND;
>>> - if (c->segment_type == SEGMENT_TYPE_WEBM) {
>>> - if ((!c->single_file && check_file_extension(c->init_seg_name, c->format_name) != 0) ||
>>> - (!c->single_file && check_file_extension(c->media_seg_name, c->format_name) != 0) ||
>>> - (c->single_file && check_file_extension(c->single_file_name, c->format_name) != 0)) {
>>> + if (os->segment_type == SEGMENT_TYPE_WEBM) {
>>> + if ((!c->single_file && check_file_extension(c->init_seg_name, os->format_name) != 0) ||
>>> + (!c->single_file && check_file_extension(c->media_seg_name, os->format_name) != 0) ||
>>> + (c->single_file && check_file_extension(c->single_file_name, os->format_name) != 0)) {
>>> av_log(s, AV_LOG_WARNING,
>>> "One or many segment file names doesn't end with .webm. "
>>> "Override -init_seg_name and/or -media_seg_name and/or "
>>> @@ -1031,7 +1140,7 @@ static int dash_init(AVFormatContext *s)
>>> }
>>> }
>>> - ctx->oformat = av_guess_format(c->format_name, NULL, NULL);
>>> + ctx->oformat = av_guess_format(os->format_name, NULL, NULL);
>>> if (!ctx->oformat)
>>> return AVERROR_MUXER_NOT_FOUND;
>>> os->ctx = ctx;
>>> @@ -1075,7 +1184,7 @@ static int dash_init(AVFormatContext *s)
>>> return ret;
>>> }
>>> - if (c->segment_type == SEGMENT_TYPE_MP4) {
>>> + if (os->segment_type == SEGMENT_TYPE_MP4) {
>>> if (c->streaming)
>>> av_dict_set(&opts, "movflags", "frag_every_frame+dash+delay_moov+global_sidx", 0);
>>> else
>>> @@ -1140,7 +1249,7 @@ static int dash_write_header(AVFormatContext *s)
>>> // Flush init segment
>>> // Only for WebM segment, since for mp4 delay_moov is set and
>>> // the init segment is thus flushed after the first packets.
>>> - if (c->segment_type == SEGMENT_TYPE_WEBM &&
>>> + if (os->segment_type == SEGMENT_TYPE_WEBM &&
>>> (ret = flush_init_segment(s, os)) < 0)
>>> return ret;
>>> }
>>> @@ -1311,7 +1420,7 @@ static int dash_flush(AVFormatContext *s, int final, int stream)
>>> }
>>> if (!c->single_file) {
>>> - if (c->segment_type == SEGMENT_TYPE_MP4 && !os->written_len)
>>> + if (os->segment_type == SEGMENT_TYPE_MP4 && !os->written_len)
>>> write_styp(os->ctx->pb);
>>> } else {
>>> snprintf(os->full_path, sizeof(os->full_path), "%s%s", c->dirname, os->initfile);
>>> @@ -1501,7 +1610,7 @@ static int dash_write_packet(AVFormatContext *s, AVPacket *pkt)
>>> }
>>> //write out the data immediately in streaming mode
>>> - if (c->streaming && c->segment_type == SEGMENT_TYPE_MP4) {
>>> + if (c->streaming && os->segment_type == SEGMENT_TYPE_MP4) {
>>> int len = 0;
>>> uint8_t *buf = NULL;
>>> if (!os->written_len)
>>> @@ -1597,9 +1706,7 @@ static const AVOption options[] = {
>>> { "timeout", "set timeout for socket I/O operations", OFFSET(timeout), AV_OPT_TYPE_DURATION, { .i64 = -1 }, -1, INT_MAX, .flags = E },
>>> { "index_correction", "Enable/Disable segment index correction logic", OFFSET(index_correction), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, E },
>>> { "format_options","set list of options for the container format (mp4/webm) used for dash", OFFSET(format_options_str), AV_OPT_TYPE_STRING, {.str = NULL}, 0, 0, E},
>>> - { "dash_segment_type", "set dash segment files type", OFFSET(segment_type), AV_OPT_TYPE_INT, {.i64 = SEGMENT_TYPE_MP4 }, 0, SEGMENT_TYPE_NB - 1, E, "segment_type"},
>>> - { "mp4", "make segment file in ISOBMFF format", 0, AV_OPT_TYPE_CONST, {.i64 = SEGMENT_TYPE_MP4 }, 0, UINT_MAX, E, "segment_type"},
>>> - { "webm", "make segment file in WebM format", 0, AV_OPT_TYPE_CONST, {.i64 = SEGMENT_TYPE_WEBM }, 0, UINT_MAX, E, "segment_type"},
>>> + { "dash_segment_type", "DASH segment files type. Syntax: \"<type> <type>:<ids>\" where <type> is one of mp4 or webm and <ids> are comma-separated stream ids", OFFSET(segment_types_str), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, E },
>>> { NULL },
>>> };
>>>
>>
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
More information about the ffmpeg-devel
mailing list