[FFmpeg-devel] [PATCH] Playlist API

[Geza Kovacs]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/06/2009 02:10 PM, Michael Niedermayer wrote:
> well ...
> this is leading to a problem that is similar to the seeking one
> demuxers arent really supposed to change streams, its possible to
> add streams (increase nb_streams) but existing streams shouldnt change.
> That is the current API ....
> Nothing really prevents us from changing the API but simply not following
> it is not a good idea ...
> and changing could be tricky as well, i dont know, ive not really considered
> it ...
> 
> We also have plenty of other things depending on this API, like the seeking
> code, i mean it takes a stream_index and timestamp in that
> stream index but if streams and thus their timebases can change this really
> turns the sementics of the seeking code upside down.
> 

Given that in a playlist the timestamp refers to the global, media-wide
time, one can infer the stream type from stream_index (discussed below),
and the playlist item from the timestamp (just convert the timestamp to
AV_BASE_TIME and use PlaylistContext->durations to find out its index).
This is demonstrated in ff_concatgen_read_seek in my previous patch,
which seemed to work with streams with different time bases.

> and then there is simply transcoding to a new file, a stream changing type
> from video to audio or such is likely not supported by the target container
> ...
> 

Couldn't there just be a stream numbering convention along the lines of:

streams[0] is primary video stream
streams[1] is primary audio stream
streams[2] is primary subtitle stream
streams[3] is alternative video stream
streams[4] is alternative audio stream
streams[5] is alternative subtitle stream
...etc...

And if one of the streams are not present (ie no subtitle, audio, or
video) just set the codec type for that stream to CODEC_TYPE_UNKNOWN and
it'll be ignored in a playlist? Or in the context of concatenation,
it'll just be set to a "dummy" stream (silent audio, black screen for
video, or empty subtitles)? Since the only mainstream container that
stores multiple audio/video/subtitle streams is Matroska and most
libavcodec-based applications I've encountered seem to already order
streams in this manner when all are present, I don't think this would
break backwards compatibility with anything.

Alternatively if you don't like that approach, I could have the playlist
API first check all the playlist items to see what kinds of streams are
available, create them all for all items, and if one type of stream not
present in one playlist item but is present in another, then it'll just
be set to a "dummy" stream, though this would of course require opening
all streams to determine the types that are available before starting to
play any so I don't think it's optimal.

> 
>>
>> If an application checks if nb_streams (or the types of each stream) has
>> changed and adapts accordingly to accommodate the addition or removal of
>> streams, then such functionality as you described would indeed work
>> as-is with the current API. The only reason you can't do this with
>> ffmpeg right now is that ffmpeg doesn't currently check and adjust its
>> settings if nb_streams or the types of streams has changed while reading
>> packets; it's not an issue with my API; ffmpeg.c just needs another
>> patch to handle changing the number and types of streams after it's
>> started reading packets (but since that change to ffmpeg.c is rather
>> intrusive I'd rather submit it after my playlist API is accepted upstream).
> 
> yes, but would this be less work than having ffmpeg check if a stream hit
> its end (check per AVStream.duration or some end of stream packet) and then
> search for another not ended/new stream
> 
> The problem iam simply seeing, independant of the API is, that if
> streams can change arbitrarly how could one really refer to a stream ...
> 

The "streams" array in the playlist demuxer is of course provided only
for convenience so that a playlist can be used transparently as a
standard container without additional code to support it. For common
purposes, I think this is enough and so long as the streams aren't
changing types (which can be accomplished in one of the above
conventions are followed) then the developer shouldn't really need to
worry that the stream just changed, other than if the codec changed
which would need manual intervention no matter what the API does.

As for "really referring to a stream", while a combination of stream
index or stream type as well as a timestamp would work for uniquely
identifying a stream in the context of seeking, if a developer really
wants fine-grained control over the streams contained in a playlist
they'd just be using the API to access streams
(PlaylistContext->icl[itemnum]->streams[streamnum]) anyhow.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkp7WuoACgkQs6+5LdXBeIIcfwCeJXKMGUhyfL3fCVpocnCHVgMn
WCsAoIm4SceXSOkST91/5SZ542ykeATD
=LJpb
-----END PGP SIGNATURE-----