[FFmpeg-devel] [RFC PATCH 3/3] HACK: avformat: rawenc: allow to output a raw PRFT

[Paul B Mahol]

On 9/22/23, Clément Péron <peron.clem at gmail.com> wrote:
> Hi Paul,
>
> On Fri, 22 Sept 2023 at 15:17, Paul B Mahol <onemda at gmail.com> wrote:
>>
>> On Fri, Sep 22, 2023 at 2:39 PM Clément Péron <peron.clem at gmail.com>
>> wrote:
>>
>> > Hi Paul,
>> >
>> > On Fri, 22 Sept 2023 at 13:41, Paul B Mahol <onemda at gmail.com> wrote:
>> > >
>> > > On Fri, Sep 22, 2023 at 12:06 PM Clément Péron <peron.clem at gmail.com>
>> > wrote:
>> > >
>> > > > Hi Paul
>> > > >
>> > > > On Fri, 22 Sept 2023 at 11:27, Paul B Mahol <onemda at gmail.com>
>> > > > wrote:
>> > > > >
>> > > > > On Fri, Sep 22, 2023 at 10:38 AM Clément Péron
>> > > > > <peron.clem at gmail.com
>> > >
>> > > > wrote:
>> > > > >
>> > > > > > Hi Andreas,
>> > > > > >
>> > > > > > On Fri, 22 Sept 2023 at 09:58, Andreas Rheinhardt
>> > > > > > <andreas.rheinhardt at outlook.com> wrote:
>> > > > > > >
>> > > > > > > Clément Péron:
>> > > > > > > > Hi Michael, Andreas,
>> > > > > > > >
>> > > > > > > > On Thu, 21 Sept 2023 at 22:50, Andreas Rheinhardt
>> > > > > > > > <andreas.rheinhardt at outlook.com> wrote:
>> > > > > > > >>
>> > > > > > > >> Michael Niedermayer:
>> > > > > > > >>> On Thu, Sep 21, 2023 at 02:17:00PM +0200, Clément Péron
>> > wrote:
>> > > > > > > >>>> Output the producer reference time to a dirty raw output.
>> > > > > > > >>>>
>> > > > > > > >>>> Signed-off-by: Clément Péron <peron.clem at gmail.com>
>> > > > > > > >>>> ---
>> > > > > > > >>>>  libavformat/rawenc.c | 122
>> > > > > > +++++++++++++++++++++++++++++++++++++++++++
>> > > > > > > >>>>  1 file changed, 122 insertions(+)
>> > > > > > > >>>
>> > > > > > > >>> this breaks fate-filter-volume and others
>> > > > > > > >>> (Segmentation fault)
>> > > > > > > >>>
>> > > > > > > >>> I can rerun it with debug symbols and provide peoper gdb
>> > output
>> > > > > > > >>> but i suspect given this has "HACK" in the title you are
>> > aware of
>> > > > > > this
>> > > > > > > >
>> > > > > > > > The "HACK" tag meaning was not supposed to be: "it's ok if
>> > > > > > > > it
>> > > > > > > > segfaults", but more to trigger a discussion is it possible
>> > > > > > > > to
>> > > > > > > > properly support an output timestamp in the raw video demux,
>> > and if
>> > > > > > > > yes how to do it :)
>> > > > > > >
>> > > > > > > If you need a timestamp for raw video, then use a proper
>> > container
>> > > > and
>> > > > > > > not raw video. In fact, this patch basically creates new
>> > > > > > > formats
>> > > > > > > different from all the raw formats.
>> > > > > >
>> > > > > > Yes I agree, but I do not want to add too much overhead nor
>> > > > > > computation processing or memory copy to my pipeline just to mux
>> > and
>> > > > > > demux between ffmpeg and my python script.
>> > > > > >
>> > > > > > The idea is to have a very light structure to easily pipe it.
>> > > > > >
>> > > > > > I'm not familiar with audio/video container but it seems to me
>> > > > > > that
>> > > > > > parsing containers are not very light no?
>> > > > > >
>> > > > > >
>> > > > > Containers range from almost no processing like .y4m to complex
>> > monsters
>> > > > > like .mxf
>> > > >
>> > > > .y4m doesn't contain a timestamp either, and I don't want to use a
>> > > > complex container :),
>> > > >
>> > >
>> > > I doubt storing clock time in container for each frame is correct
>> > approach.
>> > > Is this variable frame rate video?
>> >
>> > Why would it not? The python takes as an input a list of frames.
>> > It then runs an inference model on it and could discard some frames if
>> > needed.
>> > Taking the absolute timestamp of each frame avoids relying on a
>> > supposed frame rate (based on a free running clock) that is not
>> > guaranteed.
>> > Also the result gives inference results which are properly timestamped
>> > with the correct data acquisition time and I can easily resynchronize
>> > with other sensor acquisition.
>> >
>> > >
>> > > One can always add another, trivial container with just one field
>> > > having
>> > > whatever you want and with optional magic string in header.
>> >
>> > Yes, if this is acceptable by FFMpeg community I could propose a
>> > trivial container where the format is user defined.
>> > Else I can keep this dirty patch downstream, but I'm not a big fan.
>> >
>>
>> FFmpeg usually supports files found in wild, and not inventing own formats
>> unless there is very good reason for it.
>
> Indeed, It will be hard to find a file in the wild as it's supposed to
> be used when you read from a pipe.
>
> So if I propose a patch to support a new demux where the format is
> user defined is there a chance to be accepted?

If it passes code review and is not blocked by other means, yes.

>
> Thanks,
>>
>>
>> >
>> > Regards,
>> >
>> > >
>> > > Or can try/explore NUT container in FFmpeg.
>> > >
>> > >
>> > > >
>> > > > >
>> > > > > This patch is hack and approach/solution it tries is flawed.
>> > > >
>> > > > 100% agree with you that's why I prefix the patch with "HACK:",
>> > > >
>> > > > Regards,
>> > > > Clement
>> > > >
>> > > >
>> > > > >
>> > > > >
>> > > > > > Thanks,
>> > > > > > Clement
>> > > > > >
>> > > > > >
>> > > > > > >
>> > > > > > > - Andreas
>> > > > > > >
>> > > > > > > _______________________________________________
>> > > > > > > ffmpeg-devel mailing list
>> > > > > > > ffmpeg-devel at ffmpeg.org
>> > > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > > > >
>> > > > > > > To unsubscribe, visit link above, or email
>> > > > > > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> > > > > > _______________________________________________
>> > > > > > ffmpeg-devel mailing list
>> > > > > > ffmpeg-devel at ffmpeg.org
>> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > > >
>> > > > > > To unsubscribe, visit link above, or email
>> > > > > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> > > > > >
>> > > > > _______________________________________________
>> > > > > ffmpeg-devel mailing list
>> > > > > ffmpeg-devel at ffmpeg.org
>> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > > >
>> > > > > To unsubscribe, visit link above, or email
>> > > > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> > > > _______________________________________________
>> > > > ffmpeg-devel mailing list
>> > > > ffmpeg-devel at ffmpeg.org
>> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > > >
>> > > > To unsubscribe, visit link above, or email
>> > > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> > > >
>> > > _______________________________________________
>> > > ffmpeg-devel mailing list
>> > > ffmpeg-devel at ffmpeg.org
>> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> > >
>> > > To unsubscribe, visit link above, or email
>> > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> > _______________________________________________
>> > ffmpeg-devel mailing list
>> > ffmpeg-devel at ffmpeg.org
>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> >
>> > To unsubscribe, visit link above, or email
>> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> >
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>