[FFmpeg-devel] [PATCH] Complete rewrite of the "fps" video filter section. More accurate.

[Josh de Kock]

On 27/04/2020 07:17, list+ffmpeg-dev at jdlh.com wrote:
> From: Jim DeLaHunt <from+ffmpeg-dev at jdlh.com>
> 
> This is a complete rewrite of the documentation for the "fps" video
> filter. It describes the filter's behaviour more clearly and accurately.
> I based the rewrite on reading the source code in vf_fps.c closely.
> 
> No code, or other documentation files, are touched by this change.
> 
> Signed-off-by: Jim DeLaHunt <from+ffmpeg-dev at jdlh.com>
> ---
>   doc/filters.texi | 167 ++++++++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 149 insertions(+), 18 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 71a6787289..bd4a1ad2a9 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -11139,27 +11139,34 @@ format=pix_fmts=yuv420p|yuv444p|yuv410p
>   @anchor{fps}
>   @section fps
>   
> -Convert the video to specified constant frame rate by duplicating or dropping
> -frames as necessary.
> +Generate a video, having the specified constant frame rate, from the frames of
> +the input video, by copying or duplicating or dropping input frames based on
> +their input presentation time stamps (PTSs). The output video has new PTSs. You

Should just be `PTS'.

> +can choose the method for rounding from input PTS to output PTS.
>   
>   It accepts the following parameters:
>   @table @option
>   
>   @item fps
> -The desired output frame rate. The default is @code{25}.
> +The output frame rate, as a number of frames per second. This value can be an
> +integer, real, or rational number, or an abbreviation. The default is @code{25}.
>   

I don't think this change adds any value here.

>   @item start_time
> -Assume the first PTS should be the given value, in seconds. This allows for
> -padding/trimming at the start of stream. By default, no assumption is made
> -about the first frame's expected PTS, so no padding or trimming is done.
> -For example, this could be set to 0 to pad the beginning with duplicates of
> -the first frame if a video stream starts after the audio stream or to trim any
> -frames with a negative PTS.
> +The time, in seconds from the start of the input stream, which is converted to
> +an input starting PTS value and an output starting PTS value.
> +If set, @var{fps} drops input frames
> +which have PTS values less than the input starting PTS. If not set, the input
> +and output starting PTS values are zero, but @var{fps} drops no input frames based
> +on PTS.
> +(See details below.)
>   
>   @item round
> -Timestamp (PTS) rounding method.
> +Rounding method to use when calculating output Presentation Timestamp
> +(PTS) integer values from input PTS values. If the calculated output PTS value
> +is not exactly an integer, then the method determines which of the two
> +neighbouring integer values to choose.
>   
> -Possible values are:
> +Possible method names are:

They are still just values for the option.

>   @table @option
>   @item zero
>   round towards 0
> @@ -11170,43 +11177,167 @@ round towards -infinity
>   @item up
>   round towards +infinity
>   @item near
> -round to nearest
> +round to nearest (and if exactly at midpoint, away from 0)
>   @end table
>   The default is @code{near}.
>   
>   @item eof_action
> -Action performed when reading the last frame.
> +Action which @var{fps} takes with the final input frame. The input video passes
> +in an ending input PTS, which @var{fps} converts to an ending output PTS.
> + at var{fps} drops any input frames with a PTS at or after this ending PTS.
>   
>   Possible values are:
>   @table @option
>   @item round
> -Use same timestamp rounding method as used for other frames.
> +Use same rounding method as for other frames, to convert the ending input PTS
> +to output PTS.
> +
>   @item pass
> -Pass through last frame if input duration has not been reached yet.
> +Round the ending input PTS using @code{up}. This can have the effect of passing
> +through one last input frame.
>   @end table
> +
>   The default is @code{round}.
>   
>   @end table
>   
> -Alternatively, the options can be specified as a flat string:
> +Alternatively, the options may be specified as a flat string:
>   @var{fps}[:@var{start_time}[:@var{round}]].
>   
> + at var{fps} generates an output video with integer Presentation Time Stamp (PTS)

You already specify what PTS is above.

> +values which increment by one each output frame, and with a time base set to
> +the inverse of the given frame rate. @var{fps} copies, duplicates, or drops
> +input frames, in sequence, to the output video. It does so according to their
> +input PTS values, as converted to seconds (via the input time base), then
> +rounded to output PTS values.
> +
> + at var{fps} sets output PTS values in terms of a time line which starts at
> +zero. The integer PTS value multipled by the output time base gives a point

multiplied

> +in seconds of that output frame on that timeline. If the @var{start_time}
> +parameter is not set, or is zero, the first output frame's PTS value is zero.
> +Otherwise, the first PTS value is the output starting PTS value calculated
> +from the @var{start_time} parameter.
> +
> + at var{fps} interprets input PTS values in terms of the same time line. It
> +multiplies the input PTS value by the input time base time, to get a frame
> +position in seconds on the time line. It rounds that position to an integer
> +output PTS value. For example, if the input video has a frame rate
> +of 30 fps, a time base of 1/30 seconds, and its first frame has a
> +PTS of 300, then @var{fps} treats that frame as occurring 10 seconds (300 * 1/30
> +seconds) after the start of the video, even though it is the first frame.
> +
> +Setting a @code{start_time} value allows for padding/trimming at the
> +start of stream. For example, @code{start_time} could be set to 0 to pad the
> +beginning with duplicates of the first frame if a video stream starts after
> +the audio stream or to trim any frames with a negative PTS. When
> + at code{start_time} is not set, the @var{fps} filter makes no assumption about
> +the first frame's expected PTS, and does not pad or trim input frames which
> +have a PTS set.
> +
>   See also the @ref{setpts} filter.
>   
> + at subsection Details
> +
> + at var{fps} emits exactly one frame for each output PTS value. If there is
> +exactly one input frame with an input PTS which converts to the current output
> +PTS value, @var{fps} emits that frame. If there are multiple frames which
> +convert to the same output PTS value, @var{fps} emits the final frame of that
> +group, and drops the previous frames. If no input frame PTS converts to a given
> +output PTS value, @var{fps} emits another copy of the previously emitted frame.
> +When the first input frame converts to an output PTS after the first frame, then
> + at var{fps} emits copies of that first frame until the output PTS reaches the
> +converted value of that first frame's input PTS.
> +
> + at var{fps} always drops input frames which have no PTS value set, regardless
> +of the @var{start_time} parameter.
> +
> +The @var{frame rate} value may be provided in a variety of forms. Each form is
> +converted into a rational number, with an integer numerator and denominator.
> +Each value must be zero or greater.
> +
> + at itemize
> +
> + at item
> +An integer number, e.g. @code{25}. This converts to the rational number
> + at code{25/1}.
> +
> + at item
> +A real number, e.g. @code{3.14145926}. This converts to a rational number,
> +e.g. @code{954708/303893}
> +
> + at item
> +A rational number. The numerator and denominator may be either integers or real
> +numbers. e.g. @code{30/1.001} or @code{30000/1001}, which both convert to
> + at code{30000/1001}.
> +
> + at item
> +An abbreviation. e.g @code{ntsc} as @code{30000/1001},
> + at code{ntsc-film} as @code{24000/1001}. See the complete list at
> + at ref{Video rate,,the "Video rate" section in the ffmpeg-utils(1) manual,ffmpeg-utils}.
> +
> + at end itemize
> +
> + at var{fps} defines a sync point, where one input PTS value is treated as occuring

occurring

> +at the same moment as one output PTS value. IT calcluates other PTS values

calculates

> +as positive or negative time offsets from this sync point.
> +This affects the details of how rounding works. If @var{start_time} is set,
> +then the input and output PTS values which @var{fps} calculates from this
> +become the sync point. Otherwise, input and output PTS values of zero are
> +the sync point.
> +
> +Note that @var{fps} does not assume that input frames are separated by exactly
> +1/frame_rate seconds. It takes the input PTS values literally. If the increment
> +of PTS between frames varies along the video, fps will treat those frames as
> +happening at varying time intervals.
> +
> +As a consequence, if you have a video which is supposed to be at a certain frame
> +rate, but in reality the frames were not always captured at the exactly right
> +moment, and if the input PTS values reflect that variation, then you can pass
> +this video through an @var{fps} filter set to the same frame rate. A @var{round}
> +method of @code{near} will pass through each input frame exactly once, and
> +the output video will have new PTS values which reflect the exact time interval
> +(1/frame rate) between frames.
> +
> +Because @var{fps} treats input PTS values at face value when converting them to
> +time on the time line, and because that time line starts at zero, an input
> +video with PTS values that do not start at zero might yield unexpected results.
> +Suppose the input PTS values start with the value 300, which converts to 10
> +seconds after the sync point. Then @var{fps} will repeat the first frame of the
> +input video to fill the first 10 seconds of the output video. (However,
> + at command{ffmpeg} may suppress those repeated frames, depending on the
> + at option{-vsync} setting.) If you set @var{start_time} to 10 seconds, then
> + at var{fps} sets the sync point to the PTS values converted from the 10 second
> +moment on the time line. It no longer repeats the first frame. And, it starts
> +the output PTS at the value corresponding to 10 seconds, instead of zero.
> +
>   @subsection Examples
>   
>   @itemize
>   @item
> -A typical usage in order to set the fps to 25:
> +A typical usage in order to emit a video with a frame rate of 25 frames per second: >
>   @example
>   fps=fps=25
>   @end example
>   
>   @item
> -Sets the fps to 24, using abbreviation and rounding method to round to nearest:
> +Emit a video with a frame rate of 24 frames per second, using an abbreviation,
> +and rounding method to round to nearest:
>   @example
>   fps=fps=film:round=near
>   @end example
> +
> + at item
> +If an input video is supposed to have a frame rate of 29.97 frames per second
> +(NTSC standard), but the time base is 3003/90000, and the PTS values increment
> +variably at slightly more and less than that rate, this emits a video with the
> +same frames, but with a new time base and PTS values that increment at exactly
> +the NTSC rate. If some frames were dropped by the recorder, but the PTS values
> +still reflect when the remaining frames were captured, this will also repeat
> +frames to fill the gaps from the dropped frames.
> + at example
> +fps=fps=30/1.001:round=near
> + at end example
> +
>   @end itemize
>   
>   @section framepack
> 

Just a few nitpicks.

Also your usage of `emit' may be confusing to non-native English 
speakers and I don't think your writing has those people in mind 
particularly. I would just s/emit/output/ for the most part (check first 
this works).

Like Marton said, it's probably a bit on the verbose side, just have a 
read over and see if you can say the same thing with less words. I don't 
think being verbose is necessarily bad either, as you said there's less 
chance that a user would have to end up looking into code to know how to 
use it.

-- 
Josh