[FFmpeg-devel] Documentation: proposed changes in the structure

[Alexander Strasser]

Hi Nicolas,
hi Clement!

Wow the discussion gets up to speed quickly :)
Thanks for bringing up all those good points!

I'm intentionally top-posting, because I want to say something
concerning the points you discuss below. Though I couldn't find
a good way to do it in a comment style, so instead I'll explain
my view and what I had in mind here and leave your discussion
for reference at the end of this mail.

First of all I find the content at [1] quite practical and on spot.
I don't know if it is the only way or the best way to structure
docs, but I find the argumentation and division into the quadrants
practical and helpful. It also states that there is overlap and
that redundancy cannot be fully avoided. We have those things
in some places seamlessly intermingled in our documentation,
which is problematic, since it is hard to find and hard to read
depending on what you are after. I think we should fix it step
by step, making sure no valuable information gets lost, but
gets into the right place instead.

The remaining part of my answer is talking about reference
documentation as described in [1].

I don't talk about API docs. We have that separated and in
Doxygen. For simplicity in writing I only talk about lavfi
below. Also I assume we chose Markdown, but that doesn't
actually matter.

I talk about the filter descriptions and the help text for
the filter parameters. I think it is important we do not have
this partially duplicated like it is now.

I think we should eradicate the copy in the source files and
keep the description in the corresponding .md file. We shall
build a tool that creates supplementary C files from the .md
files which have conventional symbol names which are referenced
from the AVFilter.description and  AVOption.help fields. For
now the generated C files would only contain rather short one
line plain text strings matching the current practice. These
strings can of course be shortened versions extracted from
the complete version in the .md file. E.g. only take the
first sentence.

When all that is achieved, we would have a working version
that removes the redundancy and is compatible with the way
our APIs currently are. The fftools and the lib users would
not be affected. The .md version can be used to create the
HTML docs.

Of course we can improve up on that, but that can come later
without holding up the work on the documentation. For example
we could consider how much reference documentation we want to
embed in the libs and how we want to evolve our APIs for that.


Best regards,
  Alexander

[1] https://documentation.divio.com/ , mentioned by Clement, see below

On 2020-08-22 14:12 +0200, Nicolas George wrote:
> Clement Boesch (12020-08-22):
> > AVOption.help and AVFilter.description (as well as AVCodec.description
> > etc) do contain redundant information with the documentation.
>
> There is truth in what you say, but I disagree this is really redundant:
> AVOption.help and AV*.description are supposed to be very short, while
> the text in the documentation is supposed to be descriptive, with
> explanations.
>
> > Additionally, we sometimes document manually the default, all the
> > constants, sometimes the ranges, which are already held within the
> > AVOption.
>
> I agree, the default value and constants can, to an extents, be
> generated automatically.
>
> > So yes I'd rather have the AVOption.help extended to contain more and have
> > a tool which is basically `ffmpeg -h filter=xxx` in the choosen markup.
>
> This is exactly what we are talking about, except for two details:
>
> - We cannot extend AVOption.help without causing an ABI and API break,
>   but we can create a new more generic system and keep .help only as a
>   small supplement and for compatibility.
>
> - C source files are not the place for lengthy text.
>
> > I meant the other way around: you have a gen-doc.c which you compile and
> > link against the library, which generates the .md (integration within the
> > make is easy).
>
> If the documentation is embedded in the library, then clearly there
> sould be an API to present it. And we could use it to build the static
> documentation. It would not work for cross-compilation, though.
>
> Another option is to have a FATE tests that checks the consistency of
> the documentation with the options. I think this better, because
> although redundancy without check leads to inconsistency, redundancy
> with checks helps make sure we write what we mean and we mean what we
> write.
>
> > But I think we have it wrong with the documentation approach. What is
> > currently described within the code corresponds to the "technical
> > documentation". While all the extra information on the filters may enter
> > within the "explanation" and "tutorial" realm. This means that we could
> > have a filter documentation that starts with the technical documentation
> > (generated from the code) followed by all the "extra" detailed info on the
> > filter themselves.
> >
> > It might get clumsy but at least we wouldn't have to paraphrase with a
> > different formatting every single option, and miss half of them on a
> > regular basis.
> >
> > You may want to read https://documentation.divio.com/ to understand better
> > the separation I'm making.
>
> I read "simple and comprehensive", I automatically translate
> "oversimplification".
>
> The distinction between reference and explanations on one side and
> tutorials and how-to guides on the other side makes sense. On the other
> hand, there is a continuum between tutorials and how-tos, and a
> continuum between reference and explanations.
>
> > To clarify just a little here: the non technical documentation would be
> > written manually outside of the code, just like currently (except we would
> > strip out all the redundant info with the tech doc).
>
> The documentation for components, which is the bulk of the documentation
> and what we are discussing now, is "reference", with just enough of
> "explanation" to make it self-contained without overlapping with
> transverse documentation.
>
> I agree that it belongs with each component. But again, a C source file
> is not the place for this, mixing syntaxes is rarely a good idea. Which
> is exactly why I suggest to have a parallel structure $lib/$component.c
> + $lib/doc/$component.md.