[FFmpeg-user] should I shoot the dog?

[Devin Heitmueller]

On Tue, Sep 29, 2020 at 12:28 PM Mark Filipak (ffmpeg)
<markfilipak at bog.us> wrote:
> Top/bottom/top/bottom, especially if full lines, seems like straightforward interlaced to me. Or do
> I misunderstand?

You do not misunderstand.  I was offering the context of how the
Y-data is organized, which is straightforward compared to the
explanation that followed on how the chroma data relates to the luma
data (and how that relationship differs depending on whether the frame
represents interlaced vs. progressive video).

> >... Because of chroma subsampling and the fact
> > that multiple lines can share chroma samples, this gets tricky. ...
>
> Our messages crossed in transit, but I'm going to assume that you've seen my "In macroblock
> format..." post (in this subject thread).
>
> >... In
> > the simple progressive case for 4:2:0, you'll have the first Chroma
> > sample corresponding to the first two luma samples on line 1 and the
> > first two luma samples on line 2. ...
>
> I assume you meant to write "and the *next* two luma samples on line 2".

No, what I wrote is what I intended.  The 4:2:0 pixel format has four
luma samples, two from the first line, and the two that are directly
below those on the second line.

> That 'sounds' like what I'm
> calling sample-quads. The following is from the glossary I'm working on (reformatted to fit email).

I would encourage you stop trying to invent new terminology, in
particular as you are still learning the basics like "What is 4:2:0
planar format?"  You're better off explaining what 4:2:0 is than
showing a diagram of some pixels and giving it a name which is the
equivalent to 4:2:0.

> YCbCr420 sampleset:
>    A sampleset with sample-quads:
>    .---.---.
>    ¦ S ¦ S ¦
>    :---:---:
>    ¦ S ¦ S ¦
>    '---'---', reduced to 1/4 chrominance resolution:
>    .---.---. .-------. .-------.
>    ¦ Y ¦ Y ¦ ¦       ¦ ¦       ¦
>    :---:---: ¦Cb     ¦ ¦Cr     ¦
>    ¦ Y ¦ Y ¦ ¦       ¦ ¦       ¦
>    '---'---' '-------' '-------', distinguished by binary metadata:
>    'chroma_format' = 01. (See "Cb420 & Cr420 macroblocks", "Y macroblock".)
>
> YCbCr422 sampleset:
>    A sampleset with sample-quads:
>    .---.---.
>    ¦ S ¦ S ¦
>    :---:---:
>    ¦ S ¦ S ¦
>    '---'---', reduced to 1/2 chrominance resolution:
>    .---.---. .-------. .-------.
>    ¦ Y ¦ Y ¦ ¦Cb     ¦ ¦Cr     ¦
>    :---:---: :-------: :-------:
>    ¦ Y ¦ Y ¦ ¦Cb     ¦ ¦Cr     ¦
>    '---'---' '-------' '-------', distinguished by binary metadata:
>    'chroma_format' = 10. (See "Cb422 & Cr422 macroblocks", "Y macroblock".)
>
> YCbCr444 sampleset:
>    A sampleset with sample-quads:
>    .---.---.
>    ¦ S ¦ S ¦
>    :---:---:
>    ¦ S ¦ S ¦
>    '---'---', having full chrominance resolution:
>    .---.---. .---.---. .---.---.
>    ¦ Y ¦ Y ¦ ¦Cb ¦Cb ¦ ¦Cr ¦Cr ¦
>    :---:---: :---:---: :---:---:
>    ¦ Y ¦ Y ¦ ¦Cb ¦Cb ¦ ¦Cr ¦Cr ¦
>    '---'---' '---'---' '---'---', distinguished by binary metadata:
>    'chroma_format' = 11. (See "Cb444 & Cr444 macroblocks", "Y macroblock".)

The diagrams are probably fine, but probably not how I would draw them
given they blur the relationship between packed and planar.  Either
it's packed, in which case you should probably show 4:2:2 as YCbYCr,
or it's planer in which case the Cb/Cr samples should not be adjacent
per line (i.e. have all the Y lines followed by all the Cr/Cb lines).
You may wish to take into your account your newfound understanding of
packed vs. planar to redo these diagrams representing the data as
either one or the other.

I would probably also refrain from using the term "macroblock" to
describe the raw decoded video, as macroblocks are all about how the
pixels are organized in the compressed domain.  Once they are decoded
there is no notion of macroblocks in the resulting video frames.

> >... If the video frame is interlaced
> > however, the first chroma sample corresponds to the first two luma
> > samples on line 1 and the first two luma samples on line 3.  The first
> > chroma sample on the second line of chroma corresponds with the first
> > two luma samples on line 2 and the first two luma samples on line 4.
>
> I have pictures of those, too. What do you think of the above pictures? Do you a, like them, or b,
> loathe them, or c, find them unnecessary?

I would probably see if you can find drawings already out there.  For
example the Wikipedia article on YUV has some pretty good
representations for pixel arrangement in various pixel formats.  So
does the LinuxTV documentation.

> > This is known as "interlaced chroma" and a Google search will reveal
> > lots of cases where it's done wrong and what the effects are.  This is
> > the article I usually refer people to:
> >
> > https://hometheaterhifi.com/technical/technical-reviews/the-chroma-upsampling-error-and-the-420-interlaced-chroma-problem/
> >
> > The above article does a really good job explaining the behavior (far
> > better than I could do in the one paragraph above).
>
> I've seen that produce mild combing. I'll read your reference.

Yes, it often manifests visually as a combing effect, but it's not as
pronounced as a typical deinterlacing artifact since it's only the
chroma lines that are rendered in the wrong order.  Hence it is most
obvious with certain content types like animation (where you are more
likely to have sharp transitions between certain colors).  Also,
deinterlacing artifacts are most visible with high motion, whereas
interlaced chroma artifacts can be visible even with static non-moving
video.

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller at ltnglobal.com