[MPlayer-users] MPlayer-users Digest, Vol 146, Issue 5

[Ratin]

> Message: 3
> Date: Tue, 9 Feb 2016 00:22:02 +0100
> From: Reimar Döffinger <Reimar.Doeffinger at gmx.de>
> To: "MPlayer usage questions, feature requests, bug reports"
>         <mplayer-users at mplayerhq.hu>
> Subject: Re: [MPlayer-users] h.264 packetization and decoding
> Message-ID: <20160208232202.GD12438 at reimardoeffinger.de>
> Content-Type: text/plain; charset=iso-8859-1
>
> I kind of don't think discussing mpv and libavcodec belongs on this
> list, so I think you'll have to look for help elsewhere beyond
> a few hints.
>
> On Sun, Feb 07, 2016 at 09:15:54AM -0800, Ratin wrote:
> > I send complete (non-FU-A) SPS, PPS, SEI
> > frames in their individual chunks and I combine all Fragmented IDR and P
> > frames (FU-A) into one chunk after re-constructing them per h.264 spec.
>
> Not sure how you mean that, but if you put the SPS/PPS/SEI in packets
> separately from the frames that won't work.
>
>

Talking about how the network abstraction layer recombine frames based on
fragmentation bits set in the NALU headers
of RTP H264 payload. It seems that when mpv uses ffmpegs demuxers that
handles rtp/rtsp, it combines not just fragmented
NALS but also a bunch of reconstructed  P frames, dump all of them in one
aggregated packet to libavcodec for processing.
Putting P frames together is not per h.264 spec, the spec doesnt say that
network abstraction layer should have to put together
bunch of p frames in one packet, it just says that the NAL layer needs to
re-construct frames from their fragments and dump the
reconstrcuted frames to the decoder. However when I submit just individual
p frames (re-constructed from fragments with start codes),
libavocodec doesnt like that, I get the "illegal short term buffer state
detected" error message (per the code below).

if (!current_ref_assigned)
    {

        printf("\nff_h264_execute_ref_pic_marking:  nal_unit_type: %d:
current_ref_assigned: %d\n",
                h->nal_unit_type, current_ref_assigned);
        /* Second field of complementary field pair; the first field of
         * which is already referenced. If short referenced, it
         * should be first entry in short_ref. If not, it must exist
         * in long_ref; trying to put it on the short list here is an
         * error in the encoded bit stream (ref: 7.4.3.3, NOTE 2 and 3).
         */
        if (h->short_ref_count && h->short_ref[0] == h->cur_pic_ptr)
        {
            /* Just mark the second field valid */
            h->cur_pic_ptr->reference |= h->picture_structure;
        }
        else if (h->cur_pic_ptr->long_ref)
        {
            av_log(h->avctx, AV_LOG_ERROR, "illegal short term reference "
                                           "assignment for second field "
                                           "in complementary field pair "
                                           "(first field is long term)\n");
            err = AVERROR_INVALIDDATA;
        }
        else
        {
            pic = remove_short(h, h->cur_pic_ptr->frame_num, 0);
            if (pic)
            {
                printf("\nillegal short term buffer state detected.
short_ref_count: %d, current frame num: %d\n",
                       h->short_ref_count, h->cur_pic_ptr->frame_num);

                fflush(stdout);

                av_log(h->avctx, AV_LOG_ERROR, "illegal short term buffer
state detected\n");
                err = AVERROR_INVALIDDATA;
            }

            if (h->short_ref_count)
                memmove(&h->short_ref[1], &h->short_ref[0],
                        h->short_ref_count * sizeof(H264Picture*));

            h->short_ref[0] = h->cur_pic_ptr;
            h->short_ref_count++;
            h->cur_pic_ptr->reference |= h->picture_structure;
        }
    }



> > My question again is why it dumps  first 879322 in one chunk in the
> > beginning, what logic does it use to determine the boundary? How does
> > libavcoded handle a huge packet of this size 879322 bytes while I am
> > getting some kind of buffer overflow (green overlay is an indication of
> > that)?
>
> The packetization rules are in libavcodec/h264_parser.c.
> And it has nothing to do with overflow, you're just confusing
> the decoder (particularly the reference frame tracking) with the
> input it gets, probably triggering error concealment (which
> normally should result in grey, not green though, unless you
> overwrite some buffers with 0s when you shouldn't).
>
>
I am using hardware accelerated decoding and direct rendering, so I guess
greying will not be happening. But sounds something like what you
mentioned, probably hardware gets part of the data and then my rendering
loop will just dump whatever it got on the video frame buffer.