[FFmpeg-devel] [RFC] H264 Error Resilience

[Michael Niedermayer]

Hi all

Kieran asked me about h264 ER TODO 

so here are some ideas

H264 features:
Redundant pictures. It would be possible to encode a low bitrate image
    for each access unit and use it when the primary picture has lost
    slices. This can be limited to slices which would be concealed poorly
    if lost. This would increase the bitrate of the h264 stream

Data partitioning. We support data partitioning for MPEG4(-ASP) and
    it provided there useful quality gains. This would seperate the
    high bitrate textures from low bitrate motion vectors. The later is
    much more important for maitaining ok looking images


Error concealment features:
Interlacing support is basically missing, this is obviously
    important when the input is interlaced

Future frames are currently only used with B frames, but it would
    certainly improve other frames too. For example knowing the next P
    frame would mean we know the next frames motion vectors and the previous
    frames motion vectors which would help in estimating the motion
    vectors any lost area. This would require redecoding the next frame
    and thus possibly that is tricky

Finer grained detection of initial error position, currently errors
    are detected during decoding and a heuristic is used to cover
    several previous MBs so as to cover the whole damaged area.
    But it would be possible to analyze the MBs itself  to detect
    mismatches accorss MB edges and use this to better estimate
    from where concealing must start to maximize quality. Also if
    outside information as lost packets is known this too if it does
    not 1:1 match slices could be used to make the concealed area
    tighter

Testing:
We have little error concealment testing. A few more fate tests to
cover various videos (high/low complexity motion/ no motion) with
different errors (lost slices, reordered slices, damaged bytes/bits)
and comparing (PSNR) vs undamaged input would be usefull.
These could also be usefull to evaluate any changes done to the error
concealment code.


-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Opposition brings concord. Out of discord comes the fairest harmony.
-- Heraclitus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20171110/f4b8ab8a/attachment.sig>