[FFmpeg-devel] VP9 RTP encoder and WebRTC

[Alex Protasenko]

Jan,


Yes, that's the one, the latest draft I believe. The descriptor byte is 
the bare minimum required for VP9, which could optionally be followed by 
one or two byte serial picture ID (used by receiver to identify 
problematic frames in its RTCP feedback to the sender), and some more 
opaque scalability structure information (which could store frame 
resolution among other things). The optional parts don't seem to affect 
my use case (I added the gstreamer take on it at some point but didn't 
see any changes for better or worth),


Thanks

Alex

On 6/1/19 2:14 PM, Jan Ekström wrote:
> Hi,
>
> On Sat, Jun 1, 2019 at 8:35 PM Alex Protasenko <aprotasenko at bkmks.com> wrote:
>> Hello,
>>
>>
>> I'm trying to play some realtime video sources (web/IP cam) using WebRTC
>> in a browser. I'm sending RTP stream via Janus gateway using VP9 codec,
>> hardware transcoded using ffmpeg.
>>
>> Everything works fine except random frame corruption happening around
>> moving objects, portions of frame "flowing off the screen" and such
>> until next keyframe fixes it. This happens consistently especially at
>> higher framerates.
>>
>> It turns out the issue  could be narrowed down to the VP9 RTP
>> packetizer. The problem is it's not marking P frames vs I frames in the
>> VP9 payload descriptor octet (the P bit). Gstreamer does that and
>> doesn't experience any such corruption issues.
>>
>> I added this simple change and now WebRTC plays any stream 100% solid
>> and corruption free for me.
>>
>>
>> Could somebody implement this simple fix in the upstream. Basically
>> in libavformat/rtpenc_vp9.c add something to the effect of the following
>> two lines (to set the P bit for all but I frames):
>>
>>       /* mark the first fragment */
>>       *rtp_ctx->buf_ptr++ = 0x08;
>>
>> +    if (!keyframe) {
>> +        rtp_ctx->buf[0] |= 0x40;
>>
>>
>> Where the "keyframe" is an additional boolean parameter to the
>> ff_rtp_send_vp9 function which could be called as:
>>
>> ff_rtp_send_vp9(s1, pkt->data, size, pkt->flags & AV_PKT_FLAG_KEY);
>>
> Is https://tools.ietf.org/html/draft-ietf-payload-vp9-06#page-7 the
> correct specification for VP9 in RTP?
>
> Best regards,
> Jan
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".