[FFmpeg-user] Struggling with audio of multipart stream

Chris Shaw chris at chrisshaw.co.uk
Thu Dec 2 19:01:24 EET 2021 

I have a TPLink Kasa IP camera, which can be coaxed into sending a live or
recorded stream. It looks like this:

--data-boundary--

Content-Length: 9

User-Id: 01q8uiog

time: 1638463771

X-PacketType: file-fragment-alive

Content-Type: keepaliveframe


keepalive

--data-boundary--

Content-Length: 11681

User-Id: 01q8uiog

X-CurrentFrameTimestamp: 1638462462.263000

X-SampleDuration: 67

X-PacketType: video

X-FrameType: 1

Content-Type: video/x-h264


###### BINARY DATA ######

--data-boundary--

Content-Length: 480

User-Id: 01q8uiog

X-CurrentFrameTimestamp: 1638462462.280000

X-SampleDuration: 60

X-PacketType: audio

Content-Type: audio/g711u


###### BINARY DATA ######



And so on, interleaving video and audio. There's a short sample file here

https://www.dropbox.com/s/kzq78l6kaai3qi1/Kasa_stream.mp4?dl=0


I can view or transcode the video, but it throws a lot of errors and
doesn't seem to see the audio stream. Can anybody help?



A sample transcode looks like this:

>ffmpeg -framerate 15 -i Kasa_stream.mp4 -y tmp.mp4


ffmpeg version N-104708-g8a18db3ec6-tessus
https://evermeet.cx/ffmpeg/  Copyright
(c) 2000-2021 the FFmpeg developers

  built with Apple clang version 11.0.0 (clang-1100.0.33.17)

  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg
--extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl
--enable-libaom --enable-libass --enable-libbluray --enable-libdav1d
--enable-libfreetype --enable-libgsm --enable-libmodplug
--enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg
--enable-libopus --enable-librubberband --enable-libshine
--enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora
--enable-libtwolame --enable-libvidstab --enable-libvmaf
--enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp
--enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid
--enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3
--pkg-config-flags=--static --disable-ffplay

  libavutil      57.  9.101 / 57.  9.101

  libavcodec     59. 14.100 / 59. 14.100

  libavformat    59.  9.102 / 59.  9.102

  libavdevice    59.  0.101 / 59.  0.101

  libavfilter     8. 17.100 /  8. 17.100

  libswscale      6.  1.101 /  6.  1.101

  libswresample   4.  0.100 /  4.  0.100

  libpostproc    56.  0.100 / 56.  0.100

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

[h264 @ 0x7fad00e03cc0] non-existing PPS 0 referenced

    Last message repeated 1 times

[h264 @ 0x7fad00e03cc0] decode_slice_header error

[h264 @ 0x7fad00e03cc0] no frame!

Input #0, h264, from 'Kasa_stream.mp4':

  Duration: N/A, bitrate: N/A

  Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive),
1920x1080, 15 fps, 15 tbr, 1200k tbn

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))

Press [q] to stop, [?] for help

[libx264 @ 0x7fad00d0aec0] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2

[libx264 @ 0x7fad00d0aec0] profile High, level 4.0, 4:2:0, 8-bit

[libx264 @ 0x7fad00d0aec0] 264 - core 164 r3075 66a5bc1 - H.264/MPEG-4 AVC
codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options:
cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6
lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15
scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0
qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

Output #0, mp4, to 'tmp.mp4':

  Metadata:

    encoder         : Lavf59.9.102

  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuvj420p(pc, bt709,
progressive), 1920x1080, q=2-31, 15 fps, 15360 tbn

    Metadata:

      encoder         : Lavc59.14.100 libx264

    Side data:

      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A

frame=    8 fps=6.0 q=-1.0 Lsize=     326kB time=00:00:02.26
bitrate=1179.5kbits/s speed= 1.7x

video:325kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.281166%

[libx264 @ 0x7fad00d0aec0] frame I:1     Avg QP:20.99  size: 54904

[libx264 @ 0x7fad00d0aec0] frame P:7     Avg QP:22.63  size: 39666

[libx264 @ 0x7fad00d0aec0] mb I  I16..4: 25.5% 65.6%  9.0%

[libx264 @ 0x7fad00d0aec0] mb P  I16..4:  7.4% 17.6%  4.1%  P16..4: 46.1%
3.2%  2.6%  0.0%  0.0%    skip:19.0%

[libx264 @ 0x7fad00d0aec0] 8x8 transform intra:62.1% inter:74.5%

[libx264 @ 0x7fad00d0aec0] coded y,uvDC,uvAC intra: 42.8% 86.4% 62.7%
inter: 11.8% 52.0% 20.7%

[libx264 @ 0x7fad00d0aec0] i16 v,h,dc,p: 37% 20% 10% 33%

[libx264 @ 0x7fad00d0aec0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 22% 16%  2%
2%  3%  2%  3%  3%

[libx264 @ 0x7fad00d0aec0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 53% 21%  9%  2%
3%  5%  3%  2%  2%

[libx264 @ 0x7fad00d0aec0] i8c dc,h,v,p: 36% 20% 28% 16%

[libx264 @ 0x7fad00d0aec0] Weighted P-Frames: Y:0.0% UV:0.0%

[libx264 @ 0x7fad00d0aec0] ref P L0: 93.4%  4.4%  1.7%  0.4%

[libx264 @ 0x7fad00d0aec0] kb/s:4988.49

More information about the ffmpeg-user mailing list