Woes from the RPi 5 USB Bus
9 min readJan 11, 2025
I am trying to set up two USB cameras on one Raspberry Pi 5. I remember having issues several years ago with the previous Raspberry Ii when Itried to build stereoscopic vision with ROS. TL;DR: looks like the same problem exists today.
docker logs
working container:
PS C:\Users\jgens\Downloads\iac\rpi5-1> docker compose logs -f ffmpeg_2
ffmpeg_2-1 | ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
ffmpeg_2-1 | built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
ffmpeg_2-1 | configuration: --disable-debug --disable-doc --disable-ffplay --disable-libdrm --enable-alsa --enable-cuda-llvm --enable-cuvid --enable-ffprobe --enable-gpl --enable-libaom --enable-libass --enable-libdav1d --enable-libfdk_aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libkvazaar --enable-liblc3 --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librav1e --enable-librist --enable-libsrt --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-nonfree --enable-nvdec --enable-nvenc --enable-openssl --enable-stripping --enable-version3
ffmpeg_2-1 | libavutil 59. 39.100 / 59. 39.100
ffmpeg_2-1 | libavcodec 61. 19.100 / 61. 19.100
ffmpeg_2-1 | libavformat 61. 7.100 / 61. 7.100
ffmpeg_2-1 | libavdevice 61. 3.100 / 61. 3.100
ffmpeg_2-1 | libavfilter 10. 4.100 / 10. 4.100
ffmpeg_2-1 | libswscale 8. 3.100 / 8. 3.100
ffmpeg_2-1 | libswresample 5. 3.100 / 5. 3.100
ffmpeg_2-1 | libpostproc 58. 3.100 / 58. 3.100
ffmpeg_2-1 | Input #0, video4linux2,v4l2, from '/dev/video2':
ffmpeg_2-1 | Duration: N/A, start: 410002.408055, bitrate: 147456 kb/s
ffmpeg_2-1 | Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 640x480, 147456 kb/s, 30 fps, 30 tbr, 1000k tbn
ffmpeg_2-1 | Stream mapping:
ffmpeg_2-1 | Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
ffmpeg_2-1 | Press [q] to stop, [?] for help
ffmpeg_2-1 | [libx264 @ 0x55b931ec80] using cpu capabilities: ARMv8 NEON
ffmpeg_2-1 | [libx264 @ 0x55b931ec80] profile High 4:2:2, level 3.0, 4:2:2, 8-bit
ffmpeg_2-1 | [libx264 @ 0x55b931ec80] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2024 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=1 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
ffmpeg_2-1 | Output #0, hls, to '/hls/stream_2_.m3u8':
ffmpeg_2-1 | Metadata:
ffmpeg_2-1 | encoder : Lavf61.7.100
ffmpeg_2-1 | Stream #0:0: Video: h264, yuv422p(tv, progressive), 640x480, q=2-31, 30 fps, 90k tbn
ffmpeg_2-1 | Metadata:
ffmpeg_2-1 | encoder : Lavc61.19.100 libx264
ffmpeg_2-1 | Side data:
ffmpeg_2-1 | cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[hls @ 0x55b9320480] Opening '/hls/stream_2_0.ts.tmp' for writingdup=33 drop=0 speed=1.15x peed= 1.6x
ffmpeg_2-1 | [hls @ 0x55b9320480] Opening '/hls/stream_2_.m3u8.tmp' for writing
[hls @ 0x55b9320480] Opening '/hls/stream_2_1.ts.tmp' for writingdup=33 drop=0 speed=1.07x peed=1.14x
ffmpeg_2-1 | [hls @ 0x55b9320480] Opening '/hls/stream_2_.m3u8.tmp' for writing
[hls @ 0x55b9320480] Opening '/hls/stream_2_2.ts.tmp' for writingdup=33 drop=0 speed=1.05x peed=1.07x
ffmpeg_2-1 | [hls @ 0x55b9320480] Opening '/hls/stream_2_.m3u8.tmp' for writingbad container:
PS C:\Users\jgens\Downloads\iac\rpi5-1> docker compose logs -f ffmpeg_0
ffmpeg_0-1 | ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
ffmpeg_0-1 | built with gcc 13 (Ubuntu 13.3.0-6ubuntu2~24.04)
ffmpeg_0-1 | configuration: --disable-debug --disable-doc --disable-ffplay --disable-libdrm --enable-alsa --enable-cuda-llvm --enable-cuvid --enable-ffprobe --enable-gpl --enable-libaom --enable-libass --enable-libdav1d --enable-libfdk_aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libkvazaar --enable-liblc3 --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librav1e --enable-librist --enable-libsrt --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-nonfree --enable-nvdec --enable-nvenc --enable-openssl --enable-stripping --enable-version3
ffmpeg_0-1 | libavutil 59. 39.100 / 59. 39.100
ffmpeg_0-1 | libavcodec 61. 19.100 / 61. 19.100
ffmpeg_0-1 | libavformat 61. 7.100 / 61. 7.100
ffmpeg_0-1 | libavdevice 61. 3.100 / 61. 3.100
ffmpeg_0-1 | libavfilter 10. 4.100 / 10. 4.100
ffmpeg_0-1 | libswscale 8. 3.100 / 8. 3.100
ffmpeg_0-1 | libswresample 5. 3.100 / 5. 3.100
ffmpeg_0-1 | libpostproc 58. 3.100 / 58. 3.100top
weird! one ffmpeg is getting way more cpu time than the other. also, the bad ffmpeg *is* using cpu but not logging anything.
59323 root 20 0 1459376 10820 8192 S 0.0 0.6 0:00.08 `- /usr/bin/containerd-shim-runc-v2 -namespace moby -id 13ff8673183de635cb5d5e5a3aadf86afe8aac8d8547bb1cb36f+
59396 root 20 0 166688 38272 35072 R 50.0 2.0 4:44.34 `- /usr/local/bin/ffmpeg -f v4l2 -i /dev/video0 -c:v libx264 -threads 1 -preset ultrafast -f hls -hls_ti+
59341 root 20 0 1459120 10848 8192 S 0.3 0.6 0:00.08 `- /usr/bin/containerd-shim-runc-v2 -namespace moby -id 3c2af5dc17c93ae65b04ab61ef8cada24a296ef760aeb777d1e9+
59403 root 20 0 11160 7168 6016 S 0.0 0.4 0:00.06 `- nginx: master process nginx -g daemon off;
59567 _rpc 20 0 11164 2612 1408 S 0.0 0.1 0:00.00 `- nginx: worker process
59372 root 20 0 1459120 10732 8064 S 0.0 0.6 0:00.28 `- /usr/bin/containerd-shim-runc-v2 -namespace moby -id c00a405534eed1c0d7576f36d506a32792257e17368e8f42cf78+
59409 root 20 0 549664 57392 37888 S 29.5 3.0 2:53.68 `- /usr/local/bin/ffmpeg -f v4l2 -i /dev/video2 -c:v libx264 -threads 1 -preset ultrafast -f hls -hls_ti+ vmstat
hmm, shows a lot of idle cpu (~78) and some block out. maybe something related to file system?
root@rpi5-1:/var/lib/docker/volumes# vmstat -t 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- -----timestamp-----
r b swpd free buff cache si so bi bo in cs us sy id wa st CST
3 0 0 190160 107372 1359544 0 0 1 46 346 470 17 2 81 0 0 2025-01-11 11:04:53
2 0 0 190160 107372 1359544 0 0 0 0 1123 1616 14 9 77 0 0 2025-01-11 11:04:54
2 0 0 190160 107372 1359544 0 0 0 2060 1363 1700 13 16 72 0 0 2025-01-11 11:04:55
2 0 0 190160 107372 1359544 0 0 0 4 1360 1631 13 18 69 0 0 2025-01-11 11:04:56
2 0 0 190160 107372 1359544 0 0 0 0 1125 1592 16 6 78 0 0 2025-01-11 11:04:57
2 0 0 190160 107372 1359544 0 0 0 0 1115 1574 14 8 78 0 0 2025-01-11 11:04:58
2 0 0 190160 107372 1359544 0 0 0 0 1134 1607 15 8 78 0 0 2025-01-11 11:04:59
2 0 0 190160 107372 1359544 0 0 0 0 1339 1674 13 14 73 0 0 2025-01-11 11:05:00usbtop
now, i don’t know the specs of the raspberry pi 5 but i did try changing the resolution on the cameras thinking it would lower the bytes to/from the device and hopefully lessen the load on all subsequent parts of the system.
available format:
root@rpi5-1:/var/lib/docker/volumes# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 176x144
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 160x120
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 960x720
Interval: Discrete 0.033s (30.000 fps)
[1]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 1280x720
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 176x144
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 160x120
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.050s (20.000 fps)
Size: Discrete 960x720
Interval: Discrete 0.067s (15.000 fps)after:
root@rpi5-1:/var/lib/docker/volumes# v4l2-ctl --get-fmt-video --device 0
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'YUYV' (YUYV 4:2:2)
Field : None
Bytes per Line : 1280
Size Image : 614400
Colorspace : sRGB
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Default (maps to Limited Range)
Flags :
root@rpi5-1:/var/lib/docker/volumes# v4l2-ctl --get-fmt-video --device 2
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'YUYV' (YUYV 4:2:2)
Field : None
Bytes per Line : 1280
Size Image : 614400
Colorspace : sRGB
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Default (maps to Limited Range)
Flags :However, modifying the camera to use the lower format resulted in the same kb/s to and from the device. hmm. maybe ffmpeg is polling the device over and over to try and get the latest frame?
Bus ID 0 (Raw USB traffic, all USB buses) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/s
Device ID 2 : 0.00 kb/s 0.00 kb/s
Device ID 14 : 141.73 kb/s 24156.96 kb/s
Device ID 15 : 141.74 kb/s 24047.76 kb/s
Bus ID 1 (Raw USB traffic, bus number 1) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/s
Device ID 2 : 0.00 kb/s 0.00 kb/s
Device ID 14 : 141.73 kb/s 24156.25 kb/s
Device ID 15 : 141.74 kb/s 24047.28 kb/s
Bus ID 2 (Raw USB traffic, bus number 2) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/siotop
using ‘a’ key to swap between bandwidth and cumulative (to view cumulative), we can definitely see that only one ffmpeg process is writing to disk.
Total DISK READ: 0.00 B/s | Total DISK WRITE: 0.00 B/s
Current DISK READ: 0.00 B/s | Current DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE> COMMAND
59574 be/4 root 0.00 B 5.93 M ffmpeg -f v4l2 -i /dev/video2 -c:v libx264 -threads 1 -preset ultra~te_segments+round_durations+temp_file /hls/stream_2_.m3u8 [mux0:hls]
172 be/3 root 0.00 B 80.00 K [jbd2/mmcblk0p2-8]
915 be/4 root 0.00 B 8.00 K dockerd --containerd=/run/containerd/containerd.sock
1251 be/4 root 0.00 B 4.00 K dockerd --containerd=/run/containerd/containerd.sockstrace
working container
root@rpi5-1:/var/lib/docker/volumes# strace -p 59409 | tail
strace: Process 59409 attached
restart_syscall(<... resuming interrupted io_setup ...>) = -1 ETIMEDOUT (Connection timed out)
futex(0x55b931dcc0, FUTEX_WAKE_PRIVATE, 1) = 0
pselect6(1, [0], NULL, NULL, {tv_sec=0, tv_nsec=0}, NULL) = 1 (in [0], left {tv_sec=0, tv_nsec=0})
read(0, "", 1) = 0
write(2, "frame=13975 fps= 30 q=23.0 size="..., 95) = 95
futex(0x55b931dd18, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, {tv_sec=1736614953, tv_nsec=958157000}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
futex(0x55b931dcc0, FUTEX_WAKE_PRIVATE, 1) = 0stalled container
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE}) = -1 EAGAIN (Resource temporarily unavailable)interesting!
configuring the kernel module
some suggestion from: https://forums.raspberrypi.com/viewtopic.php?t=35689
ffmpeg_0-1 | [video4linux2,v4l2 @ 0x558f6a7260] Dequeued v4l2 buffer contains 520744 bytes, but 614400 were expected. Flags: 0x00012001.
ffmpeg_0-1 | [video4linux2,v4l2 @ 0x558f6a7260] Dequeued v4l2 buffer contains 496312 bytes, but 614400 were expected. Flags: 0x00012001.
ffmpeg_0-1 | [video4linux2,v4l2 @ 0x558f6a7260] Dequeued v4l2 buffer contains 514636 bytes, but 614400 were expected. Flags: 0x00012001.maybe we are still reading too much data?
lowering again:
v4l2-ctl --set-fmt-video=width=320,height=240,pixelformat=MJPG --device /dev/video0
v4l2-ctl --set-fmt-video=width=320,height=240,pixelformat=MJPG --device /dev/video2next i reloaded the kernel driver without the args:
rmmod uvcvideo
modprobe uvcvideoand containers are once again working.
root@rpi5-1:/var/lib/docker/volumes/rpi5-1_hls/_data# lsmod | grep uvc
uvcvideo 110592 2
uvc 12288 1 uvcvideo
videobuf2_vmalloc 12288 2 uvcvideo,bcm2835_v4l2
videobuf2_v4l2 32768 6 bcm2835_codec,uvcvideo,bcm2835_v4l2,rpivid_hevc,v4l2_mem2mem,bcm2835_isp
videodev 303104 9 bcm2835_codec,videobuf2_v4l2,uvcvideo,bcm2835_v4l2,rpivid_hevc,v4l2_mem2mem,bcm2835_isp
videobuf2_common 69632 10 bcm2835_codec,videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,uvcvideo,bcm2835_v4l2,rpivid_hevc,v4l2_mem2mem,videobuf2_memops,bcm2835_isp
mc 61440 9 videodev,bcm2835_codec,snd_usb_audio,videobuf2_v4l2,uvcvideo,videobuf2_common,rpivid_hevc,v4l2_mem2mem,bcm2835_ispusbtop still showing similar (yet a tad bit smaller):
Bus ID 0 (Raw USB traffic, all USB buses) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/s
Device ID 2 : 0.00 kb/s 0.00 kb/s
Device ID 14 : 141.70 kb/s 23719.82 kb/s
Device ID 15 : 141.69 kb/s 23717.12 kb/s
Bus ID 1 (Raw USB traffic, bus number 1) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/s
Device ID 2 : 0.00 kb/s 0.00 kb/s
Device ID 14 : 141.70 kb/s 23719.88 kb/s
Device ID 15 : 141.69 kb/s 23717.04 kb/s
Bus ID 2 (Raw USB traffic, bus number 2) To device From device
Device ID 1 : 0.00 kb/s 0.00 kb/sSummary
It seems like 352x288 YUYV and medium ffmpeg preset was the highest I could reliably get out two usb cameras on the raspberry pi 5. Given that the raspberry pi 5 is quite expensive and can only run a single “high res” webcam (or two very low res cameras), I wonder if single purpose internet webcams are the best approach given their ability to optimize buffer locations and frame rate timing and avoid other userland processes. I’ll probably end up running some of the detection models alongside, though I would prefer their API be the m3u8 stream and not the physical usb device. This would allow potentially better models to run elsewhere.
