WebRTC performance and quality evaluation tool. It allows to validate the audio/video quality and the client CPU/memory usage when multiple connections join the same WebRTC service.
Main features:
./chromium
directory) that allows
to disable the video decoding, lowering the CPU requirements when running multiple
browser sessions.The tool can be executed from sources, using the pre built executables or using the Docker image.
Using Npm:
echo '@vpalmisano:registry=https://npm.pkg.github.com' >> ~/.npmrc
npm install -g @vpalmisano/webrtcperf
# Install FFMpeg:
sudo apt install ffmpeg # Linux
# or:
brew install ffmpeg # MacOS
# Run a Jitsi test:
webrtcperf \
--url="https://meet.jit.si/${JITSI_ROOM_URL}#config.prejoinPageEnabled=false" \
--display='' \
--show-page-log=false
# Press <q> to stop.
Using Docker:
docker pull ghcr.io/vpalmisano/webrtcperf
docker run -it --rm \
-v /dev/shm:/dev/shm \
ghcr.io/vpalmisano/webrtcperf \
--url="https://meet.jit.si/$JITSI_ROOM_URL#config.prejoinPageEnabled=false" \
--show-page-log=false \
--sessions=1 \
--tabs-per-session=1
Stop the tool pressing q
(normal browser close) or x
(it will close the
process immediately).
See the config documentation.
Example output:
-- Mon, 06 Feb 2023 20:46:34 GMT -------------------------------------------------------------------
name count sum mean stddev 5p 95p min max
System CPU 1 15.89 0.00 15.89 15.89 15.89 15.89 %
System GPU 1 0.00 0.00 0.00 0.00 0.00 0.00 %
System Memory 1 72.18 0.00 72.18 72.18 72.18 72.18 %
CPU/page 1 84.42 84.42 0.00 84.42 84.42 84.42 84.42 %
Memory/page 1 1206.90 1206.90 0.00 1206.90 1206.90 1206.90 1206.90 MB
Pages 1 1 1 0 1 1 1 1
Errors 1 0 0 0 0 0 0 0
Warnings 1 0 0 0 0 0 0 0
Peer Connections 1 2 2 0 2 2 2 2
-- Inbound audio -----------------------------------------------------------------------------------
rate 2 28.73 14.36 14.36 0.00 28.73 0.00 28.73 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
jitter 2 0.00 0.00 0.00 0.00 0.00 0.00 s
avgJitterBufferDelay 1 35.29 0.00 35.29 35.29 35.29 35.29 ms
-- Inbound video -----------------------------------------------------------------------------------
received 2 2.66 1.33 1.32 0.01 2.64 0.01 2.64 MB
rate 2 967.41 483.71 483.71 0.00 967.41 0.00 967.41 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
jitter 2 0.01 0.01 0.01 0.02 0.01 0.02 s
avgJitterBufferDelay 1 50.48 0.00 50.48 50.48 50.48 50.48 ms
width 2 960 320 640 1280 640 1280 px
height 2 540 180 360 720 360 720 px
fps 1 15 0 15 15 15 15 fps
-- Outbound audio ----------------------------------------------------------------------------------
rate 2 42.84 21.42 21.42 0.00 42.84 0.00 42.84 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 1 0.001 0.000 0.001 0.001 0.001 0.001 s
-- Outbound video ----------------------------------------------------------------------------------
sent 2 3.25 1.62 1.58 0.04 3.21 0.04 3.21 MB
rate 2 1131.25 565.63 565.63 0.00 1131.25 0.00 1131.25 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 1 0.001 0.000 0.001 0.001 0.001 0.001 s
qualityLimitResolutionChanges 2 2 1 1 0 2 0 2
qualityLimitationCpu 2 0 0 0 0 0 0 0 %
qualityLimitationBandwidth 2 20 10 10 0 20 0 20 %
sentActiveEncodings 2 2 1 1 3 1 3 encodings
sentMaxBitrate 2 3700.00 1850.00 350.00 1500.00 2200.00 1500.00 2200.00 Kbps
width 2 640 640 0 1280 0 1280 px
height 2 360 360 0 720 0 720 px
fps 2 12 12 0 25 0 25 fps
pliCountReceived 2 1 0 1 2 1 2
Statistics values:
Name | Count | Description |
---|---|---|
cpu | Total sessions | The browser process cpu usage. |
memory | Total sessions | The browser process memory usage. |
tabs | Total sessions | The browser current opened tabs. |
received | Total inbound streams | The bytesReceived value for each stream. |
sent | Total outbound streams | The bytesSent value for each stream. |
retransmitted | Total outbound streams | The retransmittedBytesSent value for each stream. |
rate | Total streams | The stream bitrate. |
lost | Total streams | The stream lost packets %. |
jitter | Total streams | The stream jitter in seconds. |
avgJitterBufferDelay | Total decoded tracks | The inbound average jitter buffer delay. |
qualityLimitResolutionChanges | Total outbound video streams | The qualityLimitationResolutionChanges value for each outbound video stream. |
width | Total sent or received videos | The sent or received video width. |
height | Total sent or received videos | The sent or received video height. |
fps | Total sent | The sent video frames per second. |
See the prometheus stack.
Starts one send-receive participant:
docker run -it --rm --name=webrtcperf-publisher \
-v /dev/shm:/dev/shm \
ghcr.io/vpalmisano/webrtcperf \
--url=$MEDIASOUP_DEMO_URL \
--url-query='roomId=test&displayName=Publisher($s-$t)' \
--sessions=1 \
--tabs-per-session=1
Starts 10 receive-only participants:
docker run -it --rm --name=webrtcperf-viewer \
-v /dev/shm:/dev/shm \
ghcr.io/vpalmisano/webrtcperf \
--url=$MEDIASOUP_DEMO_URL \
--url-query='roomId=test&displayName=Viewer($s-$t)&produce=false' \
--sessions=1 \
--tabs-per-session=10
Starts one send-receive participant, with a random audio activation pattern:
docker run -it --rm \
-v /dev/shm:/dev/shm \
-v $PWD/examples:/scripts:ro \
ghcr.io/vpalmisano/webrtcperf \
--url=$EDUMEET_URL \
--url-query='displayName=Publisher($s-$t)' \
--script-path=/scripts/edumeet-sendrecv.js \
--sessions=1 \
--tabs-per-session=1
Starts 10 receive-only participants:
docker run -it --rm \
-v /dev/shm:/dev/shm \
-v $PWD/examples:/scripts:ro \
ghcr.io/vpalmisano/webrtcperf \
--url=$EDUMEET_URL \
--url-query='displayName=Viewer($s-$t)' \
--script-path=/scripts/edumeet-recv.js \
--sessions=1 \
--tabs-per-session=10
Starts one send-receive participant:
docker run -it --rm \
-v /dev/shm:/dev/shm \
ghcr.io/vpalmisano/webrtcperf \
--url=$JITSI_ROOM_URL \
--url-query='#config.prejoinPageEnabled=false&userInfo.displayName=Participant($s-$t)' \
--sessions=1 \
--tabs-per-session=1
Starts 10 receive-only participants:
docker run -it --rm \
-v /dev/shm:/dev/shm \
ghcr.io/vpalmisano/webrtcperf \
--url=$ROOM_URL \
--url-query='#config.prejoinPageEnabled=false&userInfo.displayName=Participant($s-$t)' \
--sessions=1 \
--tabs-per-session=10
The DEBUG_LEVEL
environment variable can be used to enable debug messages;
see debug-level for syntax.
git clone https://github.com/vpalmisano/webrtcperf.git
cd webrtcperf
# Optional: build the chromium customized version
# cd chromium
# ./build.sh setup
# ./build.sh apply_patch
# ./build.sh build
# install the package (on Ubuntu/Debian)
# dpkg -i ./chromium-browser-unstable_<version>-1_amd64.deb
# cd ..
yarn build
# sendrecv test
DEBUG_LEVEL=DEBUG:* yarn start \
--url=https://127.0.0.1:3443/test \
--url-query='displayName=SendRecv($s/$S-$t/$T)' \
--script-path=./examples/edumeet-sendrecv.js \
--sessions=1 \
--tabs-per-session=1
# recv only
DEBUG_LEVEL=DEBUG:* yarn start \
--url=https://127.0.0.1:3443/test \
--url-query='displayName=Recv($s/$S-$t/$T)' \
--script-path=./examples/edumeet-recv.js \
--sessions=1 \
--tabs-per-session=10
--script-params="{timestampWatermarkVideo:true,saveSendVideoTrack:'0',saveRecvVideoTrack:'1'}"
--server-port=5000
--server-use-https=true
--server-data=/data
With saveSendVideoTrack
and saveRecvVideoTrack
you can specify the sessions that will be saved at sender and receiver side (in this example it will save all the video streams sent in the session with index 0
and received in session 1
)./data
directory.docker run --rm \
-e DEBUG_LEVEL=INFO \
-v $PWD/data:/data \
ghcr.io/vpalmisano/webrtcperf:devel \
--vmaf-path /data \
The tool will generate a .vmaf.json
and a .vmaf.png
files in the data/vmaf
directory. Adding the --vmaf-preview
option, a .mp4
file containing the side-by-side video comparison will be generated.--script-params="{saveSendAudioTrack:'0',saveRecvAudioTrack:'1'}"
--server-port=5000
--server-use-https=true
--server-data=/data
With saveSendAudioTrack
and saveRecvAudioTrack
you can specify the sessions that will be saved at sender and receiver side (in this example it will save all the audio streams sent in the session with index 0
and received in session 1
)./data
directory.docker run --rm \
-e DEBUG_LEVEL=INFO \
-v $PWD/data:/data \
ghcr.io/vpalmisano/webrtcperf:devel \
--visqol-path /data \
The tool will generate a visqol.csv
in the data
directory.