Load Testing Real-Time Applications: Why Video Calls Need More Than JMeter

Most load testing assumes a simple shape: fire a request, get a response, measure how long that took, repeat it thousands of times. That shape is close enough to reality for a huge share of real applications. It isn't close enough for a real-time one. An earlier post described a massive live video-distribution platform built on ERTSA, our elastic serverless architecture; testing something like that is a genuinely different problem than testing an API, and video calls make a useful case study for why.

The current state of testing, three separate worlds

Traditional web-app load testing is a mature, well-solved problem. Tools like JMeter, k6, Gatling, Locust, and Artillery replay protocol-level requests cheaply, thousands or millions of them, from a single machine. That works because an HTTP request/response is short-lived and stateless; simulating it doesn't require anything resembling a real client.

Functional end-to-end testing is a separate discipline: proving a complete user journey actually works, not how it performs under load. Frameworks like Selenium, Puppeteer, and Cucumber drive a real browser through a full real flow, click by click, and most teams already have a suite written in one of them. Older HTTP-scripting tools like Badboy covered adjacent ground for simpler cases, though the field has moved on; Grafana's k6 browser module is a more current example worth knowing about, and tellingly, Grafana's own documentation recommends a hybrid approach, a handful of real-browser virtual users alongside a much larger number of protocol-level ones, because a browser VU costs roughly 10x a protocol VU to run at any real scale. That's the industry itself acknowledging the exact gap this post is about.

WebRTC-specific testing sits closest to what an approach like ours actually needs, worth looking at last for that reason. A video call is a long-lived, bidirectional connection with a real ICE/DTLS/SRTP stack encoding and decoding actual media, not a request/response pair, so neither of the two worlds above tests it properly on its own. Existing WebRTC testing tools split roughly into two camps: some take an API approach, exercising a platform's signaling or session-management endpoints without a genuine media stack behind them; others take the functional-testing approach and extend it toward load, the way OpenVidu's own load-testing tool drives real Chrome and Firefox instances via Selenium instead of simulating packets. That second camp is the closer relative of what this post is actually about, and it's also where the real gap shows up: coordinating a large number of those real-browser instances, and gathering rich client-side information from every one of them, isn't something these tools extend massively. That fidelity also has a real cost either way: a 2022 study on WebRTC load testing found that browser-less approximations of the WebRTC stack cut that cost by roughly 96% versus genuine browser instances, at the cost of some fidelity, a trade-off worth naming rather than assuming one side is simply correct.

Four ways to actually load-test something like this

1. Protocol-level simulation. Cheap and genuinely scalable, but it never touches a real client stack, so for a real-time application it can't exercise the thing that's actually being tested.
2. Small-scale, fully real E2E testing. High fidelity, real browsers doing real things, but expensive enough per instance that it doesn't scale to thousands of concurrent users on its own, per Grafana's own guidance above.
3. Hybrid load testing. A practical compromise, a few real browsers plus a large protocol-level crowd, but the protocol-level side still can't report real client-side telemetry, since it was never a real client to begin with.
4. Purpose-built, protocol-specific load tools (like openvidu-loadtest). Real browsers at real scale, built specifically for one protocol, closer to what a real-time application actually needs than the other three. But coordinating a large number of those instances and gathering rich client-side information from every one of them isn't something these tools extend massively, and each is typically a bespoke tool built for its own purpose, not something meant to reuse a team's existing E2E suite.

How we approach end-to-end testing for real-time media (and similar scenarios)

Our approach breaks down into four pieces.

Defining the actors. Not every simulated user needs to be identical. A realistic scenario often needs multiple roles, presenters and viewers in a video call, say, each with its own actions (join, mute, share a screen, switch view) and its own assertions (what that role should actually see, hear, or receive for the test to count as passing). Many of those actions and assertions don't need to be written from scratch, either: an existing E2E suite already encodes them for the functional-correctness case, and defining an actor around that same suite means reusing it as-is, not building a separate script.

Scaling browsers or clients. A "simple" version of ERTSA, the same elastic, ephemeral compute pattern behind the platform it's testing, is what makes running however many of those actors a scenario calls for, real browser instances among them, economically viable only for the duration of the test, instead of a permanent, idle fleet.

Coordinating user flows. Thousands of independent, simulated actors don't naturally act in a coherent order; something has to sequence what each of them does, and reason correctly about when each of them actually did it, not just when a report about it happened to arrive. That's the same problem the ERTSA case study described for fairly ranking a live viewer's response to an in-stream event, a relative queue instead of a simple one, showing up again here for the test's own correctness rather than fairness to a live audience.

Gathering events, information, and evidence. A meaningful share of what actually matters, bitrate, jitter, packet loss, the quality actually experienced, only exists inside each actor's own client; the server can confirm it sent the data, not what it looked like on arrival. Every actor reports its own client-side numbers (the same kind of data a real browser's getStats() API exposes) back into that same coordinated flow, and where quality is genuinely subjective, recording video of the simulated navigation itself gives a reviewer real evidence to judge, not just a number to trust.

The objective decides the details

None of those four pieces has one fixed shape; what a test is actually trying to prove decides all of it. Sometimes the goal is finding the breaking point, how far this can go before it fails. Other times it's guaranteeing a parameter stays within bounds, QoE, server performance, specific network conditions, rather than finding a limit at all. In our experience, that objective is what actually conditions everything downstream: what the actors need to look like, what has to be checked on the client side, what gets measured on the system side, how load ramps up over the course of the test, and how long the test needs to run. Fine-tuning those details to the actual question being asked is most of the real work.

This isn't a platform, it's an approach

Nothing described here ships as a single, off-the-shelf tool. It's an architecture pattern, built from pieces, existing E2E frameworks, ERTSA's elastic compute, and the coordination and information-gathering layer above, adapted to whatever real-time system and whatever existing test suite a team already has. Would your application benefit from this approach? Let's talk.