Upload
tsahi-levent-levi
View
2.033
Download
2
Embed Size (px)
Citation preview
Why is that?
• WebRTC is still niche/experimental to their business
• They are a startup with limited resources
• Adding features takes priority over stability
• Is this VoIP or Web?
The world of VoIP testing
• Focused on DUTs (Devices Under Test)
• Rely heavily on slow moving standards and implementations
• Vast majority of testing solutions are on premise
• Assume limited geographies
5
The world of Web testing
• Focused on different browsers
• Rely heavily on the GUI
• Responsive design causes GUI changes
• GUI and API testing treated as separate
• Geography doesn’t matter for functionality
6
Browsers update cycle in 2015
This translates to a regression test needed on a monthly basis – not taking into account changes you make to the service
40
41
42
43
44
45
46
35
36
37
38
39
40
Dec-14 Feb-15 Apr-15 May-15 Jul-15 Sep-15 Oct-15 Dec-15
Chrome
Firefox
Stable, Beta, Dev and Canary
Stable What your customers are using
Beta What you should expect in the next release
DevWhere you can complain about breakage – and expect fixes in time
CanaryA nightly build of whatever is available at that moment in time
Testing only against the Stable version of browsers means you find out issues only after the service breaks for your customers.
BrowserYour app
A Word about Mobile
At what pace do you update you app’s WebRTC lib?
WebRTC WebRTC
6-8 weeks update cycle
WebRTC isn’t always P2P
Anywhere between 5-50% of your calls get relayed via TURN servers
client client
Webserver
TURNserver
Location, Location, Location
When a call in Paris gets relayed through a TURN server in the US – you lose a customer…
What do you test at scale?
• Signaling?
• Media relays?
• Backend media processing?
• Single session multiparty scale?
• End-to-end service scale?
• Do you add variance to your scale testing?
“
A typical conversation…
Oh, it should work for 100 people in the same conference. We did test it with 12 people – the number of machines we had available – and it worked perfectly fine.
End-to-end
• There are many moving parts in your service
• WebRTC requires 3 or more servers to operate
• You need to make sure it runs end-to-end
Web testing is simpler (most of the time)
1. Write your script for a browser
2. Run it once
3. Run it in parallel on 10,000 browsers at once
4. You’re done
WebRTC is about coordinating people
1. A user and an agent
2. A user and a PSTN/mobile call
3. N users joining in on a single call
4. Automatic/dynamic/manual pairing of the users
5. …
Thank You!Tsahi Levent-Levi
@tsahil