Upload
olivier-etienne
View
1.983
Download
0
Tags:
Embed Size (px)
Citation preview
Test Automation on an infrared Set-top box
Olivier ETIENNE - Orange
What is a STB
a Set Top Box ?
Basically just a device to manage IP TV and associated services
Tuner
TNTHDD
RAM CPU
Network adapter
Drivers
Native
SDK
IR Radio
Video card
OS (Linux ou Android)
SDK
HTML
Javascript
Browser…
La STB… pour faire simple
OS (Linux ou Android)
SDK
HTML
Javascript
Browser
La STB… pour faire simple
OS (Linux ou Android)
SDK
HTML
Javascript
Browser
La STB… pour faire simple
La STB… pour faire simple
La STB… pour faire simple
La STB… pour faire simple
La STB… pour faire simple
A Browser
(Webkit)
La STB… pour faire simple
A Browser
(Webkit)
HTML
pages
La STB… pour faire simple
A Browser
(Webkit)
HTML
pages
But all
interactions
made via
remote control
Many countries
FrancePoland SpainSenegal
Many versions for each countries
Maintenance Evolutive
Many devices
Sagem Samsung Airties
So many devices…
How would you test
your TV app ?
Test
Status
How would you test your TV app ?
TEST1 sprint =
Accuracy
relative to
tester
Boring
Activity
Many tests
DEV DEV DEV
• Impossible to test everything
• Not confident in what has been tested
• Non regression test are expensive
• Fear to make any change in code
How would you test your TV app ?
We need to
automate that !
Automation objectives
• Reduce test campaign duration–non regression tests
–functional tests
• Get confident in refactorings
• Use target environnement
• Make reproductible tests
Low
cost
Easy
to use
Previous experiences
• Witbe
• TAKT Engine
• STB-tester
ExpensiveHard to maintain testSensitive to UI change
• Record IR signal
• Send IR signal
• Video capture and analysis
• Expensive hardware
A different approach
Infrared
Automate, automate, but… how ?
OrangeTV
12:32
?
Raspberry pi as a mediator
IN : readIR OUT: sendIR
Infrared
Raspberry pi as a mediator
IN : readIR OUT: sendIR
Infrared
RPi
Raspberry pi as a mediator
IN : readIR OUT: sendIR
Infrared
RPi
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
IN : sendKey
IN : runSequence
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
Raspberry pi as a mediator
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
DOM
Raspberry pi as a mediator
OrangeTV
12:32
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
DOM
Raspberry pi as a mediator
OrangeTV
12:32
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
DOM
DOM
Raspberry pi as a mediator
OrangeTV
12:32
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
DOM
DOM
Raspberry pi as a mediator
OrangeTV
12:32
IN : readIR OUT: sendIR
HTTP
Infrared
RPi
GET HTTP
IR Signal
Synchronous
HTTP
response
IN : sendKey
IN : runSequence
IN : readDump
OUT: send IR IN : regIR
DOM
Pimp my Pi
Raspberry Pi
Apache
PH
P
Scripts
SH
Java
GPIO
Driver
LIRC
HT
TP
Send a key
Record a keypi4j
screen
Screenshot or dump
RPi
Start Sequence
OrangeTV
12:32
PC
(Browser)
TV
Video 1:Record an infrared sequence and replay it from firefox
From specific to
standard
Screenshot or dump
« Screenshot »
of the STB
Content
DOM
« Screenshot »
of the STB
Content
Selenium IDE
TestSuite
TestCaseSend orders
to STB
Make
assertions
on state
Video 2 :Replay a sequence from Selenium IDE
interne Orange44
HTTP
IN : sendKey
IN : runSequence
IN : askDump
IN : runShellCmd
IN : softReboot
IN : hardReboot
RPi
Call HTTP verbs
Make assertions
A raspberry pi as an HTTP proxy to the STB
OrangeTV
12:32
Why Selenium IDE ???
• Quite as good as Selenium
• But much simpler
– Simplified script language
– Firefox Plugin
• No need to be a developer to write a test
every night
Big pictureOn demand
Code
Repo
updateTestSuitesdeploy
updatereset
selenium
Delivery
Status
Test
Repo
Results
Results
• First prototype developed in 4 weeks
• Over 300 functional tests running every night
• VOD tests takes 3h in the night instead of 5
working days
• Team has gained
– confidence in its code
– a better image
Lessons learned
• Keep control on your test data
• Think test soon
– when you write specs
– when you design your architecture
– when you code
• Simple is better
• Everything can’t be automated
And now, …
how do you test
your TV app ?
Like this !
Questions ?
Olivier [email protected]://linkedin.com/profile/view?id=7988978@olivier_etienne
Contact
Bonus slide : What kind of interaction does the
pi provides with STB
❏ send a key
❏ send a key sequence
❏ ask a screen dump
❏ wait for a specific screen to be visible
❏ execute a shell command on the stb
❏ upload and execute a sheel script on stb
❏ put a file on stb
❏ get a file on stb
❏ get logs from stb console
❏ wait for specific log
❏ …
All these verbs are exposed through HTTP interface of the raspberry pi
Bonus slide : What can you retrieve on test failure
❏ Logs from STB Console
❏ Screen dump
Available through selenium execution report in Jenkins
Bonus slide : What is java used for
Java is used for the key sequence recorder and player.
It support a script language
KEY_PRESS
WAIT
REPEAT
:
KEY_RANDOM
DUMP_SCREEN