-
mobile measurements:the mobile app / OS perspective
Why understanding mobile app performance is hard
-
cellular performance matters to some apps
•most demanding apps• multiplayer gaming• VoIP•
video chat
• somewhat demanding• web search•
app interfaces with web services
• reasonably irrelevant•
network transfers that aren’t in the critical path
26.March.2014 [email protected] 2
-
app development environment
26.March.2014 [email protected] 3
-
Start screen icons
26.March.2014 [email protected] 4
-
The Weather Channel
26.March.2014 [email protected] 5
-
MittRating
MittRating
26.March.2014 [email protected] 6
-
MittRating
MittRating
MittRating
MittRating
26.March.2014 [email protected] 7
-
MittRating
MittRating
26.March.2014 [email protected] 8
-
Internet
Mitt Romney
MittRating
MittRating
26.March.2014 [email protected] 9
-
Internet
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney
with my money.”
“If I had a nickel for every time
Mitt Romney
said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 [email protected] 10
-
Internet
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney
with my money.”
“If I had a nickel for every time
Mitt Romney
said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 [email protected] 11
-
Internet
53%
“Mitt Romney might vote for Obama as well”
“We recommend Mitt Romney for president”
“I would definitely trustMitt Romney
with my money.”
“If I had a nickel for every time
Mitt Romney
said something stupid I'd be in his tax bracket”
Mitt Romney
MittRating
MittRating
Tweets
26.March.2014 [email protected] 12
-
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 13
-
ClickHandler() {
}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 14
-
ClickHandler() {
tweets = HttpGet(url);
}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 15
-
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);
}
ProcessTweets(tweets){
...}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 16
-
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 17
-
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
Click HandlerStart
Click HandlerEnd
Thread
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 18
-
ClickHandler() {
tweets = HttpGet(url);rating = ProcessTweets(tweets);display.Text = rating;
}
ProcessTweets(tweets){
...}
Click HandlerStart
Click HandlerEnd
Thread
User Perceived Delay
53%
MittRating
MittRating
Hypothetical Synchronous Code
26.March.2014 [email protected] 19
-
MittRating
MittRating
53%
Asynchronous Code
26.March.2014 [email protected] 20
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...}
MittRating
MittRating
53%
Asynchronous Code
26.March.2014 [email protected] 21
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
53%
User Click
Asynchronous Code
26.March.2014 [email protected] 22
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
53%
User Click
Asynchronous Code
26.March.2014 [email protected] 23
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
Background Thread
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 [email protected] 24
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...} UI Thread
Background ThreadUI Dispatch
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 [email protected] 25
-
ClickHandler() {
AsyncHttpGet(url, DownloadCallback);}DownloadCallback(tweets) {
rating = ProcessTweets(tweets);UIDispatch(DisplayRating, rating);
}DisplayRating(rating){
display.Text = rating;}ProcessTweets(tweets){
...}
DisplayRating
UI Thread
Background ThreadUI Dispatch
ClickHandlerEndClickHandler
Start
MittRating
MittRating
System
Async Get Call
ProcessTweets
DownloadCallback
53%
User Click
Asynchronous Code
26.March.2014 [email protected] 26
-
Apps are highly asynchronous
30 popular apps167,000 transactions from user study
26.March.2014 [email protected] 27
-
Apps are highly asynchronous
30 popular apps167,000 transactions from user study
• On average, 19 asynchronous calls
per user transaction• On average,
8 parallel threads per user transaction
26.March.2014 [email protected] 28
-
Apps are highly asynchronous
For each user transaction, what was the critical path, and did the network matter?
30 popular apps167,000 transactions from user study
• On average, 19 asynchronous calls
per user transaction• On average,
8 parallel threads per user transaction
26.March.2014 [email protected] 29
-
MittRating
MittRating
26.March.2014 [email protected] 30
-
Tweets MittRatingMitt
Rating
26.March.2014 [email protected] 31
-
Tweets
Posts
MittRating
MittRating
26.March.2014 [email protected] 32
-
Tweets
Posts
MittRating
MittRating
26.March.2014 [email protected] 33
-
Tweets
Posts
MittRating
MittRating
47%
26.March.2014 [email protected] 34
-
UI Thread
Background Thread
Background Thread
Background Thread
User Transaction
Tweets
Posts
MittRating
MittRating
47%
26.March.2014 [email protected] 35
-
UI Thread
Background Thread
Background Thread
Background Thread
User Transaction
Tweets
Posts
MittRating
MittRating
47%
User Click
26.March.2014 [email protected] 36
-
UI Thread
Background Thread
Background Thread
Background Thread
Twitter
Thread Blocked
User Transaction
Tweets
Posts
Facebook
MittRating
MittRating
47%
User Click
26.March.2014 [email protected] 37
-
UI Thread
Background Thread
Background Thread
Background Thread
Twitter
Thread Blocked
Fire
User Transaction
Callback
Tweets
PostsTwitter
Process Tweets
Facebook
MittRating
MittRating
47%
User Click
26.March.2014 [email protected] 38
-
UI Thread
Background Thread
Background Thread
Background Thread
Twitter
Thread Blocked
Fire
Fire
User Transaction
Callback
Callback
Tweets
PostsTwitter
Facebook Process Posts
Process Tweets
Facebook
MittRating
MittRating
47%
User Click
26.March.2014 [email protected] 39
-
UI Thread
Background Thread
Background Thread
Background Thread
Twitter
Thread WakeupThread Blocked
Fire
Fire
DisplayUser Transaction
Callback
Callback
Tweets
PostsTwitter
Facebook Process Posts
Process Tweets
Facebook
MittRating
MittRating
47%
User Click
26.March.2014 [email protected] 40
-
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Twitter
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Twitter
Facebook Process Posts
Process Tweets
Facebook
User Transaction Display26.March.2014
[email protected] 41
-
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Twitter
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Twitter
Facebook Process Posts
Process Tweets
Facebook
User Transaction Display26.March.2014
[email protected] 42
-
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Twitter
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Twitter
Facebook Process Posts
Process Tweets
Facebook
Critical Path
User Transaction Display26.March.2014
[email protected] 43
-
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Twitter
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Twitter
Facebook Process Posts
Process Tweets
Facebook
Critical PathOptimizing the critical path reduces the user perceived delay
User Transaction Display26.March.2014
[email protected] 44
-
UI Thread
Background Thread
Background Thread
Background Thread
User Click
Twitter
Thread WakeupThread Blocked
Fire
Fire
Callback
Callback
Twitter
Facebook Process Posts
Process Tweets
Facebook
Critical PathOptimizing the critical path reduces the user perceived delay
User Transaction Display26.March.2014
[email protected] 45
-
key points
• app pages designed in GUI•
async interfaces in code to UI elements•
async interfaces to network•
tendency to fetch many network objects at launch•
fetch‐parse‐fetch pattern•
critical path is hard to determine•
in which user transactions does network matter?•of those, what caused network delay, if any?
26.March.2014 [email protected] 46
-
cellular network complicates this even more
• radio power states
26.March.2014 [email protected] 47
-
cellular network complicates this even more
• radio power states•
latency varies by load
26.March.2014 [email protected] 48
-
cellular network complicates this even more
• radio power states•
latency varies by load
26.March.2014 [email protected] 49
0
0.2
0.4
0.6
0.8
1
120 170 220
empi
rical
CD
F
RTT (ms)
-
cellular network complicates this even more
• radio power states•
latency varies by load• latency varies
over space
26.March.2014 [email protected] 50
0
0.2
0.4
0.6
0.8
1
0 50 100 150 200
empi
rical
CD
F
RTT difference at 90th percentile (ms)
-
why am I telling you all this?•wireless BW, latency, RSSI, etc. data is useful for research
• relate it to user experience•
this is hard because of how apps are built•
need detailed & efficient app instrumentation
•our mobile OS throttles some network behavior•
based on app performance need and cellular data limits•
research systems to do this on a per app transfer basis, using detailed & lightweight instrumentation
•need detailed, predictive wireless performance info & how individual parts of apps are affected by net perf
26.March.2014 [email protected] 51
-
for more details
•
cellular performance variations in short timescales•
Switchboard paper in ACM MobiSys 2011
•
asynchronous nature of apps & perf
measurement• AppInsight
paper in USENIX OSDI 2012
•network fetching behavior of apps•
Procrastinator paper in ACM MobiSys 2014
26.March.2014 [email protected] 52
