Experiences with Client-based Speculative Remote Display

John R. Lange & Peter A. DindaDepartment of Electrical Engineering and Computer Science

Northwestern University

Sam RossoffDepartment of Computer Science

University of Victoria

Prescience Labhttp://presciencelab.org

http://empathicsystems.org

2

Speculative Remote Display

• Remote display interactivity fundamentally constrained by network latency– Critical for WANs

• Speculation can improve interactivity – Can be done without server support

• Significant predictability exists in remote display protocol streams

• Speculation can be introduced into remote display clients– User-controlled tradeoff between latency and display

errors– User studies currently give mixed results for tradeoff

3

Outline

• Speculative Remote Display• Predictability study of Remote Display

protocol streams– RDP and VNC

• Design and implementation of a client-based speculative remote display system– VNC-SRD

• User study of VNC-SRD– Examination of user-controlled speculation

• Conclusions

4

Remote Display Systems

• Historically popular in many forms– VNC, RDP, THINC, Citrix, X– Thin clients, remote administration, mobile users, etc

• Implemented at all levels of the stack– Virtual hardware, device drivers, window system

extensions– Large variance in semantic properties of the protocols

• Lots of work has gone into optimizing Remote Displays– Predominantly to decrease bandwidth requirements

5

Latency vs. BandwidthOptimization

• Large amount of work in optimizing bandwidth usage of Remote Displays– Compression/encoding, caching, protocol design

• Bandwidth is important…• But so is interactivity

• User interaction implies waiting for server updates– Network latency is a critical component

6

Prospects for Speculative Remote Display

User Events

Screen Events

• Can we predict screen events well?• If so, can we speculatively execute screen

events before server responds?

(Windows RDP)

7

Optimizing Latency Through Speculation

• Speculation can improve interactivity– Display screen events before

they arrive • RTT is no longer limiting factor

• UI Interaction intuitively appears to be predictable– Menus, dialog boxes, etc– Often tied to specific user action

8

Questions

• Is there predictability?• How do we design and implement such a tool

– Client– Client/Server– Predictor– …

• Can users tolerate display errors?• Can users control tradeoff between display

errors and interactivity?

9

Analysis of Predictability

• Effectiveness of speculation requires predictable event sequences

• Examined predictability with state-limited kth order Markov models– Simple– General prediction symbols

• Unique event signatures as symbols

– Can provide multi step ahead predictions

10

Trace Collection

• Instrumented versions of rdesktop and VNC• User studies

– Northwestern students & faculty participants• 5 users for each protocol

– Single study model• Full screen display of remote Windows machine• Private local network• Standardized task sequence

• Event traces from typical Windows workloads– Word, PowerPoint, Web browsing– 15 minute periods– VNC: 12K-24K user events, 8.5K-17K screen events– RDP: 47K-77K user events, 712K-1M screen events

11

Analysis of Predictability

• Generated symbols from event sequences• Ran symbol streams through Markov model

– Model is continuously being updated, just as in system we later built

– Screen->Screen, Screen+User->Screen– Different model orders considered– Different constraints on state space size considered

• Analyzed one-step-ahead predictability– Percentage of predictions that were correct

• Predictor Accuracy

– Percentage of total events correctly predicted• Protocol Predictability

12

RDP Prediction AccuracyScreen+User -> Screen

13

RDP PredictabilityScreen+User -> Screen

14

VNC Prediction AccuracyScreen+User->Screen

15

VNC PredictabilityScreen+User->Screen

16

Implications

• RDP displays a high degree of predictability– Screen events are highly correlated

• RDP results are much better than VNC– Most likely due to number of messages in an update– Differences in protocol semantics– RDP screen event rate much higher than VNC

• Errors are possible– Users will experience inconsistent screen states– Implementation must correct mis-predictions

– Can users tolerate this?

17

User Control

• Errors will occur, but…• Sensitivity to errors varies…

– Between applications– Between users

• Let user decide tradeoff between aggressive use of predictions to reduce latency, and increasing display errors

• Can users make that decision?– What is the best way to present the control to users?

18

Why not RDP/SRD?

• Implementation simplicity– VNC has much less client state– VNC has fewer event types– VNC execution is straightforward

• If it works with VNC then it surely works with RDP

19

VNC-SRD

• Implementation of SRD inside VNC client– ~2300 LOC (C++) added to VNC client– ~850 LOC (Perl) Markov predictor

implementation

• User-driven prediction application policy– Allows user to adjust amount of speculation

• Generic predictor interface– Event signature-based predictions

• Error correction and rollback

20

21

Prediction Application Policy

• System predicts event signatures– Constructs events from signatures

• Events directed to one of two queues– Applied Queue

• Applied events currently displayed to the user• Held until validated by arrival of server events

– Pending Queue• Predictions waiting to be displayed

• Events applied to screen at a given rate– Rate: predicted server event rate– Rate * (desired latency reduction) = # of steps ahead

22

User Control

• Users/workloads have varying error tolerance– Allow direct user control

• VNC-SRD adds a user adjustable scale to the interface– User specifies desired network

latency reduction

23

Error Handling

• Speculation can result in wrong display– System must determine and correct errors

• Correction– VNC-SRD fetches entire modified region

• All applied predictions are rolled back• Local rollback is straightforward

• Validation– VNC-SRD validates prediction against next event

• Error on mismatch

– Conservative approach• Out of order predictions may be displayed correctly• All errors have equal weight

24

User Study

• Students from Northwestern and University of Victoria– 16 total users

• 4 network scenarios– Varied latency and drop rate

• 300ms, 100ms, 50ms, 10ms + 10% drop rate

– 3 randomly presented to each user

• 3 Application workloads– Word, PowerPoint, and Web Browsing– Performed in each of the network scenarios– 5 minute runs

• Predictor state cleared for each run

25

Purpose of Study

• Determine whether users can tolerate screen artifacts due to speculation

• Determine whether users can trade off using our interface

26

Study Questionnaire

• Users rated the system after each scenario– Ability to find a comfortable predictor setting– Responsiveness at the most comfortable

setting– Acceptability of display errors at the most

comfortable setting

– Scale: 1 (very bad) to 10 (very good)

27

User Study Caveats

• Few predictions were made– Study runs not long enough to sufficiently

seed predictor– VNC has low predictability

• Ratings should thus be taken with a large grain of salt

• Future work planned to improve this

28

Able to find a comfortable setting?

29

Are Display Errors Acceptable?

30

Is the System Responsive?

31

Results

• We still do not know for sure whether users…– Can tolerate the screen artifacts– Can trade off between screen artifacts and

latency

• More work needed

32

Alternative Design Approaches

• Speculative support on the server– Synchronized predictors

• Predictions used for other purposes– Cache hints– Update regions– Update ordering

• Other prediction algorithms/symbols– Time series predictions of event coordinates

33

Future Work

• Exploration of predictability in Remote Display protocols– How far ahead can predictions be made?– What prediction algorithm should be used?– To what degree can SRD decrease latency?– Comparison of more protocols

• Exploration of different SRD designs

34

Conclusions

• Remote display interactivity fundamentally constrained by network latency– Critical for WANs

• Speculation can improve interactivity – Can be done without server support

• Significant predictability exists in remote display protocol streams

• Speculation can be introduced into remote display clients– User-controlled tradeoff between latency and display

errors– User studies currently give mixed results for tradeoff

35

• Prescience Lab– http://presciencelab.org

• Empathic Systems Project– http://empathicsystems.org

• John Lange– http://www.artifex.org/~jarusl