Games on Demand: Are We There Yet?

Kuan-Ta Chen

Institute of Information ScienceAcademia Sinica

Games on Demand:Are We There Yet?

Academia Sinica

31 research institutes in 3 major divisions 1) mathematics, physics, and applied sciences; 2) life sciences; 3) humanities and social sciences.

1000 tenure-tracked researchers

5,000 research associates and technicians

Games on Demand / Sheng-Wei “Kuan-Ta” Chen 3

Institute of Information Science

Members40 Principal Investigators40 post-doctoral researchers300 technicians and RAs

Research Areas

•Bioinformatics •Network System and Service

•Data Management and Information Discovery •Multimedia Technologies

•Natural Language and Knowledge Processing •Computer System

•Programming Languages and Formal Methods •Computation Theory and Algorithms

Multimedia Networking and Systems Lab


Multimedia Networking and Systems Lab

Research AreasMultimedia Systems

Quality of Experience Management

Computational Social Science

http://mmnet.iis.sinica.edu.tw

http://mmnet.iis.sinica.edu.tw/

Area 1: Multimedia Systems

5

Area 2: Quality of Experience

Using physiological measurements to predict the market performance of online games

6

[1] Jing-Kai Lou, Kuan-Ta Chen, Hwai-Jung Hsu, and Chin-Laung Lei, Forecasting Online Game Addictiveness, IEEE/ACM NetGames 2012.

Area 3: Computation Social Science

“The emerging intersection of the social and computational sciences, an intersection that includes analysis of web-scale observational data, virtual lab–style experiments, and computational modeling” [1].

[1] Duncan J. Watts, Computational Social Science Exciting Progress and Future Directions, Frontiers of Engineering, Winter 2013.

Area 3: CSS (cont.)

Area 3: CSS (cont.)

Help people reduce weight by providing visual incentives

lost 5 kg lost 4 kg


GAMES ON DEMAND


Tough Life of Gamers

Games are becoming way too complexThe overhead of setting up a game is significantOften locked on a specific computer

Games may not be incompatible with some software/hardwareComputer hardware constantlydemands upgrading


On-demand services


Games on Demand: Approaches

Painless game installation

e.g., on Xbox 360

Cloud gamingCloud-supported instantgame play


Cloud Gaming: File Streaming

Instant game play supported by a minimal, playable code base (~ 5%)

Progressive downloading of game code and data during game play

3D mesh streaming can be seen a special instance

(Figure courtesy of Wei Tsang Ooi from “Scalable View-Dependent Progressive Mesh Streaming”)

Cloud Gaming: Video Streaming

Video-based remote desktop specialized for Games running in cloudHigh-definition real-time game play

Game servers

Internet

Streaming

Streaming

StreamingPC

Laptop

Mobile


The Selling Points

Gamers’ perspectivesFrees gamers from indefinitely upgrading their computersEnables gamers to play games anywhere, anytime

Game manufacturers’ perspectivesAllows developers to support more platformsReduces the production costPrevents pirating

Cloud gaming is expected to lead the future growth of computer games: 9 times in 6 years

Cloud Gaming is Hot

[CGR] http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf


Challenges


Challenge #1

Unavoidable extra delaysVideo encoding at the serverVideo decoding and playout buffering at the client

Less opportunities for delay compensationGame states (e.g., game objects’ positions and velocity) sare not available at the client side

A Comparison with “Traditional” Online Games


Challenge #1 (cont.)

OnLive dictates a server rendering/processing latency of nearly 100 ms, and partially copes with it by setting up 7data centers merely in North America

Only people who live in 1000 mile radius from a data center are encouraged to playSimilarly, Sony/Gaikai has 8 data centers in NA

(Figure courtesy of Mark Claypool from “Latency and Player Actions in Online Games)


Challenge #2

For a regular x264 zerolatency implementation, 3--5 Mbps is required for a quality 720p cloud gaming session (on desktop / TV)

Playout buffering is commonly used to absorb packet delivery disorders (loss, re-orders) not applied here as short latency is critical


Challenge #3

Investing thousands of cloud servers was partly the reason for OnLive’s bankruptcy in 2012.

GPU virtualization is getting more mature, but the degree of multiplexity is still around 10—20

i.e., to support 10000 current users, 500—1000 servers are required


Challenge #3 (cont.)

The state of the practice

OnLive Sony NVIDIA ODM

Specification 2 MB in 2U 4 PS4 MB in 1U 2U with 6 Graphic cards 2 MB in 1U # GPU 2 4 12 8GPU/U 1 4 6 8


Outline

An Open-Source Cloud Gaming Testbed

Quantifying the Susceptibility of Games to Latency

Quantifying User Satisfaction in Mobile Cloud Games

QoE-aware Auto-Reconfiguration

Placing Virtual Machines to Optimize Cloud Games

Future Perspectives


An Open-Source Implementation

Researchers have tons of ideas to improve cloud gaming services, but all existing cloud gaming systems are proprietary and closed


System Architecture

The client and the server, with many componentsImplement by leveraging open-source packages


Process Video Frames in Parallel

Suppose the targeted inter-frame delay is ∆tThe response delay may greater than ∆t

frame capture + color space conversion + encoding

It could degrade encoding bitrateProcess in parallel


Video Playout Buffering

The 1-frame buffering strategyBased on the RTP marker bitAn H.264 frame can be split into different numbers of packetsThe marker bit (with a value of 1) indicates the last packet of a frame


GA Has Lower Response Delay

Low response delay * network delay has been excluded forFAIR comparisons

GA Provides (Relatively) Better Video Quality


http://gaminganywhere.org/

56k+ visitors, 100k+ downloads since April 2013

Visitor Distribution

Geo-distribution

/ Day

July2015


Outline






Future Perspectives


The Question

Are games equally susceptible to

latency?


Definition

Real-time strictness (RS)The degree a game’s QoE degrades when the latency is higher

Cloud-gaming friendlinessA cloud game’s susceptibility to latency in terms of its QoE


Selected Games

ACTLEGO Batman (Batman)Devil May Cry (DMC)Sangoku Musou 5 (Dynasty Warriors 6) (SM5)

FPSCall of Duty: World at War (COD)F.E.A.R 2 (FEAR)Unreal Tournament 3 (Unreal)

RPGYs Origin (Ys)Loki: Heroes of Mythology (Loki)Torchlight (Torch)


Facial EMG approach

1. Continuous emotion measures (can be at a rate of 1000 Hz or even higher)

2. Does not disturb game play

3. Objective since the emotional indicators are directly measured rather than told by subjects

(EMG: Electromyography)


Facial EMG Measurement Setup

The corrugator supercilii muscle

Negative emotions

The amount of annoyance caused by latency


Measurement devices

PowerLab 16/30

Electrodes

Wires


During game play…


Trace Summary

Subjects

Trace


Overall EMG potentials


EMG Potentials for each game

1. Diverse baseline EMG potentials for each game2. The increasing rates of EMG potential are game-dependent as

well


Deriving real-time strictness (RS)


RS of the studied games

In general, FPS > RPG > ACT in terms of RSGame pace↑, RS↑, latency-critical↑


Our conjecture

How a game responds to players’ commands is associated with its real-time strictness

If its commands are “lightweight”Simple, fast, local moves Timing is important higher RS

If its commands are “heavy”Associated with long and large amounts of animationsTiming is not critical lower RS


Illustrations for “light” commandshttps://www.youtube.com/watch?v=ycYDDBKrv4I

https://www.youtube.com/watch?v=ycYDDBKrv4I


Illustrations for “heavy” commandshttps://www.youtube.com/watch?v=GGm1YNJNWbo

https://www.youtube.com/watch?v=GGm1YNJNWbo


RS prediction


Application #1: Balance games’ QoE degradation due to latency

ScenarioN users are playing different games at the same timeUsers experience different latencies and games have different RS Each player experiences different levels of QoE degradation

UsageUse the model to infer which players are having a worse gaming experience than othersPrioritize the server’s resources, such as CPU and GPU, to reduce those players’ latencies and thereby mitigate QoE degradation they would otherwise experience


Application #2: Co-optimizing data center cost and gaming experience

ScenarioN data centers, each has distinct operation cost (electricity and labor)Whenever a user signs in, we need to assign a data center to him for real-time game playQuestion: Which data center should we assign to the player?

UsageUse the model to predict users’ QoE in all the cases and choose the data center which provide a “just good enough” gaming experience

Data center A: Lower cost, longer delay

Data center B: Higher cost, shorter delay


Outline






Future Perspectives


Mobile games are !in 2011, 59% smartphone users played mobile games [1]by 2016, mobile game market will grow to 16 billion USD [2]

Mobile games are less visually appealing, because of the limitations on

CPU/GPU powermemory space/speedbattery capacity

Possible solution: mobile cloud gaming

Mobile Games

[1] http://www.infosolutionsgroup.com/popcapmobile2012.pdf[2] https://www.abiresearch.com/research/product/1006313-mobile-gaming

http://www.infosolutionsgroup.com/popcapmobile2012.pdf

https://www.abiresearch.com/research/product/1006313-mobile-gaming


Testbed for User Studies

Nintendo 64 Limbo


DemoOnline

http://youtu.be/BFpThxOnlU8


Questions

Is mobile cloud gaming energy efficient?

How to tune video parameters in an energy-conserving way?

What components are energy-hungry?

Mobile gaming experience comparable to PC?


Cloud gaming is energy efficient

Independent of game genres Energy saving (50% in CPU and 30% in energy)


Energy consumptionsImpact of tunable parameters

Frame rate > Bit rate > Resolution

3G consumes 30%--45% more energy than WiFiInput event processing incurs non-trivial energy consumption


Comparison on Gaming ExperiencePCs have many

physical keys Implementations are

efficientReally? Mobile is

better?


Why Mobile Performs Better in Graphics?

First, subjects may have lower expectation on graphics of mobile devicesSecond, smaller screen sizes make graphics imperfection less noticeable

Observation: The satisfaction levelsare based on observed flaws thanabsolute quality!


Outline






Future Perspectives


The need for auto reconfigurationThe provided QoE is normally poor when our video packets experience loss events

We will have to voluntarily reduce bandwidth usage when network is (temporarily) overloaded

Due to network dynamics, the provisioning of network bandwidth may vary in sub-seconds

An automatic reconfiguration mechanism is required that can respond to changes in run time


Our Goal

Assuming N users playing different games

A mechanism to select the best (bitrate, frame rate) configuration for each user given the current game he/she is playing

Two explicit objectivesMaximize the average gaming experience (i.e., utilitarian)Maximize the worst gaming experience (i.e., fairness)


Crowdsourced user study


QoE vs. QoS factors

Our intuitionsBitrate , frame rate graphics quality Frame rate interactivity


Game Genre MattersAction Game

Car Racing Game


Many cloud gaming users share a bottleneck link to a data center

Maximize average MOS by choosing bitrate and frame rate for each user

Problem Formulation


The proposed system

A passive bandwidth estimator for 802.11 networkA quadratic QoE model for each game

An approximate algorithm for solving the optimization problem efficiently


Achieved Performance(Efficiency = MOS score / bandwidth consumed)

(Running time in seconds)


Outline






Future Perspectives


The Research Problem

Assuming each VM handles one game sessionConsolidating VMs in different ways results in different profits and gaming quality

For example, different data centers have different prices and offer different quality of service

Hence, we propose VM placement policies to maximize the profits or gamer QoE


Notations• Frame per Second: • Processing Delay: • Network Latency: • CPU Utilization:• GPU Utilization:• Hourly fee:• Operational Cost: • Memory of Server:• Uplink of Datacenter:


Problem Formulation: Provider Version

Objective Function: Maximize Profits

Constraint: QoE Degradation

Frame Per Second

Delay

Decision variable:

……


Quality-Driven Heuristic (QDH): Provider Version

Intuition: put as many VMs on a server as possibleCondition: Do not exceed the user-specified maximal tolerable QoE degradation


QDH’: Gamer Version

A similar formulation but here we minimize QoE degradation as possible

Objective function:


Our Testbed

Physical ServersCPU: i5 GPU: NVIDIA Quadro 6000Memory: 16GB

BrokerCPU: i7 3.2 GHzMemory: 16GB

ClientsCPU: i5Memory: 4 GB


Baseline AlgorithmLocation Based Placement (LBP) algorithmplaces each VM on a random game server that is not fully loaded and the data center geographically closest to the gamer


QDH Increases Profits

Save money (by shutting down more servers and relocating servers to a less expensive data center)

Always satisfy the specified QoE requirement


QDH’ Improves QoE

Outperforms LBP algo. by providing much higher QoE


Both Algorithms Run in Real Time

Both algorithms terminate in < 2.5 sec on a commodity PC even for large services with 20,000 servers and40,000 gamers

Outline






Are We There Yet?


Technical Reasons

Technical Reason #1

Explore possible next states Render possible frames and send to userUser chooses one based on inputManage to hide latency up to 384 ms at the cost of 4.5x higher bandwidth (and extra computation/rendering cost)

[1] Chu, K. L. D., Cuervo, E., Kopf, J., Grizan, S., Wolman, A., & Flinn, J. Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Cloud Gaming, ACM MobiSys 2015.

Pre-render future frames seems possible


Technical Reason #2

Objects that are far away or near peripheral vision can be coded with fewer bitsLeads to ~50% bit rate reduction with 4.75% MOS reduction

[1] Ahmadi, H., Khoshnood, S., Hashemi, M. R., & Shirmohammadi, S., Efficient bitrate reduction using a Game Attention Model in cloud gaming. In IEEE HAVE 2013.

Game info (e.g., camera and object positions) can be used to better encode


Technical Reason #3

GPU virtualization is getting more matureNVIDIA and AMD design specialized GPUs and drivers for cloud gamingCloud-gaming-friendly game engines would further boost the scalability (by planned GPU & VRAM sharing, etc)

Degree of multiplexity keeps increasing


Marketing Reasons

As a complement, rather than a replacement solutionE.g., Playstation Now uses cloud gaming to provide backward compatibility and cross-platform support

As a playable adStartups such as mNectar, Agawi, Voxel, provide playable ad services (mainly for mobile apps)


Marketing Reasons (cont.)

B2B2C business modele.g., G-cluster Global provide turnkey solutions to telecom operators around the world

to solution providers: almost risk-free and more scalableto local service providers: low-cost investment as they can use existing infrastructuresSeems a sustainable model which is key to success


Game IntegrationVideo CodecVirtualizationUser Interface

QoE Measurement and ModelingServer SelectionParameter AdaptationResource Scheduling

[1] Kuan-Ta Chen, Chung-Ying Huang, and Cheng-Hsin Hsu, "Cloud Gaming Onward: Research Opportunities and Outlook," Proceedings of IEEE C-Game 2014, July 2014.


Conclusion

Cloud gaming shares similar fundamental problems with many interesting applications

ScreencastingMobile smart lensTele medicineImmersive remote communications

Thus, cloud gaming seems a rewarding entrance to fundamental multimedia system challenges!


My special thanks to…

GamingAnywhere team

Dr. Chun-Ying Huang Dr. Cheng-Hsin Hsu Chih-Fang Hsu

Hua-Jun Hong Ching-Ling Fang Tsung-Han Tsai


Kuan-Ta ChenAcademia Sinica

cloud gaming rocks!

Thank You!

http://www.iis.sinica.edu.tw/~swc

http://www.iis.sinica.edu.tw/%7Eswc