Narrowing the Beam: Lowering Complexity in Cellular Networks by Scaling Up

Energy Management in the Cinder Operating SystemControlling energy allocation is crucial feature for mobile OS'sIntroduces abstraction of reserves and tapsModification of HiStar OS running on ARM processor (Android G1)

Used to achieve 3 properties of controlIsolationDelegationSubdivision 1

Smart Phone Virtualization: on Servers/PCsBluestacks (Demo)MegaDroidOff-the-shelf PCs emulates a town of 300,000 Android phones Emulate cellular and GPS behaviorUsage: Tracing the wider effects of natural disastersHacking attemptsSoftware bugsAssist real phone (e.g. offload computation, virus check)Video: http://www.engadget.com/2012/10/03/sandia-labs-megadroid-project-simulates-300-000-android-phones/

Smart Phone Virtualization: on DevicesCells video: http://www.youtube.com/watch?v=i_F92AS9a6s

Personal Phone

Business Phone

Developer Phone

Childrens PhoneCourtesy: Jason Nieh et al.Bare-Metal HypervisorOSKernelOSKernelOSKernel

Hypervisor / VMMHardwareServer Virtualizationpoor device support / sharingCourtesy: Jason Nieh et al.OSOSHost OS KernelOSHypervisor / VMM

Hosted HypervisorDesktop VirtualizationkernelmoduleHardware

host user spacepoor device performanceemulateddevicesCourtesy: Jason Nieh et al.OS KernelUser Space SDKNon-Virtualization

no standard appsless securecustom user space API for isolated appsHardware

Courtesy: Jason Nieh et al.

device diversity

mobile usage modelgraphics-accelerated UIKey ChallengesPowerButtonsWiFiGPSCell RadioFramebufferGPUBinder IPCTouchscreenAccelerometerCompasspmemmicrophoneheadsetspeakerscamera(s)h.264 accel.

RTC / AlarmsCourtesy: Jason Nieh et al.CellsmicrophoneKey Observation

large: lots of windows/appssmall:one app at a time

Courtesy: Jason Nieh et al.LinuxKernel

VP 3

VP 2

VP 1

Single Kernel: Multiple VPsvirtualize at OS interfaceisolated collectionof processesCourtesy: Jason Nieh et al.LinuxKernelSingle Kernel: Device SupportPowerCell RadioBinder IPCAccelerometerCompasspmemspeakerscamera(s)hw codecRTC / AlarmsVP 3

VP 2

VP 1

ButtonsWiFiGPSFramebufferGPUTouchscreenmicrophoneheadset

Courtesy: Jason Nieh et al.LinuxKernelSingle Kernel: Device Supportall VPs access the same device simultaneouslyVP 3

VP 2

VP 1

SensorsInputAndroid...Audio/Video

RTC / AlarmsWiFiCell RadioFramebufferPowerGPUCourtesy: Jason Nieh et al.LinuxKernelPowerWiFiCell RadioFramebufferGPURTC / AlarmsSensorsInputAndroid...Audio/VideoDevice Namespacessafely, correctly multiplex access to devices

device namespacesVP 3

VP 2

VP 1

Courtesy: Jason Nieh et al.Cells+device namespacesforeground/ background=Complete, Efficient, TransparentMobile VirtualizationCourtesy: Jason Nieh et al.LinuxKernel

PowerWiFiCell RadioFramebufferGPURTC / AlarmsSensorsInputAndroid...Audio/VideoGPUFramebufferdevice namespacesCell Radioefficient basic graphics virtualizationhardware accelerated graphicsproprietary/closed interfaceCourtesy: Jason Nieh et al.

virtual addressesphysical addressesscreen memoryApproach 1: Single Assignment

FramebufferCourtesy: Jason Nieh et al.Framebuffervirtual state

screen memoryApproach 2: Emulated Hardware

emulated framebuffer

Courtesy: Jason Nieh et al.virtual addressesphysical addressesmux_fb

screen memory

VP 3

backgroundmux_fb presentsidentical deviceinterface to all VPsusing device namespacesswap virt addr mappings:point to different phys addr

Cells: Device NamespacesVP 1

FramebufferVP 2

backgroundforegroundCourtesy: Jason Nieh et al.

screen memoryAccelerated GraphicsMMUcontextVP 2

OpenGLVP 1

contextOpenGLOpenGLcontextVP 3

graphics virtual addressesphysical addressesFramebufferGPUVP: just a setof processes!process isolationCourtesy: Jason Nieh et al.

GPUMMUgraphics virtual addressesphysical addresses

screen memoryDevice Namespace + Graphics ContextcontextVP 2

OpenGLVP 1

contextOpenGLOpenGLcontextVP 3

background

foregroundbackgroundCourtesy: Jason Nieh et al.LinuxKernelDriversBaseband: GSM / CDMARilDVendor RIL

VoIPVoIPVoIP?Courtesy: Jason Nieh et al.LinuxKernelGSM/CDMARilDDual-SIM?Vendor RIL

GSM / CDMADriversDriversRilDVendor RILCourtesy: Jason Nieh et al.Root NamespaceLinuxKernelvendor APIDriversBaseband: GSM / CDMACells RILRilDVP 2

Cells RILRilDVP 1

RilDCells RILVP 3

Cells: User-Level Namespace ProxybackgroundbackgroundforegroundVendor RILproprietary hardware/software requires a well-defined interface.CellDCourtesy: Jason Nieh et al.More Infocellrox.com

cells.cs.columbia.edu

Courtesy: Jason Nieh et al.Revisiting Storage for Smartphones

Hyojun KimCristian UngureanuNitin Agrawal

32Understanding Mobile PerformanceNetwork performance can impact user experience3G often considered the bottleneck for apps like browsingService providers heavily investing in 4G and beyondCPU and graphics performance crucial as wellPlenty of gaming, video, flash-player apps hungry for computeQuad-core CPUs, GPUs to appear on mobile devices

Does storage performance impact mobile experience?For storage, vendors & consumers mostly refer to capacityWell understood!Not well understood!Courtesy: Nitin Agrawal et al.33Wireless Network Throughput Progression

Flash storage on mobile performs better than wireless networksMost apps are interactive; as long as performance exceeds that of the network, difficult for storage to be bottleneck

Standard (theoretical)Mobile Flash 802.11 a/g3G

Measured in LabCourtesy: Nitin Agrawal et al.34Introduction

Why storage is a problem

Android storage background and setup

Experimental results

SolutionsOutlineCourtesy: Nitin Agrawal et al.35Why Storage is a ProblemPerformance for random I/O significantly worse than seq; inherent with flash storageMobile flash storage classified into speed classes based on sequential throughputRandom write performance is orders of magnitude worseVendor(16GB)Speed ClassCost US $Seq WriteRand WriteTranscend2264.21.18RiData2277.90.02Sandisk4235.50.70Kingston4254.90.01Wintec62515.00.01A-Data63010.80.01Patriot102910.50.01PNY102915.30.01Consumer-grade SD performancePerformance MB/sHowever, we find that for several popular apps, substantialfraction of I/O is random writes (including web browsing!)Random versus Sequential DisparityCourtesy: Nitin Agrawal et al.36Storage coming under increasingly more scrutiny in mobile usageRandom I/O performance has not kept pace with network improvements802.11n (600 Mbps peak) and 802.11ad (7 Gbps peak) offer potential for significantly faster network connectivity to mobile devices in the future

Mobile Flash Rand Shifting Performance BottlenecksWhy Storage is a Problem

Standard (theoretical)

Mobile Flash Seq 802.11 A/G3GMeasured in LabCourtesy: Nitin Agrawal et al.37Deconstructing Mobile App PerformanceFocus: understanding contribution of storageHow does storage subsystem impact performance of popular and common applications on mobile devices?Performed analysis on Android for several popular appsSeveral interesting observations through measurementsStorage adversely affects performance of even interactive apps, including ones not thought of as storage I/O intensiveSD Speed Class not necessarily indicative of app performanceHigher total CPU consumption for same activity when using slower storage; points to potential problems with OS or appsImproving storage stack to improve mobile experienceCourtesy: Nitin Agrawal et al.38Introduction

Why storage is a problem

Android storage background and setup

Experimental results

SolutionsOutlineCourtesy: Nitin Agrawal et al.39Storage Partitions on Android

/systemyaffs2145MBread-only/cacheyaffs295MBread write/datayaffs2196.3MBread writeInternal NAND Flash Memory (512MB)/sdcardFAT3216GBread write

/misc896KBsettings/recoveryrootfs4MBalternate boot /bootrootfs3.5MBkernelExternal SD

PartitionFunctionMiscH/W settings, persistent shared space between OS & bootloaderRecoveryAlternative boot-into-recovery partition for advanced recoveryBootEnables the phone to boot, includes the bootloader and kernel SystemContains the remaining OS, pre-installed system apps ; read-onlyCacheUsed to stage and apply over the air updates; holds system imagesDataStores user data (e.g., contacts, messages, settings) and installed apps; SQLite DB containing app data also stored here. Wiped on factory resetSdcardExternal SD card partition to store media, documents, backup files etc Sd-extNon-standard partition on SD card that can act as data partitionCourtesy: Nitin Agrawal et al.40Custom Experimental SetupAbility to compare app performance on different storage devicesSeveral apps heavily use the internal non-removable storageTo observe and measure all I/O activity, we modified Androids init process to mount all internal partitions on SD cardMeasurement study over the internal flash memory and 8 external SD cards, chosen 2 each from the different SD speed classesObserve effects of shifting bottlenecks w/ faster wireless networksBut, faster wireless networks not available on the phones of today Reverse Tethering to emulate faster networks: lets the smartphone access the host computers internet connection through a wired link (miniUSB cable)Instrumentation to measure CPU, storage, memory, n/w utilizationSetup not typical but allows running what-if scenarios with storage devices and networks of different performance characteristicsRequirements beyond stock AndroidCourtesy: Nitin Agrawal et al.41Apps and Experiments PerformedWebBench BrowserVisits 50 websitesBased on WebKitUsing HTTP proxy server

App InstallTop 10 apps on Market

App LaunchGames, Weather, YouTubeGasBuddy, Gmail, Twitter, Books, Gallery, IMDB

RLBench SQLiteSynthetic SQL benchmark

Facebook

Android Email

Google Maps

Pulse News Reader

BackgroundApps: Twitter, Books, GmailContacts, Picasa, Calendar Widgets: Pulse, YouTube,News, Weather, Calendar,Facebook, Market, Twitter

Courtesy: Nitin Agrawal et al.42Introduction

Why storage is a problem

Android storage background and setup

Experimental results (talk focuses on runtime of apps)Paper has results on I/O activity, CPU, App Launch behavior, etc

SolutionsOutlineCourtesy: Nitin Agrawal et al.43WebBench Results: RuntimeRuntime on WiFi varies by 2000% between internal and Kingston Even with repeated experiments, with new cards across speed classesEven without considering Kingston, significant performance variation (~200%)Storage significantly affects app performance and consequently user experienceWith a faster network (USB in RT), variance was 222% (without Kingston)With 10X increase in N/W speed, hardly any difference in runtimeTime taken for iPerf to download 100MBWiFiUSBTime (s)Courtesy: Nitin Agrawal et al.44WebBench Results: RuntimeRuntime on WiFi varies by 2000% between internal and Kingston Even with repeated experiments, with new cards across speed classesEven without considering Kingston, significant performance variation (~200%)Storage significantly affects app performance and consequently user experienceWith a faster network (USB in RT), variance was 222% (without Kingston)With 10X increase in N/W speed, hardly any difference in runtimeTime taken for iPerf to download 100MBWiFiUSBTime (s)Courtesy: Nitin Agrawal et al.45Runtimes for Popular Apps (without Kingston)We find a similar trend for several popular appsStorage device performance important, better card faster apps

Apart from the benefits provided by selecting a good flash device, are there additional opportunities for improvement in storage?Courtesy: Nitin Agrawal et al.46

WebBench: Sequential versus Random I/OFew reads, mostly at the start; significantly more writesAbout 2X more sequential writes than random writesSince rand is worse than seq by >> 2X, random dominatesApps write enough randomly to cause severe performance dropPaper has a table on I/O activity for other appsI/O BreakdownVendorSeq:Rand perf ratioRand IOPSTranscend4302Sandisk8179RiData3955Kingston4902.6Wintec15002.6A-Data10802.6Patriot10502.6PNY15302.6Courtesy: Nitin Agrawal et al.47How Apps Use Storage?Exactly what makes web browsing slow on Android?Key lies in understanding how apps use SQLite and FS interface/data/data/com.necla.webviewlib (empty)cachewebviewCache6aaa3f00, 03051d8d, many files (5.5MB)databaseswebview.db (14KB)webviewCache.db (129KB)These files written to SQLite in syncThese files written to FS in write-behindWebBench Storage SchemaApps typically store some data in FS (e.g., cache files) and some in a SQLite database (e.g., cache map)All data through SQLite is written synchronously slow!Apps often use SQLite oblivious to performance effectsCourtesy: Nitin Agrawal et al.48What-If Analysis for SolutionsWhat is the potential for improvements?E.g., if all data could be kept in RAM?Analysis to answer hypothetical questionsPlacing Cache on Ramdisk does not improve perf. muchDB on Ramdisk alone improves perf. significantlyBoth Cache and DB in RAM no extra benefitBoth Cache and DB on SD without sync recoups most perfA. Web Cache in RAMB. DB (SQLite) in RAMC. All in RAMD. All on SD w/ no-syncWebBench on RiDataABCDCourtesy: Nitin Agrawal et al.49Implications of Experimental AnalysisStorage stack affects mobile application performanceDepends on random v/s sequential I/O performanceKey bottleneck is ``wimpy storage on mobile devicesPerformance can be much worse than laptops, desktopsStorage on mobile being used for desktop-like workloadsAndroid exacerbates poor storage performance through synchronous SQLite interfaceApps use SQLite for functionality, not always needing reliabilitySQLite write traffic is quite random further slowdown!Apps use Android interfaces oblivious to performanceBrowser writes cache map to SQLite; slows cache writes a lotCourtesy: Nitin Agrawal et al.50