THE C ASE FOR THE HOLISTIC L ANGUAGE R UNTIME S …maas/slides/wrsc14-maas-talk.pdf · Talk Outline 1. Holistic Runtime Systems Details, Advantages, Programming Model 2. Cloud Data

THE CASE FOR THE HOLISTIC LANGUAGE RUNTIME SYSTEM

Martin Maas* Krste Asanovic* Tim Harris John Kubiatowicz*

*University of California, Berkeley Oracle Labs Cambridge

Cloud Data Centers in 2020

3"

Talk Outline 1. Holistic Runtime Systems

Details, Advantages, Programming Model

2. Cloud Data Center Trends Holistic Runtimes tackle the challenges of 2020

3. Challenges & Future Work Research Directions & Opportunities

17"

PART I Holistic Runtime Systems

18"

Holistic Runtime Systems

21"

CPUs" Accelerators" DRAM"

High-speed Interconnect"

CPUs" Accelerators" DRAM"…"

Profiling"

JIT" GC"

Objects"

Program 1 Program 2"Per-node Runtime"

Node" Node"

Target accelerators"

Objects"

Threads"Profiling"

JIT" GC"

Objects"

Program 1 Program 2"

Objects"

Threads"

Bulk Storage"

Peripherals""

I/O, External Network Interface"

"

Fine-grained sharing by coordinating within each node and across (e.g. GC)"

"Dynamic service fusion/inlining/etc." Adding resources at finer granularity"

Ultra-low latency RPCs under load"

Holistic Runtime Systems

23"

CPUs" Accelerators" DRAM"

High-speed Interconnect"

CPUs" Accelerators" DRAM"…"

Profiling"

JIT" GC"

Objects"

Program 1 Program 2"Per-node Runtime"

Client #1" App A" App B" Client #2" App C" App D"Service Interfaces"

Coordinate GC"

Share JIT/profiling results"Transparent relocation of execution and data"

Schedule work on available runtime instances!

Distributed Runtime"

Node" Node"

Manage distributed heaps"

Failure tolerance"

Target accelerators"

Enforce SLAs"

Objects"

Threads"Profiling"

JIT" GC"

Objects"

Program 1 Program 2"

Objects"

Threads"

Bulk Storage"

Peripherals""

I/O, External Network Interface"

"

Application"

Programming Model

24"

Distributed Runtime System"

Service A" Service B"

Per-nodeRuntime"

Per-nodeRuntime"


Application"

Service C"

Bulk Storage"

Service C" Service A"

RDMA-style transfers

Service interfaces

Legacy Application Support

29"

Rack-scale Machine"

Legacy Code VMs"

Commodity OS"

Native"App"

Native"App"

Holistic Runtime System"

Java Language Compatibility Layer"

Python Language Compatibility Layer"

Java App" Python App"

Applicability • Would be useful today to run managed workloads on Infiniband clusters

• But really benefit from tightly coupled nodes and hardware support

• Excellent fit for future data centers 31"

PART II Cloud Data Center Trends

32"

Cloud Forecast for 2020

Hardware Workloads Languages

33"



35"

Chanpipat,"FreeDigitalPhotos.net"



Workloads Languages


36"

• Custom machines from commodity parts • Redundant/unused components (I/O interfaces, peripherals, etc.)

• Inefficiency in energy and hardware cost

Hardware



Languages


Hardware 37"

• Mostly developed in-house (e.g. Hotmail) • Some interactive, mostly batch jobs • Interleaved with external workloads

Workloads




Hardware Workloads 38"

• Workloads written by mixture of systems programmers and domain experts

• Mix of native and managed languages (external workloads often managed)

Languages

Critical Workloads in 2014 • Tune workloads to underlying cluster

• Provision lightly loaded nodes for jobs with low-latency requirements

• Write latency-critical applications in native languages (usually C++)

40"



41"


Workloads Languages Hardware 42"




Workloads Languages


• Volume of cloud market growing ! eco-nomically feasible to design custom SoCs

• Need to reduce energy and hardware cost • Software benefits from hardware support

Hardware 44"



Workloads Languages


Hardware 45"

CPU" CPU" x10B100"

Vector Core"

Accelerators"

Crypto"

Custom"SystemBonBChip"

SoC SoC SoC SoC …"

~1,000"SoC’s"

High-speed Interconnect

RAM RAM RAM RAM …"

NVM Storage NVM Storage NVM Storage

NIC"

RDMA unit (bulk storage)"



Languages


Hardware

• New workloads: e.g. sensor interactions, AR, live translation, remote gaming

• More interactive (require low-latency) • Mostly from external customers

Workloads 46"



Languages


Hardware Workloads 47"

Cloud Service Provider (Platform-as-a-Service) – Cloud API

Customer A


Customer B





Hardware Workloads

• Mostly written by external application developers (cloud will be main platform)

• Will almost exclusively use high-level languages and frameworks

Languages

48"


Implications

Mostly interactive

Productivity languages

Rack-scale machines

50"

Tune applications the underlying cluster


Implications

Mostly interactive


Rack-scale machines

51"

Tune applications the underlying cluster The Cloud is becoming more opaque ! fine-tuning infeasible (and not portable)


Implications

Mostly interactive


Rack-scale machines

52"

Provision lightly loaded nodes for jobs with low-latency requirements


Implications

Mostly interactive


Rack-scale machines

53"

Provision lightly loaded nodes for jobs with low-latency requirements

Radical over-provisioning will cease to be cost-effective ! sharing


Implications

Mostly interactive


Rack-scale machines

54"

Write latency-critical applications in native languages (usually C++)


Implications

Mostly interactive


Rack-scale machines

55"

Write latency-critical applications in native languages (usually C++)

Cloud will be exclusively programmed with high-level languages

Cloud Workloads written in managed languages, latency-sensitive and not tuned to the underlying platform

56"

Holistic Runtime Systems exploit rack-scale machines to run them efficiently

57"

PART III Challenges & Future Work

58"

Garbage Collection • Garbage Collection of tera- or peta-byte sized heaps unsolved problem

• Bulk+local storage (e.g. RAMCloud )

• Cross-node references ! Distributed Garbage Collection

59"

Fault Isolation • Faulting application or SoC must not bring down rack-scale machine

• Isolation/lifecycle support in Java: JSR-121, Multi-tasking VM

• Potential for HW support (Mondriaan ) 60"

Performance Guarantees • Probabilistic performance and tail latency guarantees for service calls

• High-level goals (e.g. Tessellation OS )

• Need predictable GC performance (HW support is work in progress)

61"

Conclusion

62"

Conclusion • Cloud data centers are changing:

• Rack-scale machines, interactive/ex-ternal workloads, managed languages

• Current software stack is a bad fit

• Are Holistic Runtimes the solution? 63"

Thank you! Any Questions?"

Martin Maas, Krste Asanovic, Tim Harris, John Kubiatowicz

[email protected], [email protected], [email protected], [email protected]

64"

Backup Slides

65"

Frameworks & Extensibility

66"

Application"

Storage APIs"Parallel

Computing APIs"Replication APIs"

StorageLibraries"

Job-scheduling & Resource Allocation"

Unified Failure Handling"

Application Developers"

System Programmers"

Auto-Tuners"

HolisJc"Language"RunJme"System"

Application"

Why Managed Languages? • Much better productivity and safety

• Abstract away hardware details and can transparently tune to platform

• Semantics allow fine-grained sharing

• Good for service-oriented architecture 68"

Programmability Crisis • Productivity programmers...

• ...programming for an increasingly complex but opaque platform...

• ...with strict latency requirements under high sharing of machines

67"

Problems with current stack • Current software stack is a bad fit:

• Interference: Intra- and inter-node

• Redundancy: JIT, class library, etc.

• Composability: RPC latencies

• Elasticity: Start-up/boot times 71"

Documents

THE C ASE FOR THE HOLISTIC L ANGUAGE R UNTIME S …maas/slides/wrsc14-maas-talk.pdf · Talk Outline 1. Holistic Runtime Systems Details, Advantages, Programming Model 2. Cloud Data