Upload
hadassah-ware
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
S(o)OS in a Nutshell. Towards the Large-Scale OS. Structure of this presentation. Content. Background / Introduction: current problems of OS and HPC / tightly coupled systems The basic idea behind S(o)OS a quick overview over the main concepts Project structure & management approach. - PowerPoint PPT Presentation
Citation preview
S(o)OS
Realising resource-independent execution support on tera-device systems
S(O)OS IN A NUTSHELLTowards the Large-Scale OS
Presenter
04/19/20232
S(o)OSService-oriented Operating Systems
1. Background / Introduction: current problems of OS and HPC / tightly coupled systems
2. The basic idea behind S(o)OSa quick overview over the main concepts
3. Project structure & management approach
CONTENTStructure of this presentation
S(o)OSService-oriented Operating Systems
BACKGROUNDissues for large scale systems
Presenter
04/19/20234
S(o)OSService-oriented Operating Systems
• We face systems with– hundred of thousands of cores– Heterogeneous cores– Accelerators attached using wide range of technologies– Myriad of connection options
• Hypertransport/Quickpath• PCIExpress• Gigabit Ethernet• Infiniband• Lot of vendor specific connectivity
• This systems cannot easily be programmedThe potential cannot be exploitedCurrent approaches are working on the symptoms (PGAS, CUDA,
OpenCL, etc.)
HPC TRENDSBrief overview over current
Presenter
04/19/20235
S(o)OSService-oriented Operating Systems
• Focus on homogeneous resource infrastructures• Scale well wrt. processes but not wrt. processing
units• Are essentially centralistic => bottleneck, OS jitter ... Future environments will be
large-scale, heterogeneous and potentially widely distributed
Current OS architectures can not deal with this
ISSUESCurrent Operating Systems
Presenter
04/19/20236
S(o)OSService-oriented Operating Systems
COMMUNICATION
Current Operating Systems
Proc. Unit
Proc. Unit
Cach
eCa
che
Proc. Unit
Assign Process
Ope
ratin
g Sy
stem
Procedure CallContext SwitchMemory Access
Cross Core Comm.Load ProcessProcess Load
Assign ProcessProcedure CallContext SwitchMemory AccessCross Core Comm.Load ProcessProcess Load
Massive communication and management overhead
Presenter
04/19/20237
S(o)OSService-oriented Operating Systems
• Require a lot of knowledge about– The resource infrastructure– The relationship between algorithm and data– The potential distribution of the algorithms and – Its connectivity / communication– Etc.
Large scale tightly coupled systems will become a common good
Programming models must become more efficient and manageable
ISSUESDistributed Programming
S(o)OSService-oriented Operating Systems
TOWARDS S(O)OS
Revising OS Architectures
Presenter
04/19/20239
S(o)OSService-oriented Operating Systems
• Distribute operating system and code across resource infrastructure
• Rearrange running code and operating system according to– Availability of resources– Requirements of code– Linkage between data
GENERAL CONCEPT
Presenter
04/19/202310
S(o)OSService-oriented Operating Systems
OVERALL CONCEPT
Virtu
al M
emor
y SegmentedCode
deploymentresourcecapabilities
virt
ual e
xecu
tion
ProcessCode
actu
al e
xecu
tion
resourcerequirements
resourcerequirements
resourcerequirements
Phys
ical
Mem
ory
match
resourcecapabilities
resourcecapabilities
Presenter
04/19/202311
S(o)OSService-oriented Operating Systems
REAL VON NEUMANN
PU
ALU Control
In / Out
MU
Memory Control
In / Out
Data Bus
In / Out
Data Bus
Control
Memory
ALU
Presenter
04/19/202312
S(o)OSService-oriented Operating Systems
THE OS MONSTER
Presenter
04/19/202313
S(o)OSService-oriented Operating Systems
• Limited cache size• Communication is costly• Resource environment is dynamic or changes
between executions• Data consistency• Code consistency
OBSTACLES
Presenter
04/19/202314
S(o)OSService-oriented Operating Systems
• Distributed, Self-Managed Microkernels– Following the SOA / Grid principle OS functionalities are
separate “services”– “Code is where the data is” – Dynamic composition of elements according to code
segment requirements
BASIC PRINCIPLES 1
Scheduler Job Mgr
HAL I/OM
emor
yFile Management
Presenter
04/19/202315
S(o)OSService-oriented Operating Systems
• Runtime Code Behaviour Analysis – Identify code parts with strong
relationships– Primary and secondary data-
sets– OS relation– Distribute segments according
to requirements and availability
– Annotate memory with analysis results
BASIC PRINCIPLES 2
OS Modules
Code Segments
Procedure Calls etc.
Self-referencing
OS1 OS2
Annotated Virtual Memory
P1.B1
P1.B2
P2.B1
P2.B1
P1.B2P1.B1
D.B2D.B1
w:0.8
w:0.5w:0.9
w:0.7 w:0.3 w:0.6
f:0.5
f:0.2
f:0.8f:0.3
w:0.9
w:0.3
f:0.9
Address Type
&x00000000
Process 1
&x00000100&x00000200
&x00000300
&x00000400&x00000500&x00000600
Process 2&x00000700&x00000800
...
“Hacking” ApplicationsAddress Add. Inf. Type
&x00000000
accessed from: &x00000600&x00000C00&x00000F00
calls:&x0002AB00&x001D1F00
reads from: &x00011F00&x000BAE30
writes to:&x000BAE30
Process 1
&x00000100 accessed from: &x00000800
jumps to:&x00000600 ...
&x00000200&x00000300&x00000400&x00000500&x00000600
Process 2&x00000700&x00000800
...
Presenter
04/19/202317
S(o)OSService-oriented Operating Systems
• Distributed Execution Model– Whole code can be distributed, not just threads– Execution context may move between cores– Distribution may vary with infrastructure– Essential distribution information is maintained with code– Reduce communication overhead
BASIC PRINCIPLES 3
Virtual process space
Exec Env Exec Env
Presenter
04/19/202318
S(o)OSService-oriented Operating Systems
• New OS architectures / paradigms• New approaches and algorithms to deal with future
distributed execution systems • Proof-of-concept implementation of distributed
execution support tools
PROJECT GOALS
S(o)OSService-oriented Operating Systems
MANAGING S(O)OS
Notes on Project Structure
Presenter
04/19/202320
S(o)OSService-oriented Operating Systems
Two main strands:• Design Strand
Development of algorithms, architectures, reference implementations– WP2: looks at the development from a hardware perspective– WP3: examines (communication) protocols– WP4: deals with distributed execution models
• Integration & Testing StrandAligns the models, performs integrated, application related tests– WP5: OS model– WP6: Application
WORK PACKAGES
Presenter
04/19/202321
S(o)OSService-oriented Operating Systems
• HLRS, University of Stuttgart
• Instituto de Telecomunicações Aveiro
• RETIS Lab, Sant'Anna School of Advanced Studies
• CTIT, Universiteit Twente• Ecole Polytechnique
Fédérale de Lausanne• European Microsoft
Innovation Centre
PARTICIPANTS