View
214
Download
0
Category
Tags:
Preview:
Citation preview
1Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Interactive Problem Solving: The Polder Meta Computing Inititiative
Peter Sloot
Computational Science
University of Amsterdam, The Netherlands
2Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Ariadne’s Red-Rope
– From PSE to Virtual Laboratory and Motivation
– Architecture• Infrastructure• Job Level: Hierarchical Scheduling• Resource management: Task-migration
– Interaction && Case implementation
– Interactive Algorithms
3Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Virtual Laboratory Environment
Internet 2 Wide Area Network
ViSE
Net Client App. User MRI/CT
Distributed Computing & Gigabit Local Area Network
Local User
Local User
Physical apparatus
Virtual-lab Information Management for Cooperation (VIMCO)
Communication & collaboration (ComCol)
Virtual Simulation & Exploration Environment (ViSE)
Advanced Scientific Domains
Computational Physics
System Engineering
Computational Bio-medicine
4Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Interactive Computing: Why?– Goal: From Data, via Information to Knowledge
– Complexity: Huge data-sets, complex processes
– Approach: Parametric exploration and sensitivity analyses:• Combine raw (sensory) data with simulation• Person in the loop:
• Sensory interaction • Intelligent short-cuts
5Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Intro: Case study from biomedicine...
6Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Changing the Paradigm
In Vivo
In Vitro
In Silico
7Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Changing the Paradigm
In Vivo
In Vitro
In Silico
8Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Changing the Paradigm
In Vivo
In Vitro
In Silico
9Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Current Situation
Observation
Diagnosis & Planning
Treatment
10Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
New Possibilities in the VL
Fast, High-throughputLow Latency
Internet
High Performance Super Computing
• Time and Space Independence
• 3D Information
• Simulation based planning
• Surgeon ‘in the loop’
11Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Experimental set-up
12Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Architecture
13Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Architecture Continued: Hybrid system
– Host: The DAS• 24 node parallel cluster in a
200 node wide area machine
• 200 MHz Pentium Pro
• Myrinet 150MB/s
• ATM wide-area interconnect between clusters
9 10 11
8
7
6
141312
15
16
17
2 1 0
3
4
5
212223
20
19
18
GRAPE1 GRAPE0
ATM
Origine 2000
Cave
14Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Immersive Environments
15Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
3D Information and Interaction
16Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Problem: Curse of dynamics:
Static task load Dynamic task load
Static resource load Dynamic resource load
Static task allocation Predictable
reallocationDynamical reallocation
17Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Solution To Curse
– Performance of a parallel program usuallydictated by slowest task• Task resource requirements and available resources
both vary dynamically• Therefore, optimal task allocation changes• Gain must exceed cost of migration
– Resources used by long-running programs may be reclaimed by owner
18Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Dynamite Initial State
Two PVM tasks communicating through a network of daemonsMigrate task 2 to node B
Node A Node B
Node C
PVMDA
PVMDB
PVMDC
PVMtask 1
PVMtask 2
19Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Prepare for Migration
Create new context for task 2Tell PVM daemon B to expect messages for task 2Update routing tables in daemons (first B, then A, later C)
Node A Node B
Node C
PVMDA
PVMDB
PVMDC
PVMtask 1
Program
PVM
Ckpt
Newcontext
20Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Checkpointing
Send checkpoint signal to task 2Flush connectionsCheckpoint task to disk
Node A Node B
Node C
PVMDA
PVMDB
PVMDC
PVMtask 1
Program
PVM
Ckpt
Newcontext
21Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Cross-cluster checkpointing(design)
Send checkpoint signal to task 2Flush connections, close filesCheckpoint task to disk via helper task
Node A Node B
Node C
PVMDA
PVMDB
PVMDC
PVMtask 1
Program
PVM
Ckpt
Helpertask
22Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Restart Execution
Restart checkpointed task 2 on node BResume communicationsRe-open & re-position files
Node A Node B
Node C
PVMDA
PVMDB
PVMDC
PVMtask 1
NewPVMtask 2
23Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Special considerations
– Preserve communication• PVM should continue to run as if nothing happened• Use location independent addressing
– Open files• Preserve open file state
24Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Performance
– Migration speed largely dependent on the speed of shared file system• and that depends mostly on the network
– NFS over 100 Mbps Ethernet• 0.4 s < Tmig < 15 s for
2 MB < sizeimg < 64 MB
– Communication speed reduced due to added overhead• 25% for 1 byte direct messages
• 2% for 100 KB indirect messages
25Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Current status: Dynamite Part
– Checkpointer operational under• Solaris 2.5.1 and higher (UltraSparc, 32 bit)
• Linux/i386 2.0 and 2.2 (libc5 and glibc 2.0)
– PVM 3.3.x applications supported and tested• Pam-Crash (ESI) - car crash simulations• CEM3D (ESI) - electro-magnetics code• Grail (UvA) - large, simple FEM code• NAS parallel benchmarks• BloodFlow
– MPI and socket (Univ. of Krakow) libraries available– Scheduling not yet satisfactory
26Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Architecture: Revisited
27Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Design Considerations
– High Quality presentation
– High Frame rate
– Intuitive interaction
– Real-time response
– Interactive Algorithms
– High performance computing and networking...
28Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Problem: Time, time what has become of us?
29Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Solution: Asynchronicity
30Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
A police officer to guide the asynchronous processes
31Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Runtime Support
– Need generic framework to support modalities
– Need interoperability
– High Level Architecture (HLA):• data distribution across heterogeneous platforms• flexible attribute and ownership mechanisms• advanced time management
32Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Provoking a bit…
Progress in natural sciences comes from taking things apart ...
Progress in computer science comes from bringing things together...
33Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Proof is in the pudding...
– Diagnostic Findings
• Occluded right iliac artery
• 75% stenosis in left iliac artery
• Occluded left SFA
• Diffuse disease in right SFA
34Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Problem: From Image to Simulation
MR Scan of Abdomen MR Scan of Legs
35Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Solution: 3DManual initialization
Place start point
Wave propagates from start- to end point
Place one or more end points
Backtrack = first estimation of the
centerline
Distance Transform from vessel wall to center centerline
Wave propagates from ‘centerline’ vessel
wall
36Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Wavefront PropagationPlace start point
Wave propagates from start- to end point
Place one or more end points
Backtrack = first estimation of the
centerline
Distance Transform from vessel wall to center centerline
Wave propagates from ‘centerline’ vessel
wall
37Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
MRA: BacktrackPlace start point
Wave propagates from start- to end point
Place one or more end points
Backtrack = first estimation of the
centerline
Distance Transform from vessel wall to center centerline
Wave propagates from ‘centerline’ vessel
wall
38Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
MRA: Wavefront PropagationPlace start point
Wave propagates from start- to end point
Place one or more end points
Backtrack = first estimation of the
centerline
Distance Transform from vessel wall to center centerline
Wave propagates from ‘centerline’ vessel
wall
39Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
MRA: Distance TransformPlace start point
Wave propagates from start- to end point
Place one or more end points
Backtrack = first estimation of the
centerline
Distance Transform from vessel wall to center centerline
Wave propagates from ‘centerline’ vessel
wall
40Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
3-D selection of region of interest
41Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Tracking the vessels
42Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Building the Geometric Models
43Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
VR-Interaction
44Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Alternate Treatments
Angio w/ Fem-Fem &
Fem-Pop
AFB w/ E-S Prox.
Anast.
Angio w/Fem-Fem
AFB w/ E-E Prox.
Anast.
Preop
45Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Problem: Flow through complex geometry
– After determining the vascular structure simulate the blood-flow and pressure drop…
– Conventional CFD methods might fail:• Complex geometry• Numerical instability wrt interaction• Inefficient shear-stress calculation
46Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Solution to interactive flow simulation
– Use Cellular Automata as a mesoscopic model system:• Simple local interaction• Support for real physics and heuristics• Computational efficient
47Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Mesoscopic Fluid Model
– Fluid model with Cellular Automata rules
– Collision: particles reshuffle velocities
– Imposed Constraints• Conservation of mass• Conservation of momentum• Isotropy
Details...
48Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
...Equivalence with NS– For lattice with enough symmetry: equivalent to the continuous incompressible Navier-Stokes equations:
uuu
u
2 1
0
Pt
u
Implicit parallel and complex geometry support.
49Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Efficient Calculation of Shear-Stress
AND the momentum stress tensor that is linearly related to the shear stresses
i
if i
iif cu
i
iiif ccΠ
x
u
x
u
~
Perpendicular momentum transfer:
From LBE scheme:
50Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Velocity Magnitude
10 cm/sec
0 cm/sec
51Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Peak Systolic Pressures - Rest
150 mmHg
50 mmHg
Angio w/ Fem-Fem &
Fem-Pop
AFB w/ E-S Prox.
Anast.
Angio w/Fem-Fem
AFB w/ E-E Prox.
Anast.
Preop
52Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
… last slides...
53Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Internet and Web Software
Distributed Computer infrastructure
Central-part Virtual Laboratory
Physical Apparatus
User
ViSE ComCol VIMCO
Internet and Web Software
Distributed Computer infrastructure
Central-part Virtual Laboratory
User
Computing in Physics
VL for Material Science
ViSE ComCol VIMCO
Meta data Integration
Computing in Engineering
Traffic Payment for mobility
Combining problem solving & data intensive
environments
Bio-medicalComputation
Study of blood flow through
veins
Integration of simulation &
visualization by man in the loop
Bio- informaticsEnvironment
DNA Research
Combing datamining & intelligent data bases
Cultural Inheritance
Environment
Art objects preservationrestoration
Collaborative data
integration
Computing in Engineering
Apply VL in non-quality of service
environment
Modeling VL in non-QoS situation
environment
Other Virtual Laboratory Applications @ UvA
54Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
AcknowledgementsStanford:
Charley Taylor, PhD.
Christopher K. Zarins, PhD. M.D.
UvA:
Robert Belleman
Alfons Hoekstra, PhD
Dick van Albada, PhD
Benno Overeinder, PhD
Krakow
Marian Bubak, PhD
Kamil Iskra
RUL/AZL:
H. Reiber, PhD.
Bloem, PhD, M.D.
SARA:
A. de Koning, PhD.
Arcobel:
S. ten Den
IBM:
J. Geise
55Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Support
ICES-KIS-1
ICES-KIS-2
KNAW
NWO/FOM
IBM
SARA
SGI
Platform HPCN
56Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
http://science.uva.nl/~sloot
sloot@science.uva.nl
57Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
58Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
1955 1965 1975 1985 19950.1
1
10
100
1000
10.000
100.000
2005
MFlop/s
1000.000
IBM 704
CDC 6600
Cray X-MP
Cray Y-MP
CM-5
ASCI-RedASCI-Blue
2D Plasma
48 hr Weather
72 hr Weather
Pharmaceutical
?
Structural Biology
Oil reservoir
59Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Results - Mean Flow Rates (ml/min) - Rest
Artery Preop AFB/E-S AFB/E-E Angio/FF Angio/FF-FP
Abd. Aorta 3009 3460 3460 3460 3460
Aortic Bifurc. 670 307 15 1310 1385
R. Int. Iliac 134 156 212 144 132
L. Int. Iliac 464 148 150 335 360
R. Femoral 216 615 663 444 546
L. Femoral 464 550 781 747 618
R. Popliteal 30 283 309 162 303
L. Popliteal 272 351 362 290 277
60Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Cellular Automata
– 1966 Introduced by John von Neumann
– 1985 Stephen Wolfram suggested CA are capable of Universal Computation
– 1990 Lindgren et al., proved UC in 1D CA
61Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
1 0 1 0 0 1 1 0 0t=0
1 1 1 0 1 1 1 0 1t=1
0000
0011
0101
0111
1000
1011
1101
1110
Productie Regel 110
Time Evolution of 1D Cellular Automata 110
Back to Mesoscopic Models
63Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
The Lattice Gas model
– Fluid model with Cellular Automata rules
– Collision: particles reshuffle velocities
– Imposed Constraints• Conservation of mass• Conservation of momentum• Isotropy
tntntcn iiiii ,,1, xxx
64Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
The Hexagonal Lattice
65Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Collision rules examples
Two body collision N1 AND N4 => N2 AND N5 && N3 AND N6
Three body collision N2 AND N4 AND N6 => N1 AND N3 AND N5
Streaming and Colliding
67Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
From LGA to LBM
– Average LGA equation to get continuous values instead of boolean values
– Boltzmann molecular chaos assumption to factorize products in collision operator:
=> Iterate:33and iiiiii NNNNNf
),(),(1
),()1,( tftftftf eqiiiii xxxcx
68Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
From Micro Dynamics to Macro Dynamics (1)
– Taylor expansion to get continuous differential operators:
itiii
itiiiti
ff
fff
ee
e
221
221
69Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
From Micro Dynamics to Macro Dynamics (2)
– Chapman Enskog expansion of equilibrium Distribution Function:
– With imposed constraints:
221ii
eqii ffff
1,0 and 0
and
6
1
)(6
1
)(
6
1
6
1
jff
ff
ii
ji
i
ji
ii
eqi
i
eqi
e
eu
70Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
From Micro Dynamics to Macro Dynamics (3)
– Multi-scale expansion of time and space derivatives:
– Solve collision/flow equation for different order of
0
:balance)order nd-(2
.0
:balance)order st -(1
0021
1
Su rtrt
rt u
1
2 and 21
rttt
71Peter Sloot: Computational Science, University of Amsterdam, The Netherlands.
Back to mesoscopic models
Recommended