Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
LGT
Software &
Multigrid
Rich Brow
erBoston U
niversity
Lattice QC
D C
omputational Science W
orkshopO
ak Ridge
April 30, 2013
Tuesday, April 30, 13
“Algorithm
s” for “solving” Quantum
Field Theory
Path Int = Integral[ exp[ - A
ction]
Feynman
Diagram
sRenorm
alizationG
roup
PDE/FEM
Schwartz
Schurs
Wilson
Lattice QC
D
‘tHooft
Dim
Reg
Multi-
Grid
Real-Space
GR
OPE
AdS/C
FTTw
istersBootstrap
DD
Dom
ainW
allW
ilsonFlow
Tuesday, April 30, 13
USQ
CD
Software Stack Stack
On line distribution: http://usqcd.jlab.org/usqcd-softw
are/
The application codes C
hroma/C
PS/MILC
and a new Q
DP LU
A code base provide a rich set of tools.
QLA
/perl = 23000 files
!"#$%&'#&("')
!"#$%&'()
*+
!""#$!
%&'()#*** Chrom
a = 4856 files
CP
S = 1749 files
MILC
= 2300 files
QU
DA
/python = 221 files Level 3
Level 4
Level 2
Level
!"#$%&
!'(&#)*(+,&!(')-.*(&}
Apps Softw
arenot
SciDA
C funded
Tuesday, April 30, 13
SciDAC
LGT contributors
•ANL: Jam
es Osborn, M
eifeng Lin, Heechang Na, (George T. Flem
ing)•
BNL: Frithjof Karsch, Chulwoo Jung, Hyung-Jin Kim,Yu M
aezawa•
Columbia: Robert M
awhinney, Hantao Yin•
FNAL: James Sim
one, Alexei Strelchenko, Don Holmgren, Paul M
ackenzie•
JLab: Robert Edwards, Balint Joo, Jie Chen, Frank Winter, Chip W
atson•
W&M
/UNC: Kostas Orginos, Andreas Stathopoulos, Rob Fowler (SUPER)
•LLNL: Pavlos Vranas, Chris Schroeder, Rob Faulgot (FASTm
ath) •
NVIDIA: Ron Babich, Mike Clark
•Arizona: Doug Toussaint, Alexei Bazavov
•Indiana/NCSA: Steve G
ottlieb, Ran Zhou
•Utah: Carleton DeTar, Justin Foley
•BU: Richard Brower, M
ichael Cheng, Oliver W
itzel•
MIT: Pochinsky Andrew, John Negele,
•Syracuse: Sim
on Catterall, (David Schaich in fall)•
Washington: M
artin Savage, Saul Cohen•
Others: Peter Boyle, Jim
Hetrick, Massim
o Di Pierro, Patrick Dreher, et al•
“Team of Rivals” (apologies to contributors and projects *NO
T* mentioned in 6 slides!)
Tuesday, April 30, 13
Highest Priority is m
oving to 3 new architecture!
(May you live in I
nteresting Times!)
Tuesday, April 30, 13
•!ab initio C
hemistry!
1."1930+50 = 1980!
2."0.1 flops !
10 Mflops!
3."G
aussian Basis functions!
•!ab initio Q
CD!
1."1980 + 50 = 2030?*!
2."10 M
flops ! 1000 Tflops!
3."C
lever Collective Variable?!
vs!
Lattice Field Theory is just now
meeting the W
ilson criterion for relevance to Experim
ental Physics!
Tuesday, April 30, 13
We are just now
able to resolve the pion scale!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
"#$%!&
'()
!*+#,)
*!#
()!-
#./.$
0!.$
!,.$)1
23#(4
*!&
#**)*1!!53
6*+7/!8
!9:!;
:!<==:!!<
>==:!?
<@=:!9
=====!"
)AB
C,)
+/('
�$)/.*
&!5D
('/'
$E$
3+,)
'$!*D,.//.$
0:!0
E9B
F.$
6.$
0!)
$)(0
%!'
G!$3+,)
.!59H9
!")I!G'
(!6)3/)
('$B
J)A!!K
/('$0!L
#30)!F
K"
!5$)#(!+
'$G'
(&#,B!6
%$#&
.+*!G'
(!+'&
D'*./)
!M.0
0*!
New
Algorithms: C
rucial for both New
Physics & Architectures
a(lattice)
⌧1/M
proton
⌧1/m
⇡
⌧L(box)
0.06ferm
i⌧
0.2ferm
i⌧
1.4ferm
i⌧
6.0ferm
i
=)L=
O(100)orMinimum
LatticeVolume
100
4!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
a(lattice)
⌧1/M
proton
/M
proton
/M
⌧1/m
⇡
⌧L(box)
0.06ferm
i⌧
0.2ferm
i⌧
1.4ferm
i⌧
6.0ferm
i
=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=)=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!L
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!L
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!=
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!O
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!O
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(100)orMinimum
LatticeVolume
100
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!(100)orMinimum
LatticeVolume
100
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!4!
Tuesday, April 30, 13
* “MG
is always the Future”: Joke from
JLab 2008
** “The future has arrived
!”: Message at Oak
Ridge 2013
•History Lessons (1989-1992) *
-Cause of early failure for 20 years!
•M
odern Era (2008-2013)
-5 years to put into production the Q
CD MG
Solver for Wilson-clover
•Future** (2013-2018)
-Dom
ain Wall & Staggered Solvers, HM
C evolution, etc
-Adaptation to heterogeneous architectures, etc.
Multigrid: C
ase History in A
lgorithm D
evelopment
Tuesday, April 30, 13
Past History
Tuesday, April 30, 13
QC
D M
G attem
pts in 1990’sS
ee Thomas K
alkretuer hep-lat/9409008 review
on “MG
Methods
for Propagators in LG
T”.
Israel: Ben-Av, M
. Harm
atz, P.G
. Lauwers &
S.S
olomon
Boston: B
rower, E
dwards,
Rebbi &
Vicari
Am
sterdam: A
. Hulsebos,
J Sm
it J. C. Vick
Am
sterdam: A
. Hulsebos,
J Sm
it J. C. Vick
Tuesday, April 30, 13
QC
D M
G “failure” in 1990’s:
Tuesday, April 30, 13
2-D W
ilson Dirac O
perator
Tuesday, April 30, 13
Universal “failure” of C
ritical Slowing dow
n:
⌧=
F(m
l�)
�=
3(cross),10(
plus),100(
squares)
Gauss-Jacobi(Diamond),CG
(circle),
Multi-Grid3levels(square
star)
Tuesday, April 30, 13
!"##$%%&'&()*+",$%&-.&/
0&)11$2
31%&*4&56678%&9&:;<=&
•K3++)**1
•"
#.$
/#.$
!L#30)!N$
I#(.#
$+)!
•"
#.$
/#.$
)6!L
#&
&#E;
!M)(&
./.+./%
•O'+#,!#
6#D/.I
)!7
,'+4.$
0!5-
./P!Q
('R)
+/.I
)!"
LB
•Q#(/.#
,!*3++)**!-
./P!5S
LB!#
/!-)#4!+'3D,.$
0
•T#.,)
6!#
/!*/('
$0!+'3D,.$
0H!
•L#,)
(4.$
!G'(&
corseoperator:D
c=
RDP
Boris Grigoryevich G
alerkin (Russian: !"#$%& '#$("%#)*+$, '-./#0$1, surnam
e more
accurately romanized as G
alyorkin; March
42[O.S. February 20, 1871]218712– July 12, 1945),
Tuesday, April 30, 13
Why D
idn’t It work?
Tuesday, April 30, 13
!"#$%&'()**&'+,,-$$.&&&/
00*"-1&2)%34536$",$&&78**)98#)%"8:
First Success: Applied M
ath/Physics Collaboration C
ollaboration
•! Saul C
ohen • S
aul Cohen
•!INT S
eattle •! S
aul Cohen
•M
ichael Cheng
• NV
IDIA
•O
liver Witzel
•! Ron B
abich
• MIT• A
ndrew Pochinsky
• Mike C
lark
• Saul C
ohen •! S
aul Cohen
Chris Schroeder
•! Ron B
abich
Tuesday, April 30, 13
Present Break Through
Tuesday, April 30, 13
Adaptive Sm
ooth Aggregation A
lgebraic Multigrid
smoothing
Fine Grid
Sm
aller Coarse G
rid
restriction
prolongation (interpolation)
The Multigrid
V-cycle
Spilt the vector space into near null space S and the com
plement S
!!
D: S !
0!
Tuesday, April 30, 13
What is the N
ew Idea?
•M
ath Speak: A Schur/Schwarzian DD splitting of the vector space:-
How do you spit the space into Fine vs Coarse Space?
-Classical M
G vs Adaptive M
G
(see Front cover of Strang’s Undergraduate MIT m
ath text!)
•In Physics Speak: The W
ilsonian Renormalization G
roups:-
How to separate UV (short scales) from IR (long scales)
-Conform
al (Scale Inv) vs Non-perturbative RG
But P†P = 1cc so Ker(P) = 0!
S!!
S span(P)!
ker(P†)!
P! P†!
span(P†)!
UV!IR
!
fine space!K
arl Herm
ann Am
andus Schw
arz (25 January 1843 – 30 N
ovember 1921)
Tuesday, April 30, 13
Devils in the D
etails!•
To turn the new Mathem
atical idea into a competitive M
G
solver took a lot of work!-
First in principle demonstration of coarsening used the norm
al equations: no dependence on the m
ass (lowest e.v.) but was not com
petitive.
-Needed to go to Red/Black precondition non-Herm
itian operator -- not conventional M
G.
-O
(25) near null vector on 4^4 blocks plus chirality.
-G
CR smoother etc etc.
•See:
•Code is available: Docum
entation this sumer?
Adaptive multigrid algorithm
for the lattice Wilson-Dirac operator, R. Babich, J.
Brannick, R. C. Brower, M. A. Clark, T. A. Manteuffel, S. F. McCormick, J. C. Osborn and C. Rebbi, Phys. Rev. Lett. 105 (2010) 201602.
Tuesday, April 30, 13
Adaptive Sm
ooth Aggregation A
lgebraic Multigrid
SciDA
C com
pliant production code on BG/P
Tuesday, April 30, 13
Good N
ews/B
ad New
s
More D
ata: Should Save M
G
projectors with
lattice
Actually M
G error
is smaller at fixedR
esidual
Tuesday, April 30, 13
Tuesday, April 30, 13
Near Future Projects
Tuesday, April 30, 13
What about D
omain W
all and Staggered?
•Dom
ain Wall attem
pt Saul Cohen, M. Clark and J. O
sborn -
Must use Norm
al Equations (so far).
-But Blocking elim
inate 5th dimension!
-The algorithm
is still not tested.
-New effort with O
liver Witzel and Andrew Pochinsky in Q
LUA
•Staggered. -
Red/Black precondition is the Normal Eq.
-Theoretical straight forward
-BUT “double” is a 4 fold increase in near null space.
-M
ay have to have blocks preserving the tastes?
-Carleton ?
Multigrid A
lgorithms for D
omain-W
all Fermions, S. D. Cohen, R. C. Brower,
M. A. Clark and J. C. Osborn, PoS LATTICE 2011, (2011) 030
Tuesday, April 30, 13
Non N
ormal !
Non H
ermitian!
Non Pos. D
ef.!
Non-norm
alnon-H
ermitian
Non Pos. D
ef.
!"#$%&''()*+,-'
Tuesday, April 30, 13
What about M
ultigrid HM
C?
New
Project with M
eifeng Lin and James O
sborn: VERY
EA
RLY
TEST
S on Wilson Lattice from
Kostas.
• Looks promising w
ith much developm
ent• C
hronological Preconditioner• R
etuning HM
C algorithm
• Bigger Lattices• N
ote: Lattice will be stored W
ITH
the near NU
LL vector so there is no set up cost to M
G in analysis
Tuesday, April 30, 13
Herterogeneous A
rchitecture•
Problem: W
ilson Clover for Light Quark is FASTER on the
CPU than using the QUDA solver on G
PUs!
•Solution put M
G on G
PU of course !!!"#$%&'%(#%)*+,-*+%./%0%10-$")%"1%0$%2*0#$%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3456678%
%%%%9:;
%%%345678%%%%%%%%%%%%%%%%%%<
9%%%3
45678%
="'%>0.&-?%%%%%%%%0'+%%<
&@*%!20)@%%%%%%%0'+%%%%A-&B
C!%
%%%%%%0'+%%%%%D
E&+&0%
(now with Mike Clark at NVIDIA and M
ichael Cheng on NSF grant)Tuesday, April 30, 13
Rapid Prototyping and Tuning? •
HYPER in FastMath with Pavlos Vranas, Chris Schroeder,
Rob Faulgot (FASTmath)
•Q
LUA = QCD + LUA is FUN
-https://usqcd.lns.m
it.edu/w/index.php/Q
LUA
_tutorials
Integration of HY
PRE and Q
lua is well underw
ay. Rob Falgout, C
hristopher Schroeder and Andrew
Pochinsky have completed an overall design of a H
YPR
E/USQ
CD
interface (HQ
L) and begun its
implem
entation. RF is largely finished extending H
YPR
E to handle more than 3 dim
ensions and fully
expects to finish implem
enting complex num
bers on schedule. CS and R
F are making progress on
the implem
entation of the HQ
L interface, and RF and A
P are proceeding with the H
QL-Q
lua
interface. AP is finishing extending Q
lua to handle data types and procedures required to support
HQ
L.
Tuesday, April 30, 13
FUEL: Jam
es Osborn for BSM
HM
C developm
ent
Tuesday, April 30, 13
RxSherical Lattice: R
x S3•
Radial Quantization requires “spatial” spheres!
-Conform
al Fix Points for BSM theories
-(or partial wave expansion & Scattering Length?)
-Need Finite Elem
ents Method to do 3d Ising on curves space!
exact icosahedral symm
etryTuesday, April 30, 13
Mu
ch m
ore A
lgorith
mic D
evelop
is a critica
l investm
ent
•* M
G D
omain W
all &Staggered
•* M
ulti-grid/HM
C Evolution
•* M
ulti-grid QU
DA
on GPU
s
•R
esearch Area for “real” D
D
• M
ultiGrid/D
D “deflation”
• *W
ilson Flow
•D
isconnected Diagram
s
•EigC
G D
eflation & A
ll mode averaging
(48^3 96 24: Big Data, 600 ev
7.2TBytes)
•M
ulti-quark contraction codes
•*R
adial Quantization: C
onformal T
heory
•LO
Ts M
OR
E to DO
.
•A
lgorithmic advances are difficult and tim
e consuming
Need separate D
evelopment and O
ptimization Softw
are Environment
Need M
odularity to move new
algorithm into A
pplication Codes.
Conclusions: “T
he Sun is But a Morning Star*”!
* Henry
David
#oreau
in Conclusion
% Walden
, 1845
Tuesday, April 30, 13