SciDAC Meeting, San Francisco, June 27-30, 2005 Scalable Molecular Dynamics T.P.Straatsma Laboratory Fellow and Associate Division Director Computational

SciDAC Meeting, San Francisco, June 27-30, 2005

Scalable Molecular DynamicsScalable Molecular DynamicsScalable Molecular DynamicsScalable Molecular Dynamics

T.P.StraatsmaLaboratory Fellow and Associate Division Director

Computational Biology and BioinformaticsComputational Sciences and Mathematics Division

Pacific Northwest National Laboratory

2 SciDAC Meeting, San Francisco, June 27-30, 2005

NWChem Molecular Science SoftwareNWChem Molecular Science Software

ENERGY

GRADIENT

OPTIMIZE

DYNAMICS

THERMODYNAMICS

QMD

QM/MM

ET

QHOP

INPUT

PROPERTY

PREPARE

ANALYZE

ESP

VIB

Classical Force Field

DFT

SCF: RHF UHF ROHF

MP2: RHF UHF

MP3: RHF UHF

MP4: RHF UHF

RI-MP2

CCSD(T): RHF

CASSCF/GVB

MCSCF

MR-CI-PT

CI: Columbus Full Selected

Integral API

Geometry

Basis Sets

PEigS

pFFT

LAPACK

BLAS

MA

Global Arrays

ecce

ChemIO

NWCHEM


Domain DecompositionDomain DecompositionDomain DecompositionDomain Decomposition

Two-dimensional representationDistributed data

reduces memory use

Locality of interactionsreduces communication

Fluctuating number of atomsrequires atom redistribution

Inhomogeneous distributionrequires dynamic load balancing

local nodenon-local node domain within short rangenon-local node domain within long rangenon-local node domain outside interaction range


Force EvaluationForce EvaluationForce EvaluationForce Evaluation

1. Asynchronous ga_get of coordinates in box on neighboring node

2. Calculation forces with all local boxes within cutoff radius

3. Accumulate local forces

4. Asynchronous ga_acc to accumulate forces in box on neighboring node

All data transfer by means of one-sided, asynchronous communication

coordinates x

forces f

node i node j

F= -U(x)

12

34


Particle-mesh EwaldParticle-mesh EwaldParticle-mesh EwaldParticle-mesh Ewald

1. Charge grid construction

2. Block to slab decomposition

3. 3D-fast Fourier transform

4. Reciprocal space energy & forces

5. 3D-fast Fourier transform

6. Slab to block decomposition

7. Atomic forces

1

2

3

4

5

6

7

All nodes Node sub-set


FlowchartFlowchartFlowchartFlowchart

get coordinates

accumulate forces

pair lists

forces

PME charge grid

FFT & PME k-space

PME forces

redistribution

load balancing

record trajectory

properties

record properties

synchronous communication

asynchronous communication

processor-sub set communication

no processor communication

global wait for processor sub set

time step


Timing AnalysisTiming AnalysisTiming AnalysisTiming AnalysisHaloalkane dehalogenase, force evaluation timings

Synchronization PME forces

Non-local forces

Local forces

Reciprocal PME (fft, f-grid)

PME node subset synchronization PME charge grid construction

PME wait


Load BalancingLoad BalancingLoad BalancingLoad Balancing

LocalRedistribution

CollectiveResizing


Dynamic Load BalancingDynamic Load BalancingDynamic Load BalancingDynamic Load Balancing

Haloalkanedehalogenase: Time per Step

0.1

1

10

100

1 10 100

Number of Processors

Sca

lin

g

IBM-SP

HP

Haloalkanedehalogenase: Scaling

1

10

100

1 10 100

Number of Processors

Sca

lin

g

IBM-SP

HP

Linear


Challenges for the DOEChallenges for the DOEChallenges for the DOEChallenges for the DOE

Environmental Legacy at Hanford and other DOE sites• Bioremediation

Environmental and Health Impact of Energy Use• Carbon sequestration• Nitrogen fixation

Production of Energy• Biofuels• Hydrogen


Molecular Basis for Microbial Adhesion Molecular Basis for Microbial Adhesion and Geochemical Surface Reactionsand Geochemical Surface Reactions

Molecular Basis for Microbial Adhesion Molecular Basis for Microbial Adhesion and Geochemical Surface Reactionsand Geochemical Surface Reactions

Microbes in the subsurface mediate a number of environmental, geochemical processes:

Uptake of metal ions, including environmentally recalcitrant metals

Adhesion to mineral surfaces Reduction and mineralization of ions at the microbial surface

Pseudomonas aeruginosa: Cu, Fe, Au, La, Eu, U, Yb, Al, Ca, Na, KShewanella putrefaciens: Fe, S, MnShewanella alga: Fe, Cr, Co, Mn, UShewanella amazonensis: Fe, Mn, SShewanella oneidensis MR1External reduction involving OM

cytochromes


Project ObjectivesProject ObjectivesProject ObjectivesProject Objectives

Molecular level characterization of:Microbial adhesion to mineral surfacesMetal ion concentration in microbial membranes

Focus on Gram-negative bacterial Outer Membrane

Computational Approach:Molecular modeling and molecular dynamics simulationsQuantum mechanical description of key functional groupsThermodynamic Modeling


Gram Negative Cell WallsGram Negative Cell WallsGram Negative Cell WallsGram Negative Cell Walls


LPS ofLPS of Pseudomonas aeruginosa Pseudomonas aeruginosaLPS ofLPS of Pseudomonas aeruginosa Pseudomonas aeruginosa

NAG1 NAG2 PP

KDO1 KDO2

HEP1

HEP2

PP

P

CONH2

GALL-ALA

GLC*

GLC1

GLC2

GLC3

RHA

RHA

FUC

MAN

MAN

30-50

Lip

id A

Core

LP

SO

ch

ain 1. Design of the Rough LPS Molecular Model

2. Determination of Electrostatic Model


LPS Membrane ConstructionLPS Membrane ConstructionLPS Membrane ConstructionLPS Membrane Construction

Distribution of functional groups and water in the outer membrane of P. aeruginosa.These results are used for thermodynamic modeling of ion adsorption in microbial membranes.


Phosphate ClusteringPhosphate ClusteringPhosphate ClusteringPhosphate Clustering

Outer Core Inner Core

These results lend support to the interpretation of recent XAS experiments carried out by J. Bargar at SLAC indicating that uranyl ions take up by microbial membranes exists in clusters involving phosphates.


Membrane Electrostatic PotentialMembrane Electrostatic PotentialMembrane Electrostatic PotentialMembrane Electrostatic Potential

Average Potential Across Membrane

Calc.: 100 mVExp.: 80 mV


Atomic Charges from 2D SCF-HF ESP FitAtomic Charges from 2D SCF-HF ESP FitAtomic Charges from 2D SCF-HF ESP FitAtomic Charges from 2D SCF-HF ESP Fit

Slab: Periodic

Hartree Fock

Fragment: Point Charges

Blue: 25 e·kJ/molRed: -25 e·kJ/mol


Membrane-Mineral InteractionsMembrane-Mineral InteractionsMembrane-Mineral InteractionsMembrane-Mineral Interactions

123

4 5


P. AeruginosaP. Aeruginosa Outer Membrane Proteins Outer Membrane ProteinsP. AeruginosaP. Aeruginosa Outer Membrane Proteins Outer Membrane Proteins

E. coli membrane protein FecA (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein FecA (Straatsma, unpublished)

E. coli membrane protein TolC (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein OprM (Wong et al., 2001)

E. coli membrane protein OmpA (Pautsch and Schultz, 1998) and homology modeled P. aeruginosa membrane protein OprF (Brinkman et al., 2000)


P. aeruginosaP. aeruginosa OprF OprFP. aeruginosaP. aeruginosa OprF OprF


Electron transfer in bacterial respirationElectron transfer in bacterial respirationElectron transfer in bacterial respirationElectron transfer in bacterial respiration

• Under anaerobic conditions, Shewanella frigidimarina is able to use extra-cellular iron as the electron acceptor in its respiration. The electron transfer pathway involves a number of cytochromes which deliver electrons from the cytoplasmic membrane to the periplasmic membrane, where iron reduction occurs.

• The electron transfer (ET) between the membranes is carried out by the respiratory enzyme flavocytochrome c3

fumarate reductase (Fcc3), which

contains four bis(histidine) hemes.


Electron Transfer in FccElectron Transfer in Fcc33 and Ifc and Ifc33 Electron Transfer in FccElectron Transfer in Fcc33 and Ifc and Ifc33

Flavocytochrome c3 fumarate reductase of Shewanella frigidimarina


Marcus’ theory of electron transferMarcus’ theory of electron transferMarcus’ theory of electron transferMarcus’ theory of electron transfer

4

1

2 3

electronic coupling relaxation energy activation energy

dxz, dyz

dx2-

y2

Fe(II)1A1 Fe(III)2A2

Low-spin electron transfer

dx

y

dz2


B3LYP Characterization of a model hemeB3LYP Characterization of a model hemeB3LYP Characterization of a model hemeB3LYP Characterization of a model heme

Ehs/ls

Ehs/ls

Ehs/ls

AEAls AEAhs

DehsDels

Dels Dehs

Ehs/ls

r(Fe-N) Å

En

erg

y kcal/

mol

Fe(III)

Fe(II)

Heme-801 Heme-802

Heme-803 Heme-804

ET donor/acceptor orbital dπ


• Computational protein structure prediction • Protein-protein complexes: cell signaling• Protein-membrane and mineral-membrane complexes • Extension to microsecond simulation times• Statistically accurate thermodynamic properties• Comparative trajectory analysis

• Enzyme catalysis using hybrid QM/MM methods• Extension toward millisecond simulation times• Protein folding and unfolding• Membrane transport of simple ions and small molecules• Membrane fusion, vesicle formation

• Scalability on next generation MPP and hybrid architectures

Computational Structural Biology ChallengesComputational Structural Biology ChallengesComputational Structural Biology ChallengesComputational Structural Biology Challenges


AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgementsDr. Roberto D. Lins, ETH Lausanne, CH

Dr. Robert M. Shroll, Spectral Sciences, Boston, MA

Dr. Wlodek K. Apostoluk, Wroclaw University, Poland

Dr. Andy R. Felmy, Chemical Sciences Division, PNNLDr. Kevin M. Rosso, Chemical Sciences Division, PNNL

Professor David A. Dixon, University of Alabama

Dr. Erich R. Vorpagel, EMSL

DOE Office of Advanced Scientific Computing ResearchDOE Office of Basic Energy Science, Geosciences Research ProgramDOE Office of Biological and Environmental Research

EMSL Molecular Sciences Computing FacilityComputational Grand Challenge Application Projects

Documents

SciDAC Meeting, San Francisco, June 27-30, 2005 Scalable Molecular Dynamics T.P.Straatsma Laboratory Fellow and Associate Division Director Computational