National Center for Supercomputing Applications GridChem Science Gateway In Production National Science Foundation 23 May 2007 Sudhakar Pamidighantam NCSA,

National Center for Supercomputing Applications

GridChem Science Gateway

In Production

National Science Foundation 23 May 2007

Sudhakar PamidighantamNCSA, University of Illinois at Urbana-Champaign

[email protected]


Outline

• GridChem: A Brief Overview• Scientific Impact• Resources High End Computing Systems Applications Software• Middleware Infrastructure• Portal/Gateway Client • User Community • Usage• Critical Gateway Issues• Future Plans


The Big Picture


Science Enabled• Chemical Reactivity of the Biradicaloid (HO...ONO) Singlet

States of Peroxynitrous Acid. The Oxidation of Hydrocarbons, Sulfides, and Selenides. Bach, R. D et al. J. Am. Chem. Soc. 2005, 127, 3140-3155.

• The "Somersault" Mechanism for the P-450 Hydroxylation of Hydrocarbons. The Intervention of Transient Inverted Metastable Hydroperoxides. Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006, 128(5), 1474-1488.

• The Effect of Carbonyl Substitution on the Strain Energy of Small Ring Compounds and their Six-member Ring Reference Compounds Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006,128(14), 4598.


Science Enabled• Azide Reactions for Controlling Clean Silicon Surface

Chemistry: Benzylazide on Si(100)-2 x 1Semyon Bocharov et al..J. Am. Chem. Soc., 128 (29), 9300 -9301, 2006

• Chemistry of Diffusion Barrier Film Formation: Adsorption and Dissociation of Tetrakis(dimethylamino)titanium on Si(100)-2 × 1 Rodriguez-Reyes, J. C. F.; Teplyakov, A. V.J. Phys. Chem. C.; 2007; 111(12); 4800-4808.

• Computational Studies of [2+2] and [4+2] Pericyclic Reactions between Phosphinoboranes and Alkenes. Steric and Electronic Effects in Identifying a Reactive Phosphinoborane that Should Avoid Dimerization Thomas M. Gilbert and Steven M. Bachrach Organometallics, 26 (10), 2672 -2678, 2007.


Possible H-bonds network for P450cam

hydroperoxy intermediate

C

O

H

OH

OO H

Fe3+

O

H

CH3

HO

H

H

N

GLY248 peptide

VAL253 peptide

THR252

2.98Å2.79Å

2.99Å

2.75Å

3.16Å

3.07Å

3.32Å

Suggested:

THR252 accepts an H-bond from the hydroperoxy (Fe(III)-OOH that promotes thesecond protonation on the distal oxygen, leading to the O-O bond cleavage

Nagano, S.; Poulos, T.L. J. Biol. Chem. 2005, 250, p.1668• Auclair, K.; Hu, Z.; Little, D. M.; Ortiz de Montellano, P. R.; Groves, J. T. J. Am.

Chem. Soc. 2002, 124, 6020.


The Somersault Isomerization of Model Cpd0

TS

Fe

O

O H

SHFe

O

SH

H

O

E24.8 (24.3) 20.3 kcal/mol

1.447Å

2.226Å

2.408Å

1.869Å116.7

102.0

97.9

1.665Å

2.473Å

1.662Å

97.2

177.9

E = 17.5 (17.8) kcal/mol

EH-bonding = 17.0 kcal/mol

77.0

127.4

97.8

2.437Å

vi=101.5i cm-1

GS MIN

CCCC CC N

C CC

S

C

O

NFeNC

O

CC CN CC CC

CC

CC

C CCN

CC

CN

S

CC Fe

O

CC N

O

CCC N CC C

CC

CC CC

CCC NCC N

S

2.487Å

CFe

1.658Å

C

O

2.186Å

O

NCC N CCC CC CC

vi=93.7i cm-1

Robert Bach and Olga Dmytrenko, 2006


Energy Diagram for the Concerted Non-synchronous Hydroxylation of Isobutane

Fe

SH

O

O(H3C)3C

H

H

Fe

SH

O

OH

H(H3C)3C

Fe

SH

O

OH

H(H3C)3C

MIN-26a

Fe

SH

O

OHH

(H3C)3C

SH

O

OH

H

(H3C)3C

Fe

Fe

SH

OH

OH

(H3C)3C

-4.0

-19.2

17.2

-83.7

MIN-24bTS-25

MIN-26b

PRODUCT 28

3.848Å5.5

11.7

TS-27

GS-24a

19.5

Fe

SH

O

(H3C)3COHH

Energy diagram (kcal/mol) for the oxidation of the isobutane with ground state, 24a (GS-8 hydrogen bonded to isobutane). MIN-24b [model oxidant MIN-10 (PorFe(SH)OHO) hydrogen bonded to isobutene] is not necessarily on the reaction pathway.


Somersault Mechanism Summary for Isobutane Hydroxylation

S

FeIV

O

CH

CH3

CH3H3C

HO

S

FeIV

O

C

HCH3

CH3H3C

O

H

S

FeIV

O

C

H CH3

CH3H3C

O

H

S

FeIV

O

C

H

CH3

CH3H3C

O

H


TetrakisDimethylAminoTitanium and its derivatives on Si(100)-2x1 Surface: Diffusion Barrier Thinfilms on Silicon

Rodrigues-Reyes and Teplyakov


Benzylazide on Si(100)-2x1 SurfaceDeposition of Aromatic Moieties on Silicon for Lateral Electron

TransferBocharov et al..


[2+2] Cyclo Additions involving B=P BondsGilbert and Bachrach

Dimerization

Ethyne Addition

Ethene Additions


[4+2] Cycloadditions involving B=P BondsGilbert and Bachrach

Cis-Butadiene Addition

Cyclopentadiene Addition


GridChem Resources

http://portal.gridchem.org:8080/gridsphere/gridsphere?cid=home


GridChem Resources

New Computing Systems System Capacity (Cpus/Cores) Capability

Mercury(NCSA) 1774 Small/Large Parallel Runs

Abe(NCSA) 9600 Massively Parallel Runs

DataStar(SDSC) 2368 SharedMemory Large Runs

Bluegene/L(SDSC) 3456 Cluster Large Parallel Runs

TeragridCluster(SDSC) 564 Small/Large Parallel Runs

BigRed(IU) 1024 SharedMemory Small/Large Runs

BCX (UKy) 1360 Shared/Distributed Memory small/Large Parallel Runs


Application Software ResourcesCurrently Supported

Suite Version LocationGaussian 03 C.02/D.01 Many Platforms

MolPro 2006.1 NCSA

NWChem 5.0/4.7 Many Platforms

Gamess Jan 06 Many Platforms

Amber 8.0 Many Paltforms

QMCPack 2.0 NCSA


GridChem Software ResourcesNew Applications

Integration Underway

• ADF Amsterdam Density Functional Theory• Wien2K Linearized Augemented Plain wave (DFT)• CPMD Car Parinello Molecular Dynamics • QChem Molecular Energetics (Quantum Chemistry)• Aces3 Parallel Coupled Cluster Quantum Chemistry• Gromacs Nano/Bio Simulations (Molecular Dynamics)• NAMD Molecular Dynamics• DMol3 Periodic Molecular Systems ( Quantum Chemistry)• Castep Quantum Chemistry • MCCCS-Towhee Molecular Confirmation Sampling (Monte

Carlo)• Crystal98/06 Crystal Optimizations (Quantum Chemistry)• ….


Gridchem Middleware Service (GMS)


GridChem User Services• Allocationhttps://www.gridchem.org/allocations/index.shtmlCommunity and External Registration Reviews, PI Registration and Access Creation Community User Norms Established

• Consulting/User Serviceshttps://www.gridchem.org/consultTicket tracking, Allocation Management

• Documentation, Training and Outreachhttps://www.gridchem.org/doc_train/index.shtmlFAQ Extraction, Tutorials, Dissemination

Help is integrated into the GridChem client

https://www.gridchem.org/consult


GridChem Client Download Statistics

http://download.gridchem.org/usage/


Users and Usage• 222 Users Under 168 ProjectsInclude Academic PIs and colleagues,

graduate classes and

training users (typically used in tutorials/workshops/demo sessions etc..)

More than a 380000 CPU Wallhours Delivered since Jan 05 for over 5690 Individual Jobs


Distribution of GridChem User Community


Job Distribution Job Distribution by Time

0

50

100

150

200

250

300

350

100

300

500

700

900

1100

1300

1500

1700

1900

2100

2300

2500

2700

2900

Wall Clock x CPUs (~SUs)

Num

ber o

f Job

s

Job Distribution by Time

0

20

40

60

80

100

120

140

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50

Wall Clock Time x CPUs (~SUs)Nu

mber

of Jo

bs


System Wide Usage

HPC System Usage (SUs)Tungsten(NCSA) 5507

Copper(NCSA) 86484

CCGcluster(NCSA) 55709

Condor(NCSA) 30

SDX(UKy) 116143

CCGCluster(UKy) .5

Longhorn(TACC) 54

CCGCluster(OSC) 62000

TGCluster(OSC) 36936

Cobalt(NCSA) 2485

Champion(TACC) 11

Mike4 (LSU) 14537


GridChem Client Enhancements

• New Molecular Editor

JMolEditor (ANU) Integration

• VMD Is integrated

• Nanotube Generator (Tubegen) Will be available

• Gamess Graphical User Interphase


Java Molecular Editor• JMolEditorThree Dimensional Visual with Java 3D

Intuitive Molecule ManipulationInteractive Bond, Angle and Dihedral Settings

A Gaussian input generator Interface


Nanotube Generator:Tubegen

Courtesy: Doren Research Group at the University of Delaware Crystal Cell Types

Output Formats


GridChem Gamess GUI


GridChem Post Processing• IR/Raman Spectra now accessible from G03, MolPro,

NWChem and Gamess Suites

VCD/ROA To be Included


GridChem Post Processing

• Normal Mode Viewing in 3D VRML

• Other Spectra With MO Integration

NMR

Electronic Spectra


GridChem UsabilityDynamic Information


GridChem Usability

• Information on Potential Start and End Time for a given set of Job parameters

• Automated Resource Selection

• Possible Job Migration In case of dropped nodes or incomplete job

• Monitoring Multiple Jobs

• Automated Monitoring Job Output


• Implementation of GRMS resource management Service http://www.gridlab.org/WorkPackages/wp-9

• Moving toward Service based job submission eliminating gateway interfaces

• Infrastructure for multiple input files for single application

• Infrastructure for multiple inputs in High Throughput processing

• Integrated workflow for multi scale coupled modeling

• Meta-scheduling for High Throughput Processing Match Making Round-robin scheduling Preferred Host Set usage

GridChem Middleware InfrastructureImplementation Currently underway

http://www.gridlab.org/WorkPackages/wp-9


GridChem In New CollaborationsResource Providers

• New Resource Providers Open Science Grid Initially for Bio-related applications (open

source preferably)

• PRAGMA Partner sites University of Hyderabad

• ORNL (Could be via TeraGrid)

• International Partners KISTI, APAC, Daresbsry Labs


Scientific Collaborations

• GridChem Extension to Molecular Sciences (Bio, Nano, Geo and Materials Sciences) (NSF Proposal)

• Parameter Sweep for Potential Energy Hyper Surfaces (Faculty Fellows, NCSA)

• Automated Parameterization of Force fields (NSF Proposal)

• Ab initio Molecular Dynamics (Faculty Fellows, NCSA)

• Education (CI-TEAM) (NSF Proposals)

• Multi-Scale Modeling (IACAT, UIUC)


Some New GridChem Infrastructure• Workflow Editors• Coupled Application Execution• Large Scale Computing• Metadata and Archiving • Rich Client Platform Refactorization• Intergrid Interactions

• Open Source Distribution http://cvs.gridchem.org/cvs/

• Open Architecture and Implementation details http://www.gridchem.org/wiki

http://cvs.gridchem.org/cvs/


Critical Gateways Issues

• Science Gateways compete with business as usual for the end user research scientist

• No direct access to HPC systems may be possible leading to apparent lack of control for users

• No “End to end solutions” If part of the research needs require old ways

Gateways may be avoided• Learning to use Gateways should provide substantial

added benefit –Cost/Benefit Issues for users• Flexibility to integrate new applications as needed by

community quickly is critical to keep the user community engaged

Documents

National Center for Supercomputing Applications GridChem Science Gateway In Production National Science Foundation 23 May 2007 Sudhakar Pamidighantam NCSA,