1. GridChem A Computational Chemistry Cyber-infrastructure Sudhakar Pamidighantam NCSA, University of Illinois at Urabana Champaign sudhakar@ncsa.edu

2. Acknowledgements

3. Outline Historical Background Current Status Science Stories Future

4. Motivation Software - Reasonably Mature and easy to use to address chemists questions of interest Community of Users - Need and capable of using the software Some are non traditional computational chemists Resources - Various in capacity and capability

5. Background Qauntum Chemistry Remote Job Monitor ( Quantum Chemistry Workbench) 1998, NCSA Chemviz 1999-2001, NSF Technologies Web Based Client Server Models Visual Interfaces Distributed computing

6. GridChem NCSA Alliance was commissioned 1998 Diverse HPC systems deployed both at NCSA and Alliance Partner Sites Batch schedulers different at sites Policies favored different classes and modes of use at different sites/HPC systems

7. Extended TeraGrid Facility www.teragrid.org

8. Grid and Gridlock Alliance lead to Physical Grid Grid lead to TeraGrid Homogenous Grid was planned but it was difficult to keep it homogenous Things got more complicated and we have heterogeneous grids now! Interoperability and Standards and Openness Are Critical

9. Current Grid Status Grid Hardware Middleware Scientific Applications

10. User Community Chemistry and Computational Biology User Base Sep 03 – Oct 04 NRAC AAB Small Allocations ------------------------------------------------------------- #PIs 26 23 64 #SUs 5,953,100 1,374,100 640,000

12. User Issues New systems meant learning new commands Porting Codes Learning new job submissions and monitoring protocols New proposals for time Computational modeling became more popular and users increased Batch queues are longer / waiting increased Find resources where to compute - probably multiple distributed sites Multiple proposals/allocations/logins Authentication and Data Security Data management

13. Computational Chemistry Grid Integrated Cyber Infrastructure for Computational Chemistry Integrates Applications, Middleware, HPC resources, Scheduling and Data management Allocations, User Services and Training

14. Resources Over 400 processors and 3,525,000 CPU hours available annually System (Site)Procs AvailTotal CPU Hours/Year HP Intel Cluster (OSC)12100,000 Intel Cluster (OSC)36315,000 Intel Cluster (UKy)96840,000 HP Integrity Superdome33290,000 Intel Cluster (NCSA)64560,000 SGI Origin2000 (NCSA) IA32 Linux Cluster 128 64 1,000,000 560,000 Intel Cluster (LSU) 32 1024 280,000 1,000,000 IBM Power4 (TACC)16140,000 Teragrid30,000

15. Other Resources Extant HPC resources at various Supercomputer Centers (Interoperable) Optionally Other Grids and Hubs/local/personal resources These may require existing allocations/Authorization

17. Grid Middleware Proxy Server GridChem System user Portal Client Grid Services Grid applicationapplication Mass Storage http:// www.nsf.gov/awardsearch/showAward.do?AwardNumber=0438312

18. Applications GridChem supports some apps already –Gaussian 98/03, GAMESS, NWChem, Molpro, QMCPack, Amber Schedule of integration of additional software –ACES-2 –Crystal –Q-Chem –Wein2K –MCCCS Towhee –More …..

19. Gridchem Middleware Web Services Oriented

20. Job Editor

21. Job Monitoring

22. Resource Status

23. Gradient Monitoring

24. Energy Monitoring

25. Visualization Molecular Visualization Electronic Properties Spectra Vibrational Modes

26. Molecular Visualization Better molecule representations (Ball and Stick/VDW/MS) In Nanocad Molecular Editor Third party visualizer integration Chime/VMD Export Possibilities to others interfaces Deliver standard file formats (XML,SDF,MSF,Smiles etc…)

27. Eigen Function Visualization Molecular Orbital/Fragment Orbital MO Density Visualization MO Density Properties Other functions Radial distribution functions

28. Some example Visuals Arginine Gamess/6-31G* Total electronic density 2D - Slices

29. Electron Density in 3D Interactive (VRML)

30. Orbital 2D Displays N2 6-31g* Gamess

31. Orbital 3D VRML

32. Spectra IR/Raman Vibrotational Spectra UV Visible Spectra Spectra to Normal Modes Spectra to Orbitals

33. GridChem Use Allocation Community and External Registration Consulting/User Services Ticket tracking, Allocation Management Documentation Training and Outreach FAQ Extraction, Tutorials, Dissemination

34. Users and Usage 160 Users Include Academic PIs, two graduate classes And about 15 training users NCSA 57000 SUs + A 7 node dedicated system UKy around 106766 SUs OSC 13,820 SUs + A 14 node dedicated system Usage at LSU and TACC as well More than a 1.5 Mil Normalized units during 8 months since Jan 06.

35. Science Enabled Chemical Reactivity of the Biradicaloid (HO...ONO) Singlet States of Peroxynitrous Acid. The Oxidation of Hydrocarbons, Sulfides, and Selenides. Bach, R. D.; Dmitrenko, O.; Estévez, C. M. J. Am. Chem. Soc. 2005, 127, 3140-3155. The "Somersault" Mechanism for the P-450 Hydroxylation of Hydrocarbons. The Intervention of Transient Inverted Metastable Hydroperoxides. Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006, 128(5), 1474-1488. The Effect of Carbonyl Substitution on the Strain Energy of Small Ring Compounds and their Six-member Ring Reference Compounds Bach, R. D.; Dmitrenko, O. J. Am. Chem. Soc. 2006,128(14), 4598.

36. Science Enabled Azide Reactions for Controlling Clean Silicon Surface Chemistry: Benzylazide on Si(100)-2 1 Semyon Bocharov, Olga Dmitrenko, Lucila P. Mendez De Leo, and Andrew V. Teplyakov* Department of Chemistry and Biochemistry, UniVersity of Delaware, Newark, Delaware 19716 Received April 13, 2006; E-mail: andrewt@udel.edu http://pubs.acs.org.proxy2.library.uiuc.edu/cgi- bin/asap.cgi/jacsat/asap/pdf/ja0623663.pdf [May require ACS access] Acknowledgment. This work was supported by the National Science Foundation (CHE-0313803 and CHE-0415979). GridChem is acknowledged for computational resources and services for the selected results used in this publication. http://pubs.acs.org.proxy2.library.uiuc.edu/cgi- bin/asap.cgi/jacsat/asap/pdf/ja0623663.pdf

37. Metalla Crown Ether Modeling Via GridChem Sudhakar Pamidighantam NCSA, University of Illinois at Urbana-Champaign Scott Brozell Ohio Supercompter Center

38. Unsymmetrical Mo(CO) 4 Crown Ethers

39. Dibenzaphosphepin based  bis(phosphorous)polyether chelated Mo(CO) 4

40. Crystal Structures CSD:XAPZAP cis-(6,6'-((1,1'-Binaphthyl)-2,2'- diylbis(oxy))bis(dibenzo(d,f)(1,3,2)dioxaphosp hepin))-tetracarbonyl-molybdenum(0) C48 H28 Mo1 O10 P2 CSD:DEQDOS cis-Tetracarbonyl-(P,P'-(6-(2'-oxy-2-biphenyl)-3,6- dioxa-hexanolato)-bis(dibenzo (d,f)(1,3,2)dioxaphosphepine)-P,P')-molybdenum C44 H32 Mo1 O12 P2

41. Reference Structure for Comparison

42. Starting Structure

43. Optimized Structure

44. Reference Structure for Comparison 8 7

45. Structural Comparisons C-C Torsion Angles for the OCH 2 CH 2 O Fragments and for the Axially Chiral Biaryl Groups Atoms PCMODEL* UFF Ab Initio Amber C37-C42-C43-C48 -49.9 -26.4 -43.0 -40.4 C1-C6-C7-C12 45.4 22.3-22.3 -72.8 C13-C22-C23-C32 75.6 74.7-85.9 -81.2 C32-O-C33-C34 -178.4 -140.8 159.7 -171.2 O-C33-C34-O 62.4 -64.5 -87.3 -82.4 C33-C34-O-C35 -80.6 -118.9 67.8 64.9 C34-O-C35-C36 174.6 118.9 -153.4 60.1 O-C35-C36-0 66.2 56.0 64.0 67.3 *Hariharasarma, et al. Organomet., 1232-1238, 2000. Ab Initio=B3LYP/3-21G* Amber9 ff03, GAFF, chloroform, 300K, median over 1ns MD

46. MD OCH 2 CH 2 O Structure 8 7

47. MD Biaryl Structure

48. 1 H NMR Chemical Shift Comparison For Aromatic Protons Reference 32ppm (from TMS B3LYP/6-31g*) Atom Exp.AbinitioAtomExp.Abinitio H27.0255.6H256.5785.7 H37.0265.8H266.7375.9 H47.0495.9H277.0186.1 H57.1816.0H287.6236.5 H87.1106.1H307.7906.7 H96.8906.0H317.2896.9 H106.7216.0 H116.2375.7H387.3276.2 H397.2746.1 H147.9255.8H407.1696.0 H157.8086.3H417.3506.3 H177.7416.0H447.3606.1 H187.2545.6H457.1605.9 H197.0915.1H467.1766.0 H206.9894.6H477.0607.0

49. 13 C Chemical Shift Comparison Reference 190ppm (B3LYP/6-31g* TMS) AtomExp. AbinitioAtom Exp. Abinitio AtomExp. Abinitio C1149.57127.3C17 127.78100.3C37149.85124.0 C2121.98 97.3C18124.74 96.5C38122.33 99.5 C3128.92101.3C19126.15 99.9C39129.50103.0 C4125.10 97.4C20126.13 99.5C40125.57 99.9 C5129.95105.5C21134.08108.9 C41130.14103.2 C6129.93105.0C22123.88 92.9C42130.19106.8 C7129.73106.3C23 118.62104.5C43129.59105.2 C8129.13102.8C24 134.02101.9C44129.95103.9 C9125.05 99.5C25 125.05100.0C45125.45 98.1 C10128.81103.1C26126.10 99.4C46129.50101.3 C11122.28 99.5C27123.06101.0C47122.26101.1 C12148.00122.2C28127.62103.2C48150.22129.6 C13 147.66128.5C29128.88103.2 C14121.06 95.9C30129.53103.2 C15128.56102.6C31114.35 98.6 C16130.65101.0C32 154.31125.5

50. P1 and P2 Are around 166ppm with a P-P Coupling of 49 Hz. Isotropic Shielding Const P1 P2 B3LYP 248.6 261.0 BPW91 251.0 265.0 Mo Isotropic Shielding Const B3LYP 1396 BPW91 1510 (Mo(CO) 6 ) Exp. -1856 B3LYP -2350 BPW91-2294 B3LYP Hybrid Not satisfactory; BPW91 “Pure” functionals give better results; Buehl, Chem. Eur. J., 3514 (1999). 31 P and 95 Mo Chemical Shifts

51. 17 O Chemical Shifts for Phosphonite/Ether Oxygens* B3LYP/3-21G*(BPW91) P-O-PhShielding Chemical Shift O 8162.3(147.7)154.3(168.9) O 9171.5(155.8)145.1(160.8) O 10153.9(139.4)162.7(177.2) (Naphthyl) O 11162.7(147.8)153.9(168.8) O 12171.7(156.1)144.9(160.5) P-O-C O95 201.9(189.5)114.7(127.1) Napth-O_C O57215.1(202.8)101.5(113.8) C-O-C O92 287.0(279.5) 29.6( 37.1) With reference to B3LYP/6-31g* H2O 316.6

52. 17 O Chemical Shifts B3LYP/3-21G*(BPW91) Carbon Monoxide Oxygens Shielding Const.Chemical Shift* O 4-79.4(-61.5)396.0(378.1) O 5-85.8(-68.1)402.4(384.7) O 6 -61.0(-42.8)376.6(359.4) O 7-73.1(-55.5)389.7(372.1) *with reference to H2O(B3LYP/6-31G*) 316.6 ppm

53. Summary Mo Crown Ethers are tricky to converge in standard DFT and HF Methods due to degenerate set of states and fluxional crown ether moieties GridChem can be successfully used to access resources and applications for computational chemistry

54. Acknowledgments Rion Dooley Stelios Kyriacou Chona Guiang Kent Milfeld Kailash Kotwani Antitsa Stoycheva Terry Lang Tack Kuntz, UCSF

55. Acknowledgements

56. Job Monitoring

57. Vacuum MD

60. Third Year Plans Post Processing New Application Support Expansion of Resources Extension Plan