Accelerating Discovery in Science and Engineering 11/9/2018 3:46 AM Accelerating Discovery in Science and Engineering Fabrizio Gagliardi Director – EMEA & LATAM Technical Computing Microsoft Corporation

Introduction Some personal introductory remarks Progress in grid computing Microsoft progress in HPC Microsoft technology for science Engagements in science Conclusions I am again here: since 2001 I have not missed this event a single time! I am happy to be associated with the pioneering work of Poland in HPC and Grid computing. Honoured to have helped in establishing the first projects with the EU when Poland was not yet a EU member state. Excellent Polish tradition in science and technology all the way back to Copernicus, who btw studies in Cracow and his theories were followed up by Galileo who was born and stayed in my home town Pisa. Many things in common between myself and Poland. Last year I spoke about e-infrastcuture and Grids and I explained the reasons for me to move to MS and what my plans were to bring MS to science and science to MS. Today one year later it is a good time to review my personal progress and the impact in the company.

Some personal introductory remarks I am again here: since 2001 I have not missed this event a single time! Happy to be associated with the pioneering work of Poland in HPC, networking and Grid computing Honoured to witness the present success Good opportunity to review the progress of my activity since last year Last year I spoke about e-infrastrcuture, Grids and Microsoft plans for Science Let’s review the progress now I am again here: since 2001 I have not missed this event a single time! I am happy to be associated with the pioneering work of Poland in HPC and Grid computing. Honoured to have helped in establishing the first projects with the EU when Poland was not yet a EU member state. Excellent Polish tradition in science and technology all the way back to Copernicus, who btw studies in Cracow and his theories were followed up by Galileo who was born and stayed in my home town Pisa. Many things in common between myself and Poland. Last year I spoke about e-infrastcuture and Grids and I explained the reasons for me to move to MS and what my plans were to bring MS to science and science to MS. Today one year later it is a good time to review my personal progress and the impact in the company.

Progress in grid computing Microsoft has sponsored GGF16 and GGF17 and took the initiative of proposing a HPC profile within the OGSA WG; a Data Management profile is also being discussed On the application side we were prime sponsor at HealthGrid in Valencia with a key note by David Heckerman (AIDS vaccine research) Rapid adoption from IT industry is essential for the future of Grid technology : GGF and EGA have merged in the Open Grid Forum (OGF) and held the first conference in Washington early September this year Industry is now represented in a board of directors: all major vendors including Microsoft (Tony Hey) Microsoft is also participating in the AdCom (myself) and in some of the WGs (OGSA and Security) Last year I spoke about pushing MS to develop a Window personal productive environment for the scientist and the engineer and promoting a faster adoption of Grid computing in the company.

Progress in grid computing 2 Major issues which still remain to bring grid computing from academy to industry and commerce are: Security Interoperability Easy to integrate and use Reliability of the infrastructure Adequate new business models Microsoft is now considering most of those issues in the context of OGF Last year I spoke about pushing MS to develop a Window personal productive environment for the scientist and the engineer and promoting a faster adoption of Grid computing in the company.

Microsoft progress in HPC Windows Computer Cluster Software released Microsoft HPC institutes successful experience around the world

High speed, low latency interconnect Microsoft Compute Cluster Server What it does : Solution for High-Performance Computing application at a medium-low range of the scale Simplified administration and job management Built-in job scheduler and MPI lib Four basic job scheduling policies supported in V1 Key advantages: Fully integrated cluster solution Interoperability with Unix systems Leverages existing Windows infrastructure and security Data Input Job Policy, reports Management DB/FS High speed, low latency interconnect User Admin Job Mgmt Resource Mgmt Cluster Mgmt Scheduling Desktop App Cmd line Head Node Node Manager User App Job Execution Active Directory MPI Admin Console User Console

HPC Innovation Centers Institutes for High Performance Computing Center Cornell Theory Center Ithaca, NY USA University of Tennessee Knoxville, TN USA TACC – University of Texas Austin, TX USA University of Virginia Charlottesville, VA USA University of Utah Salt Lake City, UT USA Tokyo Institute of Technology Tokyo, Japan HLRS – University of Stuttgart Stuttgart, Germany Southampton University Southampton, UK Shanghai Jiao Tong University Shanghai, PRC Nizhni Novgorod University Nizhni Novgorod, Russia Cornell Theory Center Ithaca, NY USA University of Tennessee Knoxville, TN USA University of Virginia Charlottesville, VA USA University of Utah Salt Lake City, UT USA TACC – University of Texas Austin, TX USA Southampton University Southampton, UK HLRS – University of Stuttgart Stuttgart, Germany Shanghai Jiao Tong University Shanghai, PRC Tokyo Institute of Technology Tokyo, Japan Nizhni Novgorod University Nizhni Novgorod, Russia

HPC Market Trends

Top 500 Supercomputer Trends Clusters over 50% Industry usage rising GigE is gaining x86 is leading Movement from research/government to industry solutions Movement from large SMP boxes (Cray) to clusters of commodity machines Movement from proprietary interconnects to GigE and Infiniband Intel architecture, like many industries, is growing. Taking advantage of price pressure and ecosystem. All of these trends coincide with Microsoft strengths and drive our strategy moving forward.

Supercomputing Goes Personal 1991 1998 2005 System Cray Y-MP C916 Sun HPC10000 Small Form Factor PCs Architecture 16 x Vector 4GB, Bus 24 x 333MHz Ultra-SPARCII, 24GB, SBus 4 x 2.2GHz Athlon64 4GB, GigE OS UNICOS Solaris 2.5.1 Windows Server 2003 SP1 GFlops ~10 Top500 # 1 500 N/A Price $40,000,000 $1,000,000 (40x drop) < $4,000 (250x drop) Customers Government Labs Large Enterprises Every Engineer & Scientist Applications Classified, Climate, Physics Research Manufacturing, Energy, Finance, Telecom Bioinformatics, Materials Sciences, Digital Media

Technology challenges Moore’s law continues but power consumption and heat dissipation are reaching their limits Memory and data access gap widen Applications become more data intensive

The Future: Supercomputing on a Chip IBM Cell processor 256 Gflops today 4 node personal cluster => 1 Tflops 32 node personal cluster => Top100 MS Xbox 3 custom PowerPCs + ATI graphics processor 1 Tflops today $300 8 node personal cluster => “Top100” for $2500 (ignoring all that you don’t get for $300) Intel many-core chips “100’s of cores on a chip in 2015” (Justin Rattner, Intel: http://www.hpcwire.com/hpc/629783.html ) “4 cores”/Tflop => 25 Tflops/chip We are at an important transition point in computing: Single CPU performance is limited by power and heat dissipation Out of cache memory access has not scaled up with the increase of power in the CPU Only way out is parallelism all the way from the CPU multi-core to the system and application S/W Many applications which are embarrassingly parallel can easily migrate to Grid computing infrastructures Other applications will require new computing development environments (compilers, parallel libraries, O/S, etc…) See recent article from Justin Rattner: http://www.hpcwire.com/hpc/629783.html

The Microsoft project in Barcelona Microsoft is interested in helping computer scientists to develop new computing architectures with a high level of parallelism Mateo Valero and his BSC centre in Barcelona are leaders in this field in Europe Microsoft will collaborate with BSC to research and develop an entirely new parallel computing ecosystem http://www.hpcwire.com/hpc/633342.html

Microsoft Technical Computing: Radical Computing Research in potential breakthrough technologies Advanced Computing for Science and Engineering Application of new algorithms, tools and technologies to scientific and engineering problems High Performance Computing and tools Application of high performance clusters and database technologies to industrial applications Application of existing and new tools for science Other TCI projects: vaccine development in Chile with top scientist and Chiron founder Pablo Valenzuela, simulation and tools for automotive industry at HLRS, engineering applications for aerospace at Southampton, Cancer and climate research at Oxford…

Can “Here and Now” technologies accelerate discovery? Can “Business” Tools and techniques for dealing with be used in scientific research to allow researchers to be scientists and not computer scientists…

Data Mining & Algorithms Real-world Data Interpretation & Insight Persistent Distributed Workflow, Data Mining & Algorithms Computational Modeling

Data Mining & Algorithms Real-world Data Interpretation & Insight Persistent Distributed Workflow, Data Mining & Algorithms Computational Modeling

The Problem for the e-Scientist Experiments & Instruments Simulations facts answers questions ? Literature Other Archives Data ingest Managing a petabyte Common schema How to organize it? How to reorganize it? How to coexist & cooperate with others? Data Query and Visualization tools Support/training Performance Execute queries in a minute Batch (big) query scheduling

Persistent Distributed Storage Visual Programming

Distributed Computation Interoperability & Legacy Support via Web Services

Searching & Visualization Live Documents RSS updates for document Reputation & Influence

Windows Compute Cluster Server 11/9/2018 3:46 AM Faster Time to Insight Better integration to existing Windows infrastructure Integrated and familiar development environment © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Research Integrate Analyze Report Data acquisition from source systems and integration Data transformation and synthesis Data enrichment, with business logic, hierarchical views Data discovery via data mining Data presentation and distribution Data access for the masses

Comparison of soil moisture From Gretchen Miller UCB Excel 2003 running against cube on SQL 2005 Analysis Services Individual plots didn’t take more than 15 mins each The familiarity of the tool allowed the scientists to do this on their own… Thanks to Gretchen Miller – UC Berkeley & Catharine Van Ingen (MSR)

11/9/2018 3:46 AM SharePoint Products and Technologies Microsoft Office SharePoint Server 2007 Server-based Excel spreadsheets and data visualization, Report Center, BI Web Parts, KPIs/Dashboards Docs/tasks/calendars, blogs, wikis, e-mail integration, project management “lite”, Outlook integration, offline docs/lists Business Intelligence Collaboration Platform Services Workspaces, Mgmt, Security, Storage, Topology, Site Model Enterprise Portal template, Site Directory, My Sites, social networking, privacy control Rich and Web forms based front-ends, LOB actions, enterprise SSO Business Forms Portal Content Management Search Integrated document management, records management, and Web content management with policies and workflow Enterprise scalability, contextual relevance, rich people and business data search © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Excel Services Overview 11/9/2018 3:46 AM 11/9/2018 3:46 AM Excel Services Overview Browser High quality web rending Zero-footprint Interactive: Set parameters, sort, filter, explore Limit to browser access View and Interact Excel 2007 Design and author Open in Excel for rich exploration and analysis Open snapshots Excel 2007 Export/Snapshot into Excel Spreadsheets stored in document libraries Spreadsheet calculation and rendering External data retrieval and caching 100% calculation fidelity SharePoint platform and Excel services Publish Spreadsheets Custom applications Set values, perform calculations, get updated values via web services Retrieve full workbook file Programmatic Access © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. 27 © 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

.NET & Visual Studio F# Iron Python SQL Sever SQL Server analysis Services Windows Workflow SharePoint Server 2007 Instant Messenger ConferenceXP Academic Live, Onfolio… Development: Data: Workflow: Collaboration: Publications:

Questions to our scientist colleagues? Can these tools/technologies provide value/insight to scientists? What’s missing? Ie. on HPC, analysis, etc? How best to test/integrate these technologies? How to communicate these ideas? Conferences, Workshops, Website? Sharecode, Samples

Conclusions Industry is moving HPC to commodity Microsoft is world leader in commodity computing and will play a major role in scientific and technical computing solutions Key figures in scientific computing such as Burton Smith, Tony Hey have recently joined the company in senior strategic positions We are interested in getting your opinion and collaborating with you to develop the most productive computing environment for science Thanks again for the invitation and see you next year!!!

More info: Windows HP www.windowshpc.net Data mining www.sqlserverdatamining.com/ Develop without Borders Challenge www.developwithoutborders.com Technical Computing Blog http://blogs.msdn.com/eScience