DANSE Diffraction Software for the SNS: DiffDANSE S.J.L. Billinge Dept. Physics and Astronomy Michigan State University.

Slides:



Advertisements
Similar presentations
The essentials managers need to know about Excel
Advertisements

Clicking on the link for the.tns file gives you will get the following screen: Select Save File and click OK.
Process Monitoring is only the first step in improving process efficiency.
Computer Basics Hit List of Items to Talk About ● What and when to use left, right, middle, double and triple click? What and when to use left, right,
PIIT Computer Science Summer Camp - Alice July 10, 2012 Brenda Parker Computer Science Department MTSU.
® IBM Software Group © 2010 IBM Corporation What’s New in Profiling & Code Coverage RAD V8 April 21, 2011 Kathy Chan
DANSE – DiffDANSE report and Community Engagement S.J.L. Billinge Department of Applied Physics and Applied Mathematics Columbia University, CMPMS, Brookhaven.
Calendar Browser is a groupware used for booking all kinds of resources within an organization. Calendar Browser is installed on a file server and in a.
DANSE Diffraction sub-group report S.J.L. Billinge Dept. Physics and Astronomy Michigan State University.
Software Engineering.
1 of 7 This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation.
Chapter 1 Program Design
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. © 2007 Microsoft Corporation. All.
Microsoft Excel 2003 Illustrated Complete Excel and Advanced Worksheet Management Customizing.
TRACK 3™ The ultimate process management software.
Calendar Browser is a groupware used for booking all kinds of resources within an organization. The software is totally integrated in Outlook. Calendar.
Working with SharePoint Document Libraries. What are document libraries? Document libraries are collections of files that you can share with team members.
ADVANCED MICROSOFT POWERPOINT Lesson 6 – Creating Tables and Charts
PYP002 Intro.to Computer Science Microsoft Word1 Lab 07 Creating Documents with Efficiency and Consistency.
McGraw-Hill/Irwin The Interactive Computing Series © 2002 The McGraw-Hill Companies, Inc. All rights reserved. Microsoft Excel 2002 Exploring Formulas.
TrendReader Standard 2 This generation of TrendReader Standard software utilizes the more familiar Windows format (“tree”) views of functions and file.
Microsoft Windows LEARNING HOW USE AN OPERATING SYSTEM 1.
The Agile Primer July 2008 © ThoughtWorks 2008.
An Introduction to Software Architecture
CIS—100 Chapter 15—Windows Vista 1. Parts of a Window 2.
Objectives Understand what MATLAB is and why it is widely used in engineering and science Start the MATLAB program and solve simple problems in the command.
MagicInfo Pro Scheduler Now that a template has been created from content imported into the Library, the user is ready to begin scheduling content to.
DANSE Diffraction Software for the SNS: DiffDANSE S.J.L. Billinge Dept. Physics and Astronomy Michigan State University.
An Introduction to Designing and Executing Workflows with Taverna Katy Wolstencroft University of Manchester.
Advanced Project Plan Formatting Lesson 14. Skills Matrix SkillsMatrix Skill Customize the calendar view Format bar styles for tasks in the Calendar view.
Introduction to Excel Line Graphing The ‘Quick’ and ‘Easy’ guide to using Microsoft Excel for Line Graphing * Created by: Bunch of BHS science teachers.
The Fundamentals of Using Windows 95. Windows 95 ã operating system that performs every function necessary for the user to communicate and control computer.
Computing Fundamentals Module Lesson 3 — Changing Settings and Customizing the Desktop Computer Literacy BASICS.
RMS Importer Status MACS Week March 2011 PP b-ABR_RMSImporterStatus Angela Brett RMS Importer Status 1.
Key Applications Module Lesson 21 — Access Essentials
An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik materials by: Katy Wolstencroft University of Manchester.
Graphing and statistics with Cacti AfNOG 11, Kigali/Rwanda.
CTS130 Spreadsheet Lesson 19 Using What-If Analysis.
Microsoft Project – Tutorial 4 Leveling Overallocations In some cases, a resource could be assigned more work in a given time period than it has working.
240-Current Research Easily Extensible Systems, Octave, Input Formats, SOA.
Microsoft ® Office Excel 2003 Training Using XML in Excel SynAppSys Educational Services presents:
Your Final Concert Movie Project How you will create a movie about the song you took “pictures” of.
Processing Lab 2 – Geometry Bryce Hutchinson Objectives: Add a dictionary Gain a better understanding of header issues Display geometry correctly Visualize.
#ZOLLSummit. RescueNet Billing Batch Posting Presented by Stacey Bickford & BJ Terrill.
How to Check Circulation Statistics by Campus by Month A tutorial for librarians using Agent VERSO.
L.T.E :: Learning Through Experimenting Using google-svn for MtM Docs Development Denis Thibault Version 3.2 Mar 12 th, 2009.
Compare and Contrast : Blackboard & a Personal Web Page www3.ltu.edu/~s_schneider/howto/faculty.htm You’ll find this presentation (and another) here :
IAEA Training Course on Safety Assessment of NPPs to Assist Decision Making Temelin NPP Risk Panel A PSA and Safety Monitor Application Workshop Information.
WS 9-1 ANSYS, Inc. Proprietary © 2009 ANSYS, Inc. All rights reserved. February 27, 2009 Inventory # Workshop 9 Taylor Impact Test – “What if” Study.
TEAM FOUNDATION VERSION CONTROL AN OVERVIEW AND WALKTHROUGH By: Michael Mallar.
MICROSOFT WORD PRESENTATION. Word Processing  Software that is designed for the entry, editing, and printing of documents.  Windows Version = Microsoft.
File and File Systems Compiled by IITG Team Need to be reorganized and reworded.
Program Design. Simple Program Design, Fourth Edition Chapter 1 2 Objectives In this chapter you will be able to: Describe the steps in the program development.
3M Partners and Suppliers Click to edit Master title style USER GUIDE Supplier eInvoicing USER GUIDE The 3M beX environment: Day-to-day use.
Editing and Debugging Mumps with VistA and the Eclipse IDE Joel L. Ivey, Ph.D. Dept. of Veteran Affairs OI&T, Veterans Health IT Infrastructure & Security.
McGraw-Hill/Irwin The Interactive Computing Series © 2002 The McGraw-Hill Companies, Inc. All rights reserved. Microsoft Excel 2002 Using Macros Lesson.
COMPREHENSIVE Excel Tutorial 12 Expanding Excel with Visual Basic for Applications.
Template Package  Presented by G.Nagaraju.  What is Template Package?  Why we use Template Package?  Where we use Template Package?  How we create.
The Next Step Hudson Fare Files 102 – Import & upload Rev. 10/14.
Chapter 8 Using Document Collaboration, Integration, and Charting Tools Microsoft Word 2013.
Data Virtualization Demoette… ODBC Clients
with a few tips and tools for managing mail
Tips and tools for managing the new interface
Lecture 4 : Windows 7 By MSc. Manar Joundy Hazar 2017
Microsoft Office Access 2003
Microsoft Office Access 2003
Introduction to TouchDevelop
AGILENT TECHNOLOGIES RESTRICTED
An Introduction to Designing and Executing Workflows with Taverna
MAINTAINING FILES AND CUSTOMIZING WINDOWS Section 2
Presentation transcript:

DANSE Diffraction Software for the SNS: DiffDANSE S.J.L. Billinge Dept. Physics and Astronomy Michigan State University

DiffDANSE goals Make diffraction data analysis software that SNS users (and other people) want to use! Make sure that it has a long life Have fun doing it

DiffDANSE goals Make diffraction data analysis software that people want to use! –Solves people’s scientific problems, opens new scientific horizons –Shortens time to publication –Flexible –Easy to use Good user documentation –Robust and Stable

DiffDANSE goals Make sure that it has a long life –Maintainable (esp. by others, i.e., SNS!) Good developer documentation Requirements/use cases/tests stored with the code –Extensible modular –Scalable Dynamic memory allocation Distributable/parallelizable –Quality controlled –Open source

Outline Goals Progress Planning

Progress Management and Organisation Software

Management and Organisation Sub-contracts –In place –MSU hasn’t invoiced Caltech yet – need to clear up what to bill for –Second period – not in place, should be quick. Need to follow up after my trip to Europe Hiring Project controls Training Software process

DiffDANSE development team

Management and Organisation Sub-contracts Hiring Project controls –Emil Bozin handling reporting functions –Use Google spreadsheets for collecting developer input –Cut and paste to MS excel for computing information to forward to Mike Not completely trouble-shot, but hopeful that this will work well with little overhead for everyone. See Emil for demo. Training Software process

Management and Organisation Sub-contracts Hiring Project controls Training –On-the-job training Works ok because of interactive group –Pyre training is a priority. Working with MM and MA after this meeting to jump-start this Software process

Management and Organization Sub-contracts Hiring Project controls Training Software process –Objective: satisfy the needs of project reporting the requirement of usable, extensible, maintainable code quality assurance requirements –Requirements: Minimize developer/management overhead

Progress Management and Organisation Software

PyMol Mostly we used this to develop our software development process! However, software mostly done, Dmitriy waiting for some assistance with pyrolization Still not satisfied with all aspects such as documentation and review Attention focusing away a bit as one thing we learned from this is that such having such small “bite-sized” projects is not amenable to agile development and group activity Ask developers for more information

PDFgui Prototype application Very successful from our perspective –Took one application “all the way” and learned a lot in the process –Resulting application we believe is good… Meets the success definition Expect that we can reuse much of the gui design –Excellent feedback from alpha-testers Status –Alpha0 and alpha1 releases were successful –Now have roughly 10 installations in different groups. Very positive feedback –Todo: beta by first week in April (documentation, windows)

Configuring a set of fits for a T- series on LaMnO 3 data.

Plotting pane of PDFgui

PDFgui Prototype application Very successful from our perspective Status –Alpha0 and alpha1 releases were successful –Now have roughly 10 installations in different groups. Very positive feedback –Todo: beta by first week in April (documentation, windows) Lessons learned –Too many to list; a very valuable experience –Need to work on build and release infrastructure now!

Outline Goals Progress Planning

Planning Management and Organization –Hiring is ok for now –Need a bit more work on project reporting etc., but getting close. Plan is to have it working smoothly by the end of March –Software process: Made a lot of progress. Need to consolidate what we have done and fine-tune it. Need to work harder on software review and developer documentation. Not just passing external reviews but having developers work on each others code. Seek to have it finalized by end of the semester –Need better ways to do builds Software

Planning Management and Organization Software –DiffPy –DiffLab –SrRietveld –SrReal

DiffPy Python/Pyre component libraries for diffraction

DiffLab Build your own application (Application Machine-tool Application) –Very much the motivation behind DANSE –This will be an application, initially, that just builds configurable regression modeling applications, –….but it is extensible for other sub-groups Basic idea: model-centric modeling -> Cost-function-centric modeling drag and drop -> program introspection and resolution

Modeling Need a model –Different models Need a cost function Need data –Different data

Run DiffLab application –Difflab: What do you want in your cost function? –Difflab: Here are the possibilities that I can find –Scientist: selects tof Neutron S(Q) chi^2 –Difflab: looks into this chi 2 module and checks for dependencies –Difflab: looks into DiffPy/DANSE libraries for all components that can satisfy those dependencies and presents a drop-down list of dependency resolutions –Scientist selects desired component –Repeat exhaustively to complete the tree

Motivation Some motivating science problems in complex materials structure analysis: Molecules intercalated in zeolite –Z-matrix representation for molecules, periodic crystal for host –Two different models have to interact self-consistently Energy minimization in a regression loop –Nesting Cu 3 Au –Monte Carlo ising-type moves then efficient relaxation steps Amorphous model builder…use energetics and relaxation in the model building process

ConfigureRuntime Cost function: Adding data Run Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Presents a list of currently supported data types which is dynamically generated. User selects and next a list of required and optional configuration items is shown. This list is generated by looking for dependencies. Items with complete configuration are green, unfinished ones red. Items where defaults are used might have there own color.  Neutron S(Q)  Neutron G(r)  X-ray constant wavelength Rietveld data  … ConfigureRuntime Cost function: Adding data Run Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Neutron G(r) on the right is red…it has unresolved dependencies. DiffLab introspects to find these dependencies and makes a list below. The  sign means that clicking on that will expand the tree in that direction, similar behavior to microsoft explorer file agent.  Data source  Cost function  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

By expanding Data source we see the suggestions that DiffLab makes for resolving the dependencies. It generates these lists dynamically by looking at the library to see what is available. The + sign changes to a minus sign (but I can’t do it easily in ppt). The amber color (between red and green) means that the dependency will be resolved with user- input at run-time  Data source  Local file  Lujan NPDF data server  Enter manually  Cost function  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Selecting “read from file” causes “Data source” to turn green as this now has its dependencies resolved and can run (arguably the behavior just described is just a dialog between DiffLab and the user so maybe should take all take place at run-time and not at configuration time, but we can discuss that later. This at least illustrates the behavior). Now compress “Data source” (if you want) and expand “Cost function”…  Data source  Local file  Lujan NPDF dataserver  Enter manually  Cost function  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Let’s do normal least squares, SUM(w ij (G calc -G meas ) 2 so select CHI^2  Data source  Cost function  CHI^2  Bill David’s Robust statistics  Custom  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Only one thing left to get green so click it  Data source  Cost function  CHI^2  Data  Calculated function  weights/errors  Bill David’s Robust statistics  Custom  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Again, Difflab finds what is available and lists them. We want to fit a model which we do in the model tab on the right, so let’s click it.  Data source  Cost function  CHI^2  Data  Calculated function  Model (expand model at right)  User supplied  weights/errors  Bill David’s Robust statistics  Custom  … ConfigureRuntime Cost function: Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

Cost function will now turn green when we satisfy all the “model” dependencies (maybe we need a different color as Amber means the program will run and will give a dialog box to get the user input. Here the color indicates that it is waiting for some dependencies to be resolved, but not in “Data” but in “Model”. Notice that Neutron G(r) has turned yellow as all it doesn’t need any more “configuring” from the user, so we can go on by adding another data-type (e.g., RAPDF G(r) or RAPDF F(Q)), or move to model…  Data source  Cost function  CHI^2  Data  Calculated function  Model (expand model at right)  User supplied  weights/errors  Bill David’s Robust statistics  Custom ConfigureRuntime Cost function: A1*[Chi2 Neutron G(r)] Adding data Run Neutron G(r) Data - Model + Plotting + RemoveAdd click DIFF Lab 0.01 X

ConfigureRuntime Cost function: CHI^2(Neutron G(r))+ Adding model Run Data + Model - Plotting + RemoveAdd click Open the model pane and click add… DIFF Lab 0.01 X

Difflab looks for all the model possibilities and gives the list here. Lets make a PDFfit type program so we want crystal structure model: Crystalline structure (small unit cell, periodicity) Large box, periodic BC’s Large box, hard boundary molecular (Z-matrix) … ConfigureRuntime Cost function: A1+CHI^2(Neutron G(r))+ Adding model Run Data + Model - Plotting + RemoveAdd click DIFF Lab 0.01 X

Difflab looks for all the model possibilities and gives the list here. Lets make a PDFfit type program so we want crystal structure model: Crystalline structure (small unit cell, periodicity) Large box, periodic BC’s Large box, hard boundary molecular (Z-matrix) … ConfigureRuntime Cost function: A1+CHI^2(Neutron G(r))+ Adding model Run Data + Model - Plotting + RemoveAdd click DIFF Lab 0.01 X

Now, let’s build PDFfit by clicking on Crystal model ConfigureRuntime Cost function: A1*CHI^2(Neutron G(r)) +.. Adding model Run Crystal model Data + Model - Plotting + RemoveAdd click DIFF Lab 0.01 X

SrRietveld Status: –Collecting brief use-cases –Training Wenduo in Rietveld and profile fitting methods –Time to begin discussing more seriously with Jason Hodges and Engineering sub-group and other stake-holders Had a visit from Jason, will follow up with more

SrReal Real-space fitting clone of SrRietveld –May be combined into one application, or left separately as two –Replacement for PDFgui which is not modular and not using DiffPy libraries (though much of the PDFfit2 codes will be reused in DiffPy)

PDFgui Make sure that it has a long life –Modular –Clear separation of gui from program logic. Interaction between the program and the gui through proxies –Team development results in code that is readable by more than one human being –Coding standards for docstrings and basic syntax structure –Project under subversion control under TRAC

Thanks!