Re-stating the obvious: To solve structures from powder diffraction, first they must be indexed! Lachlan M. D. Cranswick CCP14 (Collaborative Computation Project No 14 for Single Crystal and Powder Diffraction) Department of Crystallography; Birkbeck College, University of London, Malet Street, Bloomsbury, London, WC1E 7HX, UK. WWW:

Based on a co-authored presentation at the ACA 2002 conference in San- Antonio, Texas, USA Lachlan M. D. Cranswick CCP14 (Collaborative Computation Project No 14 for Single Crystal and Powder Diffraction) Department of Crystallography; Birkbeck College, University of London, Malet Street, Bloomsbury, London, WC1E 7HX, UK. WWW: Robin Shirley School of Human Sciences University of Surrey Guildford, Surrey, GU2 7XH, U.K. Tel: (+44) Fax: (+44)

Lachlan M. D. Cranswick Slide 3 Notes Free Zone - they are on the web

Lachlan M. D. Cranswick Slide 4 Talk Aims Some history of Indexing of Powder Diffraction Data Peak finding and peak profiling software Fundamental Parameters Peak Fitting Available powder indexing software and indexing suites Crysfire in action Chekcell in action Mmap with Crysfire 2002 (released August 2002) MCMaille by Armel Le Bail (released Sep 2002) Possibilities for the future and conclusion

Lachlan M. D. Cranswick Slide 5 Some History of Powder Indexing First published manual powder indexing method: –Runge, C. (1917), "Die Bestimmung eines Kristallsystems durch Rontgenstrahlen", Physik. Z., 18, Candidates for first published computer based powder indexing –Haendler, H. M. & Cooney, W. A. (1963), "Computer Determination of Unit-Cell from Powder-Diffraction Data", Acta Cryst., 16, –Lefker, R. (1964), "Indexing of tetragonal and hexagonal X-ray powder photographs with the aid of a small computer", Anal. Chem., 36,

Lachlan M. D. Cranswick Slide 6 Some History of Powder Indexing - 2 Candidates for first published publicly available powder-indexing program –Haendler, H. M. & Cooney, W. A. (1963), "Computer Determination of Unit-Cell from Powder-Diffraction Data", Acta Cryst., 16, –Lefker, R. (1964), "Indexing of tetragonal and hexagonal X-ray powder photographs with the aid of a small computer", Anal. Chem., 36, –Werner, P.-E. (1964), "Trial and error computer methods for the indexing of unknown powder patterns", Z. Krist., 120, –Hoff, W. D. & Kitchingman, W. J. (1966), "Computer indexing of x-ray powder patterns from crystals of unknown structures", J. Sci. Instrum., 43,

Lachlan M. D. Cranswick Slide 7 Some History of Powder Indexing - 3 Candidates for first publically available general indexing program –Ishida, T & Watanabe, Y. (1967), "Probability Computer Method of Determining the Lattice Parameters from Powder Diffraction Data", J. Phys. Soc. Japan, 23, –Roof, R. B. (1968), "INDX: A Computer Program to Aid in the Indexing of X-Ray Powder Patterns of Crystal Structures of Unknown Symmetry", Los Alamos Laboratory, University of California, Report LA –Taupin, D. (1968), "Une Methode Generale pour l'Indexation des Diagrammes de Poudres", J. Appl. Cryst., 1, –Visser, J. W. (1969), "A Fully Automatic Program for Finding the Unit Cell from Powder Data", J. Appl. Cryst., 2,

Lachlan M. D. Cranswick Slide 8 Checking Sample Purity Powder indexing requires line-position data of the highest available quality An important aspect of this is purity, since indexing programs don’t like to struggle with impurity lines, and several will not accept them at all. Some suggestions from Robin for checking sample purity: –Optical (polarising) microscope –Density analysis in a gradient column –Electron microscopy and diffraction of selected crystallites –And, if time permits, see whether properties are changing with time

Lachlan M. D. Cranswick Slide 9 Peak Profiling Software For Overall Summary of available peak profiling software refer to: Some of the available freeware includes: –CMPR –DRXWin –EFLECH –GPLSFT –pearson.xls –Rawplot (with GSAS) –SHADOW –Powder v2.00 –PowderX –Winfit –Winplotr (with Fullprof) –XFIT

Lachlan M. D. Cranswick Slide 10 Empirical peak fitting : Some tricks may be required for stability and reasonable results Peaks over small ranges may have to be linked to have the same shape and width Usually fit small ranges of data - groups of peaks Might have to fit large and medium peaks first, followed by trace peaks

Lachlan M. D. Cranswick Slide 11 Peak Profiling: high accuracy peak positions using Fundamental Parameters peak profiling Example of Fundamental parameters (convoluting in the geometry elements of the diffractometer) that can provide accurate peak positions as though your sample was being run on an “ideal” diffractometer. Tutorial at: – Available Fundamental Parameters Peak Profiling and Rietveld software: XFIT (no longer maintained) – Topas (Commercial - sequel to XFIT) – BGMN (Commercial - academic demonstration version is freely downloadable) – EFLECH/Index freeware from BGMN website –

Lachlan M. D. Cranswick Slide 12 XFIT Fundamental Parameters Peak Profiling mode : fit to a low angle peak on a Bragg-Brentano diffractometer

Lachlan M. D. Cranswick Slide 13 Powder Indexing FOM Results XFIT Fundamental Parameters peak profiling compared to empirical Peak Fitting

Lachlan M. D. Cranswick Slide 14 Powder Indexing - indexing programs Overall Summary of available powder indexing software refer to: Powder Indexing: –Classic (widely used): Ito, Dicvol, Treor –High symmetry (exhaustive): Taup (=Powder), Dicvol –Other powerful (general symmetry): Lzon, Kohl (=TMO), Fjzn –Dominant-zone cases: Lzon, Losh, Mmap –Semi-automatic (via SIW basis set): Losh, Mmap –Fundamental parameters & covariance matrix: EFLECH/Index –Incommensurate, etc: Supercell –Monte Carlo / Random search: EFLECH/Index, McMaille –Others (better with some user guidance): Scanix, Autox

Lachlan M. D. Cranswick Slide 15 Powder Indexing - indexing suites Overall Summary of available powder indexing software refer to: Integrated Suites: –Crysfire –Powder v2.00 –PowderX –PROSZKI –WinPlotr and after primary indexing: –Chekcell

Lachlan M. D. Cranswick Slide 16 Powder Indexing - specialist programs Supercel is a specialised indexing program by Juan Rodriguez-Carvajal for tackling incommensurate cells and super/sub-cell relationships. (available within Winplotr/Fullprof) Web: FTP: ftp://bali.saclay.cea.fr/pub/divers/fullprof.2k/

Lachlan M. D. Cranswick Slide 17 Powder Indexing - specialist programs McMaille reconstitutes a peak listing into a full profile and performs monte carlo based searching for good cells. Because it is Monte Carlo based it can be very slow, but due to it being whole profile, can be insensitive to trace impurity peaks. Web:

Lachlan M. D. Cranswick Slide 18 Powder Indexing - Using PowderX –by Cheng Dong – Peak finding Links to Treor

Lachlan M. D. Cranswick Slide 19 Powder Indexing - Using Winplotr –by Juan Rodriguez-Carvajal and T. Roisnel llb.cea.fr/fullweb/powder.htm Peak finding / profiling Links to –WinTreor, –Windicvol, –Winito and –supercell

Lachlan M. D. Cranswick Slide 20 Powder Indexing - the Crysfire suite (1) –by Robin Shirley (+ many contributing authors) – Intelligent defaults for control-parameters when launching each indexing program Data rescaling for tackling high-volume cells from macromolecular samples and zeolites Integrated under a common interface for use by non- specialists

Lachlan M. D. Cranswick Slide 21 Powder Indexing - the Crysfire suite (2) Principal Contributors: Franz Kohlbeck (Kohl [=TMO]) Daniel Louër (Dicvol, Lzon, Losh) Robin Shirley (Overall + Mmap, Lzon, Losh, Fjzn, etc) Ton Spek & A.Meetsma (Lepage) Daniel Taupin (Taup [=Powder]) Jan Visser (Ito, Fjzn, Lzon, etc) Per-Eric Werner (Treor)

Lachlan M. D. Cranswick Slide 22 Powder Indexing - the Crysfire suite (3) Crysfire 2000: 8 different indexing programs (Ito, Dicvol, Treor, Taup, Kohl, Lzon, Fjzn, Losh) It’s good to have a wide variety of methods available: – to cater for different types of problem – to give a feel for the range of solutions out there – and see which ones turn up repeatedly using different methods, either identically or as derivative cells

Lachlan M. D. Cranswick Slide 23 New Release - Crysfire 2002 August 2002 Now 9 indexing programs: Ito, Dicvol, Treor, Taup, Kohl, Lzon, Fjzn, Losh, Mmap New features: Mmap for exploring solution-space and evaluating trial cells Le Page/Niggli reduced-cell summary Greater ease of use, especially for first-time users Better resilience, especially under the Windows/NT family (master program totally rewritten)

Lachlan M. D. Cranswick Slide 24 Why bother running more than one powder indexing program? Maximize the possibility of indexing unknowns by taking advantage of different algorithms and different program implementations.

Lachlan M. D. Cranswick Slide 25 Powder Indexing - Routine Crysfire Example Run crysfire (giving the following screen)

Lachlan M. D. Cranswick Slide 26 Powder Indexing - Routine Crysfire Example Importing an XFIT peak file using the IM (Import command)

Lachlan M. D. Cranswick Slide 27 Powder Indexing - Routine Crysfire Example “lo” to load a CDT file estimate of expected unit cell volume is given

Lachlan M. D. Cranswick Slide 28 Powder Indexing - Routine Crysfire Example “in” to target an indexing program

Lachlan M. D. Cranswick Slide 29 Powder Indexing - Routine Crysfire Example target ITO with defaults

Lachlan M. D. Cranswick Slide 30 Powder Indexing - Routine Crysfire Example ITO runs

Lachlan M. D. Cranswick Slide 31 Powder Indexing - Routine Crysfire Example After Ito completes, press enter to see output file

Lachlan M. D. Cranswick Slide 32 Powder Indexing - Routine Crysfire Example Then “enter” again to see “one solution per line” summary

Lachlan M. D. Cranswick Slide 33 Powder Indexing - Routine Crysfire Example “Enter” again to see Le Page summary of reduced cells

Lachlan M. D. Cranswick Slide 34 Powder Indexing - Routine Crysfire Example Repeat the sequence to run the remaining relevant indexing programs. Recommended sequence is actually: –Taup, Dicvol high sym, Ito, FJZN, Treor, Kohl, Dicvol low sym, Lzon –in to select and launch an indexing program –look at the output file –look at the summary file –look at the Le Page summary file of reduced cells –repeat

Lachlan M. D. Cranswick Slide 35 Powder Indexing - Routine Crysfire Example Finally: the complete summary file trial solutions

Lachlan M. D. Cranswick Slide 36 Powder Indexing - Routine Crysfire Example Complete Le Page summary file of reduced cells trial solutions

Lachlan M. D. Cranswick Slide 37 Crysfire Self Calibration Bragg-Bretano Data Sometimes it’s hard to be sure how far the beam is penetrating into the sample, giving an unknown sample-displacement error This can be addressed by trying self-calibration (SC) –If indexing the uncorrected data has failed, try SC –Select T (for specimen-displacement correction ratio) –A list of candidates for T will be displayed, based on 1st/2nd-order line pairs –If a particular ratio comes up several times and looks plausible, adopt it –Save it (SA) in Crysfire format under a new name, so that it can be reloaded –Do another cycle of indexing runs, this time with the “corrected” data

Lachlan M. D. Cranswick Slide 38 Crysfire Self Calibration Example Self calibration output of an organometallic sample in histogram format

Lachlan M. D. Cranswick Slide 39 Crysfire Rescaling Most indexing program have evolved using a range of data with solutions between 500 and 1500 If you have cells outside this range, many indexing programs can fail A possible solution to still index large cells it to rescale the data by changing the wavelength

Lachlan M. D. Cranswick Slide 40 Crysfire Re-scaling Protein Data (from Bob von Dreele) Input for RAWPLOT: ( Å) Lysozyme – single peak fits for 24 reflections : Rescale factor = 0.1 Rescaled cell = a of ~7.9Å and c of ~3.8Å Unscaled cell = a of ~79Å and c of ~38Å

Lachlan M. D. Cranswick Slide 41 Crysfire Rescaling example Inorganic - on opening : Crysfire suggests rescaling

Lachlan M. D. Cranswick Slide 42 Ignoring the rescaling suggestion No obvious solutions

Lachlan M. D. Cranswick Slide 43 RS is the Rescale Command Crysfire will automatically change the filename and title

Lachlan M. D. Cranswick Slide 44 After running the indexing programs: MS, then U to unscale the summary file

Lachlan M. D. Cranswick Slide 45 Indexing using the suggested rescaling of 0.5 An obvious cubic ( A) solution found by Taup

Lachlan M. D. Cranswick Slide 46 Chekcell: Interpreting Crysfire Summary Files: Powder Indexing and Spacegroup Assignment Crysfire interlinks with Chekcell for Windows (part of the LMGP suite for Windows by Jean Laugier and Bernard Bochu). Chekcell provides a graphical interface for manually and automatically suggesting a best cell/spacegroup combination using both FOM and algorithms relating to parsimony of superfluous HKLs.

Lachlan M. D. Cranswick Slide 47 Chekcell : easy to see non-matching or impurity peaks

Lachlan M. D. Cranswick Slide 48 Chekcell : automatic cell and spacegroup searching can trudge through a single selected unitcell; or over 1000s of trial cells looking for the best cell and spacegroup combination based on parsimony of extra reflections criteria.

Lachlan M. D. Cranswick Slide 49 Chekcell: “integration” of Ton Spek and A. Meetsma’s Le Page (Crysfire 2001 now also provides a reduced-cell summary) Obtaining the Reduced Cell –which in the past many powder indexing programs have not determined either reliably or at all –Refer: "'Reduced Cells', M.J. Buerger, (Zeitschift fur Kristallographie, BD 109, S (1957)” –Crysfire 2001 gives a LePage overview Chekcell displays a fuller analysis Efficient Sub-cell and super-cell searching, then easy reviewing of newly derived cells within the Chekcell interface

Lachlan M. D. Cranswick Slide 50 Chekcell: GUI Cell transformation Easily transform cells and test them withing Chekcell Knows about common transformations Can manually look at sub-cells and super-cells

Lachlan M. D. Cranswick Slide 51 Chekcell: Density / Z/ Mol. Vol explorer Easily explore values of Z, density and estimated molecular volume - comparing with your found trial cells.

Lachlan M. D. Cranswick Slide 52 Chekcell: example of it running Load the raw diffraction data

Lachlan M. D. Cranswick Slide 53 Chekcell: example of it running Load the peak position file

Lachlan M. D. Cranswick Slide 54 Chekcell: example of it running Load the Crysfire summary file

Lachlan M. D. Cranswick Slide 55 Chekcell: example of it running Use the tools to evaluate the trial cells

Lachlan M. D. Cranswick Slide 56 Crysfire / Chekcell: indexing powder Protein data Using the “Lepage” feature in Chekcell Can find the correct rhombohedral cell as published in: R. B. Von Dreele, P. W. Stephens, G. D. Smith and R. H. Blessing, "The first protein crystal structure determined from high- resolution X-ray powder diffraction data: a variant of T3R3 human insulin-zinc complex produced by grinding", Acta Cryst. (2000). D56,

Lachlan M. D. Cranswick Slide 57 Crysfire / Chekcell: indexing powder Protein data Even on rescaled data, if Crysfire and the various indexing programs do not find the “true cell”, Chekcell and possibly can find a derivative cell which LePage (combined with Chekcell’s “parsimony and superflous reflections” criteria), can find the true cell. In the case of reindexing of the protein data: – Crysfire and Dicvol finds Orthorhomic –P222 or PMM2 or PMMM : 157 hkls to match 41 reflections –LePage followed by Chekcell “parsimony check” then finds: Hexagonal setting Rhombohedral –R-3 or R3 etc : 60 hkls to match 41 reflections

Lachlan M. D. Cranswick Slide 58 MMAP Mmap explores sections of solution-space (e.g. sections with varying alpha* and beta*, holding the other 4 parameters constant) The results are displayed as a map, with the high-ground colored It lists all the peaks in the map and refines them - each is a trial solution So it can act as an indexing program in its own right (taking the basis set containing the first 4 constants from Lzon’s output) This offers another way to search for trial cells Use the new “LC” (Load a trial cell from the Crysfire summary file) to then run MMAP (using the MM command) on it.

Lachlan M. D. Cranswick Slide 59 MMAP - what does the output look like? Example MMap thumbnail output from a high FOM trial cell

Lachlan M. D. Cranswick Slide 60 MMAP - what does the output look like? Example MMap output from a high FOM trial cell

Lachlan M. D. Cranswick Slide 61 MMAP But another use is for checking out trial solutions taken from the summary lists generated by other indexing programs The usual guides to which solutions are the most promising are ones like figures of merit, numbers of lines indexed, etc., but only at the position of the refined cell in the map That’s like having a list of high points in a landscape, and trying to tell tall buildings from natural features with only their heights to go on Knowing the surroundings of the high-point tells us which are man-made, and similarly which Mmap peak corresponds to the physical solution

Lachlan M. D. Cranswick Slide 62 MMAP - which of these objects is man made? Using peak height as the selection criteria. Height of 80 Height of 78 Height of 65

Lachlan M. D. Cranswick Slide 63 MMAP - which of these peaks is man made? 2 It’s obvious once you can see the landscape

Lachlan M. D. Cranswick Slide 64 Analogous problem with powder indexing Traditional indexing only gives you the numbers (and possibly a Chekcell style fit) where a large number of trial cells can give reasonable matches Mmap shows you the actual “landscape”

Lachlan M. D. Cranswick Slide 65 An Mmap landscape Quick scan of an alpha*/beta* section for some real data for a triclinic cell (a finely-divided triclinic biological sample showing some line broadening) There are a number of high points scattered across the map Each of these peaks is a potential solution (+others others out of section)

Lachlan M. D. Cranswick Slide 66 Mmap Looking at some real data and see how figure-of- merit lists can be illuminated by looking at the landscapes that they come from.

Lachlan M. D. Cranswick Slide 67 Mmap After refinement, though one peak has reached a figure of merit of 21, it still has a long list of competitors - no one solution really stands out There are 48 peaks in the whole list, all non-equivalent 17 of these have M>10 and index all of the first 20 lines

Lachlan M. D. Cranswick Slide 68 Mmap : Is the following a good solution? Here is the traditional set of information that we have for solution 1 from that list: a = Aalpha = deg b = Abeta = deg c = Agamma = deg V = A 3 M20 = 21.02

Lachlan M. D. Cranswick Slide 69 Mmap : Yes looks a good solution Here is a magnification of the region around map-peak 1 (M=21.0), reloaded into Crysfire from the Mmap summary file using LC and then selecting a cell. It’s relatively compact (though sitting on a lowish diagonal ridge)

Lachlan M. D. Cranswick Slide 70 Mmap : Is the following a good solution? Similarly for the solution that corresponds to map-peak 2: a = Aalpha = deg b = Abeta = deg c = Agamma = deg V = A 3 M20 = 16.05

Lachlan M. D. Cranswick Slide 71 Mmap A similar magnified map around map-peak 2 (M=16.0) This is revealed to be less well defined as well as lower, with its central region extending as a broad ridge right off the top of the map

Lachlan M. D. Cranswick Slide 72 Mmap: Is the following a good solution? And for map-peak 3: a = Aalpha = deg b = Abeta = deg c = Agamma = deg V = A 3 M20 = 14.57

Lachlan M. D. Cranswick Slide 73 Mmap Map-peak 3 (M=14.6) is broader still, and also drifts off the top of the map

Lachlan M. D. Cranswick Slide 74 Mmap We can also avoid wasting time on landscapes that are broad and featureless, indicating data problems or wrong initial assumptions, and so not worth pursuing

Lachlan M. D. Cranswick Slide 75 MCMaile by Armel Le Bail  Developed in September 2002 (on from the ESPOIR code)  GPL’d (free and you get the source code)  Whole profile Monte Carlo and Grid Search methods using a pattern reconstituted from the peak list  Columnar peak shape - can increase/decrease width  Can make “important” trace peaks more likely to be part of a solution by increasing their intensities.  Insensitive to impurity peaks  20,000 tests per second - but can still take many hours  Thus a method of last resort

Lachlan M. D. Cranswick Slide 76 A MCMaile ASCII input file

Lachlan M. D. Cranswick Slide 77 MCMaile is running

Lachlan M. D. Cranswick Slide 78 MCMaile output files  Chekcell *.ckm summary file of all trial cells and Fullprof PRF profile file of top solution: viewable in Winplotr

Lachlan M. D. Cranswick Slide 79 MCMaile : on large cubic  Finds cubic cell easily - even with trace “impurity” peaks  Via outputted Chekcell *.ckm summary file. Top solution is not correct solution (possible effect of impurity peaks)

Lachlan M. D. Cranswick Slide 80 MCMaile : on large cubic  Finds cubic cell easily - even with trace “impurity” peaks  Via outputted Chekcell *.ckm summary file. Second set of solutions is correct

Lachlan M. D. Cranswick Slide 81 SDPDRR-II (Structure Determination by Powder Diffractometry Round Robin - 2 )  WWW:  Started: Monday 9th September  Part 1 (powder indexing):  Start: Monday 9th September  Finish: Sunday 13th October  8 samples  : Powder indexing and structure solution  : Powder Indexing only  > 100 downloads of the data (samples 1 to 3)

Lachlan M. D. Cranswick Slide 82 SDPDRR-II - Indexing Results  WWW:  6 returns/100 downloads = 6% return rate

Lachlan M. D. Cranswick Slide 83 SDPDRR-II - Sample 1  WWW:  Organisers:  Reduced cell:  P1:  P2:  P3:  Reduced cell:  P4:No solution  P5:  P6:

Lachlan M. D. Cranswick Slide 84 SDPDRR-II - Sample 2  WWW:  Organisers:  Reduced cell:  P1:  P2:  P3:  P4:  P5:  P6:

Lachlan M. D. Cranswick Slide 85 SDPDRR-II - Sample 3  WWW:  Organisers: (vol 6734Å 3 )  P1: (Tet Å 3 )  P2:  P3: (Tet Å 3 )  P4:No Solution  P5:  P6:

Lachlan M. D. Cranswick Slide 86 SDPDRR-II - Sample 3 Tetragonal or Cubic?  2 participants (1 and 3) obtained the same tetragonal cell instead of the correct cubic.  Possibly due to default volume restrictions in the programs used(?)  What happens if you run LePage (via Chekcell) followed by a “parsimony of extra reflections” check on the tetragonal solution

Lachlan M. D. Cranswick Slide 87 SDPDRR-II - Sample 3: Tetragonal: 1 peak unmatched - 46 hkl’s (P42MC et al) to match 25 reflections

Lachlan M. D. Cranswick Slide 88 SDPDRR-II - Sample 3: Tetragonal: Running LePage shows a cubic cell (18.885Å)

Lachlan M. D. Cranswick Slide 89 SDPDRR-II - Sample 3: Visual inspection of cubic cell implies “bad”peak could still be an impurity

Lachlan M. D. Cranswick Slide 90 SDPDRR-II - Sample 3: Cubic: 37 hkl’s to match 25 reflections (I23 et al) (vs 46 hkl’s in tetragonal)

Lachlan M. D. Cranswick Slide 91 SDPDRR-II - Samples 4 to 8  WWW:  Only one response - using the EFLECH/Index software of Joerg Bergmann ( )  S4:none suggested  S5:  S6:none suggested  S7:  S8:  Summary: Powder indexing is not easy and not routine!

Lachlan M. D. Cranswick Slide 92 Be on the lookout for bugs in the Software! Found in LePage - within Chekcell

Lachlan M. D. Cranswick Slide 93 The Future  The message has got through that indexing has become a bottleneck for solving structures from powder diffraction data.  New Indexing programs are being developed.  MCMaille (Free GPL’d - get the source code as well)  Bruker SVD Indexing (commercial)  Accerlys Indexing program (commercial)  EFLECH/Index (new developments due to the SDPDRR-2)

Lachlan M. D. Cranswick Slide 94 Conclusion  Don’t give up if your lab’s favorite indexing program doesn’t work on a particular dataset  Suites like Crysfire offer a wide range of indexing tools  Follow on with Chekcell to help pin down the correct physical cell  If that does not work - whole profile methods such as MCMaille (still using Chekcell to evaluate trial solutions) Relevant web links: CCP14 Summary Page: –Crysfire: –Chekcell: –McMaille: