The goal of Data Processing From a series of diffraction images, obtain the intensity ( I ) and standard deviation (  ( I )) for each reflection, hkl. H K L I  Set of 360 imagesFinal intensities 1.Index 2.Integrate 3.Merge 3 x 10 9 bytes (3Gb) 8 x 10 5 bytes (800kB)

Indexing sounds like a trivial task plane L=0 b* a* (6,2,0)

How many dimensions? Proteinase K Crystal Diffraction Fourier transform ? Detector 332 Record

The Fourier transform of a 3D crystal is a 3D reciprocal lattice Unit cell lengths a, b, c Atom coordinates x, y, z Reciprocal cell lengths a*, b*, c* Reflection coordinates h, k, l

3D reciprocal lattice is projected on 2D detector (projection is in direction of X-ray beam). detector In an undistorted view of the reciprocal lattice, recorded reflections would reside on the surface of a sphere, not a plane.

A distortion-corrected representation of the reflections

Restored depth of the diffraction pattern is evident from an orthogonal view

Each circle corresponds to a different reciprocal lattice plane Start indexing process using reflections within the same plane and lying near the origin.

Draw a set of evenly spaced rows

Draw the vector representing this repeat distance, a*, between rows

Draw a set of evenly spaced columns a*

Draw the vector representing this repeat distance, b*, between columns a*

What is the angle between a* and b*? a* b*

Which of 14 Bravais Lattices has a=b and  =90° Cubic Rhombohedral Hexagonal/Trigonal Tetragonal Orthorhombic Monoclinic Triclinic

Draw a set of evenly spaced rows in orthogonal view a* b*

Draw the vector representing this repeat distance, a*, between rows a* b*

Draw a set of evenly spaced columns in orthogonal view a* b* a*

Draw the vector representing this repeat distance, c*, between columns a* b* a*

Is the length of c* related to a* and b*? What are the angles  and  ? a* b* a* c*

Which of 14 Bravais Lattices has a=b≠c and  =90° Cubic Rhombohedral Hexagonal/Trigonal Tetragonal Orthorhombic Monoclinic Triclinic

What is the index of the lowest resolution reflection? a* b*

What is the index of the highest resolution reflection in the l=0 plane? a* b* (-2,2,0) (3,6,0)

FILM X-ray beam Determine unit cell length “a” 1,0,0 reflection 1,0,0 (0,0,0) origin of reciprocal lattice, Also known as x-beam, y-beam crystal D CF =80 mm Da*=2.0 mm a* 1/ a*=D a /  D CF  a*= 2.0mm/80mm*1.54Å a*= Å -1 a= 61.6 Å a*/D a* = 1/  / D CF

Review which experimental parameters were required to index a spot. a* b* (-2,2,0) Coordinates of the direct beam, (X,Y) Coordinates (X,Y) for the spot position Unit cell parameters a,b,c,  The orientation of the unit cell axes with respect to the laboratory axes (  ). Crystal-to-detector distance The wavelength of the incident radiation

What are some reasons why indexing might be inaccurate or unreliable? The wavelength of the incident radiation Coordinates (X,Y) of the direct beam Coordinates (X,Y) for the spot position Unit cell parameters a,b,c,  The orientation of the unit cell axes with respect to the laboratory axes (  ).

Need a program that can index spots from multiple lattice planes without manually aligning crystal

Automatic indexing algorithm explained Acta Cryst. (1999), D55,

Locate reflections positions (peaks of high intensity) 1) Display first image in your data set with 2) Press “Peak Search”. Red circles indicate position of prominent peaks (spots). 3) Evaluate whether you need more or fewer peaks. 4) Press “OK” 5) Spot positions (x,y) are written to a file “peaks.file.” Peak Search 177 peaks found

Peaks.file height X Y frame Etc…………………………………………..

Project vectors onto a line. Measure the length of each projection.

Look for incremental differences in lengths Distribution of lengths is not incremental, it is continuous

Rotate, 7300 orientations tested.

Projected vectors for rotated image.

Sort the vector projections by length. Count the number of observations of each length. 2 vectors of length 28.5 mm 5 vectors of length 26.5 mm 5 vectors of length 24.5 mm 10 vectors of length 22.5 mm et cetera 1)Note: Projected vectors have a quantized values (distribution looks like steps). 2)The incremental difference  is proportional to the reciprocal cell length 

Fourier analysis of length histogram reveals cell dimension. Unit cell length = 62.5 Å 1-D Fourier Transform A cosine wave with periodicity of 62.5 Å is a major contributor to the 1-D FT.

Run autoindexing script The autoindexing script is simply named “a.” Type “denzo” to start the program. Then to pass instructions to Denzo.

Select a space group with desired Bravais Lattice (e.g. new space group P4) Predicted pattern should match observed diffraction pattern. “go” to refine

Paste parameters into integration script (integ.dat). Insert refined unit cell and crystal orientation parameters into integration script (integ.dat).Type “list” to obtain refined parameters..

It is not necessary to index following images from scratch. 1o1o Film 1, exposed over 1 to 2 degrees Film 2, exposed over 2 to 3 degrees

Integration 1)Draw boundaries of each reflection 2) Sum up the intensities recorded on each pixel within boundary. 3) Repeat for each spot on each film.

Integrated intensities are written to.x files Film 1, exposed over 1 to 2 degrees h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix ……………………………………………………….

One.x file for each film Film 1, exposed over 1 to 2 degrees Film 2, exposed over 2 to 3 degrees Film 360, exposed over 360 to 361 degrees h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix prok_001.img prok_001.x prok_002.img prok_360.img prok_002.x prok_360.x

h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix frames, 1 degree rotation each With.x files, we can map intensities onto a reciprocal lattice 1)Accuracy will improve if we Merge multiple observations of the same reciprocal lattice point 2)But, we must test if rotational symmetry exists between lattice points. h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix h k l flag I(profit) I(prosum)  2  (I) cos incid. X pix Y pix prok_001 -> 360.x

Choose point group symmetry (4 or 422)

Is it Point group 422 Or Point group 4? P4 H, K,L -H,-K,L -K, H,L K,-H,L H, K,-L H,-K,-L K, H,-L -K,-H,-L P422 Test existence of 4-fold symmetry Test existence of 4-fold Symmetry and Perpendicular 2-fold symmetry

j observations of the reflection = ( ) / 3 = 531 jHKL I R merge =  | I j - |  I j Discrepancy between symmetry related reflections  | I j - | = | |+| |+| | = = 62  I j = = 1593 R sym = 62/1593 = = 12.6% ? H, K, L= -H,-K, L= H,-K,-L

Merge Average (merge together) symmetry related reflections. Plane L=0 b* a* ? -H,-K, L= -K, H, L= K,-H, L= H, K,-L= H,-K,-L= K, H,-L= -K,-H,-L= -H,-K,-L= K,-H,-L= -K, H,-L= -H, K, L= -K,-H, L= K, H, L Plane L=0 b* a* K,-H,-L H,K,L -H,K,-L -K,-H,L K,H,L -H,-K,L H,-K,-L -K,H,-L

Discrepancy between symmetry related reflections (R sym ) increases with increasing resolution. Why? ShellR sym Å Å Å Å0.15 Statistics are analyzed as a function of resolution (N shells).

Average I/  decreases with increasing resolution High resolution shells with I/  <2 should be discarded. Shell I/   Å Å Å Å3.0 SIGNAL TO NOISE RATIO (I/  )

COMPLETENESS? What percentage of reciprocal Lattice was measured for a given Resolution limit? Better than 90% I hope. Shell completeness Å99.9% Å95.5% Å89.0% Å85.3% Overall 92.5%

Assignment

FILM X-ray beam Layer line screen blocks diffracted rays from upper layers ( that is, l≠0) 1,0,0 reflection 1,0,0 1/ (0,0,0) origin of reciprocal lattice, Also known as x-beam, y-beam crystal Da* a* 1,0,1

Knowing the orientation of the reciprocal lattice allows prediction of the position of each reflection on the detector a b a b..... b* a* 1/ (0,0) (0,-1) (0,-2) (0,-3) (-1,3) X-ray beam crystal detector

What’s the h,k,l of this spot? 3 lattice points in a* direction 2 lattice points in b* direction For a given spot on the film, we just trace the diffracted ray back to the reciprocal lattice point (h,k,l) The answer is HKL=3,2,2 What parameters must be defined to complete this construction?

2,0,0 3,0,-1 4,0,-2 5,0,-3 5,0,-4 5,0,-5 4,0,-6 3,0,-7 2,0,-7 1,0,-8 0,0,-8 -1,0,-8 Indexing Assign an h,k,l coordinate to each reflection of the first image. Indices h,k,l are coordinates of the reflections, analogous to how atom positions are described by coordinates x,y,z.

X-ray scattering from a 2D crystal a b Crystal Xrays Detector

Diffracted intensities arise as if reflecting from families of planes -- William Bragg. (0,1) planes a b Crystal (1,1) planes (2,1) planes (3,1) planes (0,2) planes (1,2) planes (2,2) planes (3,2) planes Xrays Detector

(0,1) planes Crystal (1,1) planes (2,1) planes (3,1) planes (0,2) planes (1,2) planes (2,2) planes (3,2) planes a b Xrays Detector What are the chances of observing reflections from all these planes in a single orientation? A) Excellent B) Zero

Crystal (1,1) planes a b Xrays Detector The planes must be oriented w.r.t the X-ray beam such that the difference in path length of each scattered ray is n. Difference in path lengths is not n. Scattered waves are out of phase. Total destructive interference. (1,1) reflection not observed in this crystal orientation. 2dsin 

Reciprocal Lattice a* b* a b Crystal Xrays Detector 1/ Sphere of reflection shows the relationship between Bragg plane (HKL) and location of reflection (HKL) on detector

RECIPROCAL LATTICE a* b* (0,1) planes length=1/d 0,1 (0,1) a b Crystal Detector Xrays (0,1) Bragg planes produce (0,1) reciprocal lattice point

RECIPROCAL LATTICE a* b* (1,1) planes length=1/d 1,1 (1,1) a b (0,1) Crystal Detector Xrays (1,1) Bragg planes produce (1,1) reciprocal lattice point

RECIPROCAL LATTICE a* b* (2,1) planes length=1/d 2,1 (1,1) (0,1) a b (2,1) Crystal Detector Xrays (2,1) Bragg planes produce (2,1) reciprocal lattice point

RECIPROCAL LATTICE a* b* (3,1) planes length=1/d 3,1 (1,1) (0,1) a b (2,1) (3,1) Crystal Detector Xrays (3,1) Bragg planes produce (3,1) reciprocal lattice point

RECIPROCAL LATTICE a* b* (0,2) planes length=1/d 0,2 (1,1) (0,1) (2,1) (3,1) (0,2) a b Crystal Detector Xrays (0,2) Bragg planes produce (0,2) reciprocal lattice point

RECIPROCAL LATTICE a* b* (1,2) planes length=1/d 1,2 (1,1) (0,1) (2,1) (3,1) (0,2) (1,2) a b Crystal Detector Xrays (1,2) Bragg planes produce (1,2) reciprocal lattice point

RECIPROCAL LATTICE a* b* (2,2) planes length=1/d 2,2 (1,1) (0,1) (2,1) (3,1) (2,2) (0,2) (1,2) a b Crystal Detector Xrays (2,2) Bragg planes produce (2,2) reciprocal lattice point

RECIPROCAL LATTICE a* b* (3,2) planes length=1/d 3,2 (1,1) (0,1) (2,1) (3,1) (2,2) (0,2) (1,2) (3,2) a b Crystal Detector Xrays (3,2) Bragg planes produce (3,2) reciprocal lattice point

RECIPROCAL LATTICE a* b* (1,1) (0,1) (2,1) (3,1) (2,2) (0,2) (1,2) (3,2) a b Crystal Detector Xrays And so on... to fill out reciprocal lattice. (1,0) (2,0) (3,0) (1,-2) (2,-2) (3,-2) (2,-1) (1,-1) (3,-1) (0,-1) (0,-2) (-1,1) (-1,2) (-1,0) (-1,-2) (-1,-1) (-2,1) (-2,2) (-2,0) (-2,-2) (-2,-1) (-3,1) (-3,2) (-3,0) (-3,-2) (-3,-1) In this crystal orientation Which reflections will appear on the detector?

RECIPROCAL LATTICE a* b* (1,1) planes (1,1) a b Crystal Detector Xrays What operation must we perform to the crystal in order to observe the (1,1) reflection? 33°

RECIPROCAL LATTICE (1,1) planes a* b* (1,1) Crystal Detector Xrays Rotate crystal to observe (1,1) reflection. a b (1,1) reflection is located here!

a b a b (1,1) planes (1,1) Crystal Detector Xrays A crystal rotation that brings the reciprocal lattice point (HKL) in contact with sphere of reflection, satisfies Bragg’s Law for reflection (HKL).  1/ d* d*/2 = 1/ sin  2dsin  = } (1,1) reflection is located here! RECIPROCAL LATTICE

a b a b (1,1) planes (1,1) Crystal Detector Xrays Indexing - We observe the location of a reflection on the detector. Which set of Bragg planes produced it? There is a reflection located here! What is it’s index? RECIPROCAL LATTICE a* b* How many increments of a* and b* is this point from the origin? d*=ha*+ kb*+ lc*

1/ RECIPROCAL PLANE Crystal Detector Xrays We collected a series of images, each covering 1° rotation. Here we rotate 15° per image. 5°5° D CF X,Y

RECIPROCAL PLANE Crystal Detector Xrays We collected a series of exposure while crystal rotates. 13°

RECIPROCAL PLANE Crystal Detector Xrays We collected a series of exposure while crystal rotates. 15°

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15°

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° °

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° °

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° °

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° ° °

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° ° ° °

RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° ° ° ° °

a* b* RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° ° ° ° a b

a* b* RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice Image rotation 10-15° ° ° ° ° ° (1,-2) (2,-2) (3,-2) (0,-2) (-1,-2) (-2,-2) (-3,-2) (2,-1) (1,-1) (3,-1) (0,-1) (-1,-1) (-2,-1) (-3,-1) (1,0) (2,0) (3,0) (-1,0) (-2,0) (-3,0) (1,1) (0,1) (2,1) (3,1) (-1,1) (-2,1) (-3,1) (2,2) (0,2) (1,2) (3,2) (-1,2) (-2,2) (-3,2) a b

a* b* RECIPROCAL LATTICE Crystal Xrays Map onto reciprocal lattice a b

3D reciprocal lattice is projected on 2D detector. detector

3 steps of data reduction Indexing –Assign H,K,L values to each reflection –We learn unit cell parameters a,b,c, , ,  Identify which of 14 Bravais Lattices Integration –Sum up the number of X-ray photons that intercepted the detector for each reflection, H,K,L. –We learn the intensity values for each recorded reflection, H,K,L Scale and Merge –Average together reflections related by symmetry –Calculate scale factor for each image to minimize discrepancies between measurements of symmetry related reflections. –We obtain the final symmetry averaged data set. –We learn the space group symmetry H K L I  H K L I 

Outline 1 Overview –We will process diffraction data in 3 steps. Briefly: –1) indexing- assign coordinates (h,k,l) to each reflection in the data set –2) integration- extract intensity values from the diffraction images for each reflection, h,k,l –3) scaling and merging -average together the multiple intensity measurements related by symmetry. Indexing is the most challenging step of the three. –The concept of indexing sounds trivial –locate spots on the image and assign them coordinates on the reciprocal lattice. But, complexity arises from the fact that the diffraction pattern has 3 dimensions and the detector used for data collection has only 2 dimensions. The 3D reciprocal lattice is projected on a 2D surface in our diffraction experiment. –Recall that the Fourier transform of a 3D crystal is a 3D reciprocal lattice. What are the names of the three coordinates used to index atoms in crystal space? (x,y,z) What are the names of the three coordinates used to index reflections in diffraction space? (h,k,l) What are the repeating unit dimensions along x,y,z? along h,k,l? –Indexing would be trivial if our diffraction images captured an undistorted view of the 3D reciprocal lattice like these. Ideally, reflections would be divided neatly into sections, aligned with reciprocal cell axes and without breaks in the pattern, as shown. Does our data come like this? No. It looks like these images: misoriented, distorted from projection, and discontinuous. It is possible to collect undistorted images of the 3D lattice using a precession camera, but unfortunately, it is inefficient and time consuming to collect data with a precession camera. –Our task of indexing is to assign h,k,l values to spots in our diffraction images. In so doing, we can map them onto an undistorted 3D reciprocal lattice, computationally—as shown here. Indexing concepts –We are going to use a program, Denzo, to help us index the thousands of reflections that we recorded. However, I would also like to show you how to index by inspection so you gain an intuitive feeling for indexing. –1) Take one of the many diffraction images that we recorded and eliminate from it the distortion due to projection of the 3D pattern onto a 2D detector. We can do this by taking into consideration the curvature of Ewald’s sphere of reflection. Recall that a reflection is recordable only when the corresponding planes in the crystal are oriented in the beam in such a way that satisfies Bragg’s law. That is, the photons reflected from the planes differ in path length precisely by integer multiples of the wavelength. This condition is satisfied when the reciprocal lattice point crosses the sphere of reflection. So, all the reflections recorded on the film originate from a curved surface. Eliminating the distortion from projecting this curved surface onto a 2D detector involves re- introducing the 3 rd dimension and restoring the curvature to the diffraction pattern. Like this. We can now see undistorted, but sparsely populated, 3D reciprocal lattice. Show orthogonal views. –2) Identify the sets of evenly spaced rows and columns in the reciprocal lattice. Draw a set of evenly spaced lines through columns of spots. Draw a set of evenly spaced lines through rows of spots. –3) Find the reciprocal cell lattice parameters. Draw the vector representing this repeat distance, a*, between columns. Draw the vector representing the repeat distance, b*, between rows.

Outline 2 –4) Note the angle between a* and b*? This is gamma. –5) Note the relationship between the lengths of a* and b*, if any. –6) Narrow down the possibilities of choice of Bravais lattice. Note, current info suggests either primitive tetragonal or cubic. –7) Identify the third unit cell length, c*. Identify the sets of evenly spaced columns in the reciprocal lattice. Draw the vector representing this repeat distance, c*, between columns. What angle does c* make with a* and b*? These are beta and gamma. –8) Is the crystal primitive tetragonal or cubic? –9) What is the index for this reflection? –Review. In our intuitive indexing process, the following parameters were either measured or derived from measurements. Which of the following parameters were measured directly from the diffraction experiment? Which parameters were derived from the measurements. –Why might indexing fail? Indexing in practice –Autoindexing using Denzo. It uses a computer algorithm to perform the same analysis as we just did here. –Identify spots. Uses an algorithm, not very good. You use your eyes and pattern recognition. –Identify rows and columns of spots. It performs a systematic search of all orientations of the image for periodicities among spot locations. It does this by projecting each spot on a line. In certain orientations the lengths of these projections will differ by integer multiples of a constant, corresponding to the reciprocal cell length. –A one-dimensional Fourier transform of the vector lengths identifies this increment, a*. –Our task of indexing is to assign h,k,l values to spots in our diffraction images so we can map them onto an undistorted 3D reciprocal lattice, computationally—as shown here. Indexing concepts –We are going to use a program, Denzo, to help us index the thousands of reflections that we recorded. However, I would also like to show you indexing by inspection so you become familiar with the concepts of indexing. –Distortions due to projection of the 3D pattern onto a –2) integration- for each reflection we sum the intensity values for all pixels within the reflection boundary –3) scaling and merging –for each unique reflection (h,k,l) average all intensity measurements of (h,k,l) and its symmetry mates. Determine scale factors to obtain the best agreement between I(h,k,l) values measured from different images.