1. PowerPoint File available: http://bl831.als.lbl.gov/ ~jamesh/powerpoint/ Oslo_2010.ppt

2. http://ucxray.berkeley.edu/~jamesh/elves Download Elves from:

3. Advanced Light Source

4. Beamline 8.3.1 staff Acknowledgments George Meigs Jane Tanamachi ALS 8.3.1 Team

5. Acknowledgements 8.3.1 PRT: Jamie Cate Center for Structure of Membrane Proteins Membrane Protein Expression Center II Center for HIV Accessory and Regulatory Complexes W. M. Keck Foundation Plexxikon, Inc. M D Anderson CRC University of California Berkeley University of California San Francisco National Science Foundation University of California Campus-Laboratory Collaboration Grant Henry Wheeler The Advanced Light Source is supported by the Director, Office of Science, Office of Basic Energy Sciences, Materials Sciences Division, of the US Department of Energy under contract No. DE-AC02-05CH11231 at Lawrence Berkeley National Laboratory. Ken Frankel Chris Neilson Michael Blum Joe Ferrara

6. Elves examine images and set-up data processing Elves run… mosflm scala solve mlphare dm arp/warp Elven Automation

7. Elves examine images and set-up data processing Elves run… mosflm scala solve mlphare dm arp/warp

8. Elven Automation Elves examine images and set-up data processing Elves run… mosflm scala solve mlphare dm arp/warp

30. Conversational User Interface user input ->

31. Conversational User Interface user input -> process the data in /data/semet

32. Conversational User Interface user input -> process the data in /data/semet recognition blah blah blah blah /data/semet

33. Conversational User Interface user input -> process the data in /data/semet recognition blah blah blah blah /data/semet simplify /data/semet

34. Conversational User Interface user input -> process the data in /data/semet recognition blah blah blah blah /data/semet simplify /data/semet preempt /data/semet contains image files

35. Conversational User Interface user input -> process the data in /data/semet report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays.

36. Conversational User Interface user input -> process the data in /data/semet report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays. confirm Everything look okay? [Yes] ->

37. Conversational User Interface user input -> process the data in /data/semet report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays. confirm Everything look okay? [Yes] -> user input distance was more like 110

38. Conversational User Interface user input -> distance was more like 110

39. Conversational User Interface user input -> distance was more like 110 recognition distance blah blah blah 110

40. Conversational User Interface user input -> distance was more like 110 recognition distance blah blah blah 110 simplify DISTANCE 110

41. Conversational User Interface user input -> distance was more like 110 report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays.

42. Conversational User Interface user input -> distance was more like 110 report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays. confirm Everything look okay? [Yes] ->

43. Conversational User Interface user input -> distance was more like 110 report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1.54 Å x-rays. confirm Everything look okay? [Yes] user input -> wavelength is wrong

44. Conversational User Interface user input -> wavelength is wrong

45. Conversational User Interface user input -> wavelength is wrong recognition wavelength = -NOT

46. Conversational User Interface user input -> wavelength is wrong recognition wavelength = -NOT simplify WAVELENGTH

47. Conversational User Interface user input -> wavelength is wrong recognition wavelength = -NOT query What is the x-ray wavelength? [1.54Å] -> simplify WAVELENGTH

48. Conversational User Interface user input -> wavelength is wrong recognition wavelength = -NOT query What is the x-ray wavelength? [1.54Å] user input -> 1 simplify WAVELENGTH

49. Conversational User Interface user input -> wavelength is wrong query What is the x-ray wavelength? [1.54Å] user input -> 1

50. Conversational User Interface user input -> wavelength is wrong report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays. query What is the x-ray wavelength? [1.54Å] user input -> 1

51. Conversational User Interface user input -> wavelength is wrong report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays. query What is the x-ray wavelength? [1.54Å] user input -> 1 confirm Everything look okay? [Yes] ->

52. Conversational User Interface user input -> wavelength is wrong report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays. query What is the x-ray wavelength? [1.54Å] user input -> 1 confirm Everything look okay? [Yes] user input -> Yes

53. Major Phasing techniques Molecular Replacement Multiple Isomorphous Replacement Multiwavelength Anomalous Diffraction Single-wavelength Anomalous Diffraction

54. ? Molecular Replacement correct structure and intensities http://www.ysbl.york.ac.uk/ ~cowtan/fourier/coeff.html

55. Molecular Replacement use something similar as a starting model

56. Model Building current model is missing something

57. Model Building phases from model

58. Model Building missing bits show up in “difference map”

59. Model Building missing bits show up better in F O + (F O - F C ) map

60. structure factor (F) spot index (h) Fitting data

61. structure factor (F) spot index (h) Fitting data

62. structure factor (F) spot index (h) Fitting data

63. structure factor (F) spot index (h) Fitting data

64. Major Phasing techniques Molecular Replacement Multiple Isomorphous Replacement Multiwavelength Anomalous Diffraction Single-wavelength Anomalous Diffraction

65. inverse Fourier Transform no phase

66. inverse Fourier Transform no phase

67. Major Phasing techniques Molecular Replacement Multiple Isomorphous Replacement Multiwavelength Anomalous Diffraction Single-wavelength Anomalous Diffraction

68. detector anomalous scattering sample x-ray beam

69. detector anomalous scattering sample x-ray beam

70. Independent tasks can be performed simultaneously Multiprocessing Strategy

83. SOLVE/P212121/done SOLVE/P21212/done SOLVE/P21221/done SOLVE/P22121/done SOLVE/P2221/done SOLVE/P2212/done SOLVE/P2122/busy SOLVE/P222/done Multiprocessing Strategy

84. epmr/P212121/model1/done epmr/P212121/model2/done epmr/P21212/model1/done epmr/P21212/model2/done epmr/P21221/model1/done epmr/P21221/model2/done epmr/P22121/model1/busy epmr/P22121/model2/done epmr/P2221/model1/ epmr/P2221/model2/ epmr/P2212/model1/ epmr/P2212/model2/ epmr/P2122/model1/ epmr/P2122/model2/ Multiprocessing Strategy

85. wARP/P212121/done wARP/P21212/done wARP/P21221/done wARP/P22121/done wARP/P2221/done wARP/P2212/done wARP/P2122/busy wARP/P222/done Multiprocessing Strategy

86. space group FOMR cryst P3 2 21 P3 1 21 P321 Multiprocessing Advantage

87. space group FOMR cryst P3 2 210.8360.323 P3 1 21 P321 Multiprocessing Advantage

88. space group FOMR cryst P3 2 210.8360.323 P3 1 210.8650.196 P3210.7260.347 Multiprocessing Advantage

89. table1.com

91. Elven Automation How often does it really work?

92. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected

93. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected 148datasets (15 MAD)

94. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected 148datasets (15 MAD) 31investigators

95. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected 148datasets (15 MAD) 31investigators 56unique cells

96. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected 148datasets (15 MAD) 31investigators 56unique cells 5 KDa – 23 MDaasymmetric unit

97. Apr 6 – 24 at ALS 8.3.1 Elven Automation 27,686images collected 148datasets (15 MAD) 31investigators 56unique cells 5 KDa – 23 MDaasymmetric unit 0.94 – 32 Åresolution (3.2 Å)

98. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets

99. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded

100. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours

101. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed

102. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours

103. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours 2 / 15MAD structures

104. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours 2 / 15MAD structures

105. NumberDescriptionPercent 446028Images (~7 TB)33% 2346Data sets47% 449MAD/SAD (1:2)19% 104Published4.4% 8.3.1 in 2003 How many structures get solved?

106. Why do structures fail?

107. Overlaps Why do structures fail?

108. Overlaps Signal to noise Why do structures fail?

109. Overlaps Signal to noise Radiation Damage Why do structures fail?

110. Overlaps Signal to noise Radiation Damage Why do structures fail?

111. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours 2 / 15MAD structures

112. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31 failed ~61 (0-231)hours 2 / 15MAD structures

113. unavoidable overlaps

114. detector

115. unavoidable overlaps phi detector

116. unavoidable overlaps mosaicity phi detector

117. unavoidable overlaps mosaicity phi detector c*

118. unavoidable overlaps mosaicity phi detector c* Ewald sphere

119. unavoidable overlaps mosaicity phi detector c* Ewald sphere

120. unavoidable overlaps mosaicity phi detector c* Ewald sphere

121. unavoidable overlaps mosaicity phi detector c* Ewald sphere

122. unavoidable overlaps mosaicity phi detector c* Ewald sphere

123. unavoidable overlaps mosaicity phi detector c* Ewald sphere

124. unavoidable overlaps mosaicity phi detector c* Ewald sphere

125. unavoidable overlaps mosaicity phi detector c* b c a

126. unavoidable overlaps mosaicity phi detector c* b c a

127. unavoidable overlaps mosaicity phi detector c* b c a

128. unavoidable overlaps mosaicity phi detector c* b c a

129. avoidable overlaps mosaicity phi detector c* b c a

130. avoidable overlaps mosaicity phi detector c* b c a

131. avoiding overlaps

134. c c

135. 1000 mm

136. avoiding overlaps 1000 mm 2 mrad 10 seconds

137. avoiding overlaps 1000 mm 2 mm 2 mrad 10 seconds

138. avoiding overlaps 1000 mm 1 mm 1 mrad 20 seconds

139. avoiding overlaps 1000 mm 300 um 0.3 mrad 60 seconds

140. Overlaps Signal to noise Radiation Damage Why do structures fail?

141. Overlaps Signal to noise Radiation Damage Why do structures fail?

142. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours 2 / 15MAD structures

143. Apr 6 – 24 at ALS 8.3.1 Elven Automation 148datasets 117succeded ~3.5 (0.1-75)hours 31failed ~61 (0-231)hours 2 / 15MAD structures

144. Radiation Damage why not just avoid it?

145. Holton & Frankel (2010) Acta D 66 393-408.

146. B ≈ 4 d 2 + 12 0 20 40 60 80 100 120 average atomic B factor 1 1.5 2 2.5 3 3.5 4 4.5 5 resolution (Å)

147. Simulated diffraction image MLFSOM simulatedreal

148. signal vs noise

150. “If you don’t have good data, then you have no data at all.” -Sung-Hou Kim

151. signal vs noise easy hard impossible

152. signal vs noise easy hard impossible threshold of “solvability”

153. signal vs noise “If you don’t have good data, then you must learn statistics.” -James Holton

154. Adding noise

155. 1 2 + 1 2 = 1.4 2

156. Adding noise 1 2 + 1 2 = 1.4 2 3 2 + 1 2 = 3.2 2 σ total 2 = σ 1 2 + σ 2 2

157. Adding noise 1 2 + 1 2 = 1.4 2 3 2 + 1 2 = 3.2 2 σ total 2 = σ 1 2 + σ 2 2

158. Adding noise 1 2 + 1 2 = 1.4 2 3 2 + 1 2 = 3.2 2 σ total 2 = σ 1 2 + σ 2 2

159. Adding noise 1 2 + 1 2 = 1.4 2 3 2 + 1 2 = 3.2 2 10 2 + 1 2 = 10.05 2

160. MAD phasing simulation Anomalous signal to noise ratio Correlation coefficient to correct model mlphare results

161. SAD phasing simulation Anomalous signal to noise ratio Correlation coefficient to correct model mlphare results

162. SAD phasing experiment Anomalous signal to noise ratio Correlation coefficient to published model

163. MR simulation Signal to noise ratio Correlation coefficient to correct density corrupted data

164. MR simulation Signal to noise ratio Correlation coefficient to correct density corrupted data

165. MR simulation Rmsd from perfect search model ( Å ) Correlation coefficient to correct density corrupted model

166. MR simulation Fraction of full search model Correlation coefficient to correct density trimmed model

167. “photon counting” Read-out noise Shutter jitter Beam flicker spot shape radiation damage σ(N) = sqrt(N) rms 11.5 e-/pixel rms 0.57 ms 0.15 %/√Hz pixels? mosaicity? B/Gray? signal vs noise

168. Which error dominates? Weak spots (high-res) background MAD/SAD (small differences) detector calibration ( if not rad dam! )

169. Holton & Frankel (2010) Acta D 66 393-408.

171. Background level sets needed photons/spot Moukhametzianov et al. (2008). Acta Cryst. D 64, 158-166

172. Holton & Frankel (2010) Acta D 66 393-408.

173. Optimal exposure time (faint spots) σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2 too long! σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2

174. Optimal exposure time (faint spots) σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2 too short!

175. σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2 Optimal exposure time (faint spots) σ total 2 = N photons + σ readout 2 + σ raddam 2 needlessly long

176. N photons ≈ σ detector 2 Optimal exposure time (faint spots) “optimal”

177. N photons ≈ 10x σ detector 2 Optimal exposure time (faint spots) “buried”

178. N spot + N bg ≈ 10x m Optimal exposure time (faint spots) “buried”

179. 0 + N bg ≈ 10x m Optimal exposure time (faint spots) “buried”

180. Optimal exposure time (faint spots)

182. t hr Optimal exposure time for data set (s) t ref exposure time of reference image (s) bg ref background level near weak spots on reference image (ADU) bg 0 ADC offset of detector (ADU) bg hr optimal background level (via t hr ) σ 0 rms read-out noise (ADU) gainADU/photon mmultiplicity of data set (including partials) Short answer: bg hr ~ 100 ADU for ADSC Q315r

183. What error dominates? Weak spots (high-res) background MAD/SAD (small differences) detector calibration if not rad dam!

184. Optimal exposure time (anomalous differences) I-I+ 3% 100 photons 10 photons 100 photons

185. Optimal exposure time (anomalous differences) I-I+ 3% 100 photons 14 photons 100 photons

186. Optimal exposure time (anomalous differences) 3% I-I+ 2000 photons 67 photons

187. Optimal exposure time (anomalous differences) 1% I-I+ 20,000 photons 200 photons

188. Minimum required signal (MAD/SAD)

189. Holton & Frankel (2010) Acta D 66 393-408.

190. “photon counting” Read-out noise Shutter jitter Beam flicker spot shape radiation damage σ(N) = sqrt(N) rms 11.5 e-/pixel rms 0.57 ms 0.15 %/√Hz pixels? mosaicity? B/Gray? signal vs noise

191. Optimal exposure time (anomalous differences) σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2

192. Optimal exposure time (anomalous differences) σ total 2 = σ spot 2 + σ bg 2 + σ readout 2 + σ raddam 2

193. Optimal exposure time (anomalous differences) no detector is perfectly calibrated! σ total 2 = N spot + σ bg 2 + σ readout 2 + σ raddam 2 + (f shutter N spot ) 2 + (f flicker N spot ) 2 + (f calib N spot ) 2 σ total 2 = N spot + σ bg 2 + σ readout 2 + σ raddam 2

194. Fractional error mult > ( — ) 2 R merge

195. Holton & Frankel (2010) Acta D 66 393-408.

196. Damage model system

197. 67 consecutive data sets

199. Data quality vs exposure Exposure time (min) Correlation coefficient

200. Data quality vs exposure Exposure time (min)  

201. Data quality vs exposure Exposure time (min)  

202. Data quality vs exposure Exposure time (min) Resolution limit

203. Data quality vs exposure Exposure time (min) R sym

204. Experimentally-phased map

206. Damage changes absorbance spectrum Photon energy (eV) counts

207. Damage changes absorbance spectrum Photon energy (eV) counts

208. Damage changes absorbance spectrum Photon energy (eV) counts

209. Damage changes absorbance spectrum Photon energy (eV) counts 1 0

210. fluorescence probe for damage Absorbed Dose (MGy) Fraction unconverted Wide range of decay rates seen 0.0 0.2 0.4 0.6 0.8 1.0 0 50 100 150 200 Half-dose = 41.7 ± 4 MGy “GCN4” in crystal Half-dose = 5.5 ± 0.6 MGy 8 mM SeMet in NaOH Protection factor: 660% ± 94%

211. http://ucxray.berkeley.edu/~jamesh/elves Download Elves from: