1. Cleaning up the mess

2. Scalars Name Dynamic Part Data Time stamp Alarm MetaData
Min / Max 0, 450.0
Data Type Double
Units Microns

3. N-D Array
Name Data [] /* Dynamic Part */ Time stamp Alarm Min / Max /* value MetaData */ Data Type Units Data Size Number of Dimensions Per Dimension /* Per Axis MetaData */ Size of Dimension Min / Max (offset and extent of this axis) Axis label Axis Representation (interval, Array, (pattern?)) Interval Array of values for each point on this axis or the interval value

4. N-D Array - Data Portion
Data [] /* Dynamic Part - this needs to be a reference too!! */ Time stamp Alarm Min / Max /* value MetaData */ Data Type /* this is the type for all of the samples in the array */ Units Data Size Number of Dimensions

5. N-D Array – Axis Information
Number of Dimensions Per Dimension /* Per Axis MetaData */ Size of Dimension /* how many values in this dimension Min / Max /* what is the origin and maximum */ /* this can be used for tiling */ /* or comparing results from different detectors *. Axis label /.* need to know the units for operations */ Axis Representation (interval, Array, (pattern?)) Interval /* describes the size of this pixel or cell */ Array of values for each point on this axis or the interval value

6. 1D Det., Raster Sample 1-D Detector: 6 10 eV in X
The detector is not offset – origin point is 0,0
Sample that is scanned in 2 dimensions:
300 microns x 100 microns
1 step = 10 micron motion in x and y
DCM:
energy set at 200 eV to 1000 eV ; 2 eV resolution

7. Scalars X Position Data /* Dynamic Part */ Time stamp Alarm
Min / Max 0, /* MetaData */
Data Type Double
Units Microns
Y Position
Min / Max 0, /* MetaData */
Energy
Min / Max 0, /* MetaData */
Units eV

8. 1-D Array – Interval Axis
1D-Detector Data [6] Timestamp Alarm Min / Max 0, 2400 Data Type U16 Units Photons Data Size 6 x Sizeof(UInt) Number of Dimensions 1 Axis information [0] Size of Dimension 6 Min / Max 0, 240 (offset and extent of this axis) Axis label eV Axis representation Interval Interval 10 AxisArray[] NULL

9. Energy Scan for each sample spot
Set Energy to appropriate power (
(for each 50 micron row of the sample)
For (x=0, x<=100; x+= 50)
(for each 100 micron column)
For (y=0; y <= 300; y+= 100)

10. Instrument Data Y X energy detector frame (pixel)
[6] 0 (4 Y positions)
[6] 1
[6] 2
[6] 3
[6] 4
[6] 5
[6] 6
[6] 7
[6] 8 (3 X positions)
[6] 9
[6] 10
[6] 11

11. Instrument Data Requests Each Pixel Where Energy Matches Some 1D Pattern
Y X energy detector frame
[6] 4
[6] 5
[6] 8
[6] 9

12. Export Detector Scan as an N-DArray
1D-Detector Data [] Min / Max 0, 2400 Data Type U16 Units Photons Data Size 6 x 12 x sizeof(U16) Number of Dimensions 2 Axis information [0] Size of Dimension 6 Axis Data type U16 (Type of all of the samples) Axis Units Photon Counts Axis representation Interval Interval 1 AxisArray[] NULL Axis information [1] Size of Dimension 12 Min / Max 0, 11 (offset and extent of this axis) Axis Data Type U16 Axis Units Frame AxisArray[] Null

13. Export Detector Scan – Not 4D but: 1D + Scalar + Scalar (all over time)
1DDetector Data Size 6 x 12 x sizeof(U16 Min / Max 0, 2400 Data Type U16 Units Photons Number of Dimensions 2 Axis information [0] Size of Dimension 6 Min / Max 0, 2400 Axis Data type U16 Axis Units Photon Counts Axis representation Interval Interval 1 Axis information [1] Size of Dimension 12 Min / Max 0, 11 Axis Data Type U16 Axis Units Frame AxisArray[] Null
X Position Data Size 12 x sizeof(Double) Min / Max 0, Data Type Double Units Microns Number of Dimensions 1 Axis Information [0] Size of Dimension 12 Min/Max 0,11 Axis Dtype U16 Axis Label Frame Axis Represntation Interval Interval 1 AxisArray NULL Y Position Min / Max 0, 300.0

14. Start cleaning from here…….
2-D Detector:
6x6 pickups
75 micron spacing in X
60 micron spacing in Y
450 micron by 360 micron in size
The detector is not offset – origin point is 0,0
Sample that is scanned in 2 dimensions:
300 microns x 100 microns
1 step = 10 micron motion in x and y
DCM:
energy scans 200 eV to 1000 eV ; 2 eV resolution

15. Scalars X Position Data /* Dynamic Part */ Time stamp Alarm
Min / Max 0, /* MetaData */
Data Type Double
Units Microns
Y Position
Min / Max 0, /* MetaData */
Energy
Min / Max 0, /* MetaData */
Units eV

16. 1-D Array – Interval Axis
1D-Detector Data [6] Timestamp Alarm Min / Max 0, 2400 Data Type U16 Units Photons Data Size 6 x Sizeof(UInt) Number of Dimensions 1 Axis information [0] Size of Dimension 6 Min / Max 0, 240 (offset and extent of this axis) Axis label eV Axis representation Interval Interval 10 AxisArray[] NULL

17. 2-D Array – not an interval
2D Detector Data Size 36 x Sizeof(Data Type) Data Type U16 Units Photons Display Range 0,2400 Number of Dimensions 2 Axis information [0] Size of Dimension 6 Min / Max 0, 450 (offset and extent of this axis) Axis label Microns Axis representation Array Interval .75 AxisArray[] 0.0, .75, 1.50, (gap) 3.50, 4.25, 5.0 Axis information [1] Min / Max 0, 360 Interval .60 AxisArray[] 0.0, .60, 1.20, (gap) 1.50, 2.10, 2.70

18. Energy Scan for each sample spot
(for each 50 micron row of the sample)
For (x=0, x<=100; x+= 50)
(for each 100 micron column)
For (y=0; y <= 300; y+= 100)
(energy scan)
For (energy = 200; energy <= 1000;
energy += 200)

19. Data Sets Y X energy detector frame 0 0 200 [6x6] 0 (5 energies)
[6x6] 20 (4 y positions)
…………………………..
[6x6] (3 x positions)
…. [6x6] ….
[6x6] 60

20. Data Set Requests 1 spot of the sample at every energy
Y X energy detector frame
[6x6] 6
[6x6] 7
[6x6] 8
[6x6] 9
[6x6] 10

21. Data Set Requests All spots at one energy
Y X energy detector frame
[6x6] 0 (1 energy)
[6x6] 6
[6x6] 11
[6x6] 16 (4 y positions)
[6x6] 21
………………………..
[6x6] (3 x positions)

22. Export Detector Scan as an N-DArray
Data Size [6 x 6 x 60]
Data Type U16
Units Photons
Display Range 0,2400
Nujmber of Dimensions 3
Axis information [0]
Size of Dimension 6
Min / Max 0, (offset and extent of this axis)
Axis label Microns
Axis representation Array
Interval .75
AxisArray[] 0.0, .75, 1.50, (gap) 3.50, 4.25, 5.0
Axis information [1]
Min / Max 0, 360
Interval .60
AxisArray[] 0.0, .60, 1.20, (gap) 1.50, 2.10, 2.70
Axis information [2]
Size of Dimension 60
Min / Max 0, 11
Axis label Frame
Axis representation Interval
Interval 1
AxisArray[] Null

23. Export Detector Scan as an N-DArray
Data Size [6 x 6 x 60]
Data Type U16
Units Photons
Display Range 0,2400
Nujmber of Dimensions 3
Axis information [0]
Size of Dimension 6
Min / Max 0, (offset and extent of this axis)
Axis label Microns
Axis representation Array
Interval .75
AxisArray[] 0.0, .75, 1.50, (gap) 3.50, 4.25, 5.0
Axis information [1]
Min / Max 0, 360
Interval .60
AxisArray[] 0.0, .60, 1.20, (gap) 1.50, 2.10, 2.70
Axis information [2]
Size of Dimension 60
Min / Max 0, 11
Axis label Frame
Axis representation Interval
Interval 1
AxisArray[] Null

24. Detector Scan with Energy = 3D+1D
Data Size [6 x 6 x 60]
Data Type U16
Units Photons
Display Range 0,2400
Nujmber of Dimensions 3
Axis information [0]
Size of Dimension 6
Min / Max 0, (offset and extent of this axis)
Axis label Microns
Axis representation Array
Interval .75
AxisArray[] 0.0, .75, 1.50, (gap) 3.50, 4.25, 5.0
Axis information [1]
Min / Max 0, 360
Interval .60
AxisArray[] 0.0, .60, 1.20, (gap) 1.50, 2.10, 2.70
Axis information [2]
Size of Dimension 60
Min / Max 0, 11
Axis label Frame
Axis representation Interval
Interval 1
AxisArray[] Null
Data Size 60
Data Type Double
Units Energy
Display Range 200,1000
Number of Dimensions 1
Azxis [0]
Size of Dimension 60
Axis Label Frame
Axis Representation Interval
AxisArray Null

25. Other issues
Operations done on data
Non-measured Metadata
Provenance

26. Common Operations Flat field subtraction Binning Peak detection …..
Others?

27. Other Related Variables
What about the description of the locations of the sample holder and detector to understand things relative to each other?
Should we store them in the archiver as variables with values and metadata?
Distance
Size
Angle
What about compiling the positions of the sample into one large array? The 2D data sample gets bigger – do this example.

28. Tracking Pedigree Processed data sets
Any data set can have a number of attributes/processing
The processing has source
Some action have meta data
This is an nearly exhaustive set of libraries
Operations – metadata for a data set
Read source
Flat field subtraction flat field data set
Integrate/exposure time time
Compression
Region of Interest
Binning (what if any of this is done at input time?)