1. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 1 Some Data Reduction Schemes for MicroBooNe Wu, Jinyuan Fermilab June, 2009

2. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 2 Data Reduction on Liquid Argon TPC Data Hit waveforms in TPC carry useful information. Digitizing the waveforms creates large volume of data. Data reduction without losing useful information is necessary. Drift Time Wire Number Data from BO detector of FNAL

3. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 3 Slow Variation of Raw Data More than 99% points differ from previous points by -1, 0 or +1. Huffman Coding can be applied to the differences of the data points. DFF Q A B A-B U(n+1) D U(n+1)-U(n)

4. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 4 The Huffman Coding The U(n+1)-U(n) value with highest probability is assigned to shortest code, i.e., single bit 1. Values with lower probabilities are assigned with longer codes, e.g., 01, 001, 0001 etc. Huffman coded words and regular words are distinguished by bit-15. U(n+1)- U(n) Code -4 and others Full 16 bits word -3000001 -20001 01 01 +1001 +200001 +30000001 1 00 ADC value (13-bit) Regular ADC data for first point or when U(n+1)-U(n) is outside +-3 Huffman Coded 000+1+2 Padding or Continue to Next Word In this example, 6 differences of the data samples are packed in the 16-bit data word. 111111000000000

5. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 5 The Huffman Coding Block The block is able to operate at up to 250MHz clock in Altera Cyclone III FPGA devices. The block uses 245 logic cells, taking 0.6% in an EP3C40F484C6 device ($129) containing 39600 logic cells. Raw Data Huffman Coded Data 245 Logic Cells (245/39600)*$129 = $0.80 1 00 ADC value (13-bit) 000+1+2 111111000000000

6. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 6 The Compress Ratio of Huffman Coding On typical TPC events a compression ratio of about 10 can be achieved. Compression ratio is sensitive to high frequency noise. N N/(10.7)

7. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 7 A “Mystery” of Huffman Coding Ratios on Down Sampled Data The 5MHz data is down sampled to 1MHz. The Huffman Coding compress ratio drops from 10.7 to 7.5 when the data is down sampled. N N/(10.7) (N/5) (N/5)/(7.5)

8. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 8 Averaging in Decimation: A Re-discovery Simple “down-sampling” is not good. When the decimation factor is D, an averaging over D samples is good either. An averaging over 2*D samples is necessary. There is still aliasing with averaging over 2*D samples but it is less severe than averaging over D samples. Nyquist Frequency < (1/2) Sampling Frequency

9. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 9 Weighted Average, The CIC-2 Filter Filter performance can be further improved with weighted average over 4*D samples. The filter is called Cascade-Integrate-Comb filter of order 2 (CIC-2). The CIC-1 filter is the moving average.

10. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 10 Huffman Coding Ratios for 5MHz to 1MHz The Huffman Coding compress ratio improves as the filter in Dynamic Decimation improves.

11. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 11 Dynamic Decimation (DD) Only small time intervals, i.e., region of interest (ROI) must be sampled at high rate. Most time intervals can be sampled with lower rate, without losing useful information.

12. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 12 A Mystery of Dynamic Decimation & Huffman Coding Dynamic Decimation reduces number of samples by factor of 10. Huffman Coding reduces number of bits from raw data by factor of 10. When cascaded, the combination reduces number of bits by factor of 60. Dynamic Decimation Huffman Coding NN/10.6 Dynamic Decimation Huffman Coding N N/60 NN/10.7

13. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 13 Huffman Coding Ratios for Dynamic Decimation The Huffman Coding compress ratio improves as the filter in Dynamic Decimation improves.

14. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 14 The Knobs of Data Volume Control The filtering schemes and parameters in the Dynamic Decimation block are knobs for data volume control. Most of analog noises can be filtered out. Serial to Parallel Conversion 16MHz to 2MHz Decimation Data Merging RAM Dynamic Decimation External Memory Output Interface Huffman Coding Huffman Coding Serial to Parallel Conversion 16MHz to 2MHz Decimation Serial to Parallel Conversion 16MHz to 2MHz Decimation Serial to Parallel Conversion 16MHz to 2MHz Decimation ADC Total Compress Ratio: 60 - 100 from BO events. Accelerator Neutrino Events Supernova Data

15. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 15 Any Differences ? Raw With Dynamic Decimation

16. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 16 Summary Waveform digitization is a necessary readout approach for TPC detectors but it creates large volume of data. It is necessary to reduce data volume without losing useful information. Accelerator neutrino data is compressed using lossless Huffman Coding scheme, with a typical (1/10) reduction ratio. Dynamic Decimation and Huffman coding are applied to supernova data with a (1/60) to (1/100) total reduction ratio.

17. June 2009, Wu Jinyuan, Fermilab jywu168@fnal.gov MicroBooNe Design Review 17 The End Thanks