1. Physics 114: Lecture 6 Measuring Noise in Real Data
John Federici
NJIT Physics Department

2. Physics Cartoons

3. REMINDER
Quiz on weeks 1,2,3 material in TUESDAY’s class. About 15 min.
Please be on time. No extra time will be allotted for late students.

4. Mean and Standard Deviation
Sample Mean
Parent population mean
Standard Deviation from sample mean
Standard Deviation from parent population mean

5. Averaging of Data - Example
Readout from Infrared Detector Array
Bottom curve – No averaging. Is there a signal present?
Top curve (Offset) - more averaging
Detected Voltage (Arb. Units)
Wavelength (Arbitrary Units)
Fowler and Gatly, Astro. J 353, L33-34, (1990)

6. Averaging of Data - Example
Readout from Infrared Detector Array
Bottom curve – No averaging. Is there a signal present?
Top curve (Offset) - more averaging
Detected Voltage (Arb. Units)
SIGNAL level is comparable to or smaller than NOISE Level
AVERAGING will reduce the noise to a level that SIGNAL is LARGER than noise.
Wavelength (Arbitrary Units)
Fowler and Gatly, Astro. J 353, L33-34, (1990)

7. How Many Times Should I Average?
If the Parent Distribution has a standard deviation of 10%
x=10+randn(1,1000);
% create data with a 'true' value of 10 and noise with standard deviation
% of 1. True value will vary within about 10%
for i=2:1000
j(i)=i;
MyStanDev(i)=std(x(1:i));
end
plot(j,MyStanDev)

8. How Many Times Should I Average?
If the Parent Distribution has a standard deviation of 30%
With increased noise, you need to AVERAGE MORE to get close to the PARENT Distribution standard Deviation.
Question: Are the averaging of three data values in Physics Lab sufficient to achieve the PARENT DISTRIBUTION standard deviation?
x=10+3*randn(1,1000);
% create data with a 'true' value of 10 and noise with standard deviation
% of 1. True value will vary within about 30%
for i=2:1000
j(i)=i;
MyStanDev(i)=std(x(1:i));
end
plot(j,MyStanDev)

9. Hw#3 Data – THz Transmission
Terahertz Time-Domain Data – Recall the basic experimental layout
Think of THz pulses of radiation as similar to a Radar pulse. Pulses through sample DELAYED in time because then travel SLOWER

10. Refractive Index
From time delay between pulses, one can measure the speed of light in the sample material and its ‘index of refraction’
Speed of light in vacuum (~3×108 m/s)
Speed of light in material L

11. Measuring Dt between two pulses
In order to measure Dt accurately between two pulses, you need to determine where the peak of the pulse is located in time. However, each measurement of the THz voltage at a particular time has some error associated with it.
1 Average
Blow up section of figure using MAGNIFYING GLASS icon in figure

12. How do you remove ‘Statistical’ Uncertainty?
AVERAGING will remove statistical noise, but SYSTEMATIC Noise (and the signal you are looking for) will remain.
1 Avg
10 Avg
100 Avg
3 Avg

13. How do you remove ‘Statistical’ Uncertainty?
AVERAGING will remove statistical noise, but SYSTEMATIC Noise (and the signal you are looking for) will remain.
1 Avg
100 Avg
Now it is easy to determine the time of the MINIMUM of the peak

14. Systematic and Statistical Noise Example
If one were to BLOCK the THz Beam in the transmission measurement, what waveform would you measure?

15. HomeWork #3
Is the noise in the THz transmission measurement statistical?
1 average

16. Does Averaging help?
10 Avg
1 Avg
10Avg
1 Avg

17. What if we average more?
600 Averages
What is THIS oscillatory signal? The THZ beam is blocked. What type of ERROR is this signal?
SYSTEMATIC Noise due to problem with experimental instrumentation

18. Data, Systematic Error, Random Noise
How do we know that this oscillatory waveform is not the ‘Real DATA’ ? ie. not the THz signal?
How do we know that this waveform is NOT statistical noise?
How do we know that this waveform is probably a Systematic Error?
THz signal is blocked!
After many averages, the persistent shape of the waveform (ie. sine wave) does not average out.
Statistics do not look Gaussian
THz Signal Blocked.
After many averages, waveform persists.

19. This Systematic Error is Intermittent
10 Avg
100 Avg
Best solution is to FIX the instrumentation to remove Systematic Errors.

20. More evidence of Systematic Error

21. Correct for Systematic Errors
If Systematic Errors are NOT intermittent, then we can subtract out the Systematic Errors
Subtract

22. In Class Exercise Begin on weekly HW assignment Problem 2.
(a) Download the THZ data set for Week 3 Homework from the course web site, in text format. Use the IMPORT DATA tool to read the data from the various files. Each file corresponds to a DIFFERENT number of averages as indicated in the file name.
(b) Import each data file and then calculate the standard deviation for the E Field (second column of data). Save the standard deviation values in an array. Save a second array whose elements are the NUMBER of averages. As a guide, look at the table below as a guide to the two vector arrays which you will create:
# of averages Standard deviation
1 std of AVG1 data
3 std of AVG3 data
6 std of AVG6 data
10 std of AVG10 data
… continuation of data