1. Physics 114: Lecture 12 Central Limit Theorem or Mean of Means
Averaging of averages?
John Federici
NJIT Physics Department

2. Physics Cartoons

3. Significant Digits
Previously in the course, when we discussed significant digits, we made several statements.
The uncertainty in a measurement is determined by the standard deviation of the measurements.
The number of significant digits in the uncertainty is TYPICALLY limited to one or TWO significant digits.
This ‘rule of thumb’ essentially comes from considering the ‘standard deviation of the standard deviation’
To illustrate the point, let’s consider a NORMAL PDF distribution with a 10% relative error.
Let’s repeat the ‘experiment’ 30 times (ie 30 trials) and calculate the standard deviation of trials.
Next, let’s repeat the 30 trials a total of 10 times and see how much the standard deviation varies from one set of 30 trials to another.

4. Significant Digits
for i=[1:10] % 10 such experiments of 30 trials each
x=10+randn(1,30); % 30 'trials'
% define 'measured value' with 10% relative error
myStd(i)=std(x); % standard Deviation of 30 trials.
disp(myStd(i:i))
end
std(myStd) % Standard deviation of the Standard Deviations
>> lecture12_example
0.9538
1.0183
0.8912
0.9817
0.9601
1.0716
0.8238
0.9701
1.1808
0.9161
ans =
0.0987
Note that the standard deviation is DIFFERENT for each set of 30 trials. This is expected since there are statistical fluctuations each measurement.
The standard deviation of these 10 standard deviations gives a sense for the UNCERTAINTY in the UNCERTAINTY
Measured Value=10.0±1.0
Limit to 2 significant digits since uncertainty varies between about 0.9 and 1.1 (ie. ±0.1)

5. What’s next?
Now that we discussed the standard deviation of the standard deviation, let’s now focus on the mean of the means….
Essentially, how to be COMBINE the information for different SETS of experiments. (ie. taken on different days, under slightly different noise conditions).

6. The Goal of Measurement
When we make measurements of a quantity, we are mainly after two things: (1) the value of the quantity (the mean), and (2) a sense of how well we know this value (for which we need the spread of the distribution, or standard deviation).
Remember that the goal of measurement is to obtain these two bits of information. The mean is of no use without the standard deviation as well.
We have seen that repeated measurement of a quantity can be used to improve the estimate of the mean. Let’s take a closer look at what is going on. HOW do we COMBINE data from REPEATED measurements?
Say we create a random set of 100 measurements of a quantity whose parent distribution has a mean of 5 and standard deviation of 1:
x = randn(1,100)+5;
Create a histogram of that set of measurements: [y z] = hist(x,0:0.5:10);
Here, y is the histogram (frequency of points in each bin), and z is the bin centers. Now plot it: plot(z,y,’.’). If you prefer bars, use stairs(z,y).

7. Histogram of “data” >> ylabel('# of occurances')
>> x = randn(1,100)+5;
>> [y z] = hist(x,0:0.5:10);
>> plot(z,y,'.')
>> xlabel('Data Value')
Here, the variable y is the # of occurances, while z is the ‘data value’
>> stairs(z,y)

8. Comparing Measurements
If you make repeated sets of 100 measurements, you will obtain different samples from the parent distribution, whose averages are approximations to the mean of the parent distribution.
Let’s make 100 sets of 100 measurements:
y = zeros(100,21); % here 21 represents the number of ‘bins’ in histo
for i = 1:100; x = randn(1,100)+5; [y(i,:) z] = hist(x,0:0.5:10); end
Plotting some of the histograms,
for i = 1:16; subplot(4,4,i); stairs(z-0.25,y(i,:)); axis([2,8,0,25]); end
Basic structure of “for” loops.
For loops can be nested.
To programmatically exit the loop, use a break statement.
To skip the rest of the instructions in the loop and begin the next iteration, use a continue statement.
for index = values
statements
end

9. Comparing Measurements
you can see that the samples means vary from one set of 100 measurements to another.

10. Comparing Measurements
Now the mean can be determined from the original values (xi):
mean(x)
or from the histograms themselves:
mean(y(100,:).*z)/mean(y(100,:))
Make sure you understand why these two should be the same (nearly), and why they might be slightly different.
Approximately, sum over all xi

11. Comparing Measurements
Now the mean can be determined from the original values (xi):
mean(x)
or from the histograms themselves:
mean(y(100,:).*z)/mean(y(100,:))
Make sure you understand why these two should be the same (nearly), and why they might be slightly different.
Since we have saved the histograms, let’s print out the means of these 16 sample distributions:
for i=1:16; a = mean(y(i,:).*z)/mean(y(i,:)); fprintf('%f\n',a); end
Here is one realization of those means.
We might surmise that the mean of these means might be a better estimate of the mean of the parent distribution, and we would be right!
Averaging the averages…. Gives better estimate of MEAN value of parent population

12. Distribution of Means Let’s now calculate the 100 means And plot them
a = zeros(1,100); for i=1:100; a(1,i) = mean(y(i,:).*z)/mean(y(i,:)); end
And plot them
subplot(1,1,1)
plot(a,'.')
This is the distribution of means.
The mean of this distribution
is 4.998, clearly very close to
the mean of the parent
distribution (5)
This plot should remind you of HW#3 in which NOISE is plotted for 1024 individual measurements

13. Mean of Means COMBINE data into ONE BIG DATA SET
We can think about this distribution in two different, but equivalent ways.
If we simply sum all of the histograms, we obtain a much better estimate of the parent population:
COMBINE data into ONE BIG DATA SET
mom = sum(y);
stairs(z-0.25,mom)
mean(mom.*z)/mean(mom)
gives: 4.998

14. Mean of Means
Alternatively, we can think of these different estimates of the mean of the original population as being drawn from a NEW parent population, one representing the distribution of means.
This NEW parent population has a different (smaller) standard deviation than the original parent population.
std(a) is

15. This should remind you of HomeWork #4
1 average

16. Central Limit Theorem (CLT)
The Means of Means is a manifestation of what the OpenStax Textbook (Chapter 7) calls CLT
The central limit theorem (clt for short) is one of the most powerful and useful ideas in all of statistics. There are two alternative forms of the theorem, and both alternatives are concerned with drawing finite samples size n from a population with a known mean, μ, and a known standard deviation, σ.
The first alternative says that if we collect samples of size n with a "large enough n," calculate each sample's mean, and create a histogram of those means, then the resulting histogram will tend to have an approximate normal bell shape.
The second alternative says that if we again collect samples of size n that are "large enough," calculate the sum of each sample and create a histogram, then the resulting histogram will again tend to have a normal bell-shape.
In either case, it does not matter what the distribution of the original population is, or whether you even need to know it. The important fact is that the distribution of sample means and the sums tend to follow the normal
distribution.
So…. Even if your distribution is Poisson, the distribution of the MEANS will be Gaussian

17. Central Limit Theorem (CLT)
OpenStax Textbook (Chapter 7.1): CLT applied to averages
Suppose X is a random variable with a distribution that may be known or unknown (it can be any distribution). Using a subscript that matches the random variable, suppose:
μX = the mean of X.
σX = the standard deviation of X
If you draw random samples of size n, then as n increases, MEAN values of those samples tends
to be normally distributed
The central limit theorem for sample means says that if you keep drawing larger and larger samples (such as rolling one, two, five, and finally, ten dice (OR 1, 2, 5, 10, 100 AVERAGES in the THz HW example) and calculating their means, the sample means form their own normal distribution (the sampling distribution). The normal distribution has the same mean as the original distribution and a standard deviation that equals the original standard deviation divided by n, the sample size. The variable n is the number of values that are averaged together.
is called the standard error of the mean.

18. CLT - Example OpenStax Textbook (Modification of Example 7.1):
On the NJIT campus, the number of students who daydream while in class is an unknown distribution but with a mean of 90 and a standard deviation of 15. If samples of size n = 25 are drawn randomly from the population.
a. Find the probability that the sample mean for the number of students who daydream in class is between 85 and 92.
NOTE: in this problem, you are NOT asked for the probability of a certain value of the distribution (ie. what is the probability that 100 students are daydreaming?), but rather the probability of a sample MEAN. Therefore, the Central Limit Theorem applies.
Solution: the PDF of the MEANS is a Gaussian (Normal) PDF with a mean of 90 and a standard deviation of
>> upper=normcdf(92,90,15/sqrt(25));
>> lower=normcdf(85,90,15/sqrt(25));
>> upper-lower
ans =

19. We have seen the CLT before!
This should remind you of HW #3
If one were to BLOCK the THz Beam in the transmission measurement, what waveform would you measure?

20. HomeWork #4
Is the noise in the THz transmission measurement statistical?
1 average

21. Class exersize
If one average gives a standard deviation of the noise of 0.01, how many averages are required to reduce the standard deviation of the noise to 10-6?
1 average
If each average takes 1 second, how many minutes will it take to complete the averages you calculate above?

22. Does Averaging help? 10 Avg 1 Avg 10 Avg 1 Avg
“Averaging” of 10 measurements together into 1 number is the first “MEAN”. Then repeating the measurements over time (ie repetitions over 80ps) gives a ‘population’ of MEAN measurements for the histogram. Then when you calculate the histogram of that AVERAGED trace (essentially, the MEAN OF THE MEAN), the distribution function of the mean narrows.

23. Calculation of Error in the Mean
Recall in Previous Lectures we introduced the general formula for propagation of errors of a combination of two measurements u and v as:
Generalizing further for our case of N measurements of the mean m’, and ignoring correlations between measurements (i.e. setting the cross-terms to zero), we have
We can make the assumption that all of the si are equal (this is just saying that the samples are all of equal size and drawn from the same parent population). Also so
NOTE: This is essentially a PROOF of the Central Limit Theorem.

24. Calculation of Error in the Mean
This is no surprise, and it says what we already knew, that the error in the mean gets smaller according to the square-root of the number of means averaged.
Result from HW#3
Slope=-1/2

25. Calculation of Error in the Mean
This is no surprise, and it says what we already knew, that the error in the mean gets smaller according to the square-root of the number of means averaged.
Again, this is the case when all of the errors in the means used are equal. What would we do if, say, some of the means were determined by averaging different numbers of observations instead of the same number each time?
In this case, we can do what is called weighting the data. If we know the different values of si, then the weighted average of the data points is
Even if we take the SAME number of observations each ‘experiment’, we may STILL want to weight the data…. If measurements were taken on DIFFERENT days, the noise might be different because the experimental conditions might be different.

26. Error in the Weighted Mean
In this case, if we want to combine a number of such weighted means to get the error in the weighted mean, we still have to calculate the propagation of errors:
but now the si are not all the same, and also we use the weighted mean to get the gradient of the mean
Inserting that into the above equation, we have

27. Relative Uncertainties
In some cases, we do not necessarily know the uncertainties of each measurement, but we do know the relative values of si. That is, we may know that some of our estimates of the mean used 100 measurements, and some used only 25. In that case, we can guess that the latter measurements have errors twice as large (since the standard deviations are proportional to the square-root of the number of measurements).
So, say Then
In other words, because of the nature of the ratio, the proportionality constant cancels and we need only the relative weights.
To get the overall variance in this case, we must appeal to an average variance of the data:
So the standard deviation is found using this, as
Central Limit
Theorem!!!!

28. Calculation of Error in the Mean
Why does the experiment data DEVIATE from the Central Limit Theorem?
Slope=-1/2

29. REMINDER of Limitations of Averaging
Some of the major concepts of HW#3:
Averaging will definitely help, but the trade-off is TIME to acquire the data
Even if we had enough time (and money to PAY the people making the measurements), some of the experimental conditions MAY CHANGE during experiment – Temperature changes cause materials to expand or contract, changing, for example, optical alignment of mirrors and lenses.
Systematic errors CAN NOT be removed even with averaging.

30. Previous Example of Systematic Error
February 12, 2010

31. This Systematic Error is Intermittent
10 Avg
100 Avg
Best solution is to FIX the instrumentation to remove Systematic Errors.

32. ‘Non-statistical Fluctuations’
Essentially the point is that averaging will get rid of STATISTICAL fluctuations (if we are willing to wait long enough). However, other sources of noise can become dominate which are NOT STATISTICAL in nature.
Systematic Noise
‘Drifting’ of equipment calibration… eg. temperature changes over time
Slope=-1/2
‘Non-Statistical Fluctuations’

33. ‘Severe’ Outlier Data Points
What is a 'Black Swan‘?
A black swan is an event or occurrence that deviates beyond what is normally expected of a situation and is extremely difficult to predict; the term was popularized by Nassim Nicholas Taleb, a finance professor, writer and former Wall Street trader. Black swan events are typically random and are unexpected. (ie. RARE events)
For insurance, how do you determine the risk and price of insurance for a HIGHLY unlikely event (tidal wave on coast of NJ) but if it DID happen, the effects would be catastrophic ….

34. Elimination of Data points
The Bevington textbook describes Chauvenet’s criteria which essentially says that if an outlier data point occurs with much higher rate than would be expected by statistics, you can discard the data point as a ‘valid’ data point for calculating mean and standard deviation.
HOWEVER
Bevington textbook
OpenStax textbook, Chapter 2:
A potential outlier is a data point that is significantly different from the other data points. These special data points may be errors or some kind of abnormality or they may be a key to understanding the data.
Class Discussion: If you have outliers in your data, what should you do?

35. Trivia Question
What was the name of the machine which Nazi Germany’s used to create secret code during World War II?
(a) Imitation
(b) Enigma
(c) Turing
(d) HYPO
(e) Alan

36. Trivia Question
What was the name of the machine which Nazi Germany’s used to create secret code during World War II?
(a)
(b) Enigma
(c)
(d) HYPO – Team that cracked Japanese codes during WWII
(e)
Alan Turing
After Turing and this team cracked the Enigma machine, why did not they act on EVERY message that they decrypted?

37. In Class exercise
Later in the course, we will be processing multiple THz data files to create images. The first step in this progress is to use FOR loops to generate the correct file names.
x 0y 0.txt
x 1y 0.txt
x 2y 0.txt …
Use nested FOR loops to create the FILE NAMES starting with ‘x 0y 0.txt’ and ending with ‘x 5y 7.txt’ .
Printout the names of the files as you create them.
HINT: Format integers and the letters as ‘string’ variables.

38. In Class Exercise Problem 2: You are given the following data,
Thick = [105.3, 99.5, 101.5, 104.5, 97.5],
which represent the average thickness [in microns] of a paint layer as measured from a specialized probe. The uncertainty, represented by the standard deviation, of each
measurement is [in microns]
σT = [2.0, 2.0, 2.5, 1.5, 2.0].
Compute the “mean of means” of these thickness measurements over the entire set of experiments. Give the equation you used.   
Compute the sample standard deviation for the entire set of experiments. Give the equation you used. 
Using your answer for part (b), assume that you repeated the measurements 100 times rather than 5 times. ESTIMATE what you would expect the standard deviation to be for 100 measurements for this “averages of averages”.