Statistics Time Series – Moving Average https://www.123rf.com/photo_6622261_statistics-and-analysis-of-data-as-background.html

Moving Average Go to the Portal Open the spreadsheet

Time Series Analysis Because the DJIA data is so volatile, it is hard to see the major trends

Time Series Analysis One trick we use to be able to see the trends is to draw trendlines

Time Series Analysis Smoothing I did by eye and by hand:

Time Series Analysis But… there MUST be a more “mathy” way of doing this! It’s a MATH class… right?

Moving Average Right! We use a technique called finding the “moving average” to smooth the data (sometimes this is called a “rolling average”)

Moving Average What is a moving average???

Moving Average What is an average??? Well… there are actually a whole bunch of “averages” (you’ll find out about a few in statistics class) The moving average “average” is the one we’re most used to

Moving Average What is the average of this data set: 2 2 5

Moving Average To find this average, add up the values and divide by how many values there are: 2+2+5 9 3 3 = = 3

Moving Average A moving average steps along the data through time taking an average of a small subset of the data at each step 3 7 8 2 5 1 1 4 3 9 2 3 2 4 9 7 6 6 4 1 2 6 5 9 7 8 2 6 avg 1 avg 2 avg 3 avg 4 avg 5 avg 6 avg 7

Moving Average Except you only move along one data observation each step: 3 7 8 2 5 1 1 4 3 9 2 3 2 4 9 7 6 6 4 1 2 6 5 9 7 8 2 6 avg 1 avg 5 avg 9 avg 13 avg 17 avg 21 avg 25 avg 2 avg 6 avg 10 avg 14 avg 18 avg 22 avg 3 avg 7 avg 11 avg 15 avg 19 avg 23 avg 4 avg 8 avg 12 avg 16 avg 20 avg 24

Moving Average Let’s try one!

Moving Average Suppose we had this data set: Time Price 1 $100 2 $120 3 $110 4 $105 5 $110

Moving Average The graph would look like:

Moving Average Let’s take a moving average of every two observations: Time Price 1 $100 2 $120 3 $110 4 $130 5 $120 Avg = $110 Avg = $115 Avg = $120 Avg = $125

Moving Average We align the moving average with the final observation used to calculate it: Time Price MovAvg 1 $100 2 $120 $110 3 $110 $115 4 $130 $120 5 $120 $125

Moving Average Notice the first observation doesn’t have a moving average value: Time Price MovAvg 1 $100 2 $120 $110 3 $110 $115 4 $130 $120 5 $120 $125

Moving Average Now, we make a new graph including the new column of data:

Moving Average Compare the two lines… Is the moving average line less volatile? Does it show an inherent trend?

Moving Average That’s why we do moving averages on volatile time series data!

Questions?

Moving Average But there MUST be a way for Excel to do this ... (It’s an Excel class, right?)

Moving Average To do moving averages in Excel, you need to install the Analysis Toolpak Add-In (This has been done on the classroom computers – you’ll have to do it yourself on your home computers)

Adding the Analysis ToolPak Open Excel and click on the 'File' tab at the top left of the screen At the bottom of the menu, click on the 'Options' button The Excel Options window will open

Adding the Analysis ToolPak In the column on the left, click on the 'Add-Ins' heading

Adding the Analysis ToolPak On the right side of the window, scroll down to Inactive Application Add-Ins. Click on 'Analysis ToolPak' to select it. Then click the 'Go' Button.

Adding the Analysis ToolPak An 'Add-Ins' window will open. Click in the checkbox next to 'Analysis ToolPak' Click the 'OK' button.

Adding the Analysis ToolPak Click on the “Data” tab at the top “Data Analysis” should be on the far right of the ribbon

Moving Average We already have a spikey graph of this VERY volatile data

Moving Average For information, highlight the “VALUE” data – the number of observations is at the bottom:

Moving Average It would be a HUGE PAIN to manually calculate moving averages for all 265 values!!!! Happily, Excel will do it for us (with less pain)

Moving Average Step 1: At the top, click the “Data” tab At the far right, click “Data Analysis”

Moving Average The Data Analysis window opens Highlight “Moving Average” Click “OK”

Moving Average The Moving Average window opens – click the square at the end of “Input Range”

Moving Average When Excel says “Input Range” they do not mean the input variable Usually what we want to “input” is the output variable!

Moving Average On your spreadsheet data page, highlight all of the column “VALUE” Include the label at the top

Moving Average Click the square at the end of the input area

Moving Average Click the box next to “Labels in First Row”

Moving Average Click the square at the end of “Output Range”

Moving Average Again, Excel doesn’t mean the output variable, it just wants to know where we want the moving averages it calculates to go!

Moving Average Highlight the area in the column I’ve labeled “Moving Average” Don’t include the label

Moving Average When the entire area is highlighted, click the square at the end of the input area Moving Average

Moving Average In the “Interval” box we tell Excel how many values to include in each moving average - Let’s try 2

Moving Average Click the “OK” button

Moving Average AAGH! Why does Excel HATE US???

Moving Average Actually, it doesn’t hate US… it just hates the #N/A’s in the data These were put in to show that no data is available

Moving Average The graph handles them by just drawing a straight line between the last real value and the net real value

Moving Average The moving averages function handles them by breaking and giving us an error message!

Moving Average So, we’ve got to fill in those missing slots with real values

Moving Average We’ll use averages!!!

Moving Average We’ll take the last real value and the next real value and average them to create an estimated value to fill in the missing data

Moving Average This technique is used almost universally to fill in the (inevitable) missing data values

Moving Average Get out of Moving Averages (hit the red X at the top right twice)

Moving Average We will modify the metadata to include our method of filling in missing values

Moving Average Add a line at the bottom of the metadata

Moving Average Add the title in A17: Missing data: Add the info in B17: Missing data (shown in red) have been estimated using the average of the last observed data and the next observed data

Moving Average

Moving Average Scan the data Click on the first #N/A you see Go to the fx line

Moving Average Enter the formula: =(b23+b25)/2 Hit “Enter” on your keyboard

Moving Average Tah-dah! An estimated data value! But… it looks like a REAL data value…

Moving Average Click on the cell, Click on the color icon or the arrow Change the color to something you can see

Moving Average Now you can tell which cells are estimated!

Moving Average Highlight the cell and copy to each of the #N/A cells in the data series

Moving Average Oops… now what?

Moving Average Paste the cell anyway, then tweak the formula to include the next cell containing a value: (B238+B241)/2

Moving Average Then keep pasting! The next cell is OK!

Moving Average Until all the #N/A cells are filled

Moving Average When all the #N/A’s are filled, do the Moving Average again (it should remember what you entered before)

Moving Average AAGH! NOW WHAT???

Moving Average Oops… we added a line at the bottom of the metadata and now all the line numbers need to be adjusted…

Moving Average

Moving Average Poof! It adds the moving average values! (Isn’t Excel wunnerful???)

Moving Average Note – Excel added its own #N/A at the top of the moving averages!!

Moving Average Now, create a new graph comparing the two lines – highlight all three columns of data at once

Moving Average To make it pretty: Get rid of the confusing markers and add a title

Moving Average Change the data axes: For the dates, min is 40900 max is 41300 major unit is 92 For the $, auto is probably OK

Moving Average Drag the legend into a “safe location” where it doesn’t block any of the graph Expand the graph size You may have to tweak these a bit

Moving Average Make the legend pretty! Right click the legend box Click “Format Legend” Click “Fill” Select “Solid fill” Pick a color you like

Moving Average Click “Border Color” Select “Solid line” Pick a color you like

Moving Average

Moving Average Hmm… Howcum it’s not smooth? The red line is a little smoother than the blue…

Moving Average If we do bigger intervals for our averages (more than 2 data points) it will get smoother! Let’s try 5 (that is a week of DJIA data)

Moving Average Better, but still not smooth! Try 22 (a full month of DJIA data)

Moving Average Smooth! But now it “lags” the real data…

Moving Average Comparison of my hand-drawn smoothing to the 22-period moving average

Moving Average The moving average will “lag” the observed data

Moving Average The bigger the interval the greater the lag

Moving Average Life is cruel…

Questions?

Moving Average You can make a “Centered” moving average that will not have the lag

Moving Average The MA data are formulas:

Moving Average Go to your first non#NA Moving Average value formula

Moving Average It averages data in Row 19-40 and puts the average in Row 40

Moving Average It would “center” on Row (19+40)/2 (row 29.5) If we move it to Row 29 or 30, it would be centered

Moving Average Copy the formula from the Formula Bar to cell C30

Moving Average Copy cell C30 down to the bottom

Moving Average Note that at the bottom the data goes into the future!

Moving Average That’s OK – Excel is counting those as missing!

Moving Average Now let’s fill in the early #N/A values at the beginning

Moving Average Go back to cell C30 and copy it up to the top

Moving Average Go to the top cell

Moving Average It looks fine, but it’s hiding an evil secret…

Moving Average It looks fine, but it’s hiding an evil secret… it’s adding in B8- B18 (which is not data…)

Moving Average It looks fine, but it’s hiding an evil secret… Change the first cell in the formula to B$19

Moving Average Make the corrected formula a different color and copy it down to cell C29

Moving Average Change your column title to “Centered Moving Average”

Moving Average Tah-dah! Centered MA values!

Questions?

Cyclical Moving Average A lot of time series data spread over several years

Cyclical Moving Average Time series data often have an annual cycle

Cyclical Moving Average In Excel terms, an annual cycle means: However many periods are in a year, that is the interval you’ll need to use to smooth out the data and see the annual cycle

Cyclical Moving Average Quarterly data will be smoothed by a 4-period moving average Monthly data will be smoothed by a 12-period moving average

Cyclical Moving Average What if you had data every four months? (how many times a year would that be?)

Cyclical Moving Average Open the spreadsheet in the Portal

Cyclical Moving Average On the page labeled “Quarterly” make a graph of the data

Cyclical Moving Average Is it volatile?

Cyclical Moving Average Is it cyclical?

Cyclical Moving Average Count from the max of the pattern

Cyclical Moving Average Start over on the next max

Cyclical Moving Average Is the cycle “4” for the most part?

Cyclical Moving Average Then “4” is the interval you should use for the Moving Average

Cyclical Moving Average Calculate the moving average

Cyclical Moving Average Graph the data series

Cyclical Moving Average Make it pretty and informative

Cyclical Moving Average Do we need to center this data?

Cyclical Moving Average Probably not!

Questions?

Cyclical Moving Average Go to the “Monthly” page Add a column for the Moving Average Create a graph of the data (Be sure to include the Moving Average column)

Cyclical Moving Average Yuck! What’s the problem?

Cyclical Moving Average Fill in the missing data

Cyclical Moving Average Much better!

Cyclical Moving Average Make it more visible by adjusting the axes and getting rid of the legend (I used for the date axis: Minimum 21000.0 Maximum 42000.0 Major unit 3650.0)

Cyclical Moving Average Tah-dah!

Cyclical Moving Average Is there a cycle?

Cyclical Moving Average There is a yearly (12-month) cycle!

Cyclical Moving Average Lots of things have an annual cycle (weather, schools, car license plates, etc.)

Cyclical Moving Average Smooth the CO2 data using a centered 12-month Moving Average

Questions?