1. University of South Australia The Load Forecasting Dilemma – Factors influencing progression rates at Higher Education Institutions before and after the Bradley review. Andrea Matulick, Manager: Cognos 8 Migration Project, UniSA

2. Outline of presentation Student Load forecasting overview Predicting continuing student progression rate CGS continuing load – major influencing factors Why progression rates may vary The future – what changes can we expect after the Bradley reforms

3. Student Load Forecasting Overview Purpose of Load Forecasting: Meeting DEEWR (or govt) Funding agreement arrangements Budget allocations to academic areas Determining tertiary admission centre intakes Location and Level of Load Forecasting: Mostly in planning, strategic planning or planning and quality offices Mostly by staff at level HEO8 and above (AAIR Load Management SIG Questionnaire 2009)

4. Student Load Forecasting Overview cont. Formula to forecast total load for the next year – 2 step process: Forecast the expected continuing load (using historic evidence based data) Plan the correct commencing load to give the required total load (required to meet funding targets, strategic plans etc.) Forecast Total Load =Forecast Continuing Load +Planned Commencing Load Target resultHard to controlAbility to adjust

5. Student Load Forecasting Overview cont. Factors making up each part of the total load equation: Planned Commencing Load = Number of offers * estimated Acceptance Rate * estimated EFTSL per Person Forecast Continuing Load = Previous Years Total Load * estimated Progression Rate Where Progression Rate = current year continuing load / previous year total load

6. Student Load Forecasting Overview cont Accuracy of Load Forecasting: the more accurate the prediction of continuing load, the easier it is to plan for the commencing load required to achieve the desired total load. it becomes difficult to predict and control load if the commencing load needs to vary significantly to cater for inaccuracy in forecasting continuing load or controlling commencing load

7. Student Load Forecasting Overview cont

8. Predicting Continuing Student Progression Rate Progression Rate = current year continuing load / previous year total load The analysis: Look at how actual progression rates varied over the years Use 2 different methods to predict a progression rate Look at how accurate the predictions would have been Look for reasons why progression rate may not be as predicted The data used: DEEWR aggregate student load data from 2004 to 2007 for 38 Australian Tertiary institutions with valid progression rates (PR from 0.1 to 1.0 in all 4 years)

9. Predicting Continuing Student Progression Rate cont. Progression Rates for most institutions

10. Predicting 1ate cont. Progression Rates for 11 tagged institutions:

11. Predicting Continuing Student Progression Rate cont. Variation of actual student load progression rates over the 4 year period: The average actual progression rate over all institutions was 63% The minimum variation in actual progression rate for any institution was 0.88% The maximum variation in actual progression rate for any institution was 15.59% 71% of institutions had a progression rate that varied by less than 5% 11% of institutions had variations of more than 10% How accurately can we forecast progression rate?

12. Predicting Continuing Student Progression Rate cont. Comparison of accuracy using 2 different methods of forecasting: Method 1: Estimated Progression Rate is a 3 year average from the prior 3 years. Method 2: Estimated Progression Rate is set to the same as the most recent year. Progression RatesMethod 1Method 2 20042005200620073YR Avge1YR Prev 0.66650.61240.64820.64610.64240.6482

13. Predicting Continuing Student Progression Rate cont. Funding Group Num Inst Variation in Actual Progression Rate over 4YR Range Accuracy of Forecast Progression Rate using 3YR Avge Accuracy of Forecast Progression using Prev 1YR Num Institutions Best 3Y Num Institutions Best 1Y Num Inst <5% Percent Inst <5% Num Inst <5% Percent Inst <5% Num Inst <5% Percent Inst <5% Method 1Method 2 Total Load382771%3284%3284%2117 Dom UG Load382771%3489%3797%1820 Int UG Load36719%2467%2261%1521 Dom PG Load382258%3284%3079%2414 Int PG Load38513%2258%2463%1622 for 3 of the 4 funding groups, more universities would have achieved a better continuing load prediction using the 1YR method than the 3YR average 84% of institutions could have made predictions of continuing progression rate to within 5% of the actual progression rate by using either of the two forecasting methods Actual Variation and Forecast Accuracy of progression rates over 4 years:

14. Predicting Continuing Student Progression Rate cont.

15. Points to note so far: Variation in actual progression rate For most universities progression rate is fairly constant and estimating a value to use in their forecast continuing load is not difficult. For about 10% of institutions progression rate does vary significantly over years which may cause inaccuracies in forecasting continuing load. Accuracy of 2 methods for forecasting progression rate More universities would have achieved a better continuing load prediction using the 1YR method than the 3YR average. However, 84% of institutions could have made predictions of continuing progression rate to within 5% of the actual progression rate by using either of the two forecasting methods. What are the influencing factors for variation and accuracy?

16. CGS Continuing Load – influencing factors Use the Domestic UG load group as proxy because: Large, relatively stable group that equates closely to CGS load Expect less variation in commencing and continuing load 11 universities with variation on actual PR > 5% over 4 year period (tagged) Look at these 11 universities to find out which factors may be related to the larger than usual variation in continuing student progression rate How well could they have predicted their continuing student progression rate and load? Which forecasting method would have given the the best results?

17. CGS Continuing Load – influencing factors cont. Inst Progression Rate Variation 4YR Progression Rate accuracy using 3YR avge Cont EFTSL accuracy using 3YR avge Progression Rate accuracy using 1YR prev Cont EFTSL accuracy using 1YR prev Uni22 12.13%7.55%1880.54%13 Uni90 9.40%0.63%-224.74%163 Uni23 9.12%3.94%5300.54%-73 Uni92 8.25%5.71%3942.31%160 Uni89 8.15%2.45%-1202.26%111 Uni83 7.60%6.94%11307.49%1218 Uni52 6.37%5.40%3614.35%291 Uni95 6.25%2.91%-834.27%-121 Uni67 6.15%3.24%2080.02%1 Uni98 5.56%2.98%2821.53%145 Uni39 5.31%0.92%721.46%-114 11 Unis with Dom UG PR varying >5% in 4 year period (tagged):

18. Do Total Load, State and Alliance Group relate to variation in progression rate?

19. CGS Continuing Load – influencing factors cont. A closer look at why progression rate may vary: Variation in commencing load 6 of the 11 tagged institutions were in the top 8 institutions for variation in commencing load as a percent of most recent total load. All of the 6 had a variation of more than 22% in commencing load over the previous 4 years. Difficulty in forecasting continuing load 5 of the 11 tagged institutions were in the top 6 institutions being least able to predict their most recent year progression rate. All of the 5 could not have predicted their progression rate to within less than 2.2% for the most recent year using either of the 2 forecasting methods.

20. CGS Continuing Load – influencing factors cont. 10 Unis with largest variation in commencing load over 4 years: Inst Progression Rate Variation 4YR Progression Rate accuracy using 3YR avge Cont EFTSL accuracy using 3YR avge Progression Rate accuracy using 1YR prev Cont EFTSL accuracy using 1YR prev Commencing Load Variation 4YR Uni90 9.40%0.63%-224.74%16355.71% Uni39 5.31%0.92%721.46%-11436.57% Uni83 7.60%6.94%11307.49%121833.61% Uni95 6.25%2.91%-834.27%-12132.72% Uni82 4.33%1.34%903.24%21824.96% Uni84 2.19%0.26%-340.05%-723.86% Uni52 6.37%5.40%3614.35%29123.58% Uni23 9.12%3.94%5300.54%-7322.37% Uni77 1.91%0.52%320.05%322.25% Uni96 2.44%1.61%3120.91%17722.17%

21. CGS Continuing Load – influencing factors cont. 10 Unis with least accuracy in forecasting continuing load: Inst Progression Rate Variation 4YR Progression Rate accuracy using 3YR avge Cont EFTSL accuracy using 3YR avge Progression Rate accuracy using 1YR prev Cont EFTSL accuracy using 1YR prev Commencing Load Variation 4YR Best Accuracy of forecast Progression Rate Uni837.60%6.94%11307.49%121833.61%6.94% Uni526.37%5.40%3614.35%29123.58%4.35% Uni956.25%2.91%-834.27%-12132.72%2.91% Uni653.90%2.82%1612.60%14919.46%2.60% Uni928.25%5.71%3942.31%16016.63%2.31% Uni898.15%2.45%-1202.26%11116.73%2.26% Uni873.24%2.26%2242.34%23211.02%2.26% Uni944.57%3.08%3981.84%23820.59%1.84% Uni23.72%1.77%1213.72%25515.88%1.77% Uni303.48%1.66%1282.17%16712.18%1.66%

22. CGS Continuing Load – influencing factors cont. More points to note: Variation in commencing load Large variations in commencing load as a percent of total load is a good indication that progression rate will also vary. Both of these factors mean it will be difficult to accurately predict and manage load. However, some institutions with large variations in commencing load and progression rate should not have difficulty predicting their continuing and total load Accuracy in forecasting progression rate Large variations in progression rate and commencing load are common factors in being unable to accurately predict and manage load. However they are not always present which means that in some cases other factors are more likely to be contributing – e.g. completion rate, major changes to programs or environmental factors

23. The future What to look for - Changes in metrics Large variations in progression rate over time (more than 5%) Large variation in commencing load over a short period of time (more than 20% of total load) Smaller institutions or those from regional areas Plan to react to changes (expected or unexpected) by understanding the load modelling process used at the institution.

24. The future What to expect - Changes in methods Load forecasting will increase in strategic importance and become more integrated with other planning processes such as workforce and infrastructure More universities will use sophisticated BI software to standardise and automate the load planning process Load forecasting should be actively managed and reviewed as a key strategic process Questions?