PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD? Chicago, Illinois April 26, 2004

Accounting Principles Council Strategic Business Reporting Project Project Team Jennifer Taylor, NMSU, Chair Tammy Anthony, NMSU Teresa Gordon, University of Idaho Roger Patterson, University of North Carolina-Chapel Hill Carolann Sagesse, Cornell University Sue Menditto, NACUBO Kim Dight, NACUBO

Why is NACUBO’s Accounting Principles Council concerned with Performance Measurement? Both GASB and FASB have implemented long-term projects to address the issue. Accounting Principles Council Strategic Business Reporting Project

Performance Measurement Background –GASB research effort to enhance service efforts and accomplishments reporting (SEA) –SEA is non-financial performance information for governmental organizations –SEA is rooted in GASB Concept Statement No.1, “Objectives of Financial Reporting” Assist in fulfilling government’s duty to be publicly accountable Enable users to assess accountability

Performance Measurement Background –GASB Concepts statement No. 2, “Service Efforts and Accomplishments Reporting” –Conceptually links accountability and external financial reporting Accounting Principles Council Strategic Business Reporting Project

Performance Measurement Background –GASB received Sloan Foundation research grants in 1997 and 2000 To expand performance measurement research To understand external reporting of performance information –The grants have six research phases Accounting Principles Council Strategic Business Reporting Project

Six Phases of GASB Research Project 1.Performance measurement website 2.Case studies based on interviews and surveys on uses and effects of using performance measures 3.Citizen discussion groups that evaluate users responses to performance measures 4.Develop suggested criteria to enable performance information reporting 5.Experimentation with suggested criteria (future) 6.Evaluate effectiveness of suggested criteria (future) Accounting Principles Council Strategic Business Reporting Project

NACUBO Performance Measurement Project Goals Address GASB performance reporting suggested criteria and related issues (GASB phase 4 research results) Inform GASB of and proactively represent higher education performance measurement practices (NACUBO advocacy role) Develop a simple, visual template for reporting meaningful performance information both internally and externally (GASB phase 5 – experimentation with criteria and external performance reporting) Develop a model useful to both public and private institutions (NACUBO Industry Guidance) Accounting Principles Council Strategic Business Reporting Project

Project Approach Worked with GASB for guidance Developed a survey –To determine if performance reporting currently exists –To determine measures considered important and relevant for higher education –To assess opinions and relevance of the GASB 16 criteria Accounting Principles Council Strategic Business Reporting Project

Project Timeline Conduct Performance Measurement Survey – January 2004 Analyze Survey – February 2004 Review Sample of Existing PM Reports and Prior Research Results – April 2004 Review Draft Template Content and Format with Focus Groups – Summer 2004 Develop Template – Winter 2004

Accounting Principles Council Strategic Business Reporting Project Survey Goals Identify prevalence and type of PM reporting currently in place within higher education (industry- wide and by constituent group) Identify commonly valued KPI at both public and private institutions Assess importance of non-financial indicators in higher education Provide GASB with input: –Opinions about 16 reporting criteria –Use of outcome-driven and efficiency-based measures

What is a KPI? KPI = Key Performance Indicator Types of Performance Measures: Input – selectivity or enrollment stats Output – graduation stats, national exam Outcome – perceptions surveys, employment stats Efficiency – cost per student Effectiveness (Productivity) – difference in entrance and exit scores Accounting Principles Council Strategic Business Reporting Project

GASB Performance Reporting Criteria 1. Purpose/scope is stated clearly 2. Institutional goals/objectives stated clearly 3. Citizen involvement in goal setting 4. Multiple levels of detail presented 5. Qualitative analysis of results/challenges 6. Concise yet comprehensive, focus on key issues 7. Assessment of information reliability 8. Measures are relevant, link to goals Accounting Principles Council Strategic Business Reporting Project

GASB Performance Reporting Criteria 9. Measures should relate to cost/resources/efficiency 10. Report citizen perceptions of quality/results 11. Present comparative information 12. Discuss factors affecting results 13. Show aggregation/disaggregation by user group 14. Measures should be consistent across periods, or explain changes 15. Easy to find, access, understand 16. Available annually and in timely manner pdf Accounting Principles Council Strategic Business Reporting Project

Survey Highlights Surveyed all NACUBO Primary Rep’s Response rate: 12% Survey supports continuing template development –Majority of respondents do some type of reporting, large percentage of these include internal reporting –Majority do not have mandated reporting format –Majority do not have external guidance for voluntary reporting External reports are primarily extracts of internal reports

Accounting Principles Council Strategic Business Reporting Project IndependentsPublics Report KPIs Externally 38%62% Use KPIs Internally 46%54% KPIs Reported and Used

Accounting Principles Council Strategic Business Reporting Project Survey Highlights Roughly half of respondents relate reports to audited financial statements Top priority KPI’s identified Certain KPI’s indicated as not being appropriate to report A wider variety of measures are used internally than externally A different approach to internal reporting What is “important” versus what is “recommended” to be included are not always the same KPI’s!

Survey Highlights Independents vs. Publics Independents report/use selectivity measures (externally/internally) more often than publics Publics report persistence toward graduation statistics externally more often as compared to independents But both USE persistence statistics internally at comparable levels Accounting Principles Council Strategic Business Reporting Project

Included in External Reports Number of respondents:159 NumberPercentage Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) % Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) % Efficiency or comparative financial data/ratios % Graduation statistics (e.g., degree granted by level or field of study) % Diversity measures (e.g., statistics on race or gender for students, faculty or staff) % Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) % Selectivity measures (e.g., acceptance rate, high school GPA, SAT or ACT scores) %

Used Internally by Management Number of respondents:156 NumberPercentage Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) % Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) % Graduation statistics (e.g., degree granted by level or field of study) % Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) % Efficiency or comparative financial data/ratios % Diversity measures (e.g., statistics on race or gender for students, faculty or staff) % Student satisfaction or graduating senior survey results %

Accounting Principles Council Strategic Business Reporting Project What SHOULD Be Reported Externally – top five were selected by over 67% of respondents Internally – top six were selected by 95% or more of respondents –Most of the indicators were identified as useful for internal management purposes

Top 5 Measures that SHOULD be Reported Externally Accounting Principles Council Strategic Business Reporting Project

Top 5 Measures that SHOULD be Used Internally Accounting Principles Council Strategic Business Reporting Project

Four Measures Common to External and Internal Reporting

Enrollment and Persistence Measures Top the list on what is being reported externally and what should be reported externally Top the list on what is being used internally and what should be reported internally Accounting Principles Council Strategic Business Reporting Project

Measures which Should NOT Be Reported Externally or Internally Accounting Principles Council Strategic Business Reporting Project

GASB Reporting Criteria – rating (1-7) Accounting Principles Council Strategic Business Reporting Project

Survey results tabulated. What’s next External research and development of a reporting template. Accounting Principles Council Strategic Business Reporting Project

Objectives of Reporting Template Verify external reporting versus survey responses Identify additional categories of indicators Find examples of indicators Approach to Compiling Reporting Template Approximately 15% of survey respondents provided links to external reporting Researched institutions not responding to survey who provide external reporting on their website Tallied indicators

Research Highlights 45 institutional reports reviewed 23 categories of KPI’s identified –50% not on 2004 survey 272 individual KPI’s identified –17% in Common Data Set or IPEDS –13% recommended from prior surveys/studies Accounting Principles Council Strategic Business Reporting Project

Research Highlights Most Common Measurement Categories Graduation Statistics Persistence and Graduation Outcomes Research Measures Enrollment Statistics Selectivity Measures Diversity Measures

Research Highlights Most Common Measures Freshman retention rate Percent graduating in 4-6 years Degrees granted by level or field of study External dollars received or expended through research/creative activity Undergraduate transfer data Annual giving FTE and Headcount Race/gender/nationality of students/faculty Accounting Principles Council Strategic Business Reporting Project

Research Highlights Most Common Weighted Measures Freshman retention rate Percent graduating in 4-6 years Degrees granted by level or field of study Entering test scores (ACT, SAT) FTE and Headcount Number of minority enrollments/ratio Student/faculty ratio Accounting Principles Council Strategic Business Reporting Project

Research Highlights Most Common Unweighted Measures External dollars received or expended through research Average class size Comparative exam results Total number of new students enrolled each Fall Perceived quality of degree Functional expense as % of total expense Number of sponsored awards/national ranking Accounting Principles Council Strategic Business Reporting Project

Reporting template complete. What’s next Focus Groups

Objectives of Focus Groups Develop relevant new questions not addressed by survey Consider next steps –Should the project aim for assistance by conducting experimentation as set forth in the GASB Service Efforts and Accomplishments Reporting Program? –Reaction to draft template Accounting Principles Council Strategic Business Reporting Project

Focus Groups Planned Tuesday, April 27 at 9:30 am NACUBO Integrated Planning and Budgeting Workshop - May Higher Education Association Community (ACE, AIR, SHEEO, SHEFO, etc.) Summer NACUBO Annual Meeting - July Accounting Principles Council Strategic Business Reporting Project