National Postschool Outcomes Center Florida’s Experience in Accessing Administrative Records Teleconference May 24, 2005 Jay Pfeiffer, Assistant Deputy Commissioner Division of Accountability, Research, and Measurement Florida Department of Education

Additional role: Department-wide conduit for LMS, LED, Census, BEBR publications Office of K-20 Education Information & Accountability Public Schools Community College & Technical Center University Source Information Systems K-12 Adult Education Career/Technical Associate Degree Baccalaureate Masters Ph.D./Professional Integrating Systems Source Systems Employment Assessment Financial Aid Teacher Certification Facilities Employment Earnings Postsecondary Ed. Welfare Incarceration Licensure K-20 Education Data Warehouse FETPIP K-20 Governance Organization: Alignment of Data Systems

FETPIP – The Florida Education and Training Placement Information Program

“ Instead of maintaining a separate program to collect industry occupational information, it would be more efficient in terms of resources expended to merge the data collection phase of the Occupational Employment Statistics (OES) program with the ES 202 program.” … Florida’s Occupational Information and Delivery System, Technical Assistance Project, U.S. Department of Commerce, Project Number , May 1975: Tallahassee Florida Regarding the wage record system: “The addition of a single data element, an occupational title, would provide the possibility for the most useful single data base in the State of Florida.” The Florida Occupational Information Coordinating Committee “A suggested approach to developing a comprehensive occupational information data base for Florida from existing data resource.” A concept paper, Florida Department of Labor and Employment Security, June 1981: Tallahassee Florida “ Priority should be given to examining the addition of occupational information to the UI Wage Report file.” Herndon, John, et al. Project to Coordinate Economic Development, Employment and Training: Report 1 Issues and Recommendations, Report to the Governor 1 of the EDIT Project, March 1982: Tallahassee Florida “The Unemployment Insurance Wage Report represents the best potential comprehensive source of occupational and wage information in the state, with the addition of occupational titles to the report.” This would be useful in: “labor market demand forecasting and analysis …economic analysis…evaluation and follow-up…” Polivka, Larry; et al. Vocational Education Study: Labor Market Demand Information in Florida: Policy Issues and Alternatives Office of the Governor, August 1982: Tallahassee Florida Key Background Studies and Recommendations

“Funds provided in Specific Appropriations 337A shall be used to conduct a study of the feasibility of implementing an automated system that matches social security numbers of persons completing vocational programs with wage, geographic, and occupational title data available through the Unemployment Insurance Wage Report and the Worker’s Compensation system.” Chapter Laws of Florida, Specific Appropriation 337A: Tallahassee Florida Initial Legislative Direction Specific Appropriation 337A established two important precedents: Administrative records might have uses other than that which they were originally designed for; and, Education money went to the DLES under the tutelage of the FCOVE and the SJTCC with the DOE reporting back results;

DEVELOPMENTS LEADING TO FETPIP'S CURRENT STATUS Fiscal Years and Project Description Location, Staffing, Funding and Source , Occupational Identifiers Project: The "OIP" project proposed and developed the systems necessary to add occupational titles to the Unemployment Wage Report system. State Job Training Coordinating Council (SJTCC), Two Staff People, $60,000 in State General Revenue , Occupational Identifiers Pilot Project: The project pilot tested a revised wage reporting system involving 5000 Florida employers. Ultimately, recommendations were made to focus on follow-up data collection. SJTCC Three Staff People, $61,000 in State General Revenue plus $61,000 in Federal Wagner Peyser Act Funds , Occupational Identifier Project: Redesign original software and collect follow-up employment data for 1school district, 3 community colleges. Florida Council on Vocational Education (COVE), Three Staff People*, $165,000 in State General Revenue , Occupational Identifier Project: System goes statewide for vocational education. Adds partial public continuing education, military enlistments to wage report data.. COVE Four Staff People*, $256,820 in State General Revenue , Renamed as "FETPIP": System expands into education and training other than voc ed. Expands public continuing education coverage, adds federal and state government. service, U.S. Postal Service employment. Department of Education Five Staff People*, $60,458 Federal Chapter II, $256,542 in State General Revenue

Florida Education and Training Placement Information Program (FETPIP) Annual Collection Cycle High School Vocational Center Community College Public University Independent University Other Programs Federal Programs Welfare Transition Services FL Corrections Releases Special Projects, Longitudes FL UC Covered Employment Fed Government & Postal Service Uniformed Military Service Out-of-State Employment Out-of-State Postsecondary Ed. Public University Enrollment Public Adult Ed. & Vocational Enrollment Private College & University Enrollment Public Community College Enrollment FL Corrections Incarcerations Performance and Accountability Reports Workforce Funding Formula Ad Hoc Reports “CHOICES” Career Counseling Software Internet Education & WIA Reports Merge Files Unduplicate Edit Names & SSN Create Master Follow-up File Create Tapes & Files for Record Links Public Assistance Receipt February, 2003

Input Data  Student/Participant - demographic, socio- economic, enrollment/services, educational programs, promotion, attendance, GPA, test scores…  Educational Institution - types, location, performance statistics, private/public…  Financial Aid - federal/state/local, loans/grants, amounts disbursed…  Courses - offerings, student & teacher participation, instruction type, grades…  Educational Staff - demographic, awards, certification, teaching & work activity  Educational Awards - type, program, institution, date Examples

Regular Publication of Annual Performance Accountability Data Annual performance data include employment rates, earnings, full-time employment proxies, continuing postsecondary education statistics by educational sector, public assistance participation, and incarceration…

Most High School Grads Nation-wide Go On To Postsecondary Within 2 Years Source: NELS: 88, Second (1992) and Third (1994) Follow up; in, USDOE, NCES, “Access to Postsecondary Education for the 1992 High School Graduates”, 1998, Table 2.

Information may be aggregated over long periods of time to show longer term results of continuing to pursue educational opportunities, employment, earnings, and interactions of various factors… Source: FETPIP Longitudinal Regular Publication of Longitudinal Information for Select Cohorts * 1991 Public High School Graduates in 2002 – Standard Diplomas

Source: FETPIP Longitudinal Florida Public High School Class of 1991 Earnings by Educational Credential in 2002

FETPIP Data  Public Education Funding  Performance-Based Budgeting  Federal Performance Measurement: WIA, Wagner-Peyser, Voc Rehab, Blind Services, Perkins, NRS (Adult Education)  Private School Licensure  School District Accountability Reports  Community College Performance Reports  University System Program Reviews  ICUF Accountability Reporting  Legislative Performance Reviews  Workforce Estimating Conference  Policy Research  Career Counseling, Consumer and Guidance Information Major Uses

FETPIP-Type Operations  Use existing data resources  Cheaper, more accurate, more complete, and with more capabilities, than traditional surveys  Data are regarded as objective- non-judgmental  Provide a basis for common core sets of outcome measures, comparisons, discussions of issues across many programs & operations Benefits

With such a system, what can you expect to hear? Part 1 Wow! You can do that? If I provide you with a set of student data can you…? How much will this cost me? Why should I participate in this?

With such a system, what can you expect to hear? Part 2 The System:  is too slow  is not done frequently enough  doesn’t cover students without correct SSNs  misses types of employment  misses types of postsecondary education  doesn’t cover the other, more important outcomes

Challenges (part 1)  Administrative data are designed for purposes other than performance measurement Definitions, taxonomies may differ Time periods may differ What is measured may not be exactly what is desired Administrative data change over time  Owners of Administrative Data have legal, ethical, proprietary obligations They won’t naturally want to share “Legal” beliefs may be conjured up Program interests may be different from those of technical staff The “owners” change over time

Challenges (part 2)  Confidentiality of Information Requirements may be different for each data source The issue is complicated by legal issues as well as beliefs Beliefs/requirements may change over time  Weakest Links The strength of a system of linked data sets is determined by the weakest of the data resources Opponents will focus on the weaknesses The data lag actual performance  Support On going, reliable funding On going, reliable political support Cooperative attitude among stake holders All involved will change

What has worked for FETPIP?  Statutory authority, provisions, including cross references  Core funding from State General Revenue appropriated by the Legislature  Persistently collaborate with legislative, executive staff  Persistently collaborate with data owners, stakeholders in decisions about the mechanics of exchanges, reporting of results  Constantly anticipate roles that the system can play in accountability, performance measurement, cost return analysis, evaluation  Consistently work with data owners on changes, data anomalies, problems  Steadfastly make it easy for stake holders to participate, reduce burdens, provide data in exchange  Always facilitate discussions about common data elements, definitions, core performance measures, performance funding Support

What has worked for FETPIP?  Location within the state education agency  Always surpass all requirements, expectations regarding data security  Faithfully limit uses to statistical purposes for evaluation, program improvement  Persistently keep exchange agreements current and up-to-date, continually brief employees, stake holders on security and confidentiality issues  Always refer inquiries about particular resources to the resource owners Confidentiality/Security

What has worked for FETPIP?  Strive to meet changes, challenges head-on  Persistently refresh understanding of resources, requirements, limitations  Persistently seek new data resources, expand coverage  Always revise business rules as understanding, conditions change  Establish short term and long term visions for the effort Changes

What has worked for FETPIP?  Establish archival data resources  Accommodate ad hoc interests and requirements  Seek out opportunities to demonstrate your wares  Always collaborate Other

K20 EDW – Florida’s K20 Education Data Warehouse

Employee Identity Educational Institutions Educational Curriculum Employee Demographic Includes:  Teachers  Faculty Members  Support Staff Course K-20 Budget Student Demographic Certified Staff Instructional activity Financial Assistance includes:  Federal Aid  State Aid  Local Aid  Bright Future Student Test Scores (Ex-)student Employment District Institution's Finances Chart of Accounts Institution's Revenues Active Student includes:  All enrollments  Acceleration  Attendance  Discipline  Exceptionality Program Educational Facilities Student Award Student's Identity Student's Identity Student's Courses Includes:  Public Schools  Colleges  Universities  Private Institutions Includes  SUS Application  Talented 20 Florida’s K20 Education Data Warehouse

Educational Institution Demography Educational Awards Educational Staff Courses Grades, Standardized Test Scores Financial Aid Student Employment K20 Data Warehouse: Student-Centered