NAEP, TIMSS, PISA and PIRLS September 8, 2015 DAC Annual Meeting

Update There is a new NAEP Assistant in town! Alexandra Fahnlander Alex is from Fort Myers Graduated from FSU in May with a degree in English, focusing on editing, writing, and media Worked in the arts and at the FSU radio station during college Volunteers with Florida’s YMCA Youth in Government throughout the year She can be reached at 850.245.9317 AlexFahnlander@fldoe.org 2

National Assessment of Educational Progress (NAEP) 2015 Results

NAEP 2015 Results 2015 NAEP Mathematics and Reading assessment results will probably be released the first week in November National, State, and TUDA results will all be released together - http://www.nationsreportcard.gov/ 4

NAEP 2015 Results Results will also be posted on the NAEP webpage of the FLDOE website. Best accessed through Chrome http://www.fldoe.org/accountability/assessments/na tional-international-assessments 5

Trends in International Mathematics and Science Study (TIMSS) 2015 Results

TIMSS 2015 Results 2015 TIMSS Mathematics and Science assessment results will be released December 2016 International and Florida results will be released together - https://nces.ed.gov/TIMSS/ 7

2011 TIMSS Mathematics Results 8

2011 TIMSS Science Results 9

Summary of change in average U.S. TIMSS scores over time 10

Program for International Student Assessment (PISA) 2015

PISA 2015 in Florida Oct 5 – Nov 13, 2015; Results will be released December 2016 PISA testing in Florida will occur in 11 schools from 8 districts: Duval Martin Hernando Miami-Dade Hillsborough Orange Manatee Volusia 12

What is PISA? International assessment of 15- year-old students conducted every 3 years since 2000 http://nces.ed.gov/surveys/pisa Measures how well students apply their knowledge and skills in solving problems in science, reading, mathematics, financial literacy, and collaborative problem solving

PISA 2015 in Florida, con’t Most students are assessed in only 2 of the 5 assessment areas. No student takes every PISA test question. 2015 Focus is Science 2012 Focus was Mathematics—Florida was over- sampled to provide state-level results Students from more than 70 countries and education systems have previously participated Sample size is 42 students – each student will receive a certificate for 4 hours of volunteer service

PISA 2015 in Florida, con’t PISA 2015 will be administered on computers provided by Westat PISA takes approximately 3 hours–includes instructions, the assessment, and a brief survey questioning students about themselves. Some students will also be selected to participate in an additional 1-hour assessment in financial literacy that is held after main PISA. Many PISA items are interactive. Sample questions can be found at http://nces.ed.gov/surveys/pisa/items_cba.asp

2012 Mathematics Literacy - U.S. 15-year-olds below OECD average CT 506 (*) FL 467 (*,**) * = State avg. different from U.S. ** = State avg. different from OECD Average is higher than U.S. average Average not measurably different from U.S. average Average is lower than U.S. average 16

2012 Science Literacy - U.S. 15-year-old students not measurably different from OECD average MA 527 (*,**) CT 521 (*,**) FL 485 (**) * = State avg. different from U.S. ** = State avg. different from OECD Average is higher than U.S. average Average not measurably different from U.S. average Average is lower than U.S. average 17

2012 Reading Literacy - U.S. 15-year-olds not measurably different from OECD average MA 527 (*,**) FL 492 CT 521 (*,**) * = State avg. different than U.S. ** = State avg. different than OECD Average is higher than U.S. average Average not measurably different from U.S. average Average is lower than U.S. average 18

State Results for PISA 2012 Average Scores Compared to 65 OECD Countries and Education Systems can be found at http://nces.ed.gov/surveys/pisa/items_cba.asp

National Assessment of Educational Progress (NAEP) 2016

NAEP 2016 in Florida Jan 25 – March 4, 2016 National Results: Arts in March 2017 Florida - 74 schools in 14 districts Distribution of Assessments: Grade 4: 16 schools, DBA, Mathematics and Reading Grade 8: 31 schools, DBA, Mathematics and Reading or Writing Grade 8: 10 schools, Paper-and-Pencil, Arts (Music and Visual Arts) Grade 12: 17 schools, DBA, Writing Broward Miami-Dade Pinellas Escambia Orange Santa Rosa Hillsborough Osceola Sarasota Lake Palm Beach Seminole Manatee Pasco 21

NAEP 2016 in Florida, con’t Two types of assessments Paper and Pencil Assessment Subject: Art Grade: 8 Digital-Based Assessment (DBA) Microsoft Office Pro 3 tablets provided by Westat Subjects: Mathematics, Reading, and Writing Grades 4 and 8: Mathematics and Reading Grades 8 and 12: Writing Grades 4 and 8: Two administrations of 25 students each Grade 12: One administration of 25 students 22

NAEP 2016, con’t DACs will receive a folder for each sampled school that will include: School’s MyNAEP ID Number and assessment date Directions for registering on MyNAEP and completing the Provide School Information (PSI) form—note: enrollment number is to be for the grade, not the school An Overview of NAEP NAEP in Your School NAEP Tools on the Web Strategies for Motivating Grade 8 Students DACs will receive a District MyNAEP ID number 23

Motivating Grade 8 Students Student Brochure School NAEP Coordinator Brochure 24

NAEP Pre-Assessment Activities Principals have already been given their assessment dates by you Their next important task is to appoint a School NAEP Coordinator (SNC) List of sampled students will be provided via the MyNAEP website in early December 25

NAEP Pre-Assessment Activities In December a Westat staff person will call each of the schools to Assist with registering for MyNAEP Review all MyNAEP “Prepare for Assessment Tasks” Go through the “Complete the Following:” tasks and provide a short description of each Introduce the video tutorials Provide timeline for completing tasks Schedule the Preassessment Review Call (PRC) for January

NAEP Pre-Assessment Activities, cont. SNCs or DACs are required to upload a current roster of students enrolled in the sampled grade in January Enrollment will be open for the first 3 weeks in January Schools with early assessment dates will be requested to upload their current rosters the week of January 4 Schools with later assessment dates will be requested to wait until the weeks of Jan 11 or 18 SNC will be notified if any new students have been added to the sample NAEP Help Desk is being redesigned to better meet the needs of the schools Hiring more agents More comprehensive agent training 27

NAEP Pre-Assessment Activities, con’t. On the MyNAEP website, the SNC or Principal will have to review the list of sampled students to: Identify withdrawn or ineligible students Update demographic data and SD/ELL classifications Indicate how each student identified as SD and/or ELL should be assessed on NAEP (keep exclusions to a minimum, especially for grade 8 ELL students scheduled for the Reading assessment) Indicate if a student has a Section 504 Plan and will need testing accommodations Identify any student who is not enrolled in current grade, is a foreign exchange student, or does not attend classes on campus Field representatives will make in-person visits to schools experiencing difficulty with MyNAEP tasks

NAEP Pre-Assessment Activities, cont. Copy of Parent/Guardian Notification Letters no longer collected by DACs On the MyNAEP website, SNCs will verify parent/guardian notification at least 2 weeks prior to the assessment School and Teacher Questionnaires will be completed online SNC will send questionnaires via email to school principal (or a designee) and to teachers of students in the NAEP sample SNC will track the completion of the questionnaires and be able to send reminder emails, as necessary 29

NAEP Pre-Assessment Activities, cont. SNC will finalize assessment arrangements Provide start times and testing locations Provide 2 possible dates for make-up sessions Assessment cancelled due to weather Participation is less than 90% SNC will Print and distribute Student Appointment Cards Teacher Notification Letter – date, time, location of sessions List of Sampled Students 30

NAEP 2016 Each student takes NAEP in only 1 of the 2 different formats (P&P or DBA) and in only 1 of the 4 subject areas (Art, Mathematics, Reading, or Writing) NAEP representatives will bring all necessary materials and equipment to the schools 31

NAEP 2016 http://www.fldoe.org/accountability/assessments/national-international-assessments/naep/resources-for-participating-schools.stml 32

NAEP 2017 and Beyond 33

NAEP 2017 - 2020 National Assessment Governing Board (NAGB) Goal: NAEP 2017 operational mathematics and reading assessments will be completely digital-based for the first time (http://www.nagb.org/). Grade 12 state-level NAEP has been eliminated (administered in 2009 and 2013 in 13 self-selected states including Florida). An * indicates state-level results will be available. Current schedule adopted by NAGB March 2015 34

Progress in International Reading Literacy Study (PIRLS) 2016 35

PIRLS and ePIRLS 2016 March 1 – May 27, 2016; Results will be available December 2017 8 schools in 6 districts International assessment designed to measure trends in reading achievement at the 4th grade level every 5 years since 2001 https://surveys.nces.ed.gov/Pirls/ Involves students from more than 40 countries U.S. Students from 176 schools in 42 states will be sampled in 2016 Clay Palm Beach Duval Seminole Orange St. Lucie 36

PIRLS and ePIRLS 2016, con’t PIRLS will send a letter to school principals on  September 21st reminding them to name a school coordinator if they haven’t already done so. PIRLS will contact schools again after October 5th to Obtain the name of the school coordinator (if not already named) Determine data collection logistics

PIRLS and ePIRLS 2016, con’t Students (one or two classes of approximately 18- 24 students in each school) will participate in PIRLS and ePIRLS on two successive days PIRLS is a paper-and-pencil assessment that includes reading passages containing informational and literary texts followed by open- ended and multiple-choice questions about the texts – 2.5 hours ePIRLS uses a simulated Internet environment to measure fourth-grade students’ achievement in reading for informational purposes – 2.5 hours 38

ePIRLS is to be conducted on school computers if possible PIRLS and ePIRLS 2016, con’t ePIRLS is to be conducted on school computers if possible RTI International will provide laptop computers if: School computers are not compatible with ePIRLS School computers are not available at the school The school computer lab does not have enough computers for each student RTI will work with participating schools to coordinate logistics between October 2015 and February 2016 39

Florida scored 2nd in the WORLD on Grade 4 PIRLS in 2011! Not significantly different from Florida Not significantly different from U.S.

PIRLS 2016 http://www. fldoe 41

Data and Released Questions 42

NAEP Data Explorer (NDE) http://nces. ed Analyzes NAEP data Creates statistical tables, graphics, and maps Compares the performance of jurisdictions and evaluates trends over time Shows possible relationships between variables and student performance 43

NAEP Questions Tool (NQT) http://nces.ed.gov/nationsreportcard/itmrls Contains over 1,000 released items from different content areas Sorts items by domains, objectives, cognitive ability, and difficulty level Includes multiple-choice and both short- and extended-response items Reports student performance on a specific test item by states and subgroups 44

International Data Explorer Questions Tools International Data Explorer PIRLS, PISA and TIMSS have released items from mathematics, reading, and science: http://nces.ed.gov/surveys/international/ Analyzes PIRLS, PISA and TIMSS data and can create statistical tables, graphics and maps: http://nces.ed.gov/surveys/international/ide/

Goodbye! After serving as your NAEP State Coordinator since January 2003, I am preparing to retire next July. That will make me one of the longest-serving NSCs in the country. Many thanks to all of you for being such outstanding colleagues over the years! I couldn’t have done the job without your help and support! 46

Michele.Sonnenfeld@fldoe.org http://www.fldoe.org/accountability/assessments/national-international-assessments/naep (850) 245-0787 47