1. AAC/DAC and FSAA Update Meeting
Angela Leigh Nathaniel
August 31, 2017

2. Agenda Important Dates FSAA UEB Rollout
Online Management System- Piedra Data Services
Test Security
Unreturned Materials Reports
Returning Materials – Best Practice
2017 Performance Task Results
2017 Datafolio
Service Center Calls
Contact Information

3. Important Dates Assessment View System (AVS) opens – August 30, 2017
Datafolio
Datafolio
Assessment View System (AVS) opens – August 30, 2017
nd Administration Training – September 6-8; Tampa
Collection Period 1 (CP #1) – September 1-29, 2017
Goal Setting – October 2-9, 2017
CP # 1 and Goals Locked at Midnight – October 9, 2017
CP #2 – October 30-December 15, 2017
CP #2 Evidence Upload – October 30, 2017 – March 9, 2018
CP #3 March 12 – April 6, 2018
CP #3 Evidence Upload March 12 – April 13, 2018
AVS Closes April 13, 2018

4. Important Dates Performance Task Performance Task
Preorder Materials – EOCs, Braille, One-sided – September 20-November 1, 2017
Administration Training Modules 1-3/Tutorials - October 2017-March 31, 2018
Administration Support Documents – January 13, 2018
Online System Training Module 4/Tutorials – February – April 28, 2018
Elementary and Middle School, Civics EOC
Materials in schools – February 16-22, 2018
Testing Window February 26-April 13, 2018
Return Materials – May 11, 2018
High School and Course EOCs
Materials in schools – March OR March 19-23, 2018
Upon receipt through April 27, 2018

5. UEB FSAA-Performance Task Rollout

6. Online Management System

7. Online Management System
Ordering Materials
Preorder Materials – September 20 – November 1, 2017 using the Online Management System
Braille/Tactile Graphics
Grades 3-5 UEB ONLY (NEW)
Grades 6 – 10 & EOCs EBAE ONLY Indicate Contracted or Uncontracted Students requiring tactile graphics will receive uncontracted materials

8. Online Management System
Ordering Materials (Cont’d)
One-sided Response Booklets
EOC Assessments – Early, valid & reliable counts
Materials for students completing courses with EOCs in 2018 MUST be ordered using the Online Management System (e.g., Access Algebra 1B)
EOCs are ordered by school
Indicate the course name and the number of students taking the EOC

9. Online Management System

10. Online Management System
Verification System
Must have BOTH the student’s SID and their FLEID to edit or add students in the system (NEW)
Verify survey 2 student demographic information (December, 2017)
View/edit shipping contact information

11. Online Management System
Verification System (Cont’d)
Add additional students
Grades 3-10 – Required
Grades 11 & 12 – Optional (NEW)
Order additional test materials (Available beginning February, 2018)
Schedule pickup of materials (Available beginning February, 2018)

12. Test Security

13. Test Security
Securely maintain all test materials before, during, and after test administration.
Store in locked storage room
Return all materials to School Assessment Coordinator (SAC)
Maintain Chain of Custody form

14. Test Security
ALL involved staff - Sign the “FSAA-PT Test Administration and Security Agreement”
AACs must also sign the “Test Administrator Prohibited Activities Agreement”

15. Test Security Breaches of Administration
Report possible breaches (i.e., secure test content photographed, copied or otherwise recorded) to SAC
SAC reports possible breach to district coordinator (DC) - DC contacts DOE with plan of action
DC provides written report within 10 calendar days

16. Test Security Breaches of Administration (Cont’d)
DC Completes and fax or * “Breach Administration Report” to FLDOE
*Remember confidential student information must not be communicated via

17. Test Security Missing Materials
SAC verify packing list; report missing or mis-packaged materials to the DC
School Level Responsibilities
investigate any report of missing materials
If not found, SAC contacts the DC

18. Test Security Missing Materials (Cont’d)
District Coordinator Responsibilities
Interview appropriate staff
Send written report to Bureau of K-12 Assessment within 30 calendar days, including changes to procedures to prevent reoccurrence
Complete and fax or * “Missing Materials Report” to FLDOE

19. Unreturned Materials

20. 2017 Unreturned Materials Reports
Distributed materials
Total number of test materials distributed – 117,743 to 69 districts
18,688 (18.9%) more than 2016
Missing/Reconciled materials
Total number of materials reported missing, but accounted for – 72 in 12 districts
49 (200%) more than 2016

21. 2017 Unreturned Materials Reports
Unreturned/Not Reconciled materials
Total number of test materials not reconciled – 49 in 12 districts
114 (30.1%) less than 2016
46 districts received clean missing materials reports vs 41 in 2016
Percent missing/unreturned materials .1% vs %

22. Returning Materials - Best Practices
Use sturdy boxes and secure both the top and the bottom with proper packing tape.
Districts using pallets to return materials should limit the number of boxes on the pallet to 48.
Send Missing and Soiled Materials Reports to both DOE and Piedra.

23. 2017 Performance Task Test Results

24. 2017 FSAA-Performance Task Results
ELA
23,889 valid scores in ELA grades 3 thru 10
Level 1* - 4,414 students
Level 2* - 7,832 students
Level 3* - 8,244 students
Level 4* - 4,706 students
* Does not include data from districts with N size less than 10

25. 2017 FSAA-Performance Task Results
ELA – Number assessed by grade
Grade 3 – 2,926
Grade 4 – 2,918
Grade 5 – 3,099
Grade 6 – 2,990
Grade 7 – 2,959
Grade 8 – 2,969
Grade 9 -2,999
Grade 10 – 3,288

26. ELA – Performance Level by Grade Grades 3-5*

27. ELA – Performance Level by Grade Grades 6-8*

28. ELA – Performance Level by Grade Grades 9-10*

29. 2017 FSAA-Performance Task Results
Mathematics
17,887 valid scores in Mathematics grades 3 thru 8
Level 1* - 3,885 students
Level 2* - 4,154 students
Level 3* - 5,485 students
Level 4* - 3,640 students
* Does not include data from districts with N size less than 10

30. 2017 FSAA-Performance Task Results
Math – Number assessed by grade
Grade 3 – 2,922
Grade 4 – 2,924
Grade 5 – 3,111
Grade 6 – 2,999
Grade 7 – 2,961
Grade 8 – 2,970

31. Mathematics – Performance Level by Grade; Grades 3-5*

32. Mathematics – Performance Level by Grade; Grades 6-8*

33. 2017 FSAA-Performance Task Results
Science
6,067 valid scores in Science grades 5 and 8
Level 1* students
Level 2* - 1,722 students
Level 3* - 1,959 students
Level 4* - 1,242 students
* Does not include data from districts with N size less than 10

34. 2017 FSAA-Performance Task Results
Science– Number assessed by grade
Grade 5 – 3,105
Grade 8 – 2,962

35. Science – Performance Level by Grade; Grades 3 and 5*

36. 2017 FSAA-Performance Task Results
EOCs
11,038 valid scores in Algebra 1, Biology 1, and Geometry
Number Assessed by Course
Algebra 1 – 3,627
Biology 1 – 3,113
Geometry – 4,298

37. 2017 FSAA-Performance Task Results
EOCs – Level 1*
EOCs - Level 2*
Algebra
Biology
Geometry– 555
Algebra 1- 1,401
Biology 1- 1,659
Geometry- 1,135
Algebra 1– 1,005
Biology 1– 1,135
Geometry– 845
Algebra 1 – 658
Biology 1 – 790
Geometry - 468
EOCs – Level 3*
EOCs - Level 4*

38. EOCs – Performance Level by Course; Algebra1, Biology and Geometry*

39. FSAA-Datafolio

40. FSAA-Datafolio 2017 Administration 41 districts 186 Schools
340 Teachers
602 Students

41. FSAA-Datafolio Standard Setting – July 2017
FSAA-PT Social Studies
FSAA Datafolio
Recommendations to Commissioner and State Board
3 Performance Levels
Level 3 - Satisfactory

42. FSAA Service Center

43. FSAA Service Center
Service Center hours Monday – Friday 8am – 5pm daily (excluding holidays)
(866)
Extended Hours between February and May (7:00 am to 8:30 pm)

44. FSAA Service Center Activity 2016 – Performance Task
Performance Task April 4 – 29, 2016
Total Number of Calls – 2,424
Total tickets
Resets – 1,646
Logins/password – 390
Data management – 205
Function/Administration – 132
Days with >100 calls - 12

45. FSAA Service Center Activity 2016
March 21 – May 5, 2016
Total Tickets – 3,167
Response to Service Center Calls
Improvements to the online system
Improvements in Training
Improvements to the Online System Platform
Added School Level Coordinator
Allowed ACC/SLC to reset passwords
Changed Test Form Assignment Process
Changed submit Test Process
Improvements in Training
Additional Resources to Target Areas of Concern
ie., Tutorial for Scaffolding Procedure

46. FSAA Service Center Activity 2017
Performance Task April 3 – 28, 2017
Total Number of Calls – 1,117
Total Tickets – 1,090
Resets – 201
Logins/password – 321
Data management – 268
Function/Administration – 75
Days with >100 calls - 4

47. FSAA Service Center Activity 2017
March 13 – May 5, 2017
Total number of calls 3,167

48. FSAA Service Center 2016 vs 2017 2016 2017 Change 2424 1117 -54% -41%
2016
2017
(Peak Month Activity)
4/4/ /29/2016
4/3/ /28/2017
Change
Total Number of Calls
2424
1117
-54%
Total tickets
1846
1090
-41%
Resets
959
201
-79%
Logins/password
390
321
-18% 
Data management
205
268
+31% 
Function/Administration
132 75
 -43%
Days with >100 calls 12 4
-67%
(Online Entry Window)
3/21/ /5/2016
3/13/ /5/2017
3167
1777
-49%

49. FSAA Service Center 2016 vs 2017

50. Total Number of Calls during Online Entry Window
3/21/16 - 5/5/16 versus 3/13/17 - 5/5/17
-49%

51. Considerations for 2018 Improvements to the Online System
Reduce presentation mode to facilitate faster online response entry
Additional Training Resources
Exploring additional Train the SLC training for AACs
Imbedded Scaffolding videos for Train the Trainer
Available as stand alone tutorial on the Portal

52. Contact Information
FSAA Service Center
Monday – Friday 8 am – 5 pm EST (extended hours during assessment window 7 am – 8:30 pm EST)
Phone –
Fax – –

53. Contact Information
DOE – Policy questions
Angela Nathaniel, Program Specialist
Laura Bailey, Project Manager
Fax –