1. Welcome to the AEM Quality Indicator Series, Part 4!
We’ll begin shortly
Material is available for download at
To view closed captions:
Click on the Closed Caption button in your Zoom controls
Captions will appear along the bottom of your Zoom window
@AEM_Center

2. Contribute to the Conversation
To use the text chat:
Open the chat panel
Choose all panelists and attendees from the dropdown menu above the text entry field
And please tweet out to #aem4all
@AEM_Center

3. AEM Quality Indicators for K-12 Part 4: Collecting and Using Data to Monitor and Evaluate System Effectiveness
Cynthia Curry, AEM Center Director
Kay Ratzlaff, Supervisor, Florida Instructional Materials Center for the Visually Impaired (FIMC-VI)
Daniel McNulty, State Director, PATINS Project, Indiana

4. The Quality Indicators for the Provision of AEM K-12
A coordinated system
Timely manner
Written guidelines
Comprehensive learning opportunities and technical assistance
Systemic data collection
Use of data to guide changes
Allocation of resources
@AEM_Center

5. Objectives Participants will be able to
Explain, in their own words, the intent behind AEM Quality Indicators 5 & 6.
Learn from the experiences and practices of other states and districts to improve their own data collection and data use processes.
Know where to go to build background knowledge and collect resources for further learning.
@AEM_Center

6. Presenters Cynthia Curry, AEM Center Director
Kay Ratzlaff, Supervisor, Florida Instructional Materials Center for the Visually Impaired  (FIMC-VI)
Daniel McNulty, State Director, PATINS Project, Indiana
@AEM_Center

7. About Quality Indicators 5 & 6
Data Collection and Data Use
@AEM_Center

8. Quality Indicator 5
The agency develops and implements a systematic data collection process to monitor and evaluate the equitable, timely provision of appropriate, high-quality accessible materials and technologies.
@AEM_Center

9. Quality Indicator 5: Intent
Data are collected that measure the degree to which accessible materials and technologies provide all learners with disabilities the opportunity to acquire the same information, engage in the same interactions, and benefit from the same services as learners without disabilities with substantially equivalent ease of use (U.S. DOJ/U.S. DOE).
@AEM_Center

10. Quality Indicator 5: Sample Components
Strategies for collecting data
Extent to which students receive AEM
timely manner, number of students served, disability categories, sources, feedback, etc.
Compliance
copyright and accessibility
Learning opportunities and TA provided
providers, delivery formats, impact, etc.
Plan for collecting and maintaining data
Methods
data management system
Timelines
@AEM_Center

11. Quality Indicator 6
The agency uses the data collected to guide changes that support continuous improvement in all aspects of the systemic provision and use of accessible materials and technologies.
@AEM_Center

12. Quality Indicator 6: Intent
Data are systematically analyzed to gauge effectiveness of all aspects of the current system and are used to inform actions needed to improve future practice, program planning, and resource allocation.
@AEM_Center

13. Quality Indicator 6: Sample Components
A plan for analysis and use
purpose, sharing, quantitative & qualitative analyses, student ID protection, timelines for review and implementing decisions
Cross-referencing strategies
Student independence, participation, progress
Disability categories served
Student demographics
Data use improves future practice, program planning, resource allocation
@AEM_Center

14. Pre Self-Reflection
To what extent are Quality Indicators 5 & 6 addressed in your own provision of materials and technologies for students?
Not started
Somewhat
Really cooking!
@AEM_Center

15. AEM & AIM
@AEM_Center

16. Florida
Kay Ratzlaff, Supervisor, Florida Instructional Materials Center for the Visually Impaired  (FIMC-VI)
Data Systems in Use in Florida 
Use of the Data
Security of Data
@AEM_Center

17. Florida FIMC-VI Responsibilities for AEM:
Textbooks for students with visual impairments
District Digital Rights Managers for students with print disabilities
Coordinate with other Department of Education projects
This Photo by Unknown Author is licensed under CC BY-SA
@AEM_Center

18. FIMC-VI Data Systems Comprehensive proprietary database system to:
Provide online annual registration of students with visual impairments
Online ordering of accessible textbooks
Track textbooks from order to delivery to inventory
Reports – weekly, monthly, quarterly, annually
Annual online services survey 
Online ordering and tracking for APH materials and Loan Library
District-reported data from Quality Programs for Students with Visual Impairments (QPVI) Master List 
Quarterly and annual reports to Florida DOE
@AEM_Center

19. Statewide Data Systems
From the Florida Department of Education we receive:
Student enrollment numbers per district by primary disability (unduplicated) and secondary disability (duplicated) 
Student achievement data on statewide assessments by disability for each district and grade level
Student placement data (LRE) by disability 
Teachers certified by disability areas per district 
Student exit data by disability
@AEM_Center

20. Student Data Disaggregation and Analysis
Student demographics
Disabilities
Tracking districts for over and under identification and providing appropriate technical assistance 
Correlate enrollment to textbook and materials orders 
Correlate achievement to textbook and materials orders
@AEM_Center

21. Use of Data Accountability for transcribers and vendors
Accountability for in- house production and shipping
Reported to Florida DOE quarterly and annually 
Identify professional development and technical assistance needs
Predict future textbook orders and materials purchases 
Share with stakeholders
@AEM_Center

22. Security of Data
Only the staff members authorized by their district Director of Exceptional Student Education are allowed access to the FIMC-VI database 
Authorized annually 
Once authorized, staff can register students and order materials at any time 
Digital Rights Managers are also authorized annually by ESE Directors; up to seven allowed per district
Auto back-ups and cloud-based security
@AEM_Center

23. Come teach with us in Florida and that beach chair can be yours
Come teach with us in Florida and that beach chair can be yours! For more information:
Contact
Kay Ratzlaff
@AEM_Center

24. Indiana Daniel McNulty, State Director, PATINS Project
Indiana Center for Accessible Materials (ICAM)
Statewide AEM Data Collection (Acquisition)
AEMing for Achievement Intensive District Data (Implementation and Anecdotal)
@AEM_Center

25. ICAM Indiana Center for Accessible Materials
Assist Indiana's LEAs in meeting the NIMAS regulations of the IDEA reauthorization of 2004 by securing accessible textbooks and core curriculum materials in specialized formats in a timely manner, for students with a documented print disability.
Student Registration / DRM Registration
Web-Based Ordering System
Braille, Large Print, Audio, Digital Text
Maintenance of Copyright Integrity
Conversion of File Formats
AEM Technical Assistance and Support
@AEM_Center

26. ICAM Data on Acquisition
Collected through LEA’s requests for Specialized Formats:
As NIMAS file sets are acquired from NIMAC:
Approx 5,000 NIMAS Textbooks as various conversions
4,350 Braille
Student Registrations / DRM Reg and Training
As requests are placed by LEAs
As requests are fulfilled to LEAs: 3,170 (July – Feb)
As technical assistance requests are fulfilled to LEAs: 1,469 (July – Feb)
@AEM_Center

27. AEMing for Achievement Grant
ABOUT
Application Process: 8 school districts selected per school year
Full Day of Orientation to accessibility, UDL, ICAM and AEM in August
Two PATINS Specialists assigned to each district
In-Depth Needs Assessment with each district (based on 7 AEM Quality Indicators)
Action Plan developed with each district (based on 7 AEM Quality Indicators)
Weekly meetings on Policy & Implementation with Specialists and/or Daniel
Mid-year online progress gathering in January. Each team presents for 15 min.
Weekly meetings and action plan implementation continues
Collaboration and Show & Tell day in late May, with data and stories
@AEM_Center

28. AEMing for Achievement: Implementation Data
Monthly Data Form Required
Month, District Name, Address
Percent of Students on Diploma Track This Month
Time Away from Instruction/Out of Class Tracking Form
Number of Active DRMs in District this Month
Number of Students Identified with a Print Disability this Month
Written Districtwide Policy Regarding AEM this Month
Anecdotal Evidence: An example of how AEM has impacted a student or group this month
@AEM_Center

29. AEMing for Achievement: Implementation Data, cont’d
September 2018– February 2019
Percent of Students on Diploma Track
5 out of 7 teams show increase in 6 months
Time Away from Instruction/Out of Class
Inconclusive but some show decrease
Number of Active DRMs in District this Month
All have increased
Number of Students Identified with a Print Disability
All have increased but mostly in the past 2 – 3 months
Written Districtwide Policy Regarding AEM
None at the start, all but 1 do currently
Anecdotal Evidence: An example of how AEM has impacted a student or group this month
@AEM_Center

30. Anecdotal Data
“Starting the process of creating a written policy regarding AEM for district, this will impact for years to come not just this group of students but generations hopefully.”
“It has prompted us to write and complete a written policy regarding accessible educational materials, which impact all of our students.”
”Increase AT evaluations due to AEM grant”
“It was very beneficial for a third grader who is struggling as he is in his final stretch to prepare for IREAD-3.”
@AEM_Center

31. Anecdotal Data, cont’d
“One student asked to speak to the School Board to talk about how (Text to Speech) has helped her and to thank me for writing this grant to bring it to our district.”
“An internal website is under development to provide teachers with information about UDL and to prepare for rolling out UDL on a larger scale.”
”One student reported that they have been able to write their writing assignments more easily and with less stress. This has been a huge success because she often would refuse to do written assignment work.”
@AEM_Center

32. Anecdotal Stories Kayle Sam Emily Mathew Ben
Sam: 2nd grade, PreK Level A reader. CCC met and determine he would be put into a functional life skills program and effectively taken off diploma track. 2nd GRADE! Through our AEM grant, it was realized that if sam was provided with materials that were flexible and accessible to him and tools that provided TTS, he was actually comprehending text on a 4th grade level! Sam now uses STT and TTS and is on grade level with all classes and is ON diploma track.
Emily: Jr. High… AP classes, very introverted, possibly on the autism spectrum, Struggling with vocab in AP Gov. shut down, went from being A student to a D student. Individual words were getting in the way of overall comprehension and effectively blocking growth and learning for Emily. With accessible materials, a tool that not only voiced text for her, but also could simplify text on demand, Emily could relate back to prior knowledge AND build her vocabulary. She learned that, with the right help she has still not reached the limit of her comprehension
Mathew: recent transfer into new school…6th grader. Additionally, Matthew experienced great feelings of success after intervention with Snap&Read and Co:Writer. His reading score increased by 23 points (from a level 5 to an early level 6 on the iReady diagnostic test) in just two months, bringing him into grade level range. He read over 15,000 words over two and a half days of reading. His grades increased from all B’s in four classes to three A’s and one B; these classes included Language Arts, Math, Science/Health and Social Studies. Seeing improvement and feeling empowered, he reported feeling better and more accepted. For the first time, it became evident that he was eager to come to class and to continue in Mishawaka Schools.
Ben:
@AEM_Center

33. Looking Forward: Implementation Data
Increase alignment with state results driving accountability
Graduation rate
State testing results/reports
Least Restrictive Environment (LRE) reporting
Alternative Assessment participation
Dropout rate of students with IEPs
Student growth/proficiency
@AEM_Center

34. Q&A
Data Collection & Data Use
@AEM_Center

35. Questions for Kay and Daniel
@AEM_Center

36. Post Self-Reflection
What will be your first action toward collecting and using data to improve outcomes for learners who need accessible materials and technologies?
@AEM_Center

37. AEM Progress Monitoring Tool for K-12 on the Horizon!
Accessolution?
Traccessibility?
Accessiful?
@AEM_Center

38. Join us for the final webinar of the series!
Part 5: Allocating Resources
Tue, Apr 2, 3 pm ET
@AEM_Center

39. Contact Information
Cynthia Curry, Director, AEM Center
@clcurry
Please access digital handout for contact information and resources provided by our presenters
@AEM_Center

40. Link to Our Evaluation
Please take a moment to complete our evaluation
@AEM_Center

41. OSEP Disclaimer
The contents of this presentation were developed under a cooperative agreement with the US Department of Education, #H327Z However, those contents do not necessarily represent the policy of the US Department of Education and you should not assume endorsement by the Federal Government, Project Officer, Tara Courchaine, Ed.D.
@AEM_Center