1. Missouri Department of Elementary and Secondary Education
Dynamic Learning Maps
2015 MO-CASE FALL CONFERENCE
Presented by: Bonnie Aaron & Tiffani Muessig
Missouri Department of Elementary and Secondary Education
September 2014

2. PURPOSE
To provide a basic overview of the New MAP-A developed by Dynamic Learning Maps consortium
Hello. This session on Dynamic Learning Maps is presented by the Office of Special Education. I am Tiffani Muessig and this is Bonnie Aaron.
The purpose of this session is to provide a basic overview of the New MAP-A developed by DLM consortium.
This session does not replace required DLM trainings or manuals. Test administrators must pass the required trainings with a score of 80% or more in order to administer student assessments.

3. LEARNING OUTCOMES: Learning Maps Claims and Conceptual Areas
To provide a basic overview of:
Learning Maps
Claims and Conceptual Areas
Essential Elements
Linkage Levels and Nodes
Instructionally Relevant Items (Testlets)
Item Types
We plan to provide a basic overview of Learning Maps, Claims & Conceptual Areas, Essential Elements, Linkage Levels & Nodes, Instructionally Relevant Items (Testlets), Blueprints and Test Features.

4. LEARNING OUTCOMES: What do educators need to know and do right now?
Security Agreement
Required Trainings
KITE
First Contact Survey
Personal Needs and Preferences
Released Practice Testlets
Practice Testlets for Students
Blueprint
We will also give you a basic overview of what educators need to know and do right now, such as, the security agreement, required trainings, KITE, First Contact Survey, Personal Needs Profile and Professional Development.

5. Dynamic Learning Maps (DLM)
Basic Overview
Beginning with the school year, all Missouri public education students eligible for the Missouri Assessment Program-Alternate will be taking a new form of assessment.
Missouri, along with other states, joined the Dynamic Learning Maps Consortium to build a new instructional-assessment tool for students with the most significant cognitive disabilities.

6. What is Dynamic Learning Maps?
The DLM project is guided by the core belief that all students should have access to challenging grade-level content
The new DLM Alternate Assessment System will let students with significant cognitive disabilities show what they know in ways that traditional testing cannot
The Dynamic Learning Maps
Are designed to map a student’s learning throughout the year
Uses items and tasks that are embedded in day-to-day instruction
Integrates testing and instruction to inform teaching and benefit students
The DLM project is guided by the core belief that all students should have access to challenging grade-level content. The new DLM Alternate Assessment System will let students with significant cognitive disabilities show what they know in ways that traditional testing cannot. The Dynamic Learning Maps are designed to map a student’s learning throughout the year, use items and tasks that are embedded in day-to-day instruction and integrate testing and instruction to inform teaching and benefit students.

7. What makes DLM unique? Assessments can be delivered on PC MAC Desktops
Laptops
iPads
Interactive whiteboards
DLM assessments can be delivered on PC or MAC desktops or laptops, iPads, or other devices attached to computers such as interactive whiteboards. Most students are expected to interact directly with the computer.

8. AAC Devices Have Given Freedom
Computer assessments are accessible via one-switch scanning devices, two-switch devices, alternate keyboard, mouse and iPad. If a student is not able to access the computer, then the teacher will enter the student’s answers.

9. The Big Picture
This is what the DLM big picture looks like. The next few slides will cover the learning map, claims, conceptual areas, and essential elements.

10. What are Learning Maps?
Learning maps are the many different pathways a student may take to learn a skill or concept.
Learning maps are the many different pathways a student may take to learn a skill or concept.

11. The maps feature multiple pathways toward the development of academic knowledge and understanding. For instance, a student may receive instruction using different approaches until the student learns the skill.
The DLM learning maps are highly connected representations of how academic skills are acquired, as reflected in research literature. In the case of the DLM project, the general education standards helped to specify academic targets, while the surrounding map content clarified how students could reach the specified standards via multiple and alternate pathways. Learning maps of this size had not been previously developed.

12. DLM’s Multiple Pathways
Similar to how our brains function, the Map provides different cognitive pathways and varying alternate pathways to content mastery.

13. What are Claims?
Claims are broad statements about what students are expected to learn and to be able to demonstrate within ELA and mathematics content areas.
A claim is a broad statement about what the DLM consortium expects students to learn and to be able to demonstrate within each content area. Each claim is subdivded into two or more conceptual areas.
The next few slides are a snapshot of technical terms used by DLM. To learn more about these terms, go to the DLM professional development self-directed learning module titled “Claims and Conceptual Areas.”

14. Claims . . .
Identify major domains of interest in English language arts and math for students with significant cognitive disabilities
Are broad statements about expected student learning that serve to focus the scope of the assessment
Help organize the structures in the learning map for this population of students
Claims identify major domains of interest in English language arts and mathematics for students with significant cognitive disabilities. They are broad statements about expected student learning that serve to focus the scope of the assessment. Claims help organize the structures in the learning map for this population of students. There are four claims in English language arts and four claims in mathematics.

15. English Language Arts Claims
Don’t read slide.
Here is an example of an English Language Arts Claim:
The Dynamic Learning Maps claims are overt statements about what we intend for students to learn and for our assessment to measure. For example, the first claim for English Language Arts states that students will comprehend text in increasingly complex ways.
The Dynamic Learning Maps project has specified a total of 4 claims in English Language Arts. They are: Students can comprehend text in increasingly complex ways. Students can produce writing for a range of purposes and audiences. Students can communicate for a range of purposes and audiences; and, finally, students can engage in research/inquiry to investigate topics and present information.
There are also 4 claims in Mathematics.

16. What are Conceptual Areas?
Conceptual areas further define specific knowledge and skills required to meet the broad claims identified by DLM.
Conceptual areas further define specific knowledge and skills required to meet the broad claims identified by DLM.

17. Conceptual Areas . . .
Are areas within the learning map that have nodes directly related to the Essential Elements
Represent concepts and skills that support the learning of the Essential Elements
Are groups of connected concepts and skills
Serve as models of how students may acquire and organize their content knowledge
Conceptual areas are areas within the learning map that have nodes directly related to the Essential Elements. They represent concepts and skills that support the learning of the Essential Elements. Conceptual areas are groups of connected concepts and skills and serve as models of how students may acquire and organize their content knowledge. There are 9 conceptual areas in ELA and 9 conceptual areas in math.

18. Conceptual Area ELA.C1.1
Subareas of the Claims, called Conceptual Areas, connect the learning map to the overall Claims and identify large areas of conceptually related skills. Within these Conceptual areas are Essential Element targets and the nodes reflecting the knowledge and skills development that precedes and extends beyond the targets.
For example, the first Conceptual Area of Claim 1 of English Language Arts is about Determining Critical Elements of Text and includes nodes about basic elements found within both narrative and information text.

19. What are Essential Elements?
Essential Elements are specific statements of knowledge and skills linked to the Missouri Learning Standards.
Essential Elements are specific statements of knowledge and skills linked to the Missouri Learning Standards. The next few slides provide a brief overview of the Essential Elements. To learn more, go to the DLM self-directed learning module entitled “Dynamic Learning Maps Essential Elements.”

20. Essential Elements MLS Most SCD EEs
For the students with the most significant cognitive disabilities (SCD) on the left, the Essential Elements (EEs) form a BRIDGE (middle) to the Missouri Learning Standards (MLS) on the right.

21. Essential Elements Continued . . . .
Alternate standards aligned to Missouri Learning Standards
Developed by content specialists (educators)
Provide links between the standards and grade level expectations for students with significant cognitive disabilities
Essential Elements are alternate standards aligned to the Missouri Learning Standards that were developed by content specialists. Essential Elements provide links between the standards and grade level expectations for students with significant cognitive disabilities.
The Dynamic Learning Maps Alternate Assessment uses the EEs to support the design of individualized experiences for each student by presenting questions and tasks that are appropriate for a student’s needs and abilities.
Embedded within the learning maps are EEs, which are challenging, grade-level learning targets. The EEs are specific statements of the content and skills students with significant cognitive disabilities are expected to know and be able to do in English language arts and mathematics.
EEs reflect a different approach to teaching, learning and testing that focuses on providing children with significant cognitive disabilities a deep understanding of the most important concepts in the subjects they are studying so they can apply that knowledge and skills to other subjects and in the real world.

22. Crosswalk from Missouri Learning Standards to Essential Elements
ELA.RL.4.3
English Language Arts Reading Literature Grade 4 Standard 3:
Describe in depth a character, setting, or event in a story or drama, drawing on specific details in the text
(e.g. a character’s thoughts, words, or actions).
ELA.EE.RL.4.3
Essential Element Reading Literature Grade 4 Standard 3:
Use details from the text to describe characters in the story.
The EEs provide links between the general education content standards and grade-specific expectations. The EEs are specific statements of the content and skills that are linked to the Missouri Learning Standards grade level-specific expectations for students with significant cognitive disabilities.
The EEs replace the Alternate Performance Indicators (APIs) in ELA and mathematics.
As an example of the how EEs align with Missouri Learning Standards, let’s take a look at English language arts, grade 4, standard 3.
Read the Missouri Learning Standards and the EE.

23. What are Linkage Levels?
Linkage levels are small sections of the learning map that contain one or more nodes.
Linkage levels are small sections of the learning map that contain one or more nodes.

24. Linkage Levels
Linkage levels are always related directly to grade level Essential Elements but extend back to foundational skills.

25. What are nodes?
Nodes specify individual skills and understandings that were drawn from the research in English language arts and Mathematics.
Nodes specify individual skills and understandings that were drawn from the research in English Language Arts and Mathematics.

26. What are Nodes? Essential Elements Learning Map Linkage Nodes
Nodes in the learning map represent specific knowledge, skills and understandings along the pathways. The DLM system includes nodes for English language arts and mathematics. There are also nodes that are foundational to the development of academic knowledge in both subjects.
Linkage nodes represent critical concepts or skills needed to learn the Essential Element.
Linkage Nodes

27. Five Linkage Nodes Five linkages to each Essential Element
Link the Essential Elements to the learning map
Initial precursor node
Distal precursor node
Proximal precursor node
Target node
Successor node
Where a student enters the map based on the First Contact Survey
There are five linkage levels: initial precursor, distal precursor, proximal precursor, target, and successor. The nodes at the target level (red arrow) are most closely related to the expectation in the grade-level Essential Element.
Think of the five levels in the context of a target.
The target is the bullseye or goal for which the student is striving. Essential elements are written at the target level. The other 4 levels are named in reference to the distance from the target. For example, the initial precursor level is the farthest from the target because students at that level are at the pre-symbolic stage of development. The distal precursor is still quite a distance from the target but closer than the initial level. The proximal level indicates the student is closer in proximity to reaching the target level. On the other hand, the successor node means the student has successfully passed the expectations of the target level.
Linkages identify major milestones on the pathway to achieving the knowledge and skills described by the EE and specify where a student is in relationship to that grade-level target. The linkage node is where the student enters the map based on the first contact survey.

28. This table shows the Node Linkage Progression and alignment of content standards and EEs. Nodes in the learning map represent specific knowledge, skills, and understandings along those pathways.
In this particular node linkage progression, the skill set begins at the initial precursor where the student is expected to pay attention to either the entire object, a characteristic of the object, or an action in which the object can perform after some verbal label has been attached to it. The skill set progresses to the target node where the student is expected to produce responses to questions seeking information on specific characters and what each of them did in a narrative by providing details on them.

29. What are Instructionally Relevant Testlets?
Instructionally relevant testlets are sets of 3-8 items and an engagement activity.
A testlet is short for Instructionally Relevant Testlet. These testlets are used to test the student’s knowledge of the skills required for each Essential Element.

30. Engagement Activity
Precedes a testlet and introduces the concept to be addressed
Describes a scenario and provides context for the items
Taps prior knowledge or experience
An engagement activity precedes a testlet and introduces the concept to be addressed. It describes a scenario and provides context for the items and taps prior knowledge or experience.

31. Testlets Are used for instruction AND testing
Increase the instructional relevance of the assessment
Provide a better estimate of the students’ knowledge, skills and abilities
Provide more reliable and valid indicators of student performance than current status collection of evidence
Testlets are used for instruction AND testing. They increase the instructional relevance of the assessment. The testlets provide a better estimate of the students’ knowledge, skills and abilities and provide more reliable and valid indicators of student performance than current status collection of evidence.
Most testlets are designed for direct student interaction with the computer.
Some testlets are designed for administration by the examiner, outside the system, with the test administrator recording responses in the system, rather than the student recording the response.

32. What item types will be used?32
Each testlet will feature a variety of different types of questions.
Each testlet will use a variety of different types of questions. The next few slides will discuss the specific types of items that will be used in the testlets.

33. ITEM TYPES USED IN TESTLETS
Multiple choice
Technology enhanced
Matching
Sorting
Labeling
Teacher administered
Engagement Activity
DLM uses technology-enhanced items where they are cognitively appropriate which varies with the area of the map within which the student is learning. For example, if a map node is about ordering things, then an ordering technology-enhanced item type (testlet) will be used. Three item types are currently developed: multiple choice; technology-enhanced item types that support sorting, ordering, and matching tasks; and teacher-administered items.
The next couple of slides are screen shots of different item types used in DLM testlets.

34. Multiple Choice
This is an example of what a student would see on a multiple choice item.

35. Matching
This is an example of what a student would see on a matching item.

36. Sorting
This is an example of a drag and drop sorting item.

37. Educator Administered Assessment
For students who are working in the initial precursor area of the map, their educator will get a testlet from the computer and administer it outside of the system and then enter the results into the computer.

38. What do I need to know and do right now?
Now what?
What do I need to know and do right now?
Now that we have reviewed the basic DLM vocabulary, the remainder of this session will be used to discuss what educators need to know and do right now to prepare for this year’s MAP-A testing.

39. Educator Portal Login 39
The first thing that all educators must have is access to educator portal. The Data Steward for your district is responsible for uploading all teachers, students and rosters into Educator Portal. Once the educator has access, they must log in. This is what the log in screen looks like.

40. Security Agreement 40
Once logged into Educator Portal, all educators must complete the Security Agreement. This will pop up the first time you log in. To complete the agreement, you simply need to read the document, click the bubble that states, “I have read the security agreemtn and agree to follow the standards” and then type your full name and click save.

41. Required Test Administration Training41
Before an educator is able to access a student’s test ticket, they must first complete the required test administration training. Once logged into Educator Portal, click on the “Professional Development” tab at the top and then “Modules.”

42. Missouri’s Guide to Required Test Administration Training 2014-2015
On the DESE website, on the Assessment page, there is a link to the “Missouri’s Guide to Required Test Administration Training” that can walk you through exactly how to sign up for and complete each of the modules.

43. FIRST CONTACT SURVEY
A comprehensive questionnaire completed by the student’s teacher based on the teacher’s knowledge of the student’s performance, skills, and knowledge
Used in conjunction with the Personal Needs and Preferences Profile to determine where a student initially enters the KITE Student Portal assessment system
Completed by the student’s teacher in the KITE Educator Portal
Completed once a year and edited or adjusted as needed
Once the required test administration training is complete, educators must complete a first contact survey for each student. The First Contact Survey is a comprehensive questionnaire completed by the student’s teacher based on the teacher’s knowledge of the student’s performance, skills and knowledge. It is used in conjunction with the Personal Needs and Preferences Profile to determine where a student initially enters the KITE Student Portal assessment system. The First Contact Survey is completed by the student’s teacher in Educator Portal and is completed once a year, though it can be edited or adjusted as needed throughout the year.

44. First Contact Survey GO TO EDUCATOR PORTAL
Complete the First Contact Survey for each student
After you have received your user name, password and verified that all student information is accurate, educators should complete the first contact survey for each student.
Educators do have the ability to choose the columns that are displayed and rearrange them in whatever order you prefer to look at them.

45. Personal Needs and Preferences
Student-specific information that tells the DLM test delivery system what the needs are for individual users.
The PNP includes information the system needs to make the student’s user interface compatible with his or her accessibility needs. In DLM™, the PNP profile includes information about display enhancements, language and Braille, and audio and environment supports.
Educators who know the student provide the information in the profile.
Accessibility is also supported through the Personal Needs and Preferences (PNP) profile. Each student’s IEP team decides which tools and supports should be provided. The teacher records these choices in the system and the student’s assessment experience is customized based on those tools and supports.
The practice activities have many of the accessibility features built in. Use the demo practice activities to help determine which features to choose for your students.
The PNP profile is used to select the appropriate accessibility features and supports within the system, and thus to tailor each student’s experience based on individual needs. It can be completed any time before testing begins. It can also be changed as a student’s needs change. Once updated, the changes appear the next time the student is logged in to the KITE system, which is the platform used to administer the DLM alternate assessment.
It identifies the accessibility options within the KITE system and allows the teacher to set the "preferences" for the student based on the First Contact Survey

46. PERSONAL NEEDS AND PREFERENCES
Customization for each student.
Once the First Contact Survey has been completed, the educator should complete the personal needs and preferences for each student.

47. Released Practice Testlets
These are the many different PNP features to explore. The teacher should use the demo.sue accounts to get a preview of the various PNP features available.

48. Practice Test Have the students practice taking the test
Before having students complete the instructionally embedded assessments or the spring window, educators should have the student take a practice test to become familiar with the system.

49. Planning
Plan and provide instruction using the essential elements blueprints
Do not forget the manipulatives!
On November 10th the Instructional Interface Tool will be available. The educator selects the EEs for each student from a list of available grade level EEs. Before beginning instructionally embedded assessments, the educator selects the appropriate level within each EE after reviewing the recommendations from the system. The system uses the student’s assessment results to help refine the system’s recommendations.
Then the educator instructs on that content before administering the assessment.
In the spring assessment window, the student is re-assessed on some of the EEs previously taught during the year to update and validate the student’s results. The system automatically selects five of the EEs and the levels for each student based on the student’s assessment history during the year.
An engagement activity that precedes a testlet describes a scenario, taps prior knowledge or experience, and/or introduces the concept to be addressed. In English language arts, the text being read often serves as the engagement activity. In math, the engagement activity provides context for the items.

50. What are Blueprints?
The Missouri Blueprints contain EEs that will be used for testing.
The Missouri blueprints contain Essential Elements that will be used for testing. These can be found on the DESE Assessment webpage.

51. The blueprint is a general description of the content covered for each grade. The specific options and minimum expectations for each student’s assessment are provided with each table. The specific EEs available in each grade are listed in tables provided on the blueprint. EEs are organized according to conceptual areas.
Educators do not need to select more than the minimum number of EEs specified for each conceptual area on the blueprint for grade level.
The Blueprints for ELA and math can be found on the DESE Assessment website.

52. Now You’re Ready!
Now the student is ready for the instructionally embedded operational assessment (Phase A,B,C)
Once all of this is completed, you are ready to begin work on the instructionally embedded operational assessment.

53. ASSESSMENTS
Beginning of Year
Teacher chooses Essential Elements for each student
During the Year
Teaching and instructionally embedded assessments
Spring Testing Window
Re-assess Essential Elements
Each time a student takes an instructionally embedded assessment, the system uses the results to update mastery of all the linkage levels for that EE. Scores used for summative purposes are based on the student’s final updated information for all Ees and linkage levels by the end of the year.

54. 2014-2015 DLM Assessment Calendar
NEW MAP –A Assessment Calendar
October 13-31
Field Testing
November 10 – December 19
Operational Testing
January 5 - March 13
March 30-May 22
ELA and math
During the school year, field testing and operational testing will happen simultaneously. Phase A will be strictly field testing and will take place October Phase B will be both Field Testing and Operational Testing and will take place November 10-December 19. Phase C will also be both Field Testing and Operational Testing and will take place January 5-March 13. Finally, the Spring Window will take place March 30-May 22 and will be the summative year end assessment.

55. School Year 2014-2015 MAP-A for ELA & Mathematics
Instructionally relevant testlets begin August 2014
Used for instruction and operational data
One or two per week in ELA and mathematics
Based on new Essential Elements (replace current APIs)
2015 spring window new MAP-A in ELA and mathematics
Used for federal and state accountability
ELA and mathematics grades 3-8 and grade 11
2015 spring window
Instructionally relevant testlets began in August, They can be used for instruction and operational data. One or two testlets should be used in both ELA and mathematics. The testlets are based on the new Essential Elements and replace the old APIs. The 2015 Spring Window will be for the new MAP-A in ELA and mathematics. It will be used for federal and state accountability. ELA and mathematics will be tested in grades 3-8 and 11.

56. What about Science?
The new MAP-A Science Assessment pilot testing will begin this spring
Until further notice:
Continue to test Science with the current status collection of evidence in:
Grades 5, 8 and 11
January and February collection windows
Continue to use current Science APIs.
The new MAP-A Science Assessment will begin pilot testing this spring. However, until further notice, please continue testing science with the current status collection of evidence in grades 5, 8 and 11 in the January and February collection windows. Continue using the current science APIs and ProFile.

57. Professional Development Resources
Available consortium and DESE professional development resources:
Glossary of terms
Essential Elements
DESE weekly DLM updates
DLM website
DLM Frequently Asked Questions pdf
Self directed learning modules
There are many professional development resources available to educators. The DESE Assessment webpage has many, many resources for educators. There are also 30 self-directed learning modules available on the Dynamic Learning Maps website.

58. The Next Generation of Alternate Assessments
ALL students are being prepared for life, work, and citizenship, including students with the most significant cognitive disability.
Preparation for graduation begins the day the student enters school.
Dynamic Learning Maps consortium is creating the next generation of alternate assessments as part of that preparation.
We have a bigger focus on what students CAN do rather than what students cannot do.
By using the next generation of alternate assessments, all students are being prepared for life, work and citizenship, including students with the most significant cognitive disabilities. Additionally, all students are beginning preparation for graduation the day the student enters school. The Dynamic Learning Maps consortium is creating the next generation of alternate assessments as part of that preparation. This assessment has a bigger focus on what students CAN do rather than what students cannot do.

59. THANK YOU
Questions?