1. Graduate School of Education George Mason University
Design Treatment Presentation Fall 2005 T/TAC Online Project VA Assessments December 15, 2005
Matt Humphrey
Aala’a Mashaal
Pamela Sharpe
Tim Smith
Maria Washington
Graduate School of Education
George Mason University

2. Presentation Overview
Part 1
Immersion Overview
Instructional Design Process
Part 2
Background & Introduction
Mission and Goal of Project
Part 3
Analyses
Needs Analysis
Task Analysis
Findings and Results
Part 4
Design Approach
Next Steps

3. Immersion Overview Constructivist Learning Environment
Action Learning
The Learning Process
Teamwork
Portfolios and Reflections
Challenges
Benefits
What is Immersion
Alternate modes of assessment
Reflections
Self eval
Peer eval
Challenges:
Constructivist learning environment
Ill-structured
Hands on
Working for an actual client
Work in teams: different personalities
Project due dates
Benefits
Learn by doing

4. Instructional Design Process
ADDIE MODEL
Design
Analysis
Develop
Implement
Evaluate

5. Integrated Learning Design Framework

6. Instructional Design Process Performance Centered Design Process
Persona/
User
Performance Zone
Content/
Knowledge

7. Background & Introduction
No Child Left Behind Act (NCLB)
Individuals with Disabilities Education Improvement Act of 2004 (IDEA)
VA Assessments section for T/TAC Online

8. T/TAC Online http://ttaconline.com
T/TAC Online is a web-based community linking people and resources to help children and youth with disabilities.
T/TAC Online is available to teachers, parents and professionals who can utilize the services through the convenience of a personal computer. T/TAC Online provides a central location for the community to gather information and share knowledge. Currently,the knowledge areas are categorized as SOL Enhanced, Resources, Events, Online Training, School Improvement, and VA Assessments.
The Virginia Department of Education, sponsor of T/TAC Online ( also supports eight regional T/TACs that
comprise a statewide system emphasizing collaboration in the planning and provision of services to meet state and local staff development needs. The site was designed and developed by George Mason University.

9. Problem Transition to Cognitive Standards Based Assessments
Communication Skills
State Mandated Assessments
New Process for Teachers

10. Analysis Needs Analysis Analysis Performance Analysis Learner Analysis
Design
Develop
Implement
Evaluate
Analysis
Needs Analysis
Performance Analysis
Learner Analysis
Task Analysis

11. Needs Analysis Overview
The process of determining, analyzing, and prioritizing needs and, in turn, identifying and implementing solution strategies to resolve high-priority needs.

12. Needs Analysis Objectives
The objectives of the needs analysis are to:
Determine the best solution
Generate data to support recommendations
Provide specific recommendations
Decide priorities for action

13. Needs Analysis Six Steps
The Six Steps involved in our Needs Analysis:
Determine Current Status
Performance Analysis
Analysis Results
Teacher Expectations
Identify Discrepancies
Determine the Goals
Set Priorities for Action

14. Performance Analysis Overview
Is used to determine what needs to be done in order to accomplish the client’s stated goals and objectives.
Focuses on determining the elements that support the project
Investigates the current situation (actuals) and desired performance (optimals) (Rossett, 1999).

15. Performance Analysis Clients & Information Sources
Dr. Patricia Abrams
Dr. Michael Behrmann
John Eisenberg
Nathan M. Sparks
Mary Wilds
Human:
All clients
Jocelyn DelSignore
Anya Evmenova
Teachers
Content:
T/TAC & VA DOE websites
VAAP Manual

16. Performance Analysis Current Situation/Actuals
Special Education teachers have not been doing assessments for their students in a standardized format
Special Education teachers collaboration outside the classroom is limited
The assessment documents are difficult to locate on the Virginia Department of Education website
The challenge of assessing special education students creates a barrier
Special education teachers of students with severe disabilities have not been required to conduct SOL based assessments
Teachers spend an inordinate amount of time developing portfolios which were not aligned with SOL expectations

17. Performance Analysis Desired Performance/Optimals
Special Education Teachers should assess their students using the different assessment standards including the VAAP, VGLA, and the VSEP.
The documents should be in one location as a “one stop shop” for teachers
The documents should be searchable for easy access and easy downloading
The creativity of the special education teachers should be shared with other teachers by creating and submitting lesson plans
Teachers can submit standardized assessment portfolios with streamlined evidences

18. Needs Analysis Data Collection
Review the manuals (VAAP, VGLA, VSEP)
Attend conferences to observe teachers reactions
Administer Surveys
Visit schools to interact with Special Education Teachers
Informal talk with Jocelyn
Interview with Dr. Behrmann
Informal talk with Anya
Interview with Mary Wilds
Interview with John Eisenberg

19. Needs Analysis Survey Questions
T/TAC ONLINE | VA ASSESSMENTS PROJECT SURVEY
Dear teacher:
We are conducting research in order to help teachers transition to using the new VAAP manual in the most effective and efficient way possible. Please assist by completing this survey. Thank you.
Name:
School name:
Age (Circle One): 20-30, 31-40, 41-50, 51+
Years of Experience in Special Education field:
Type of License:
On a scale of 1-5 (5 being the strongest) rate the following:
I am proficient using the internet.
I have sufficient access to a computer (in the class/at home).
I use the internet to assist me in planning lessons and/or activities.
Surveys conducted 10/25/05 and 11/3/05, 89 SPED teachers

20. Needs Analysis Survey Questions
Regarding the new VAAP (Virginia Alternate Assessment Program)
I agree with the need to assess cognitive as well as life skills for VAAP students.
I currently have a good method to assess my VAAP students.
I spend a lot of time assessing my VAAP students.
I use/have many (5 or more) tools for assessments.
My lesson plans are shaped by the current VAAP assessments.
Assessments
There are many hindrances (time, resources, lack of appropriate information, etc) that affect my assessment process.
I think the new assessments will change the way I prepare lesson plans.
I would be interested in interacting with other special education teachers online who are involved in the same VAAP transition.
I would like to share lesson plans and activities.
I would like to get help aligning the cognitive skills and life skills into the curriculum.
Surveys conducted 10/25/05 and 11/3/05, 89 SPED teachers

21. Needs Analysis Findings/Results – New VAAP
Figure 1
85.6% of the teachers indicated that they agree with the assessments.
67% of teachers surveyed indicated that they spent a lot of time assessing the students.
61% of teachers surveyed agreed that they have five or more tools for assessments
Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

22. Needs Analysis Findings/Results - Assessment
Figure 2
74%
72%
77%
75%
79%
50%
62%
Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

23. Needs Analysis Findings/Result - Age Ranges
Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

24. Needs Analysis Findings/Results - Years of Teaching Experience
Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

25. Needs Analysis Findings/Results - Licensure
Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

26. Needs Analysis Teacher Performance Expectations
For students designated to be assessed under the VAAP teachers must begin aligning lesson plans to new aligned standards of learning (ASOL) guidelines, which include cognitive skills.
Skills will be measured in four categories: Reading, Math, History & Social Science and Science.
Teachers must develop lesson plans that align to SOL guidelines and can be rated 3 or above on the established scoring rubric by locally identified scorers in the areas of 1) student performance, 2) communication context, 3) communication settings, 4) communication socialization, and 5) communication support.

27. Needs Analysis Teacher Performance Expectations
Teachers must be able to demonstrate the following skills and knowledge:
Have knowledge of ASOL guidelines and how to incorporate communication skills
Have access to VAAP Manual
Know how to complete an Individual Education Plan (IEP)
Have access to students IEP (work with IEP team)
Align lesson plans to SOL guidelines for SPED students
Develop lesson plans based on cognitive skills
Align lessons to SOL enhanced
Use computer:
Download blank forms
Internet access
Be able to use word-processing software
Be familiar with state and district-wide assessment
Have knowledge of learner’s skill level, needs, and characteristics
Have knowledge/access to proven- research based instruction techniques
Be able to develop lesson plans that fit the student’s current stage of learning for a particular skill.

28. Needs Analysis Discrepancies
Teachers do not have experience in aligning their lesson plans to the new standards.
Some of the teachers do not have experience assessing using the former VAAP standards.
The greatest discrepancy lies in the issue that this process is completely new to all teachers in the state of Virginia.
Teachers have limited resources to help them with the assessment process.
Teachers have limited time outside the classroom to dedicate to this process

29. Needs Analysis Goals
1. Provide scaffolding for teachers to develop lesson plans aligned with the SOLs.
2. Produce a searchable VAAP manual
3. Provide additional resources for teachers to utilize throughout the assessment process

30. Needs Analysis Priorities for Action
Analyze lesson plan process
Based on the needs, determine potential solution
Create prototype of the solution system

31. Task Analysis Overview
A task analysis is a process of analyzing and articulating the kind of learning that you expect the learners to know how to perform (Jonassen, Tessmer, & Hannum, 1999).
A thorough task analysis is essentially a blueprint of the instructional sequence and it assures the instructional designer that there are no gaps in the instruction.

32. Task Analysis Information Processing Analysis
The task analysis method selected was information-processing analysis (IPA). This type of analysis is used most often for procedural and cognitive tasks.
According to Smith and Ragan (1999), conducting an information-processing analysis is the first step in “decomposing” or breaking down a goal into its constituent parts, identifying what the students need to learn to attain the goal.

33. Task Analysis Process
A task analysis was conducted on the lesson plan to compile all components in an organized manner. The following task analysis is based on extant data research, observed teacher training of VAAP instruction process, as well as an expert interview with T/TAC Online Asssistant Director at GMU.
When a teacher completes a lesson plan that aligns with the task analysis all necessary components will be included to target the ASOL, CS, and IEP goals.
The completed task analysis will support the design and development of the solution system.

34. Task Analysis Challenges
Is this the only lesson plan process?
Problem types for each task
Problem solving, rule using, etc
Appropriate supports for each task
What does the final result of each task look like?

35. Task Analysis Lesson Plan Document

36. Task Analysis Lesson Plan Process
Prerequisite Tasks
Review present level of performance, assessment data, and IEP team recommendations
Review student profile document
Identify communication skills
Identify student’s entry points
Plot entry-level points and communication skills on the student and classroom matrix

37. Task Analysis Lesson Plan Process
Identify a main theme or event of the lesson plan
Rule Using
Record the date of creation of the lesson plan
Identify the student tasks and activities
Main tasks, sub tasks, prerequisites, instructional strategies
Choose an ASOL subject matter
Organizing topic, standard used, support in activity context
Problem Solving

38. Task Analysis Lesson Plan Process
5. Choose a communication skill that can be incorporated into the lesson
Can be used multiple times
Problem Solving
6. Identify the materials needed
Rule Using
7. Identify and list assistive technology needs
Ensure that all students have appropriate assistive technology throughout the lesson
8. Identify evidence collections materials

39. Task Analysis Further Analysis
Field test the lesson plan process
Check for completeness
Task analyze observed processes to determine adequate supports for each task
Task Analysis used to support design and development of the solution system

40. Needs Analysis Performance Analysis
Analysis  Design
Analysis
Develop
Implement
Evaluate
Design
Flowcharts
Schemas
Prototype
Needs Analysis Performance Analysis
Learner Analysis
Task Analysis

41. a. Flowchart 1: Searching the VAAP Manual

42. Database Schema 1: Searching the VAAP Manual

43. Flowchart 2: Lesson Plan Document

44. Design Approach Users
Using the data collected from the surveys and interviews with SME’s, teachers and administrators we found that our users were:
Novice Special Education Teacher
Intermediate Special Education Teacher
Expert Special Education Teacher

45. Design Approach EPSS
EPSS was selected as the solution system for the users
Rationale:
Teacher are familiar with the lesson plan process but not with aligning lesson plans with standards (supporting an existing process)
Teachers are very dispersed, a web based solution can reach more teachers.
Teachers time is limited, an EPSS provides just in time support for their needs.

46. Design Approach Characteristics of an EPSS
Key Characteristics of EPSS
Deborah Alpert Sleight (1993) states that an EPSS should contain these key characteristics:
Computer-based
Provide access to the discrete, specific information needed to perform a task at the time the task is to be performed
Used on the job, or in simulations or other practice of the job
Controlled by the user
Reduce the need for prior training in order to accomplish the task

47. Design Approach Attributes of an EPSS
In her classic article “Attributes and Behaviors of Performance Centered Systems”, Gloria Gery delineated numerous elements of effective performance support systems including:
Establishes and maintain a work context
Aids in establishing the goal
Structures the work process
Reflects the natural flow of work
Provides alternative views data, information, and knowledge
Provides contextual feedback
Provides support resources without breaking the task context
Embeds knowledge into the interface
Provides access to underlying logic
Automates tasks

48. Design Approach Prototype

49. Next Steps Develop Analysis Design Implement Evaluate Prototype
Formative Evaluation

50. Next Steps Identify Supports to be integrated into the EPSS
Continue Development of Prototype
Implement Prototype
Usability Testing
Formative Evaluation

51. Reference
Gery, G. (1991). Electronic Performance Support Systems. Tolland, MA: Gery Associates.
Gery, G. (1995). Attributes and Behaviors of Performance-Centered Systems. Performance Improvement Quarterly, 8(1),
Smith, Patricia L. & Ragan, Tillman J. (2005). Instructional Design. Hoboken, NJ: John Wiley & Sons, Inc.

52. Thank You for Listening
Any Questions?