1. T/TAC I Final Presentation George Mason University Graduate School of Education Immersion Program May 11, 2004 T/TAC I & II

2. T/TAC I ONLINE TRAINING SCHOOL IMPROVEMENT T/TAC I & II

3. T/TAC I SITE LAUNCH: Special Education Council APRIL 20, 2004 T/TAC I & II

4. T/TAC I T/TAC I - Online Training T/TAC II - School Improvement T/TAC I & II - Overview of T/TAC Online Site T/TAC I & II

5. T/TAC I SEMESTER II

6. Tasks to Address from Last Semester Functionality –www.Workshops Documentation –Workflow Process –Quick Start Guide –E-Learning Content Guidelines –Accessibility Guidelines –Online Tutorial

7. T/TAC I Goal To improve the overall site usability and usefulness by providing best practices along with support mechanisms and guidance in delivering good meaningful online instruction.

8. Project Plan System Usability Content Development

9. System Usability Determine if T/TAC staff can easily upload their training content to the T/TAC Online site and uncover any potential difficulties with the upload process. Determine the best practices for the inclusion of multi-media by establishing which content formats will best meet the needs of the staff and the site's audience. Assess the effectiveness of a pre-production workflow process designed to assist the T/TAC staff in the preparation of their content for uploading

10. Content Development T/TAC staff will feel confident about their ability to develop quality online content in any of a variety of formats Content posted to T/TAC Online Training site will be developed based on best practices for online learning Identify practices used to develop and delivery good online training.

11. Formative Evaluation Using formative evaluation to achieve the team goal by seeking all information necessary to help improve the system and its content.

12. Formative Evaluation Types of evaluations conducted were: –One-to-One –Expert Reviews –Usability Testing

13. T/TAC I USABILITY EVALUATION

14. Usability evaluation Experts’ perspective –Found several issues –Changed “Courses” to “www. workshops” Users’ perspective –Usability tests

15. Usability Test A structured interview focused on specific features in a user interface.

16. Usability Testing Goals Conduct a Kirkpatrick Level 3 (performance) evaluation on the site’s Online Training section Determine how to improve the site Gather data for constructing tutorials Validate tutorials

17. Usability Testing Procedure Select representative users T/TAC staff at GMU Three without the tutorial Two with the tutorial Select representative tasks View, create, edit, publish for webshops & workshops. Develop a testing script to ensure consistent testing Observe users performing the tasks

18. Gathering Usability Results Users were asked to think out loud during the testing Observers noted thoughts & users’ errors while performing tasks Observers noted areas of user confusion that could be addressed in tutorials Observers noted areas of the user interface that could be improved with some redesign Observers tracked the time of completing the tasks

19. Usability Testing Results Found several design issues that when addressed improved user performance Users completed tasks faster with fewer errors after completing the tutorials than without the tutorials Users’ ratings of ease of use & time required to perform tasks were higher for those who had used the tutorials first

20. T/TAC I TUTORIALS

21. Staff Education Earlier Performance Analysis indicated staff desire for tutorials Usability testing confirmed the need for tutorials Two aspects of staff education were considered –System training –Content creation and conversion

22. Training Deliverables The evaluation plan that was then developed called for the following training deliverables: –Quick Start Guide –Preproduction Workflow – Creating Online Workshops that Work: Introduction –Getting Started: A Framework for Designing Online Training –Removing the Teacher without Losing the Student –Implementing Multimedia Elements

23. System Tutorials We started with two tutorials –How to build a Webshop –How to build a www.Workshop Our goal: –Measure the effect of having a tutorial as opposed to not having a tutorial as part of our formative evaluation process. –Collect feedback on the tutorial itself The tutorial was only given to half of the usability test subjects.

24. Creating The Tutorials We used feedback from the initial usability tests as guidelines to create the initial tutorials. We concentrated on making the instruction clearly understandable

25. Creating The Tutorials Shown here are some of the visual devices we used Easy to see and locate step numbers Arrows pointing to buttons mentioned in the text

26. Creating The Tutorials Another visual device we used was subtle shading and a cutout window to help focus the user on the area discussed in the tutorial.

27. The Feedback We gathered a number of good suggestions. The feedback included but was not limited to items like: Breaking original tutorials in smaller sections and combining the sections into a Workshop Modifications to screen language and instructions for clarity Additional Visuals for clarity Suggestions for sequencing various parts of the tutorial Items to be defined for the end user

28. Preproduction Workflow We have also prepared a preproduction workflow checklist for content authors It will be designed as a job aid for planning and construction of Webshops and Workshops It presents ideas and things to think about in the areas of: –Project planning –Visual asset preparation and organization –Narrative creation –Copyright issues

29. Quick Guide A quick guide can now be accessed when a Webshop or a www.Workshop The Quick guide provides basic, textual instructions for creating content

30. T/TAC I CREATING ONLINE WORKSHOPS THAT WORK

31. Development Process Phase One: Analysis of Existing Workshop Materials Recommendations for Training Focus Phase Two: Incorporate Findings into Webshops and www.Workshops

32. Phase One – Expert Review 1. Identify Content Types 2. Identify Goal Types 3. Associate Goals with Techniques 4. Identify & Apply e-learning Principles

33. 1. Identify Content Types Fact Specific and unique data or instance YES Concept Category with multiple examples YES Process Flow of events or activities YES Procedure Task with step-by-step actions NO Principle Task performed by adapting guidelines YES

34. 2. Identify Workshop Goals Inform Communicate information YES Perform – Procedure Build procedural skills NO Perform – Principles Build principle-based skills YES

35. 3. Link Teaching Techniques with Goals Receptive – Inform Goals Lots of information Limited practice opportunities YES Directive – Perform Procedure Goals Require frequent responses Provide immediate feedback NO Guided Discovery – Perform Principle Goals Job-realistic problems Supporting resources YES

36. 4. E-Learning Principles Multimedia Contiguous Coherent Multiple Modes Personalization Practice Examples

37. Phase One: Summary of Results Good Presentation Techniques Well-organized Capture attention Effective use of case studies and examples Training Focus on Web Environment Ground in theory Personalization Multi-media

38. T/TAC I PHASE TWO: Incorporate Findings into Webshops and www.Workshops

39. Phase Two Create www.Workshop Conduct One-to-one Evaluation Conduct Level One Evaluation

40. Create www.Workshop Creating Online Workshops that Work Webshops: Getting Started: A Framework for Designing Online Training Removing the Teacher without Losing the Student Incorporating Multi-media Elements

41. Getting Started: A Framework for Design This Webshop provides a structure you can use when developing online training. It discusses what is the same and what is different and how your previous experience will help you.

42. Removing the Teacher without Losing the Student This Webshop introduces six techniques you can use to help keep students involved in a dialogue with the training when you are not there.

43. Incorporating Multimedia Elements Effectively This Webshop discusses how you can use multimedia elements to reinforce information transfer based on the theory of human cognition

44. Conduct One-to-one Evaluation One-to-one Evaluation –Teachers review raw content –Logs used for recording: page being reviewed, comments or suggestions and changes incorporated by the designer of Webshop –Revisions are made and content is transferred to T/TAC Online in the form of Webshops

45. Conduct Level One Evaluation Level One Evaluation –Purpose: To collect reactionary data –Surveys emailed to T/TAC staff from all regions –Feedback: T/TAC staff were pleased with the content of the www.Workshop T/TAC staff members feel more confident in their ability to create Online training

46. T/TAC I RECOMMENDATIONS

47. Tracking Site Issues Web-based quality assurance system for tracking user experience issues Issues filtered by page, category, date opened, status Issues sorted by severity, page, category Screen shots included to assist with communicating the issues

48. Types of Quality Assurance Issues Accessibility – Violations of Section 508 or WC3 WCAG Anomaly - Unexpected behavior or bug Data Entry – Using text fields, check boxes, radio buttons, etc. Information Architecture – Structure of information & labeling Management – Issues for the site manager, not programmer Navigation - Intuitive navigation menus and buttons Printing – Acquiring print outs in the most useful form Search - Easy options for filtering and sorting data User Guidance - Clarity of instructions, messages & online help Visual Design - Professional appearance and layout

49. Future Recommendations Determine where the Webshops and the www.Workshops with information for T/TAC staff will be located on the system. Determine how staff will access Webshops and www.Workshops with staff-specific information. Develop additional tutorial webshops on using T/TAC Online as users indicate their needs Review and address the issues in the Quality Assurance Tracking System

50. T/TAC I SITE LAUNCH: Special Education Council APRIL 20, 2004 T/TAC I & II

51. Site Launch Activities Audience Analysis Scope, Sequence, Strategy Review Site Consistency Orientation Video Survey Brochure New Tab – SOL Lessons

52. T/TAC I VA DEPTMENT OF EDUCATION SPECIAL EDUCATION COUNCIL PRESENTATION T/TAC I & II

53. Introduction to T/TAC Online A community linking people and resources to help children and youth with disabilities An online extension of T/TAC Services –Usage tracked the same way Categories and subcategories Same forms to collect data Track by region and category

54. Accessing the Site www.ttaconline.org Map of Virginia –Region –Drop-down county lists –Out-of-state No login required to visit most of the site - only training is tracked by user

55. T/TAC I Q&A T/TAC I & II

56. T/TAC I Thank You! T/TAC I & II