1. Simple Sequencing in Training Delivery and Management Neil Cramer, Manager Instructional Systems Northwest Airlines neil.cramer@nwa.com

2. Simple Sequencing  A method for representing the intended behavior of an authored learning experience  Consistent sequencing of learning activities  Based on rules and conditions tied to activities  Controls the launching of activities and access to learning resources  The manifestation of an instructional strategy

3. SCORM 1.3 (IMS) Simple Sequencing What it isn't:  Simple  AI based sequencing  Schedule based sequencing  Sequencing requiring data from closed systems and services (e.g. embedded simulations)  Collaborative learning  Customized learning

4. Terminology  Learning Activity – a meaningful unit of instruction (potentially with sub-activities or access to learning resources in a defined hierarchy)  Have a discreet start and end  Have well defined completion and mastery conditions  Sub-activities can be nested to any depth  May or may not have associated SCOs or SCAs  May be associated with Parent or Child activities a la SCORM Content Aggregation Model

5. Terminology  Tracking Activity – association of tracking status data with a Learning Activity impacted by learner interactions and impacting learning sequences  No specification on data persistence beyond current login session  Active/Passive Content – active content sets values of elements in data tracking model, passive content assumes default values

6. Terminology  Activity Tree – represents the organization of all elements, hierarchies, and relationships between components relative to Content Aggregation  Derived from the instructional design parameters  Implemented and controlled by the LMS

7. Terminology  Cluster – an activity that consists of child activities. Includes the parent activity and its first level children but not the descendants of the child activities.  Combined to form the Activity Tree  The building blocks of content aggregation and sequencing

8. Activity Tree and Clusters Course Lesson 3Lesson 2Lesson 1 Module 1 Module 2 Module 1 Module 2 Module 3 Module 1 Module 2 Topic 1 Topic 2 Cluster

9. Terminology  Activity Duration Tracking – the temporal model for the learning experience  Attempt – effort to complete a learning activity. Applies across all components of the parent/child hierarchy up through the root. Status parameters are changed to determine subsequent attempts versus resumption.  Suspension/Resumption – learner remains within previous attempt rather than tracking new attempt. More than one activity can be in suspension at one time.  Sequencing Session – period from start of sequencing process to completion for given activity.

10. SCORM Sequencing Definition Model Elements that may be used to define intended sequencing behavior:  Sequencing Control Modes  Sequencing Rules  Limit Conditions  Auxiliary Resource  Rollup Rules  Objectives  Objective Map  Rollup Controls  Selection Controls  Randomization Controls  Delivery Controls

11. Sequencing Conditions and Actions/Behaviors Conditions SatisfiedActivity Progress Known Objective Status Known Attempted Objective Measure Known Attempt Limit Exceeded Objective Measure Greater Than Time Limit Exceeded Objective Measure Less Than Outside Available Time Range Completed Precondition Actions Skip Disabled Hidden From Choice Stop Forward Traversal Postcondition Actions Exit Parent Exit All Retry Retry All Continue Previous Exit Actions Exit Then IF (not)

12. Sequencing in IBM/Lotus LMS 1.0  Two controls provide for management and launching of activities:  Choice – free movement through enabled activities identified as clickable links  Flow – controls the way in which a student proceeds (or recedes) through learning activities

13. Typical Sequencing Strategies  Choice – All  All learning activities can be launched at any time  Student controls what they launch and in what order  Good for overview and general information  Does not include prerequisites or assessments (at least ones that drive sequence)

14. Typical Sequencing Strategies  Flow – All  Activities can not be launched from the course list  Activities have "Continue" and/or "Previous" buttons on screen  Flow is controlled by course hierarchy  Good for highly structured course controlled by design

15. Typical Sequencing Strategies  Choice and Flow – All  Allows for free movement but encourages a sequence  Student will have clickable link in course list but also see "Continue" and "Previous" buttons on screen  Good for course that has both new and returning students

16. Typical Sequencing Strategies  Choice, then Flow  Top level activities are marked as choice, child activities are marked as flow  Student can choose any activity but must work through sub-activities in prescribed order  Typically high level activities are self contained with no prerequisites  Activity always starts at the beginning even on subsequent retries

17. Objectives Objectives link performance with sequencing  Objective Status – the objective's defined goal has been achieved through launch or completion  Score (Normalized or Objective Measure) – value from 0 to 100 (or 0.00000 to 1.00000) as a percentage of correct answers on an assessment

18. Information Passing  Local Objective  Associated with a single course item  Score and status affect that one item  One local objective is tagged as primary and contains roll up information for LMS  Global Objective  Passes information between local objectives  The only type that can be accessed outside the scope of a single course item  Must have unique ID throughout entire LMS

19. Basic Architecture Course item Primary Objective Sequencing Rule Topic Parameters Choice = False Flow = True Objective SatisfiedObjective not Satisfied Course item "Stop Forward Traversal" Objective Satisfied by Measure Minimum Normalized Measure =.80...

20. Multiple Objectives Example  Pre-test assess two types of knowledge –basic and advanced  Must repeat assessment until basic knowledge is mastered  If basic knowledge is mastered but advanced knowledge is not student receives remediation  When both basic and advanced knowledge are mastered student moves to next topic

21. Multiple Objectives Example Course Topic 1 Topic 2 Content Assessment Basic Question(s) Advanced Question(s) Quiz Score Choice = False Flow = True Choice = False Flow = True "Basic" "Advanced" By Objective Is Master Exam = True Report Objectives = True Objectives #1 Basic – Objective satisfied by measure =.80 #2 Advanced – Objective satisfied by measure =.80 Sequencing Rules #1 If objective "basic" is not satisfied then retry #2 If objective "advanced" is not satisfied then previous

22. Global Objective Example  Pre-test determines path  If pre-test is passed, student proceeds to advanced content  If pre-test is failed, student proceeds to introductory content  Global objective used to pass data from pre- test to the introductory content

23. Global Objective Example Course Topic 1 Advanced Pre-test Question(s) Quiz Score Choice = False Flow = True Global Objective – data container Primary Local Objective - Objective satisfied by measure =.80 By Objective Is Master Exam = True Sequencing Rule (Pre-condition Rule) Skip if Mastery >.80 Intro Primary Local Objective - Objective satisfied by measure =.80 Export Map Import Map

24. Sample SCORM Sequencing Models  No Sequencing  Student can choose learning activities in any order  When all components of a cluster are complete the cluster is complete  If all questions return "passed" the whole course is passed  Choice = ALL

25. Sample SCORM Sequencing Models  Linear  Guided flow  Student progresses through course in predetermined order  Must complete one component before moving to the next  Cluster is complete when all components are complete  No intermediary assessments, only final assessment  Flow = ALL

26. Sample SCORM Sequencing Models  Learner Controlled  Must complete introduction, then presented with menu of modules  Each module lists its component lessons  Student can select any module, but not a lesson  Each module has assessment  Student may not attempt module more than once  Choice, then Flow

27. Sample SCORM Sequencing Models  Knowledge Paced  Student must complete introduction, then proceed to module 1 assessment  If assessment is passed, move to module 2 assessment  If assessment is not passed, move to module, then assessment again  Etc. for all modules  Flow = ALL w/ sequencing instruction to repeat on failure

28. Sample SCORM Sequencing Models  Remediation  Student completes introduction and follows linear sequence through modules  Student takes comprehensive assessment  On pass course is complete  On fail of any given section of assessment (by objective), student is moved to associated content module  On completion of remediated module student retakes module assessment  Flow = ALL w/ sequencing rules to repeat on specific objective(s) failure

29. Sequencing Controls MeaningOptions ChoiceWhen set to True, indicates that students can choose which course items to complete. True, False Choice ExitWhen set to True, indicates that students can exit an activity that is part of a choice sequence. True, False FlowWhen set to True, indicates that the student has the ability to use Continue or Previous navigation. True, False Forward OnlyWhen set to True, students may not go back to previous course items higher in the course hierarchy. True, False

30. Rollup Controls MeaningOptions Rollup Objective Satisfied When set to True, the status of the objective associated with the course item is included in the rollup calculations. True, False Rollup Objective Measure Weight When greater than one, the objective for rolled-up items are weighted more heavily. (A weight of two would weigh these items twice as heavily as others, for example.) A number of one or greater. Rollup Progress Completion When set to True, the complete and incomplete status is included in the rollup calculations. True, False Rollup Progress Duration When set to True, the amount of time spent launching and working on the course items are included in the rollup calculations. True, False

31. Limit Controls MeaningOptions Attempt ControlWhen set to True, students are bound by the Attempt Limit.True, False Attempt LimitThe number of attempts that are permitted for the course item.A number Attempt Wallclock Duration Control When set to True, students are bound by the Attempt Wallclock Duration Limit. True, False Attempt Wallclock Duration Limit The amount of time students are allowed for one attempt on the course item. A time limit in days, hours, and minutes. Attempt Interaction Duration Control When set to True, students are bound by the Attempt Interaction Duration Limit. True, False Attempt Interaction Duration Limit The amount of time students can spend actively working on a course item in a single session. It does not include time when the course item is open, but not being worked on. A time limit in days, hours, and minutes. Wallclock Duration Control When set to True, students are bound by the Wallclock Duration Limit.True, False

32. Limit Controls MeaningOptions Wallclock Duration LimitThe total amount of time students are allowed to have a course item open, across multiple sessions. A time limit in days, hours, and minutes. Interaction Duration ControlWhen set to True, students are bound by the Interaction Duration Limit. True, False Interaction Duration LimitThe total amount of time students can spend actively working on course items for all attempts they make. A time limit in days, hours, and minutes. Time Limit ControlWhen set to True, students are bound by Begin Time Limit and End Time Limit. True, False Begin Time LimitSpecifies on which day, hour, and minute of the course timeline the course item becomes available. A time limit in days, hours, and minutes. End Time LimitSpecifies on which day, hour, and minute of the course timeline the course item becomes unavailable. A time limit in days, hours, and minutes.

33. Delivery Controls MeaningOptions TrackedWhen set to True, the objective and progress data is recorded and the data contributes to the rollup for the parent course item. True, False Completion Set by Content When set to True, indicates that the content sets the completion status. When set to False, progress data is sent by default tracking rules, rather than waiting for data sent from the content. True, False Objective Set by Content When set to True, indicates that the content sets the status for the primary objective. When set to False, objective status is set by default tracking rules. True, False

34. Rollup Rules MeaningOptions Child SetDetermines how many child items (those contained as children of the course item in the course hierarchy) must meet the Rollup Condition. At Least Count is a minimum number of child items; At Least Percent is a minimum percentage of the child items. All, Any, None, At Least Count, At Least Percent Number FieldThe number that applies to At Least Count or At Least Percent Child Sets. For "At Least Count," this is any number from 1 to 100. For "At Least Percent," this is a number from 0 to 100 and you can specify up to five digits to the right of the decimal place; for example, 57.48529. Rule Conditions Evaluate toDetermines if all conditions for the rule must be met or at least one, in order for this rule to take effect. All True, Any True Specify Condition (operator)The selected condition is either stated as is with {none} or negated with Not. {none}, Not Specify Condition (condition)The course item status for which this rule applies. Satisfied, Objective Status Known, Completed, Activity Progress Known, Attempted, Attempt Limit Exceeded, Time Limit Exceeded, Outside Available Time Range, and Always. Perform ActionThe progress that is marked for the parent item when the condition is met. Complete, Incomplete, Not Satisfied, and Satisfied

35. Sequence Rules MeaningOptions Rule Conditions Evaluate toDetermines if all conditions for the rule must be met or at least one, in order for this rule to take effect. All True, Any True Specify Condition (operator)The selected condition is either stated as is with {none} or negated with Not. {none}, Not Specify Condition (condition)The course item status for which this rule applies. Satisfied, Objective Status Known, Objective Measure Greater Than, Objective Measure Less Than, Completed, Activity Progress Known, Attempted, Attempted Limit Exceeded, Time Limit Exceeded, Outside Available Time Range, and Always. Action GroupDetermines when this rule becomes active.Pre Condition Sequencing Rule, Exit Condition Sequencing Rule, Post Condition Sequencing Rule Perform ActionStates what navigation occurs when the rule is applied. Disabled, Hidden From Choice, Skip, Stop Forward Traversal. Choices depend on the Action Group selected.