1. FREMA: e-Learning Framework Reference Model for Assessment
David Millard
Yvonne Howard
Learning Technology Group
University of Southampton, UK

2. Structure Overview Workshop: Eliciting Common Usage Patterns
Introduction to FREMA
What is a Reference Model?
Views on the Model
Do we have the right Personas – are you there?
Do our Concept Maps make sense to you?
Workshop: Eliciting Common Usage Patterns
What are the most common tasks within assessment?
What is (and is not) being addressed by current software?
What do you consider to be the core elements of the assessment domain?

3. Introduction What is FREMA? What is the ELF?
JISC funded Project between Southampton, Strathclyde and Hull
Part of the e-Learning Framework (ELF) effort
What is the ELF?
Service Oriented Architecture for e-learning systems
Layered Web Services (Domain services over Common Services)
Dynamic and evolving
FREMA will develop a Reference Model for the Assessment domain
Supporting design-time activities
Supporting run-time activities
Virtual organisations and lifelong learning
What is a Reference Model?
A description of how services behave within a particular domain
A community resource

4. Assessment and ELF Not enough to describe and define these services
Need a proper audit trail of decision making
Start by defining the domain
Work up through the services to a reference implementation
This is an ELF Reference Model

5. Anatomy of a Reference Model
Domain Definition
Overview of the domain, and how projects and standards fit within it
Identifying Common Usage Patterns
Scoping the FREMA Project
Developing Use Cases
Formal descriptions of usage patterns
Gap Analysis
Mapping of Use Cases to the Services in ELF
Reference
Impl’
Service Profiles
Formal descriptions of those services
Service Profiles
Gap Analysis
Reference Implementation
Of key/core services
Examples
Validation
Resource
Use Cases
Common Usage Patterns
Assessment Domain Definition

6. Assessment Domain Definition
What does it look like?
An evolving, cross-referenced, searchable web site
Indexed resources and narrative descriptions of the domain
UML Use Cases and Scenario documents
Service descriptions, narrative and WSDL
Service implementations to download (Java/.NET)
Different gateways into the model
according to how you want to use it
Reference
Impl’
Service Profiles
Gap Analysis
Use Cases
Common Usage Patterns
Assessment Domain Definition

7. Assessment Domain Definition
How might you use it?
Use the Reference Implementation
Build on some or all of the developed services
Use the Service Profiles
To develop your own services that will fit into the framework
Use the Use Cases
To help understand usage patterns within the domain
Develop new Service Profiles and thus Services
Use the Domain Definition
To develop a context for your own work
Understand how existing work fits together
Identify standards
Locate experts
Reference
Impl’
Service Profiles
Gap Analysis
Use Cases
Common Usage Patterns
Assessment Domain Definition

8. Use Case for a Service Oriented Reference Model (SORM)

9. A More Concrete Description
An ontology for the resources of the domain
A knowledge base of the resources
Concept Maps that define the domain and aid searching
Processes (think of verbs)
Classes of Objects (things, think of nouns)
And many more gateways into resources
Visualisations of
Concepts
Populations of resources over concepts
Searchable, flexible, dynamic website

10. Process concept map (verbs)

11. Classes concept map (nouns)

12. Another look at the SORM Use Case
How would these actors choose to view the domain?

13. Personifying the actors
Will, Web services developer
Scenario:
‘I want to lookup use cases and scenarios to help me design my application. This will help me to define my footprint in the assessment domain. I see there are some web services I could download but some are missing. What standards can I use when writing my own web services to ensure that I can interoperate with the web services I’ve chosen?’
Yvonne, Institutional Resource Manager
Scenario
‘I want an overview of what this domain is all about. I want to know what standards are applicable in the domain to ensure that we comply with quality assurance requirements. I want to examine use cases and scenarios to understand the available footprints. I also want to know who the key players are and what the key projects are.’
Other Actors
Early Adopter
Toolkit developer
Course developer
And there are others

14. Concept modelling
What mental models might these and other actors use to orient their searches?
How do they visualise the assessment domain?

15. Structure Overview Workshop: Eliciting Common Usage Patterns
Introduction to FREMA
What is a Reference Model?
Discussion
Do we have the right Personas?
Do our Concept Maps make sense?
Workshop: Eliciting Common Usage Patterns
What are the most common tasks within assessment?
What is (and is not) being addressed by current software?
What do you consider to be the core elements of the assessment domain?

16. Another concept map – based on a domain expert view
Items
Validation
Final Grade
Structuring
Assessment
Marking
Grade book
Multiplicity
Self assess
Mark
Learning
Peer Group
Student Feedback
Evidence
Tracking
Artefacts
Course
Group
Member
Transcript
Schedule
Delivery
Plagiarism
Authorise
Authenticate
Student Appeal
Help
Authoring QA
Workflow
Failover