Enhancing Nominal Group Technique using technologies: a demonstration (ID275) ALT-C September 2012 Jaye McIsaac, Educational Development Tunde Varga-Atkins, eLearning Unit University of Liverpool ELESIG-funded project

OUTLINE Introduction and purpose What is the Nominal Group Technique? What is the problem? Technologies we tried

OUTCOMES To describe the Nominal Group Technique (NGT) as a student feedback & evaluation tool. To evaluate the use of learning technologies in the process of a face- to-face NGT session. To consider the value of using technologies in online settings using NGT.

WHAT IS NOMINAL GROUP TECHNIQUE? Structured group activity One given topic Facilitates group decision-making Immediate results Reduces researcher/participant bias - Delbecq & Van de Ven (1971 )

NGT: STAGES 1) Individual responses 2) Clarification and consolidation 3) Ranking responses Example question: ‘What would you change in your course?’

STAGE 1: WHAT TO KEEP DOING (FEEDBACK TO STUDENTS) Annotated,personalised feedback. Model answers Giving model answers for essay titles Audio feedback for everythin g! Audio feedbac k giving extensive commen ts Drafts that must be submitted and marked prior final work

STAGE 2 : WHAT TO KEEP DOING (FEEDBACK TO STUDENTS) Model answers Giving model answers for essay titles Audio feedback for everythin g! Audio feedbac k giving extensive commen ts Drafts that must be submitted and marked prior final work Annotated,personalised feedback. 6 1 Audio feedback Model answers Personalised feedback

STAGE 3 : WHAT TO KEEP DOING (FEEDBACK) Item no.Item description 5 points1Annotated, personalised feedback 4 points2Giving model answers 3 points4Audio feedback 2 points7etc. 1 point10etc.

THE TOP FIVE QUALITIES OF GOOD FEEDBACK Timing, quick Detailed, on how to improve Applicable to future work PersonalConsistent bw markers

‘PROBLEMS’ Credit by Nathan T. BakerNathan T. Baker

1) Individual responses 2) Clarification and consolidation 3) Ranking responses TECHNOLOGIES WE TESTED

Web- conferencing online sticky- note: linoit.com Polling: GoogleModerator clickers mobile text-entry / MCQ polling

Don’t forget: You can copy-paste this slide into other presentations, and move or resize the poll.

MOBILE TEXT ENTRY (E.G. POLLEVERYWHERE.COM, TEXTWALL.CO.UK) 1) Individual response 2) Consolidation and clarification 3) Ranking 

CLICKERS 1) Individual response 2) Consolidation and clarification 3) Ranking 

GUIDE TO NGT & PROJECT REPORT See References On slideshare.net, search for Nominal Group Technique

GOOGLE MODERATOR Live link:

GOOGLE MODERATOR 1) Individual response 2) Consolidation and clarification 3) Ranking 

WEB-CONFERENCING (ELLUMINATE)

WEB-CONFERENCING 1) Individual response 2) Consolidation and clarification 3) Ranking 

ONLINE STICKY-NOTES: E.G. LINOIT.COM

LINOIT.COM 1) Individual response 2) Consolidation and clarification 3) Ranking 

TECHNOLOGIES TRIED FOR NOMINAL GROUP 1) Individual response 2) Consolidation and clarification 3) Ranking  so…

FOR NOW: KEEPING WITH and…

UNINTENDED OUTCOME 1: ONLINE SETTINGS

UNINTENDED OUTCOME 2: IN OTHER STUDENT FEEDBACK METHODS

UNINTENDED OUTCOME 3: PAPER AS TECHNOLOGY ubiquitous manipulable overlay multi-items text entry legibility visibility Image by Paul Duvall

IT’S YOUR TURN TO TEXT What other technologies can you suggest? In what other student feedback or evaluation contexts could you use these for? text in your thoughts if you like – see instructions

THANK YOU Contact Jaye McIsaac Tunde Varga-Atkins

REFERENCES Delbecq, A., Van de Ven, Andrew, & Gustafson, D. (1975). Group techniques for program planning: a guide to nominal group and Delphi processes. Glenview Ill.: Scott Foresman. Luff et al (2007) Augmented Paper: Developing relationships between digital content and paper Oliver, M. (2012) Learning with technology as a coordinated sociomaterial practice: digital literacies as a site of praxiological study Varga-Atkins, McIsaac et al (2011) Using the nominal group technique with clickers to research student experiences of e- learning: a project report [ ] Varga-Atkins, McIsaac et al (2011) The Nominal Group Technique – a practical guide for facilitators [