1. Enhancing Nominal Group Technique using technologies: a demonstration (ID275) ALT-C 11-13 September 2012 Jaye McIsaac, Educational Development Tunde Varga-Atkins, eLearning Unit University of Liverpool ELESIG-funded project

2. OUTLINE Introduction and purpose What is the Nominal Group Technique? What is the problem? Technologies we tried

3. OUTCOMES To describe the Nominal Group Technique (NGT) as a student feedback & evaluation tool. To evaluate the use of learning technologies in the process of a face- to-face NGT session. To consider the value of using technologies in online settings using NGT.

4. WHAT IS NOMINAL GROUP TECHNIQUE? Structured group activity One given topic Facilitates group decision-making Immediate results Reduces researcher/participant bias - Delbecq & Van de Ven (1971 )

5. NGT: STAGES 1) Individual responses 2) Clarification and consolidation 3) Ranking responses Example question: ‘What would you change in your course?’

6. STAGE 1: WHAT TO KEEP DOING (FEEDBACK TO STUDENTS) Annotated,personalised feedback. Model answers Giving model answers for essay titles Audio feedback for everythin g! Audio feedbac k giving extensive commen ts Drafts that must be submitted and marked prior final work 123 4 5 6

7. STAGE 2 : WHAT TO KEEP DOING (FEEDBACK TO STUDENTS) Model answers Giving model answers for essay titles Audio feedback for everythin g! Audio feedbac k giving extensive commen ts Drafts that must be submitted and marked prior final work 3 5 2 4 Annotated,personalised feedback. 6 1 Audio feedback Model answers Personalised feedback

8. STAGE 3 : WHAT TO KEEP DOING (FEEDBACK) Item no.Item description 5 points1Annotated, personalised feedback 4 points2Giving model answers 3 points4Audio feedback 2 points7etc. 1 point10etc.

9. THE TOP FIVE QUALITIES OF GOOD FEEDBACK Timing, quick Detailed, on how to improve Applicable to future work PersonalConsistent bw markers 1 2 3 4 5

10. ‘PROBLEMS’ Credit by Nathan T. BakerNathan T. Baker

11. 1) Individual responses 2) Clarification and consolidation 3) Ranking responses TECHNOLOGIES WE TESTED

12. Web- conferencing online sticky- note: linoit.com Polling: GoogleModerator clickers mobile text-entry / MCQ polling

13. Don’t forget: You can copy-paste this slide into other presentations, and move or resize the poll.

14. MOBILE TEXT ENTRY (E.G. POLLEVERYWHERE.COM, TEXTWALL.CO.UK) 1) Individual response 2) Consolidation and clarification 3) Ranking 

15. CLICKERS 1) Individual response 2) Consolidation and clarification 3) Ranking 

16. GUIDE TO NGT & PROJECT REPORT See References On slideshare.net, search for Nominal Group Technique

17. GOOGLE MODERATOR Live link:

18. GOOGLE MODERATOR 1) Individual response 2) Consolidation and clarification 3) Ranking 

19. WEB-CONFERENCING (ELLUMINATE)

20. WEB-CONFERENCING 1) Individual response 2) Consolidation and clarification 3) Ranking 

21. ONLINE STICKY-NOTES: E.G. LINOIT.COM

22. LINOIT.COM 1) Individual response 2) Consolidation and clarification 3) Ranking 

23. TECHNOLOGIES TRIED FOR NOMINAL GROUP 1) Individual response 2) Consolidation and clarification 3) Ranking  so…

24. FOR NOW: KEEPING WITH and…

25. UNINTENDED OUTCOME 1: ONLINE SETTINGS

26. UNINTENDED OUTCOME 2: IN OTHER STUDENT FEEDBACK METHODS

27. UNINTENDED OUTCOME 3: PAPER AS TECHNOLOGY ubiquitous manipulable overlay multi-items text entry legibility visibility Image by Paul Duvall

28. IT’S YOUR TURN TO TEXT What other technologies can you suggest? In what other student feedback or evaluation contexts could you use these for? text in your thoughts if you like – see instructions

29. THANK YOU Contact Jaye McIsaac J.Mcisaac@liverpool.ac.uk Tunde Varga-Atkins tva@liverpool.ac.uk

30. REFERENCES Delbecq, A., Van de Ven, Andrew, & Gustafson, D. (1975). Group techniques for program planning: a guide to nominal group and Delphi processes. Glenview Ill.: Scott Foresman. Luff et al (2007) Augmented Paper: Developing relationships between digital content and paper Oliver, M. (2012) Learning with technology as a coordinated sociomaterial practice: digital literacies as a site of praxiological study Varga-Atkins, McIsaac et al (2011) Using the nominal group technique with clickers to research student experiences of e- learning: a project report [http://slidesha.re/xQlBCg ]http://slidesha.re/xQlBCg Varga-Atkins, McIsaac et al (2011) The Nominal Group Technique – a practical guide for facilitators [http://slidesha.re/AmYOgv]http://slidesha.re/AmYOgv