1. © University of South Wales Technology supported literacy and numeracy enhancement workshops for trainee teachers David Longman and Kerie Green University of South Wales 4 th Annual TEAN Conference, Birmingham, May 2013 Contact: Dr Kerie Green: kerie.green@southwales.ac.ukkerie.green@southwales.ac.uk

2. © University of South Wales Assessment Development Group School of Education Dr Kerie Greenkerie.green@southwales.ac.ukkerie.green@southwales.ac.uk Barbara Kurzikbarbara.kurzik@southwales.ac.ukbarbara.kurzik@southwales.ac.uk David Longmandavid.longman@southwales.ac.ukdavid.longman@southwales.ac.uk The Assessment Development Group is a pragmatic working group of educationalists at the School of Education. It develops pedagogically embedded computer-based tools that support student learning through structured feedback guided by data.Assessment Development GroupSchool of Education

3. A merger of two existing universities launched on April 11, 2013; Now one of the largest universities in the UK and the largest university in Wales (33K students) ;33K students Five campuses across South Wales. © University of South Wales University of South Wales - the background

4. © University of South Wales Initial Teacher Training: background Many of our BSc students are from the South Wales region; parts of this region are on the extremes of European social indicators; This carries risk, e.g. that without effective support retention goes down.

5. © University of South Wales Initial Teacher Training: Our current learning and teaching strategy notes that a “widening access strategy [will] include a stronger focus on helping those who access HE to successfully complete their learning objectives”. Teaching and assessment methods must provide accurate and useful feedback on learning and be flexible and adaptive to meet the needs of ‘non- traditional’ students.

6. © University of South Wales Previous Project: enhancing participation in UG mathematics Display question Wait for all student clickers Display graph of responses Group discussion Reveal correct choice Respond-Discuss-Reveal (RDR)

7. © University of South Wales Previous Project: some outcomes Changed social conditions of classroom learning & increased confidence and engagement. Purposive teacher questioning. Promoted focused whole-class discussions. For mathematics, highlighted the role of language in understanding. Patterns of responses used to highlight areas of difficulty and to critique question design. Response data guides tutor's formative judgments in directing teaching appropriately.

8. © University of South Wales Current Project: Policy context From Sept. 2013 entry qualifications raised to GCSE grade B in English and Mathematics; Welsh Govt. requirement to test ITT candidates at interview and to be accountable for standards as a condition of QTS; To monitor, track and maintain standards at recruitment, during training and as outcome for QTS; UCET Cymru developing all-Wales literacy /numeracy interview tests. Pilot complete. Issues of definition etc. continue to be explored.

9. © University of South Wales Current Project: Course Context Regular lit. and num. audits as component of Professional Development Planning (PDP); New focus on monitoring and maintaining standards; Issues of target setting, improvement and consolidation; Issues of efficiency, cost effectiveness etc.; Targeted on those undergraduates identified in need of support for literacy and numeracy based on PDP audits.

10. © University of South Wales Initial samples Sample selected: <65% in audit (in any of 3 sections) N=33 Shape, Space, Measure (Total score = 18) IDScore%age N1/L11161% N41161% N5/L71161% N7/L91161% N81161% Number (Total score = 23) IDScore%age N21461% N10/L121461% Data handling (Total score = 14) IDScore%age N3964% N9964% N10/L12964% Lexical (Total 27) IDScore%age L31556% L41556% L51763% L9/N71452% L101763% Grammatical (Total 40) IDScore%age L1/N12563% L22255% L7/N52358% L82358% L9/N71640% L102563% Punctuation (Total Variable) IDScore%age L63337% L115163% L12/N105361% Numeracy Sample 9 students Literacy Sample 12 students 4 students in both samples

11. © University of South Wales Procedure Questions sampled from categories covered by Literacy or Numeracy audits; RDR slide shows created. Two sessions in each plus final summary sessions; Different questions per session 10 for numeracy; 15 for literacy; Data collected: responses to questions plus feedback survey.

12. © University of South Wales Immediate feedback to student via mail merge Dear Kerie Here are your responses from the Maths session on 17 th October. The lower case 'i' indicates an incorrect choice, 'c' a correct choice. The detailed description of the questions and choices is in the large table below. Qu.Response 1A c 2B c 3D c 4A i 5D c 6A c 7C c 8A c 9A i 10C c Points8 Percent80%

13. © University of South Wales Final results Variable attendance Literacy Session 1 (15 qu) Literacy Session 2 (15 qu) Literacy Final Session (16 qu) IDAuditPoints%ageIDAuditPoints%ageIDAuditPoints%age L255%1280%L255%960%L255%956% L356%853%L356%853%L456%1275% L456%1067%L456%853%N373%956% L563%747%L563%853%L563%1063% L946%853%L946%853%L9/N746%956% L1063%747%L1063%1067%N974%744% Numeracy Session 1 (9 qu) Numeracy Session 2 (5 qu) Numeracy final session (8 qu) IDAuditPoints%ageIDAuditPoints%ageIDAuditPoints%age N161%556%N364%5100%L283%338% N261%556%N461%360%L475%338% N364%667%N961%240%L572%225% N561%667%N1061%360%L9/N761%450% N964%667%N364%563% N1064%667%N964%675% RedImproved >64% GreenImproved BlueDecline PinkNo sig change (Note Student L9/N7)

14. © University of South Wales Student feedback (i) Positive elements: “ Being able to discuss … why we were getting the wrong answers greatly helped with my confidence … seeing why I was getting it wrong helped.” “In the literacy session it was good to show divided opinion about answers. It more understandable why certain things should or shouldn't be the case.” (In numeracy) “…the immediate feedback from lecturers and peers helped to discuss incorrect and correct methods how to answer particular questions. A few I got wrong all the time so the discussions were helpful to see why…”

15. © University of South Wales Student feedback (ii) Suggestions for improvement: Could be earlier in the year; Could be more closely linked in time to the audit process; Wider coverage of content; Embed in professional studies.

16. © University of South Wales Conclusions Similar positive comments by students to first project about the process and participation in session but … … quantitative data inconclusive (at best) in terms of scores. Issues: test design - simple retesting with same tests probably not a robust measure of consistency, of maintaining standards. Freedom to test is more of a headache than it appears!

17. © University of South Wales Thank you for listening Questions?