1. University of Winchester TESTA/American Studies
Engagement Revelation Change
Carol Smith

2. Data audit Engagement
Programme Audits: Comparative Overview at a glance
American Studies Winchester
Average on 9 programmes
Range over 8 programmes
Total Assessments 63 47
32 – 80
Summative 52 37
26 – 52
Formative 11
0 – 41
Variety 13
7 – 17
Exam %
9.6%
15.2%
3% - 34%
Timeliness
21 days
20.4 days
17 – 28 days
Oral feedback
3hrs 32 mins
6 hrs 40 mins
37 mins to 30 hours
Written
10,972 words
7,153 words
2, ,412 words

3. Revelation TESTA Comments: (see p13/14)
Data from the programme audit indicates that students experience the following:
High total volumes of assessment, most of which is coursework
A high variety of assessment types
Minimal formative-only assessment.
High volumes of written feedback, delivered relatively slowly, with low volumes of oral feedback.
Programme Leader /Team
Yes but ‘shocked’ at the number
Agreed and planned as such- so?
Debated – drafting of assessments throughout, especially final year thesis
Debated - variable within team plus traffic?

4. Quantity & quality of f’back
AEQ ‘Scores’
Scales
of effort
Quantity
Coverage of Syllabus
Quantity & quality of f’back
feedback
Use of
Assessment
Appropriate
and standards
Clear goals
Surface Approach
Approach
Deep
Learning from exams
satisfaction
Overall
Mean
Range
3.61
2.88
3.15
3.69
3.71
3.34
3.26
3.8
2.89
3.92
American Studies
3.91
2.72
3.76
3.97
3.74
3.25
2.56
4.24

5. Revelation Programme Leader /Team
Comments (TESTA p15):
The quantity and quality of feedback score is significantly higher than on most other programmes in the sample, reflecting the high volume of written feedback in the audit data, but suggesting good quality feedback as well. This somewhat contradicts relatively low oral feedback and slow timeliness on the audit data.
The high summative – low formative assessment regime and low formal oral feedback usually results in low scores for clarity of goals and standards. Yet AMS students are surprisingly clear about goals and standards, suggesting that some other process may be supplementing feedback processes.
Coverage of syllabus score is very low, indicating that students feel that they can be selective about what they study. This may relate to cores on the quantity of effort scale to very low scores on learning from exams, suggesting that students are ‘spotting’ and being selective about topics. Alternatively, the AMS model may be encouraging selectivity for depth rather than breadth.
Programme Leader /Team
How to capture non- written and ‘off’ campus oral feedback
Our instinct – its in the feedback on plans, critical bibliographies and drafts
Yes - Interdisciplinary subject – focus on critical knowledge and modelling of knowledge rather than curriculum transmission

6. Intuition reinforced by Focus group Engagement e. g
Intuition reinforced by Focus group Engagement e.g. Drafting- student quotes
At this stage I think they should say ‘If you don’t submit a draft - an essay or a critical piece, or whatever - you’re an idiot’.
The thing is last year every time I did hand in a draft that was generally the difference of a grade. Honestly. Like every time I handed in a draft it was the difference between a 2.1 and a First say on an essay.
Some, you have to sometimes ask ‘Are you taking drafts?’ whereas I think especially in the third year it ought to be blanket across the board that all tutors with written work, critical commentary, essays or whatever, accept at least one or two drafts from everybody.
The thing is though they have to get it back. I don’t want to nag them, but my goal for this year was to make sure I’ve written essays a week in advance so I can send it on that week deadline and have enough time to then make changes and turn it around. So I sent one on the week and then it took five days before I got a response to it. So whilst I still just about had time to do it, it almost was pointless sending it.

7. From audit to change Enabling the process
Programme level transparency
Institutional level
transparency
All data, reports, discussions held in team and reported back to students at Programme Committee and posted on Learning Network
Faculty and University wide discussion stressing the enabling and transformative aims of TESTA. Reinforced by positive & enabling operational mode of the team of Tansy Jessop, Yaz El-Hakim and Graham Gibbs.

8. First Changes – Programme
Student
Mapping of assessments, types, iterations and importance
Clarity of feedback, drafting and importance of inter-active and developmental nature of process
Staff
Stronger and less QA driven notion of assessment – in process of ‘cutting’.
Acceptance of the importance of drafting or similar formative process as whole a team.

9. Institutional Context and changes Fees, league tables, change in academic year
Management
NSS/Unistats
Change in academic year – 2011/12
Process ‘driven by quality office dates’
Programme /LTDU Winchester
Have used the TESTA process to interpret.
Ongoing debate TESTA offers model of transformative and ongoing change

10. Changes for us all?
TESTA offers us a programme based, positive, transformative and evidenced based method of engaging with the learning experience of students which is imperative in the prevailing reductive sweep of questionnaires and league tables.
TESTA offers staff a critical and collegiate space to review their pedagogy which could replace the Quality Assurance driven periodic review.

11. Further info TESTA web site http://www.testa.ac.uk/
to contact Carol Smith
Thank you for you participation.