1. Introduction to CEM and Computer Adaptive Assessments
Belfast, March 6th 2013
Neil Defty
Business & Development Manager
CEM

2. Who / What is CEM ?

3. Centre for Evaluation & Monitoring
Not for Profit Organisation
Part of Durham University
Close links to School of Education
Established 1983
Nursery / Reception  Post-16
Monitoring Systems, Research and Evaluation Projects
UK & International
Over 1/3 of UK Secondary & Post-16 establishments
Informed by Evidence from Research

4. CEM Provides Information Systems from Nursery to Post-16
Student Guidance & Chances
Performance Monitoring
Evaluation & Research Projects
Entrance Testing
Support – Phone, , documentation
Training
INSET
Roadshows
Conferences (including exhibitions)

5. The Systems

6. The Systems Alis Yellis MidYIS A / AS / Btec / IB etc Year 13 Year 12
GCSE
Computer Adaptive Baseline Test
Year 13
Year 12
GCSE
Year 11
Yellis
Computer Adaptive Baseline Test
Year 10
KS3
Combines curriculum tests with developed ability
INSIGHT
Year 9
MidYIS
Computer Adaptive Baseline Test
Year 8
Year 7

7. Early Testing & Predictions
Typical Timeline
Testing is available from June at end of previous academic year
e.g. Y6 / Y9 / Y11
Early Testing & Predictions
The baseline score is measured as early as possible and the information sent to CEM
The institution then receives feedback from CEM: baseline test results and predictions
The public examination results are sent to CEM when received by the institution
The value added data is available from September via the Institution’s secure website at CEM

8. Basic Principles: Measure Student’s ability
Intake Profiles
IPR - Strengths and Weaknesses
Project future performance from national trends
Predictions
Record Final Grade
Compare actual performance to Projected Performance
Value-Added Reports

9. Baseline Assessment and the Computer Adaptive Baseline Test (CABT)

10. Baseline Assessment Year 7 (8) Year 9 Year 10 (11) Year 12 (13+)
Computer Adaptive Baseline Test
Vocabulary
Maths
Non Verbal

11. What is an Adaptive test ?
All questions allocated to groups of different difficulty
All students in a year group get common difficulty starting question
If answer correct, next question harder
If answer wrong, next question easier
Test looks at all answers in a section and homes in on questions of a suitable difficulty for student
Try it yourself at

12. Adaptive approach
Low
Average
High

13. X          13

14. Standardisation Test scores are standardised; Mean = 100, SD = 15
Standardised Score
National Percentage
Comment
>130
Top 2.5%
Traditional classification of ‘mentally gifted’
>120
Top 10%
>100
Top 50%
<80
Bottom 10%
<70
Bottom 2.5%
Potential special educational needs?

15. The Computer Adaptive Baseline Test:
Adaptive components common to MidYIS, Yellis, and Alis are:
Vocabulary
Maths
Non- adaptive sections:
Non-verbal (MidYIS, Yellis, Alis)
Skills (MidYIS only)
Assessments are ‘curriculum free’: i.e. students are not taught prior to the test
Measure of acquired ability
There is no preparation for the test
The vocabulary and maths sections are the most important to the prediction of all subjects.
Maths most difficult section to be curriculum free
Non verbal is suitable for pupils for English is a second or additional language – allows ability to be displayed without the confounds of language
No preparation for the test – this is important to ensure that every pupil starts from the same fair point. No lesson of preparation – each pupil hears the same examples, instructions and receives the same amount of help. This standardised administration ensures that you get a measure of typical performance for each pupil an indication of how much work you will need to put into take each pupil through to the next stage of education – through KS3 and onto GCSE.
Developed ability is a set of acquired skills For example you are born with an IQ but your environment is so important. Yes you are born with a particular set of genes but you can’t read – you have to learn how to, you’re born with NV ability, but you can’t you can’t solve non verbal puzzles – you have to learn how to answer them. 15

16. Difficult vocabulary
Vocab - Closely related to reading and English – a proxy – pupils need to be able communicate ideas across the curriculum areas
Vocab is caught not taught – absorbed from the world around us. On its own is an excellent predictor for later academic achievement. Contributes well to the prediction of
English
History
Foreign Languages
Choose word or phrase with most similar meaning from those shown on right.
How is vocab relevant to my subject? 16

17. Difficult maths
x/4 + 2
And a hard question – could you do this? Seems tricky – especially for 11 year olds! However 1 in 15 pupils are getting this right!
Good maths skills will mean that any areas of the curriculum that require numeracy - the pupil to think logically, perform calculations, reading data
Maths – here’s an item of medium difficulty.
Good predictor of
Maths#
Stats
ICT
D&T
Economics 17

18. Non-Verbal: Cross-sections
Non verbal – 3D visualisation, spatial awareness, pattern recognition. NV consists of 3 sections – crss sections, block counting, pictures.
A low score here will mean that students will struggle will any elements of the curriculum that needs them to be able to think in 3 dimensions –
for example
geography,
Art
Drama
Maths
Science
D & T
Solid been cut with plane to form a cross section. 18

19. Non-Verbal: Blocks 3
Piles of blocks number of small and large blocks 19

20. Non-Verbal: Pictures Pictures 20
Pictures – addition, subtraction, sequences of pictures.
Low scores in Non verbal – anecdotally number of times PH has been told on INSET that low NV corresponds to observed problems in D&T. 20

21. Skills: Proof Reading 21 Skills = PR & PSA – skills for modern world
Proof reading - Essay writing 21

22. Skills: Perceptual Speed and Accuracy
PSA – skimming & scanning skills. Decoding skills – can pupils read and understand the question?
Low skills relate to under performance – not checking work, not able to work quickly (related to speed of work in class), not spotting errors
In PH’s school he found that pupils with a D band in vocab and a D band in Skills on many occasions turned to be pupils who were dyslexic – either already diagnosed or diagnosed following further testing.
PSA - related to speed of work in class
High ability who is a slow worker, - ma doubt thaier ability because they don’t get wok finished
Interesting to look at pupils who have low skills scores overall but the PSA element is low – here you will get pupils who do not take care intheir work – ie don’t check it or spot mistakes but who will rush. You may blame their incorrect work on their rushing but their low skills score means they probably would have got it wrong anyway!
ie low Skills – rushing to get get work finished, getting things wrong 22

23. Alis - Patterns
PSA – skimming & scanning skills. Decoding skills – can pupils read and understand the question?
Low skills relate to under performance – not checking work, not able to work quickly (related to speed of work in class), not spotting errors
In PH’s school he found that pupils with a D band in vocab and a D band in Skills on many occasions turned to be pupils who were dyslexic – either already diagnosed or diagnosed following further testing.
PSA - related to speed of work in class
High ability who is a slow worker, - ma doubt thaier ability because they don’t get wok finished
Interesting to look at pupils who have low skills scores overall but the PSA element is low – here you will get pupils who do not take care intheir work – ie don’t check it or spot mistakes but who will rush. You may blame their incorrect work on their rushing but their low skills score means they probably would have got it wrong anyway!
ie low Skills – rushing to get get work finished, getting things wrong 23

24. Alis – Logical Reasoning
PSA – skimming & scanning skills. Decoding skills – can pupils read and understand the question?
Low skills relate to under performance – not checking work, not able to work quickly (related to speed of work in class), not spotting errors
In PH’s school he found that pupils with a D band in vocab and a D band in Skills on many occasions turned to be pupils who were dyslexic – either already diagnosed or diagnosed following further testing.
PSA - related to speed of work in class
High ability who is a slow worker, - ma doubt thaier ability because they don’t get wok finished
Interesting to look at pupils who have low skills scores overall but the PSA element is low – here you will get pupils who do not take care intheir work – ie don’t check it or spot mistakes but who will rush. You may blame their incorrect work on their rushing but their low skills score means they probably would have got it wrong anyway!
ie low Skills – rushing to get get work finished, getting things wrong 24

25. INSIGHT Curriculum-based assessments covering:
Maths - Number & Algebra
- Handling Data
- Space, Shape & Measurement
Science - Biology, Chemistry, Physics,
- Attitudes to Science
Reading - Speed Reading
- Text Comprehension
- Passage Comprehension
Additionally for full INSIGHT:
Developed Ability
Vocabulary
Non verbal, Skills
Attitudinal measures

26. INSIGHT - Maths 7

27. INSIGHT - Maths

28. INSIGHT - Science
Plus Attitude to Science: Perception of Science, Relevance to Self, Environmental Actions, Enjoyment of Science

29. INSIGHT – Reading (Text Comprehension)

30. INSIGHT – Reading (Passage Comprehension)

31. INSIGHT – Reading (Speed Reading)

32. INSIGHT – Developed Ability