1. BECTa ICT Research Conference – June 2002 Intro  Survey Details  Secondary Surveys conducted July 2000 and June/July 2001  Sponsored by Fischer Family Trust and RM  3500 replies from secondary subject departments in over 2500 schools  Total of over 25,000 ratings of individual ICT resources  Pilot Primary Survey (180 schools) in 2001  Publications  High Impact ICT Resources  Fischer Family Trust Reports 2002  www.fischertrust.org  Further Developments  ‘Value-Added’ Analysis  Workshop – ‘Making a difference with ICT’  Survey 2002

2. BECTa ICT Research Conference – June 2002 Philosophy  Ask the experts  Teachers who have been using software in real environments  Key Questions  Is it easy to use, motivating etc. useful, but what really matters is  “Does it help to improve learning”  IMPACT is more important than LEVEL OF USE  Overall Aim  Is to provide feedback to teachers managers in schools about which ICT resources are felt to have the greatest impact upon pupils’ learning

3. BECTa ICT Research Conference – June 2002 Surveys  Survey asked Teachers to provide:  a list of the main ICT resources used in the teaching of their subject(s) - software packages, website, specialist peripherals  an overall rating for the impact of ICT on pupils’ learning in each subject  ratings, in terms of use and impact on pupils’ learning, for each ICT resource  An overall rating for the impact of website use (2001)  a brief summary of packages used for administration (2001)  any other comments  Ratings used were:  1 = Very Little  2 = Some  3 = Significant  4 = Substantial

4. BECTa ICT Research Conference – June 2002 Overall Summary  Schools were a representative sample:  Distribution (geographical, type, catchment area) similar to national patterns  Comments indicated that levels of resourcing varied significantly, indicating that responses were not primarily from well-resourced schools.  Less than 10% of secondary departments made very little use of ICT  Impact rated as SIGNIFICANT or SUBSTANTIAL by: Key Stage2000 Survey2001 Survey 159% 267% 340%49% 453%59%

5. BECTa ICT Research Conference – June 2002 Variations between Subjects  Impact rated as SIGNIFICANT or SUBSTANTIAL by: SubjectSecondaryPrimary Art55%38% D&T78%30% English46%79% Geography36%39% History23%51% ICT85%87% Mathematics31%75% MFL30% Music76%19% PE25%7% RE(48%)24% Science38%54% Welsh(45%)(29%) Secondary based upon: Art260 D&T360 English168 Geography 186 History147 ICT370 Mathematics348 MFL272 Music420 PE301 RE 28 Science232 Welsh 47 Primary based upon 180 schools

6. BECTa ICT Research Conference – June 2002 Value Added Analysis - Background  Purpose  To ask the question “Is there any evidence for the impact of ICT upon pupils’ progress and attainment”  Methodology  Link pupil-level value-added data (KS2-3 and KS3-4 for 2 and 3 years respectively) to software survey responses.  Note – performance data provided by government agencies for the purpose of this research is confidential. Data for individual schools will not be disclosed or published.  Analyse data to see whether pupils make better progress in schools where ICT rated as HIGH IMPACT (Significant or Substantial).

7. BECTa ICT Research Conference – June 2002 Analyses – School Level Correlation KS2->KS3 KS3->KS4 Graphs plot Actual vs Estimated Mean Score Each graph shows data for ~10,000 schools over a 3 year period

8. BECTa ICT Research Conference – June 2002 Analyses - Accuracy  Correlations  Inputs:  TA and Test Levels in each subject (Pupil Level)  Gender (Pupil Level)  Free School Meals Entitlement (School Level)  Outputs:  Total Points Score Key StagePupilsSchoolsPUPSCHLEA 1->20.1 million4,000 / 3 yrs0.820.75NA 2->31.7 million12,000 / 3 yrs0.870.950.93 3->42.2 million12,000 / 3 yrs0.890.950.87

9. BECTa ICT Research Conference – June 2002 VA for ‘HIGH Impact’ Secondary Departments Consistently Better in 2000 Improving Faster Little Difference Art Design & Technology English Geography History Mathematics MFL Music Physical Education Science Value Added Analysis

10. BECTa ICT Research Conference – June 2002 FAQ  Are the outcomes simply a reflection of overall school differences?  Using pupil-level value-added data means that we can control for pupils’ prior attainment  Detailed analysis shows that ‘high impact’ departments tend to achieve better results than other departments in the same school  Does this PROVE that ICT use has a positive impact upon standards?  NO – but it does show that pupils make better progress in departments where ICT is felt to be making a significant contribution to learning.  Why is this important?  We can now identify and work with teachers from departments where there is evidence of significant ICT impact to find out HOW this has been accomplished

11. BECTa ICT Research Conference – June 2002 Dissemination /Development  Dissemination  Leaflet in BETT catalogue  Reports provided to delegates at NAACE conference (February)  2002 reports sent to all Primary and Secondary schools in England, Northern Ireland, Scotland and Wales (June)  Reports available on Website (www.fischertrust.org)www.fischertrust.org  Development  2002 surveys included with reports (June)  Surveys available on-line  Reports on overall trends in the data, links to value-added analysis, case studies …

12. BECTa ICT Research Conference – June 2002 Opportunities and Limitations  We now have:  A database, currently with more than 25,000 evaluations by 3,500 teachers in 2,500 secondary departments  A rapidly growing database of evaluations from primary schools  Evidence that use of Teacher Evaluation (together with Value- Added Analysis) is a good mechanism for identifying examples of how the use of ICT can ‘make a difference’  What we don’t currently have:  Time and resources to go beyond the publication of reports and the development of a some case studies  Therefore  Are there others interested in working with us to build upon this work, particularly in terms of finding key ‘success factors’ in High Impact ICT departments or schools  Any suggestions – Email ( miket@fischertrust.org )miket@fischertrust.org