Background questionnaires: Why ask about social identity Ruth Childs (OISE-UT) Orlena Broomes (OISE-UT) 1

Social Identity Social Identity is a potentially important factor in our attempts to understand student achievement on large scale tests. We typically do not collect enough information about this variable. It takes time and effort to develop good questionnaire items 2

Questions How do we deal with those groups who do not want the information that could be obtained by such items? Aboriginal population What are the implications of options such as: “I prefer not to Respond” Can respondents distinguish between confidentiality and anonymity 3

Four Perspectives on the Issue of Motivation as it Relates to Large-scale Assessment Christina van Barneveld (Lakehead University) Sherri-Lynne Pharand (Lakehead University) Lori Ruberto (Lakehead Public Schools) Daphne Haggart (Lakehead Public Schools) 4

Motivation Motivation can have an impact on students’ test performance. The importance of expectation and value There are a variety of methods to detect lack of motivation. There is a real value in examining the perspectives of different stakeholders. 5

Questions Which students are not motivated? Are there other ways to raise motivation rather than through the use of incentives and support? What do you mean by the notion that “motivation is a content irrelevant factor? Have you been able to get the perspectives of others within your stakeholder groups? Does increased motivation lead to change? 6

Testing of English- and French Speaking Students in Canada Kadriye Ercikan, UBC Marielle Simon, University of Ottawa María Elena Oliveri, UBC Normand Dufour, Quebec Ministry of Education 7

Comparability of Translated Tests It is difficult to do this successfully. Minority language population are often small (<5%) There are a variety of procedures, both statistical and “descriptive” to try to address these comparability issues. We have a long way to go. 8

Questions Why is there such a large section on DIF? Should we be promoting comparability across languages or discouraging it? When can we be confident in the comparability of results across languages? 9

Complexities, Challenges and Opportunities in Scoring Provincial Assessments: A Perspective from British Columbia María Elena Oliveri (University of British Columbia) Britta Gundersen-Bryden (B. C. Ministry of Education) Kadriye Ercikan (University of British Columbia) 10

Scoring The methods used to score large-scale assessments can impact the accuracy of the results. The case of British Columbia. 11

Questions Which kinds of tests and items are more susceptible to scoring errors?. What are some of the “real” challenges of decentralized scoring? Move beyond consistency Downloading of responsibility What have other researchers found out here? 12

Improving the Ways we Report Test Scores: Research Findings, and Important Steps and Methods. Ron Hambleton (University of Massachusetts) April Zenisky (University of Massachusetts) 13

Reporting There is a need for score reports to include important technical information? Scales Measurement Error We need to develop reports that make it easier for accurate interpretations. Potential of benchmarking. Diagnostic Information 14

Questions But, students, teachers, etc. want more information! What accurate information can we provide? The need for large-scale test honesty. 15

Summary Across Papers Each chapter highlights an important issue in large-scale testing These topics often do not receive sufficient attention in large-scale testing systems The chapters are well written summaries of what we know Differences in the writing, structure, and depth of each chapter. There is not much new in the chapters. Most of the information has been reported in other forums. 16