1. Proposed student questionnaire for the main survey National Research Coordinators Meeting Windsor, June 2008

2. NRC Meeting Windsor June 2008 Contents of presentation Criteria used for selection of questionnaire material Overview of field trial questions and recommendations Length of proposed instrument Proposed changes

3. NRC Meeting Windsor June 2008 Criteria for selection Field trial analysis results –Scaling properties (for scaled items) –Item dimensionality –Face validity –Review of relationship between related variables/constructs Coverage of assessment framework Feasibility of adaptations to national contexts

4. NRC Meeting Windsor June 2008 General proposals Only one (combined) student questionnaire (with both background and perceptions questions) Only one form No “don’t know” categories

5. NRC Meeting Windsor June 2008 Review of field trial instruments Three categories of questions –To be retained without changes (in green) –To be modified (in yellow) –To be deleted (in red) Some questions are flagged as having “medium priority” Recent advice from the PAC meeting 7-8 June appears in orange

6. NRC Meeting Windsor June 2008 Student characteristics Age (A01) Gender (A02) Ethnicity (optional) (A03) Expected educational level (A04) –without “don’t know” category

7. NRC Meeting Windsor June 2008 Home environment Household composition (B01) –Medium priority only Country of birth (B02) Language use at home (B03)

8. NRC Meeting Windsor June 2008 Parent background Parental occupation (B04, B06) –Good measure of socio-economic background –Somewhat larger percentages of missing values in some countries Parental education (B05, B07) –Without “don’t know” category –Still considerable percentages of missing values Parental interest in political/social issues (B08) –Interesting results!

9. NRC Meeting Windsor June 2008 Household possessions Household items (B09) –low reliabilities (especially in more developed countries) –low correlations with other socio- economic indicators) –PAC advice: Consider as an international option Books in the home (B10) –Good indicator of socio-economic background

10. NRC Meeting Windsor June 2008 Measuring socio-economic background in the main survey It is proposed to have three indicators –Parental occupation (highest of both parents) –Parental education (highest of both parents) –Number of books at home Analysis will be undertaken to explore computation of composite index

11. NRC Meeting Windsor June 2008 Student activities Out-of-school leisure time activities (C01) –Keep only items on reading and hanging out –Add: Homework and study time –PAC advice: Consider keeping also (at least) computer and television use items Activities out of school (C02) –Discussion item form scale –Media information items do not scale but are of interest as single items (wording change for TV news) –Social participation items do not scale and should not be retained –PAC advice: Consider combining items on organised social activities (boy scouts, youth clubs etc.)

12. NRC Meeting Windsor June 2008 Students’ civic participation Civic participation in the community (C03) –Worked as scale and interesting associations with other variables –PAC advice: An additional note to include religious services was initially considered but should rather not be included Civic participation at school (C04) –Two items reflect different dimension (tutoring, school newspaper) and are proposed to be deleted –There were adaptation problems with the item on school protest, which is proposed as another deletion –PAC suggestion: Consider same categories as for previous question

13. NRC Meeting Windsor June 2008 School-related perceptions - 1 Open climate for classroom discussion (D01) –Good scaling properties and interesting correlations –PAC advice: Use rather same categories as in CIVED Student influence at school (D02) –Good scaling characteristics but negative correlations with civic knowledge –ISC sees this as question with medium priority –PAC advice: Additional item about extracurricular activities

14. NRC Meeting Windsor June 2008 School-related perceptions - 2 School climate items (D03) –Good scaling properties for student-teacher relation items (medium priority) –Students’ sense of belonging has not always satisfactory scaling properties and should be deleted Students’ confidence in school participation (D04) –Negatively worded items did not load on the same dimensions and should not be retained

15. NRC Meeting Windsor June 2008 Perceptions of citizens and society - 1 Democratic value beliefs (E01) –Agreement item format leads to highly skewed responses –Alternative format proposed asked about characteristics of “ideal society” –PAC advice: Keep original format Good citizenship beliefs (E02) –Two-dimensional structure confirmed –Proposed to delete three items that did not scale well with either of the two scales –PAC advice: Consider keeping “obeying the law” and “working hard” for single-item reporting

16. NRC Meeting Windsor June 2008 Perceptions of citizens and society - 2 General trust (E03) –Few interesting associations –Only item about trust in “people in general” should be added to question about trust in institutions Acceptance of socially undesirable actions (E04) –Some problems with item scaling properties and generally not too interesting results

17. NRC Meeting Windsor June 2008 Self-perceptions - 1 Interest in political and social issue (F01) –Generally good scaling properties –Five items to be retained and one modified Students’ self-concept (political internal efficacy) (F02) –Six items have good scaling properties –One (negatively phrased) item did not scale well and should be deleted

18. NRC Meeting Windsor June 2008 Self-perceptions - 2 Feelings of distinctiveness (F03) –Doubts that items measure what is intended Attachment to community levels (F04) –Results not very interesting –May be retained for regional instruments

19. NRC Meeting Windsor June 2008 Rights and responsibilities - 1 Gender rights items (G01) –One item did not scale well and should be deleted Rights for ethnic/racial groups (G02) –Good scaling properties –Correct adaptation need to be ensured (not minorities!)

20. NRC Meeting Windsor June 2008 Rights and responsibilities - 2 Rights for immigrants (G03) –Negatively phrased items reflect different dimension and should not be retained many [instead of enough] –PAC suggestion: Keep item h (“When there are not many [instead of enough] jobs available the number of should be limited“) for single-item reporting Reaction to threats to democracy (G04) –Items did not scale and only single-item reporting might be considered –Proposed to delete this question –PAC advice: Include three items (b, d and f) in democracy belief question (E01)

21. NRC Meeting Windsor June 2008 Perceptions of institutions and society - 1 Trust in groups and institutions (H01) –Good scaling properties for CIVED items and interest in single-item reporting for not scaled items –Addition of “people in general” –“Standardisation” of optional items –PAC suggestion: Consider asking about media separately (Newspaper, TV, radio, internet) and ask about “people in general” first Attitudes toward country (H02) –Generally, good scaling properties except for three items (two of them should be deleted)

22. NRC Meeting Windsor June 2008 Perceptions of institutions and society - 2 Government responsiveness (external political efficacy) (H03) –Analysis revealed two dimensions (“responsiveness” and “cynicism”) –Both scales have weak reliabilities Satisfaction with country’s achievements (H04) –Generally, good scaling properties –Relatively high correlations with attitudes toward country –Deletion proposed on content grounds –Mixed PAC advice

23. NRC Meeting Windsor June 2008 Perceptions of institutions and society - 3 Support for political parties (H05) –Interesting results –Sufficient proportions in each category

24. NRC Meeting Windsor June 2008 Perceptions of participation in society - 1 Citizenship self-efficacy (I01) –Good scaling properties for all but one item (“raising money for people in need”) Protest activities (I02) –Two dimensional-structure (legal and illegal protest) –Shortening of legal protest scale (two items proposed for deletion)

25. NRC Meeting Windsor June 2008 Perceptions of participation in society - 2 Participation as an adult (I03) –Two dimensions (expected electoral participation and expected active political participation) –Good scaling properties for all items Participation as a young person (I04) –First two items reflect different dimension (community-based participation) –Good scaling properties for item measuring informal participation

26. NRC Meeting Windsor June 2008 International option regarding religion Religious denomination (J01) Participation in religious services (J02) Attitudes toward religion (J03) –Good scaling properties for five items –Three items do not really reflect attitudes and should be deleted

27. NRC Meeting Windsor June 2008 Proposed student questionnaire for main survey With 5 items per minute the estimated length is 34 minutes There are about 170 to 180 items in the instrument The CIVED questionnaire had 162 items in the perceptions part (for 30 minutes)

28. NRC Meeting Windsor June 2008 Proposed changes Reduction in questionnaire length of about 10 minutes Generally, only slight modifications or deletions are proposed The only more fundamental change for democratic value beliefs a format change has been dropped following PAC advice –Risk of using an un-trialled question format in main survey!

29. NRC Meeting Windsor June 2008 Questions or comments?

30. NRC Meeting Windsor June 2008 Group discussions No complete re-writing of questions possible No additional questions –unless previously used in surveys of the same age group... Go through the original field trial material (tables in document 6) and discuss –Agreement with proposed selection –Prioritisation of questions –Feedback on proposed –Suggestions of (minor) changes to wording

31. NRC Meeting Windsor June 2008 Group allocations From six groups Three groups start from the beginning of questionnaire material (those sitting in front) Three groups start from the end (those sitting towards the back) Try to form mixed groups (from different countries and regions)