Development and Validation of a Tool for Measuring Indicators of Information and Communication Technologies (ICTs) at a Turkish State University Yavuz Akbulut Prof. Dr. Mehmet Kesim Anadolu University Eskişehir, Turkey

2 Introduction In the 21st century, skills regarding information and communication technologies (ICTs) have gained utmost importance for education, for employment and for everyday life use. ICTs provide a more colorful atmosphere in comparison to face-to-face learning to access global knowledge and support. The ability to use ICTs with confidence and efficiency is demanded from most individuals. In this respect, it is important to equip individuals with skills to use ICTs efficiently, independently and responsibly.

3 Survey of European Universities Skills in ICT of Students and Staff ICT (SEUSISS) students, staff and employers across Europe responded students, staff and employers across Europe responded ICT was considered vital for their future careers by the great majority of the participating students (Haywood, 2003) ICT was considered vital for their future careers by the great majority of the participating students (Haywood, 2003) ICT skills of students from different universities varied in accordance with some variables including age, gender, academic subject domains, year of study, family support, and the richness of facilities offered to students.. ICT skills of students from different universities varied in accordance with some variables including age, gender, academic subject domains, year of study, family support, and the richness of facilities offered to students.. However, in order to examine the problem better, there needs to be efficient, timely and meticulous measurement criteria and tools so that the ICT picture of a specific organization or community could be diagnosed better.

4 Since ICTs in teacher education constitute a dynamic field of study, which requires constant refreshment, there is always a need to measure up-to-date latent constructs of ICTs through valid and reliable tools including high quality indicators. Some measurement tools Lin (2005): ICT integration factors that are considered important for teachers Lin (2005): ICT integration factors that are considered important for teachers Liu and Huang (2005): Concerns of teachers regarding ICT integration Liu and Huang (2005): Concerns of teachers regarding ICT integration Akpınar (2003): K-12 teachers’ technology using behaviors Akpınar (2003): K-12 teachers’ technology using behaviors

5 Purpose The current study has a different scope from above studies for it purports to develop a measurement tool for diagnosing the overall picture of an institution regarding internal and external ICT indicators from prospective teachers’ points of view.

6 Indicators of Successful ICT Integration UNESCO (2002) & Odabasi et al. (2006) content and pedagogy issues content and pedagogy issues collaboration and networking issues collaboration and networking issues social and health issues social and health issues technical issues technical issues

7

8 Methods & Procedures Research Questions What are the valid and reliable indicators to diagnose the overall ICT picture of an educational institution and its members from prospective teachers’ perspectives? What is the current situation of the selected university in terms of ICT indicators?

9 Participants 359 senior education faculty students (% 43 of all education faculty students at Anadolu University)

10 Instrument A personal information form collecting independent variables needed for further analyses Six to ten statements for each indicator Revision by an IDT instructor and 8 PhD candidates Expert opinions (2 professional test developers) Number of items after evaluations: 54 5-item scales: Never, rarely, sometimes, very often and always

11 Procedure Written permissions from the rectorate and the human subjects committee of the institution Administration of the survey during students’ normal class hours at the second half of 2006 fall semester. Same type of instructions to all participants 15 to 20 minutes to complete the questionnaire

12 Data Analysis An exploratory factor analysis through SPSS 14.0 for Windows A confirmatory factor analysis through LISREL 8.51 Descriptive statistics Parametric tests of comparison

13 Results (1 of 5) Kaiser-Meyer-Oklin Measure of Sampling Adequacy (i.e..848) Bartlett’s Test of Sphericity (χ²= ; p<.0005) Values were ideal to conduct factor analysis (Hutcheson & Sofroniou,1999; Pallant, 2001).

14 Results (2 of 5) Items with inappropriate loadings (i.e. less than.40) were eliminated. Items with inappropriate corrected-item total values (i.e. less than.30) were eliminated one by one, and further factor analyses were conducted each time. Final solution with the maximum likelihood extraction method revealed 41 items indicating 10 factors with eigenvalues exceeding 1, and explained % of the total variance. This was considered appropriate according to the literature (Dunteman, 1989). Items were further processed through LISREL 8.51 (Jöreskog & Sörbom, 2001) to conduct a confirmatory factor analysis. Three further questions were eliminated during the confirmatory analysis and the final model revealed an ideal root mean square error of approximation (RMSEA) value(i.e.,.05) along with ideal fit indices (non-normed fit index:.89; comparative fit index:.90; incremental fit index:.90) (Raykov & Marcoulides, 2006).

15 Results (3 of 5) The Cronbach’s Alpha (  ) of the instrument was.888 Ten factors were labeled as the following (1) Ease of use (eigenvalue: 10,173) (1) Ease of use (eigenvalue: 10,173) (2) Teaching – learning method (eigenvalue: 4,297) (2) Teaching – learning method (eigenvalue: 4,297) (3) Ethics (eigenvalue: 3,504) (3) Ethics (eigenvalue: 3,504) (4) Special needs (eigenvalue: 1,905) (4) Special needs (eigenvalue: 1,905) (5) Infrastructure (eigenvalue: 1,616) (5) Infrastructure (eigenvalue: 1,616) (6) Professional development (eigenvalue: 1,574) (6) Professional development (eigenvalue: 1,574) (7) Access (eigenvalue: 1,338) (7) Access (eigenvalue: 1,338) (8) Health (eigenvalue: 1,216) (8) Health (eigenvalue: 1,216) (9) Policy (eigenvalue: 1,089) (9) Policy (eigenvalue: 1,089) (10) ICT in curriculum context (eigenvalue: 1,047). (10) ICT in curriculum context (eigenvalue: 1,047). Technical assistance and learning communities were not revealed through the analysis

16 Results (4 of 5) Means on each factor were calculated and these values were compared with the neutral value of 3 through one- sample t-tests as done in Warschauer (1996) Since ten t-tests were conducted, a Bonferroni Adjustment Procedure was applied as suggested by Huck (2000), and the critical p-level was determined as.005.

17 Results (5 of 5) No difference between the results and the neutral value of 3 ICT in curriculum context (  =2.982; t357=-.429; p=.668) ICT in curriculum context (  =2.982; t357=-.429; p=.668) Results were better than expected Ethics (  =3.664; t357=16.138; p<.0005) Ethics (  =3.664; t357=16.138; p<.0005) Professional development (  =3.508; t357= ; p<.0005) Professional development (  =3.508; t357= ; p<.0005) Health (  =3.975; t357=17.291; p<.0005). Health (  =3.975; t357=17.291; p<.0005). Results were worse than expected Teaching-learning method (  =2.636; t357=-8.664; p<.0005) Teaching-learning method (  =2.636; t357=-8.664; p<.0005) Ease of use (  =2.175; t357= ; p<.0005) Ease of use (  =2.175; t357= ; p<.0005) Special needs (  =1.884; t357= ; p<.0005) Special needs (  =1.884; t357= ; p<.0005) Infrastructure (  =2.657; t357= ; p<.0005) Infrastructure (  =2.657; t357= ; p<.0005) Access (  =1.916; t357= ; p<.0005) Access (  =1.916; t357= ; p<.0005) Policy (  =2.162; t357= ; p<.0005). Policy (  =2.162; t357= ; p<.0005).

18 Discussion (1 of 5) The present paper suggests a measurement tool for examining ICT indicators. The study set out with 12 factors and more than a hundred statements at the beginning and finished with 10 predetermined factors indicated by a total of 38 questions. Indicators of technical assistance were contaminated by indicators of access, ease of use and infrastructure. Since these subcategories all belonged to the construct of technical issues as mentioned in the literature review, this was not an extraordinary finding. None of the items addressing learning communities had appropriate loadings to be included in the final version of the scale. This could be explained by two possibilities. Items addressing learning communities might have been badly prepared. Items addressing learning communities might have been badly prepared. Turkish students might lack skills to work properly in learning communities, which leads to serious results. Turkish students might lack skills to work properly in learning communities, which leads to serious results.

19 Discussion (2 of 5) The analysis conducted with a single sample in the current study demonstrates first-order relationships between sub- categories of UNESCO (2002)’s four competencies and observed variables. Further analyses with new samples could be conducted to see the relationships between four competencies and subcategories, and between subcategories and observed variables, which can reveal second-order relationships. Moreover, through administering the current scale across different populations, the tool might be developed further, so that structural equation models can be suggested among reliably measured constructs.

20 Discussion (3 of 5) Means of each factor reveals that the sample institution where the data collection took place is at an average level in terms of ICT in the curriculum context. The institution is at a better situation than the neutral value in terms of ethics, professional development and health. However, the institution needs immediate action in terms of teaching-learning method, ease of use, special needs, infrastructure, access and policy.

21 Discussion (4 of 5) Since the current data were collected from an opportunity sample which was a single Turkish state university, generalizations to other state universities based on the current dataset could only be suggestive rather than definitive. Besides, the current scale investigates ICT indicators from senior students’ points of view. A parallel form of the questionnaire could be developed for instructors to investigate their perspectives in terms ICT indicators.

22 Discussion (5 of 5) As mentioned before, the target population of the current study was senior students of the Faculty of Education at Anadolu University. The scale should be used with larger samples across different faculties aside from the education faculty to develop its construct validity and generalize the results to a larger reference population. Researchers intend to administer the scale across all education faculties in Turkey, which might lead to comprehensive data on the ICT integration level of educational faculties and prospective teachers. Besides, collaboration among European universities within the scope of a joint project like SEUSISS (Haywood, 2003) to investigate ICT indicators might lead practitioners to invaluable information about the ICT situation across European universities

23 Thank You !!!