1. The Information Systems Analyst National Assessment Exam: Factors for Success Mark Segall (segallm@mscd.edu) Loren Gollhardt (gollhard@mscd.edu) Joe Morrell (morrellj@mscd.edu) Computer Information Systems, Metropolitan State College of Denver (MSCD)

2. ISA Exam Pedigree

3. ISA Exam Content Maps to the IS 2002 model curriculum for undergraduate Information Systems Reports on 6 core areas –Analysis and Design –Role of IS in Organizations –Data Management –Networking and Telecommunications –Modern Programming Language –Hardware and Software

4. ISA Exam Mechanics Student register for exam to receive a password Delivered over the internet to a proctored testing site Three hour exam with 258 Questions Score reported upon completion of exam ISA certification: 50% or above ISA master certification: 70% or above

5. ISA Exam Functionality For departments: A third party assessment exam for IS majors For students: A possible certification

6. MSCD Exam Summary Core Area# of ItemsAll SchoolsMSCD Modern Programming Language 1240.944.8 Hardware and Software 1041.847.3 Networking and Telecommunications 1245.145.3 Data Management 4446.051.2 Analysis and Design 10847.551.2 Role of IS in Organizations 7252.056.3

7. Research Question What are the characteristics for success on the ISA Exam? academic factors work experience personal factors environmental factors

8. Research Question What are the characteristics for success on the ISA Exam? Academic factors –Cumulative GPA –Major GPA –Subject –Course Number –Credit Hours –Grade –Transfer Credits

9. Research Question What are the characteristics for success on the ISA Exam? Work experience –Company type –Years of experience in line of work –Job Title –Job Skills –Amount of Work (Full Time, Part Time, None)

10. Research Question What are the characteristics for success on the ISA Exam? Personal factors –English as a second language (ESL) –Self rated emotional control –Self rated general problem solving ability –Age –Married –Number of minor children living with you –Gender

11. Research Question What are the characteristics for success on the ISA Exam? Environmental factors –How many hours did you prepare for this test? –How did you feel physically at test time? –Rate your own emotional state at test time. –How many credit hours were you taking the semester in which you took the test?

12. Sample Characteristics N = 140 students Fall 2004 to Spring 2006 Average ISA score: 49.7 ± 12.2 SD Average age: 30.1 ± 8.1 SD Male / Female Ratio: 68% / 32% ESL: 15.7%

13. Sample Characteristics Caucasian 63% Asian 17% Hispanic 6% Black 5% Other 5% Non Resident Alien 4%

14. Sample Characteristics Overall GPA: 3.03 ±.52 SD Major GPA: 2.97 ±.59 SD Credits / semester: 9.9 ± 2.39 SD Transferred courses: 16.6 ± 14.4SD –maximum of 73 –74% have transfer courses –1 to 2 semesters worth of transfer courses

15. Sample Characteristics Survey return rate: 34% (48 out of 140) Average age –Completed survey: 30.1 years –Did not complete survey: 30.1 years ISA exam score –Completed survey: 53.1 ± 12.7 SD * –Did not complete survey: 48.0 ± 11.7 SD GPA –Completed survey: 3.1 –Did not complete survey: 3.0

16. Sample Characteristics Work Status –Full time:69% –Part time:19% –None:12% Company Type –Information Technology 31 –Student worker 19 –Retail13 –Government 8 –Banking & Finance 8 –Other (7 Industries)21

17. Sample Characteristics Years of experience –Less than one year 14 –1 – 2 years 19 –3 – 5 years 21 –More than 5 years 42 –Not Applicable 4

18. Results: Personal Factors PredictornSlopeR 2 ValueF- scoreP-value Gender140-2.20.7%1.00.331 Age1400.263.0%4.30.040* ESL140-12.213.3%21.20.000**

19. Results: Personal Factors

21. Factors that are not significant –Self rated IS skill level –Self rated emotional control –Self rated problem solving ability –Marital status –# of children

22. Results: Academic Factors CourseTitle % of LU covered by Course CMS 2010 Computer Applications for Business31 CMS 2110 Business Problem Solving: A Structured Programming Approach37 CMS 3050 Fundamentals of System Analysis and Design31 CMS 3060 Database Management Systems23 CMS 3145 Business Application Development with Visual Basic41 CMS 3230 Telecommunication Systems9 CMS 4050 Advanced Systems Analysis and Design91 Learning Units (LU) covered by the ISA exam: 150

23. Results: Academic Factors PredictornSlopeR 2 ValueF- scoreP-value GPA14010.720.6%35.80.000** Major GPA 1409.320.0%34.60.000** CMS 4050 Adv. SAD 1249.538.0%74.90.000** CMS 3230 Telecom 1345.818.3%29.60.000** CMS 2110 Bus Prob. Solving 1226.017.3%25.20.000**

24. Results: Academic Factors

25. PredictornSlopeR 2 ValueF- scoreP-value CMS 3060 Database 1394.19.7%14.70.000** CMS 2010 Comp App. Bus. 1123.35.0%5.40.018* CMS 3340 Statistics 1343.04.4%6.00.015* CMS 3050 Fund. SAD 422.86.1%2.60.115 Programming 1381.00.9%1.20.274

26. Results: Environmental Factors PredictornSlopeR 2 ValueF- scoreP-value Physical state 481.910.8%5.560.023* Emotional State 483.018.4%10.40.002** Zero hours prep 4811.018.0%10.10.003** 1-5 hours prep 48-3.91.9%0.870.355 6+ hours prep 48-17.019.8%11.30.002**

27. Results: Environmental Factors Factors that are not significant –Self rated motivation level –Credit hours this semester

28. Results: Work Factors Factors that are not significant –Type of company (IT, Student, Retail) –Work experience (< 1 year, 1-2 years, 3-5 years, more than 5 years) –Time working (Full time, Part time, None)

29. Summary ESL has a substantial negative impact Age has weak positive effect Academic performance (GPA) is a positive predictor

30. Summary Performance in Advanced Systems Analysis & Design is the best predictor of performance on the ISA Exam Telecommunications, Structured Programming / Problem Solving, and Database courses are all substantial predictors General courses in Business Information Systems and Business Statistics are weak predictors

31. Conclusion Academic, personal, and environmental factors have an impact on ISA Exam scores. There is a strong relationship between grades in specific courses and performance on the ISA exam.

32. Conclusion These results are from students in just one CIS Department taking the exam over a period of two years. This study should be expanded to all CIS Departments in all institutions using the ISA Assessment Exam. This is both an invitation and a challenge for those institutions to join us in this endeavor.

33. Co Authors Loren GollhardtJoe Morrell