Quality Assurance using International Curricula and Employer Feedback Marta Lárusdóttir Reykjavik University, Iceland Mats Daniels Uppsala University, Sweden Roger McDermott Robert Gordon University, UK

Identify strengths & weaknesses Certify fulfillment Sorting 2 Quality Assurance Why?

National agency Institution/Department External organizations –ABET –ENQA 3 Quality Assurance By Whom?

Outcome based –Abilities of graduates Process based –Features of learning environment 4 Quality Assurance How?

Value for society Disciplinary Learning 5 Quality Assurance What?

Mats – Member of Swedish evaluation Marta – Planning evaluation of BSc CS program in Iceland 6 The Reykjavik Example Starting Point

National degree criteria Issues –Quite abstract –Competencies are complex –Disciplinary content unclear –Functioning in a work environment 7 The Reykjavik Example What to Evaluate?

Map national criteria to skills (ACM Curricula 2013) –“capable of interpreting and presenting scientific issues and research findings” –“Graduates should have the ability to make effective presentations to a range of audiences about technical problems and their solutions. This may involve face-to- face, written, or electronic communication. They should be prepared to work effectively as members of teams. Graduates should be able to manage their own learning and development, including managing time, priorities, and progress.” ACM/IEEE 2013, p.22 8 The Reykjavik Example Coping with “abstract & complex”

ACM Curricula 2013 norm –Knowledge Areas 18 Tier-1 Tier-2 Elective Curricular hours 9 The Reykjavik Example Unclear Disciplinary Content

Process oriented Course focus Carried out by faculty –Educational –Community creating 10 The Reykjavik Example So Far

Characteristics of CS graduates –At end of degree program –11 specified in ACM Curricula 2013 –Overarching several knowledge areas –Professional competencies 11 The Reykjavik Example Functioning in the Work Place

Outcome based Ten companies Characteristics of CS graduates –Interviews –Questionnaire 12 The Reykjavik Example Employer Assessment

Course based –Topic coverage –Learning outcome coverage –Characteristics coverage 13 The Reykjavik Example Data: Educational Setting and Delivery

Graduate based –Characteristics coverage –Capture “real” output 14 The Reykjavik Example Data: Educational Delivery

Knowledge areas –Topics Overall too low coverage –Six of 18 are barely covered in tier-1 –Four of 18 are barely covered in tier-2 –The two new in 2013 were not covered “non-technical” worse –Learning outcomes were slightly worse 15 The Reykjavik Example Findings: Educational Setting

Characteristics covered –All eleven areas were covered –Commitment to professional responsibility was not a core competency anywhere 16 The Reykjavik Example Findings: Educational Setting

From employers –Four of the eleven characteristics were deemed clearly below how important they were seen by employers Familiarity with common themes and principles Problem solving skills Commitment to life-long learning Commitment to professional responsibility 17 The Reykjavik Example Findings: Educational Delivery

It takes time to conduct ACM CS Curricula 2013 useful Faculty positive Combination of process and outcome based worked well Areas of improvement were identified The study is limited in size and local Measures are not “validated” 18 Conclusions

Anna Ingolfsdóttir Henning Ulfvarsson Faculty at the department in Reykjavik 19 Acknowledgements

Questions? Hope you’ll look like this….