Measuring the Impact WP5 & WP9 ECHO Utrecht / KinderUni Wien Hendrik Asper (ECHO)

ECHO Established in 1994, an independent NGO since 10 years. Initiatives for social inclusion and access to Higher Education. Advising on policy and implementation of strategic goals of the ministry of education. Working together with the ministry, institutions of higher education, municipalities and community organizations. Disseminating knowledge and best practices on social inclusion. 12 years of ECHO awards has created a close network of over 200 hundred first generation students and young professionals. Involved in national projects regarding the benchmarking and tracking of HE students (access, attrition and success) and the design of performance indicators. Quantitative and Qualitative analyses regarding the participation of underrepresented groups for regional/national governments, (HE) institutions and companies.

Local Defined Minorities LDM captures the diversity of ways in which under- represented groups are understood at the national, regional, city level. LDM is based in the differing socio-demographic nature of local areas. A likely starting point is that the children would be one of the first in their family to enter higher education, but this would not have to be the case for all children. The LDM can be very different depending on the SIS activity.

WP5/9 & MQT Developing a SIS-wide framework for: –Identifying and raising awareness of underrepresented groups in their respective locale (locally defined minorities) –Benchmarking the participation rate of underrepresented groups in children’s activities and HE education –Surveying and charting the children participating in the activities. –Setting up a sustainable and comparative monitoring system for SiS activities, measuring long-term impact

Goals monitoring framework The monitoring framework provides institutions and organisations insights into their commitment and readiness (newcomers) to engage children and young people into science, with a specific focus on the inclusion of underrepresented groups (LDM). This framework is determined by a set of criteria (MQTs), or dimensions, which reflects the sustainability of the models. The MQT sections are (1) Strategic development & embedding, (2) Cooperation & networking, (3) Engagement target groups, (4) Academic impact and (5) Evaluation and monitoring. Each MQT section consists of a set of measurable indicators.

Stakeholders Organisations and institutions organising the children’s activities: –Comparing the approach to children’s activities –Best practices –Securing budget and sponsoring Policy-makers (institutional or governmental) –Raising awareness –Budgetting Researchers Parents –Limited due to the regionality gfsffga

Monitoring Framework Toolbox Raising awareness: background documentation on Locally Defined Minorities and underrepresented groups. –Combined with the SIS practical guide for newcomers General data on the participation of underrepresented groups Measuring the Impact Together with institutions and policymakers : institutional benchmarking on readiness, engagement and sustainability With organizers and children : monitoring of activities Long term : the effect of children’s activities

Children activities Underrepresented groups Evaluation and monitoring Academic impact Engagement target groups Cooperation and networking Strategic development and embedding Background institutions Self-assessment and benchmarking

MQT principles (based on U-map) Empirical data, reliable and verifiable User-driven comparisons Multiple perspectives Applicable to providers of SiS-activities Non-hierarchical dimensions Efficient, comparable, data collection Challenges Participation of HE-policymakers and organizers : what information do we need, can we attain, can we compare on a EU level (best practices) Realization of a sustainable long term measuring of impact (quantitative and qualitative)

Monitoring framework

Strategic development and embedding Focus strategic development plan and children activities Alignment of agenda’s (national, regional, local) Financial sources & in kind contributions HRM: specific capacity, trainings, permanent staff Specific facilities

Cooperation and networking To realize children activities To get in contact with underrepresented groups To exchange knowledge To get an accreditation Use of (targeted) communication tools

Engaging target groups Participation and relationship (underrepresented) children Role of families, use of role modeling concepts Social & political context Unintended spin offs

Academic impact Change of learning behavior children Change of curricula, pedagogy Putting themes on the agenda

Evaluation and monitoring Evaluation activities Data collection background children and parents Monitoring goals program Analyzing and comparing (benchmarking)