1. Why can’t we be friends: Evaluating professional development fellowship programs with SNA
AEA Conference | November 10, 2017
Evgenia Valuy | Evaluation and Learning Specialist, IIE |

2. Centroamérica Adelante (CAA) Goals
Support professional skills and leadership development of social change leaders
Provide opportunities to strengthen national and regional professional networks
Enable participants to have greater impact on social change
AEA: Why can’t we be friends

3. CAA Evaluation Methodology
Utilization- focused
Primary users: Foundation sponsor & IIE program management team.
Secondary users: Program participants.
Participatory
Primary evaluation users contribute to M&E planning, instrument design, and interpretation of findings.
Longitudinal
4 data collection points: before, during, immediately after, and 10 months after the program.
Mixed methods
3 surveys: pre, post, alumni.
4 focus groups: midterm.
14 interviews: post.

4. Integration: SNA and M&E Approach
Why it matters?
Measures network creation and growth
Perfectly fits the second program goal of network increase
Exact measurement of current network
Aligns with the need for comparable data at multiple data collection points
Measures structure or pattern of the network
Allows us to measure changes over time in national and regional networks as a whole
AEA: Why can’t we be friends

5. Integration: SNA and Data Collection
Why it matters?
With groups under 100 can create a roster for a survey
Best fit for a small, personalized professional development program.
Can be valued – include rating of possible relationships
Allows for progressive impact: varying stages of networking and depth of relationship over time
AEA: Why can’t we be friends

6. List of names

7. Step back: what does it look like?
Before:
2 collaborations
Right after:
64 collaborations
10 months after:
178 collaborations
AEA: Why can’t we be friends

8. Integration: SNA and Data Analysis
Why it matters?
Has 3 levels of analysis:
Unit is a pair
Unit is a person
Unit is the network
Our primary unit of analysis was each participant. Can paste them in SPSS and treat as a regular variable
SNA allowed to better look at network as a unit. Local team contribution to the analysis: historic, sociopolitical context in the region and each country (network units).
AEA: Why can’t we be friends

9. Challenges to Integrating SNA
Program goals. SNA is a cool new methodology, but networking should be your active program goal/component if you are to use it.
Data collection. Larger cohorts cannot be analyzed as one group using a roster. The larger the list, the harder to get response rates.
Participation expectations. Need to explain and get commitment from program sponsor and management on discussion of sociograms can be used to further the networks.
Team training. It is a new methodology, and it is a strain on resources ($$, time) to train everyone on the team to apply it.

10. Key Takeaways 1 3 2 4 Evgenia Valuy evaluy@iie.org
Get training. Understand SNA concepts and how you may apply the methodology to your projects. Pick one project to start with.
Plan analysis integration. Consider how SNA analysis process can be integrated into your existing analysis process. Plan how to transform SNA data accordingly. 2 4
Plan data collection in detail. SNA requires intensive data collection, and you cannot easily modify it half- way through.
Discuss SNA opportunities with other stakeholders beforehand. Go beyond getting their buy-in on using the methodology and discuss their involvement.
Evgenia Valuy
Monitoring, Evaluation and Learning Unit
Institute of International Education iie.org