1. 2014 AmeriCorps External Reviewer Training
Assessing Need, Theory of Change, and Logic Model of Tier 1 Education Applications
2014 AmeriCorps External Reviewer Training

2. Topics Purpose of Logic Model
Assessing Need, Theory of Change, and Logic Model

3. Preview of Assessment Questions
(True/False)
The Logic Model is a visual representation of the applicant’s Theory of Change
External Reviewers should assess whether the outcomes in the Logic Model are measurable
(Multiple Choice)
Data about the number of economically disadvantaged beneficiaries in a community would most likely describe:
(Practice Your Understanding)
Complete portion of Review Assessment Form

4. Logic Model
What is a Logic Model?
A visual summary or snapshot of your program that communicates how your program works, the resources you have to operate your program, the activities you plan, and the outcomes you hope to achieve
A well-specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice and describes the relationships among the key components and outcomes, theoretically and operationally (Notice of Funding Opportunity (NOFO) p. 18)
The purpose of a Logic Model is to describe how a program will create change.

5. Certain resources are needed to operate your program
Logic Model
How a Logic Model Works
Figure 1: How to Read a Logic Model
If these benefits to participants are achieved, then certain changes in organizations, communities, or systems might be expected to occur
If you accomplish your planned activities, then you will hopefully deliver the amount of product and/or service that you intended
If you accomplish your planned activities to the extent you intended, then your participants will benefit in certain ways
If you have access to them, then you can use them to accomplish your planned activities
Certain resources are needed to operate your program
Resources/ Inputs
Activities
Outputs
Outcomes/ Impacts
Long-Term Goal j k l m n
Your Planned Work
Your Intended Results

6. Applicant Logic Models
Inputs – Resources, including AmeriCorps members
Activities (Interventions)
Outputs (Beneficiaries, other products of program services)
Outcomes
Short Term (Knowledge, Skills, Attitudes, Opinions)
Medium Term (Behavior)
Long-Term (Condition)

7. Logic Model
Key Points
The Logic Model and narrative work together to depict the program’s Theory of Change
All the information in one doesn’t have to be in the other, as long as the two are aligned
The Logic Model is simply a visual representation of the Theory of Change and program design
Applicants do not measure all outcomes in Theory of Change; they continue to measure outcomes that make sense for their program; External reviewers do not assess applicants’ performance measures and so should not be concerned with whether outcomes are measurable
CNCS provided a template for applicants’ Logic Models

8. Theory of Change Elements
Assessment: Theory of Change
Theory of Change Elements
Community Problem/Need
Specific Intervention
Intended Outcome
Statistics documenting the need
Evidence
Guides choice of intervention
Supports cause-effect relationship

9. How Do The NOFO Pieces Fit?
Assessment
How Do The NOFO Pieces Fit?
Applicant’s Overall Theory of Change
Need narrative
Theory of Change narrative
Logic Model

10. The Selection Criteria & Standards
Assessment
The Selection Criteria & Standards
Four Key Themes
Completeness (Logic Model)
Alignment/Logic
Quality of Theory of Change
Level/Quality of Evidence

11. Review Rubric Most items scored on a five-point scale
Assessment
Review Rubric
Most items scored on a five-point scale
Some items scored on a two-point scale (Satisfactory/Unsatisfactory)
Refer to the Review Rubric in the handbook to read the descriptions for each Rating

12. Completeness (Logic Model)
Assessment
Completeness (Logic Model)
Number of locations and sites where members are providing services
Number of members that will be delivering the intervention
Core activities that define the intervention or program model
Duration of the intervention
Dosage of the intervention
Target population for the intervention

13. How Assessed Series of check boxes on the review form
Assessment
How Assessed
Series of check boxes on the review form
Reviewers check the box if the item is present in the Logic Model

14. Eligible Intervention(s) in Logic Model or Narrative
Assessment
Eligible Intervention(s) in Logic Model or Narrative
Reviewers will also indicate whether or not the applicant’s Logic Model and/or Narrative incorporates at least one of the following six required member interventions for Tier 1 Education applicants (listed on this slide and the next slide):
1) Providing ongoing mechanisms for family and community engagement
2) Establishing a school culture and environment that improves school safety, attendance, and discipline and addresses other non-academic factors that impact student achievement, such as students’ social, emotional, and health needs
3) Accelerating students’ acquisition of reading and mathematics knowledge and skills

15. Eligible Intervention(s) in Logic Model or Narrative
Assessment
Eligible Intervention(s) in Logic Model or Narrative
4) Increasing graduation rates through strategies such as early warning systems, credit-recovery programs, and re-engagement strategies. 5) Increasing college enrollment rates through college preparation counseling assistance to include completing the Free Application for Federal Student Aid (FAFSA) and college applications, and educating students and their families on financial literacy for college 6) Supporting school implementation of increased learning time

16. Alignment with Logic Model
Assessment: Theory of Change
Alignment with Logic Model
The Theory of Change is described consistently in the Logic Model and application narrative
The inputs, activities and outputs are logically aligned
The Theory of Change and Logic Model cover comprehensively the applicant’s entire program

17. Quality of Theory of Change
Assessment: Theory of Change
Quality of Theory of Change
Problem
The applicant cites specific, relevant data to document the needs that the AmeriCorps members will be addressing and these needs were identified with the involvement of eligible partner school(s) and LEA leadership
The need in community and in the school(s) where the proposed AmeriCorps members will serve is prevalent and severe
Intervention
The potential contribution of AmeriCorps members is an effective way to address the needs identified by eligible school and LEA leadership
The project effectively addresses multiple student needs
Outcomes
The outcomes identified in the Logic Model are logically aligned with the problem/need and intervention

18. Assessing Alignment and Quality
Assessment: Theory of Change
Assessing Alignment and Quality
Alignment and Quality are assessed using a five-point scale:
Greatly exceeds the standard
Exceeds the standard
Meets the standard
Partially meets the standard
Does not meet the standard at all
Refer to the Review Rubric to read the descriptions for each Rating.

19. Need vs. Evidence Applicants are asked to provide data to demonstrate:
Assessment: Need
Need vs. Evidence
Applicants are asked to provide data to demonstrate:
Prevalence and severity of need versus the level and quality of evidence
Specific, relevant data to document the need
Evidence to support the choice of intervention

20. Community and School Problem/Need
Assessment: Need
Community and School Problem/Need
Data documenting problem/need should demonstrate the prevalence and severity of the problem:
Who and how many are directly affected?
How severe is the problem?

21. Examples Graduation rate
Assessment: Need
Examples
Graduation rate
Percent of students eligible for free/reduced lunch
Student academic achievement data
Data about the school culture or environment
Data about family engagement
Student attendance data

22. Assessment: Need
Data Should Be
From a reputable source such as a government agency, foundation, university or research organization
Recent
Local

23. Assessment: Need
Common Issues
State or national data is provided instead of local data describing the need in the specific communities where the service will occur
Statistics are unrelated to the problem/need
Statistics cited are not recent
Data sources are not identified or are not reputable

24. Next Steps
To check for understanding and verify that you have completed this orientation session, please reference the remaining slides for the two-part Assessment of this Orientation.

25. Practice Your Understanding
Assessment: Practice
Practice Your Understanding
Answer the Assessment Questions at the following link:
Read and assess the norming/sample application and complete the Rating sections for the Problem/Need, Theory of Change, and Logic Model sections of the review form. Do not include significant strengths, weaknesses, or other narrative sections.
You will receive a review form completed by a CNCS staff member. Compare your assessments to those in the key.
This will complete your verification requirement for this training module.

26. Assessment: Practice
Practice
SELF-ASSESSMENT When you receive the key for the practice activity from CNCS, compare your Ratings to those of the CNCS standard example. If your Rating was more than one Rating higher or lower than the example provided by CNCS, your assessment may not be normed closely enough to CNCS’s expectations for applicants