1. Office of Adolescent Health Office of Grants Management
Federal Reporting: Guidance for Completing Your Annual Progress Report & Requesting Carryover Funds
Good afternoon, everyone. Thank you for joining us for today’s webinar on completing your annual progress report and requesting carryover funds. We hope that you will find the information helpful and that many of your questions will be answered. Of course, if you still have questions at the end of this presentation, you can contact your Project Officer or your Grants Management Specialist for additional guidance.
Office of Adolescent Health
Office of Grants Management
September 12, 2011

2. Webinar Objectives
Describe the content and submission requirements for completing the OAH Annual Progress Report and Financial Status Report.
Summarize the process and requirements for requesting to carry over unobligated funds from Year One to Year Two.
The objectives for today’s webinar are to:
Describe the content and submission requirements for completing the OAH Annual Progress Report and Financial Status Report; and
Summarize the process and requirements for requesting to carry over unobligated funds from Year One to Year Two.
Office of Adolescent Health

3. OAH Grantee Reports Three primary reports due each year:
Non-competing continuation application
May 31, 2011
Annual progress report
November 30, 2011
Annual financial status report
OAH has three primary reports due each year from all grantees: a non-competing continuation application and two annual reports. All are important reports with specific guidelines to follow and regular due dates. Today, we are going to focus on the annual progress report and the annual financial status report. We will also spend some time talking about carryover funds from year 1 to year 2.
Office of Adolescent Health

4. Annual Progress and Financial Reports
Program Progress Report
12-month progress report (September 1, 2010 – August 31, 2011)
Describes the completion of objectives and activities for the entire 12 months of the year 1 budget period
Success story
Financial Status Report
SF-269
The annual progress report consists of the 12-month progress report for the project year that was most recently completed. The progress report should assess progress in achieving the stated objectives and activities for the entire 12 months of the year 1 budget period. The progress report should include an update on both programmatic and evaluation objectives and activities. You should also include one or more detailed success stories in your annual progress report.
The annual financial status report consists of the SF-269 form. This form will allow you to report on your grant expenditures for the entire year.
Office of Adolescent Health

5. Expectations for 12-Month Progress Report
Provide an update based on the objectives of your program.
Focus on the entire 12-month period of September 1, through August 31, 2011.
Describe major accomplishments.
Describe any challenges/barriers you encountered and how they were addressed.
If applicable, include the reasons that goals or objectives were not met and a discussion of assistance needed to resolve the situation.
Report on any other significant project activities, accomplishments, setbacks or modifications (e.g., change in key staff, change in scope of work) that have occurred in the current budget period.
Provide a clear and detailed update on the progress of your program and the achievement of your objectives. This report should cover the entire 12-month period of time from September 1, 2010 through August 31, This includes the initial 6 months that were reported on through the continuation application.
(2) Describe major accomplishments. Include sufficient detail that anyone picking up the report could understand what you have been doing and what has been accomplished. Be sure to include statements that include the outcomes of your actions.
(3) Describe any challenges or barriers you encountered and how they were addressed.
(4) If applicable, include reasons that goals and objectives were not met and a discussion of assistance, if any, needed to resolve the situation.
(5) Report on any other significant project activities, accomplishments, setbacks or modifications (e.g., change in key staff, change in scope of work) that have occurred in the current budget period and were not part of the program work plan. These should include legislative and/or judicial actions impacting the program, as well as agency events.
Office of Adolescent Health

6. Expectations for 12-Month Progress Report
Include sufficient detail that any one picking up the report could understand what you have been doing and what has been accomplished.
Be sure to include challenges faced; brainstorm ideas to overcome those challenges.
No specific length is required– just a solid level of detail and depth.
Recommended template is not required; the requirement is to cover the bulleted points.
Office of Adolescent Health

7. Example of Progress Report
EXHIBIT B: Example Twelve-Month Progress Report (Partial)
Grantee X; Grant #:xxxxx
September 1, 2010 – August 31, 2011
Goal: Replicate xxx evidence-based program in 60 sites across xxx County.
Objective:
By August 31, 2010 ensure all facilitators are trained in the xxx evidence-based program model.
Met
By the end of the first grant year, we trained all 60 facilitators in the xxx evidence-based program model. We identified organizations that were certified to conduct training on the evidence-based program, had a conversation with each organization about the content and cost of their training, selected and entered into an agreement with xxx organization to conduct our trainings, and have conducted four facilitator trainings. We offered the same training four times to provide options in the location and timing of the training and to limit each training to no more than 15 participants.
Activity:
Identify and secure a trainer to conduct training on xxx evidence-based program.
We identified three organizations that were certified to conduct trainings in xxx evidence-based program. We contacted each organization to learn more about the content and cost of their training. Each organization offered a 3-day training, but one organization also included 20 hours of follow-up technical assistance in their training plan. The cost estimates from the three organizations were similar. We decided that having the 20 additional hours of technical assistance from the trainer would be beneficial since this is a new program for all of our facilitators, therefore we selected xxx organization. We signed a contract with xxx organization to conduct four identical 3-day trainings for our facilitators and to provide 20 hours of follow-up technical assistance. It was agreed that our organization would take care of the logistics and registration for each training.
Conduct four, 3-day trainings in the xxx evidence-based program for program facilitators.
Training dates and locations for four 3-day trainings were secured:
March 22-24, 2010 at the xxx community organization in City
April 14-16, 2010 at the xxx community organization in City
May 2-4, 2010 at the xxx community organization in City
May 20-22, 2010 at the xxx community organization in City
Trainings were advertised to the 60 facilitators who are implementing the xxx evidence-based program. Each training includes an overview of the program model, core components, and teaching philosophy; a detailed review of the activities included in the program; time for each participant to practice delivering the program activities; review of the fidelity monitoring tools; discussion about allowable adaptations; and review of the available evaluation tools (see Appendix A – Training Agenda). Training participants completed an evaluation form after the training. Results have been analyzed indicate that facilitators are confident in their ability to implement the program with fidelity as a result of the training.
Office of Adolescent Health

8. Success Stories
One or more success stories to demonstrate impact of activities over the past 12 months.
 Grant-funded activities that have resulted in positive changes for young people during the past year.
Stories from this past 12-month period may focus on your successful efforts to plan, pilot, and raise awareness of the program within your target community.
Grantees are encouraged to provide one or more success stories with the annual progress report to communicate the impact of activities during the latest budget period. Success stories are critical in helping educate decision makers about the impact of your program, demonstrating responsible use of resources, sharing best practices with other similarly-funded programs, and attracting new partners for collaboration.
The success story should describe your grant-funded activity or activities that have resulted in positive changes for young people during the past year. The focus of your submitted story each year will change over time as your program expands and evolves. Stories from this past 12-month period may focus on your successful efforts to plan, pilot, and raise awareness of the program within your target community.
Office of Adolescent Health

9. Financial Status Report
Submitting Financial Status Reports Using Standard Form-269 (SF-269)
Good afternoon, everyone. At this time I’d like to discuss the submission requirements for the SF-269 Financial Status Report. This document is used to prepare the financial reports in support of your annual program progress report.
The SF-269 provides a snapshot of where you stand on spending funds obligated under the year one award. By examining the information on your Financial Status report with your progress report, it helps us to determine whether your organization is spending awarded funds efficiently to meet your target goals and objectives.
Let’s look at how to complete the SF-269 -
Office of Adolescent Health

10. Which Financial Status Report to Use and When is it Due?
SF-269A (Short Form)
SF-269 (Long Form)
90 days after the end of the annual budget period
There are two different Financial Status Report forms.
The SF-269A (Short Form) is the default form and most commonly used if your program does not generate Program Income.
The SF-269 (Long Form) is used to report program income, if funds are generated as a result of at least a portion of the grant funds.
Just as described within your year one NGA, the FSR is due no later than 90 days after the end of the annual budget period. (September 1, 2010 thru August 31, 2011.) **Due no later than November 30, 2011.
Office of Adolescent Health

11. Access to SF-269
Download SF- 269 Short Form at link below :
Adobe Reader
Begin by downloading a PDF version of the SF-269A at the link below:
To take advantage of some of the auto-calculations features, ensure you have the latest version of Adobe Reader. which you can download for free at the link below:
Office of Adolescent Health

12. How to Fill Out the SF-269A Let’s look at how to fill out the SF-269.
Federal Agency to which report is submitted – Office of the Assistance Secretary of Health (OASH)
1. Insert your Grant/Agreement number just as listed within box four (4) of your Notice of Grant Award. 2. Select Cash or Accrual - Most grantees are reporting on a cash basis. Your financial manager will know if you’re reporting on an accrual basis.
3. The funding period is the entire duration of your grant/agreement and for project period ending 08/31/11, it should be listed From: 09/01/2010 To: 08/31/2011.
4. The Budget period covers each budget reporting period alone and should be listed From: 09/01/2010 To: 08/31/2011.
Office of Adolescent Health

13. How to Fill Out the SF-269A
1, 2 and 3 isn’t required, because no previous period exist for this reporting period.
4. Total Outlays -The cumulative amount spent as of the beginning of the year being reported on (this box should equal the total of boxes 2 & 3 below).
5. Recipient Share - Cost share contributed as of the beginning of the year being reported on.
6. Federal Share - Funds expended as of the beginning of the year reported on.
7. Federal Share/Cumulative – Total funds that have been expended as of the end of the year being reported on.
8. Total Federal Share -Total funds that have been expended as of the end of the year being reported.
9. Federal Funds Authorized - Total amount of federal funds that have been awarded to date under this grant.
10. Unobligated Balance—Total remaining unobligated funds.
Complete bottom section 11 of the FSR (not shown) if indirect reimbursement is claimed.
Complete bottom section 12 to add remarks such as requested carryover amount or when submitting a corrected FSR to provide explaination.
Have the FSR signed by the Chief Financial Officer or accountant when completed before submitting.
Office of Adolescent Health

14. Submission Process
Due date for Year 1 Annual Progress and Financial Report submissions: November 30, 2011.
Two avenues available for submission:
Electronic via (preferred)
Hard copies (mailed or hand delivered)
Do not submit through both mechanisms!
Please select one submission mechanism.

15. Electronic Submission (preferred)
Entire report submitted to OAH and OGM electronically.
Sent directly via to the assigned OAH Project Officer and OGM Grants Management Specialist.
Grantees should include the official grant number on all submissions.
Office of Adolescent Health

16. Submission Process Hard Copies
Submit 2 hard copies of the entire annual progress and financial report.
You can find the SF-269 form on
Can be mailed or hand delivered.
Due by 5pm EST on November 30, 2011.

17. Submission Process Mail Hard Copies to: Office of Grants Management
Office of the Assistant Secretary for Health
1101 Wootton Parkway, Suite 550
Rockville, MD 20852
Subject: OAH Annual Progress Report

18. Questions?
Office of Adolescent Health

19. Carryover Requests
Office of Adolescent Health

20. Carryover Request Process
Purpose of this Training The purpose of this training is to provide detailed guidance on carryover requests submission to the Office of Grants Management (OGM), Office of Adolescent Health (OAH) and identify pitfalls when processing the carryover request.
The Purpose of this Carryover Training is to provide guidance on carryover requests, submission to OGM, OAH and identify pitfalls when processing the request.
Office of Adolescent Health

21. How Can Carryover Funds be Used?
All unobligated funds that are available for carryover must be used to support the original approved goals and objectives of the grant program based on the Funding Opportunity Announcement Funding Restrictions- Funds cannot be used for the following purposes: To supplant or replace current public or private funding; To supplant on-going or usual activities of any organization involved in the project; To purchase or improve land, or to purchase, construct, or make permanent improvements to any building; or To reimburse pre-award costs.
How Can Carryover Funds be Used- All unobligated funds that are available for carryover must be used to support the original approved goals and objectives of the grant program based on the Funding Opportunity Announcement.
As specified within the FOA, funds cannot be used for the following purposes:
To supplant or replace current public or private funding. To supplant on-going or usual activities of any organization involved in the project. To purchase or improve land, or to purchase, construct, or make permanent improvements to any building. To reimburse pre-award costs.
Office of Adolescent Health

22. Submitting your Carryover Request
your request to both of the following: Assigned OGM - Grants Management Specialist Assigned OAH – Program Project Officer Mailing address: Office of Grants Management, OASH Attn: [Assigned Grants Management Specialist] Wootton Parkway, Suite Rockville, Maryland 20852
Let’s look at where and how to submit your Carryover Request.
You may scan and the request to your assigned Grants Management Specialist and OAH Program Project Officer
Or you may mail to the address as listed using the assigned GMS:
Office of Grants Management, OASH Attn: [Assigned Grants Management Specialist] 1101 Wootton Parkway, Suite 550 Rockville, Maryland 20852
Office of Adolescent Health

23. Documentation to Support the Carryover Request
1. A written statement indicating why the carryover funds were not spent during the approved budget period signed by the Authorizing Business Official and Program Director A detailed line item Budget and Budget Narrative Justification for the amount of carryover funds A Work Plan to support the carryover request A Financial Status Report (SF 269), certifying funds are available for use signed by the Financial Officer Note: Grantees may not carryover funds that are identified as restricted in the Notice of Award’s Terms and Conditions.
The following documentation is needed to support the carryover request: A written statement indicating why the carryover funds were not spent during the approved budget period signed by the Authorizing Business Official and Program Director.
A detailed line item Budget and Budget Narrative Justification for the amount of carryover funds.
A Work Plan to support the carryover request.
A Financial Status Report (SF-269), certifying funds are available for use signed by the Financial Officer. Please note, grantees may not carryover funds that are identified as restricted in the Notice of Award’s Terms and Conditions.
Office of Adolescent Health

24. Process for Review of the Carryover Request
1. The request is received by the OGM and OAH. We’ll review the request for allowable and reasonable cost and ensure all supportive documents have been submitted The Project Officer will identify whether the grantee has performance issues that may have caused a large unobligated balance to occur over time, or if a Change of Scope has occurred and provide feedback to the Grants Management Specialist on such issues, if applicable The Grants Management Specialist will review the grantee’s financial report and reconcile it with the Division of Payment Management to determine whether the funds requested are actually available for carryover and provide feedback to the Project Officer.
Once received by OGM and OAH, we’ll process your carryover request in the following manner:
We’ll review the request for allowable and reasonable cost and ensure all supportive documents have been submitted.
The Project Officer will identify whether the grantee has performance issues that may have caused a large unobligated balance to occur over time, or if a Change of Scope has occurred and provide feedback to the Grants Management Specialist on such issues, if applicable. The Grants Management Specialist will review the grantee’s financial report and reconcile it with the Division of Payment Management to determine whether the funds requested are actually available for carryover and provide feedback to the Project Officer.
Office of Adolescent Health

25. Response to Carryover Request
1. Revise the NGA authorizing the grantee to spend the unobligated funds for approved purposes. (Carryover funds must be used to cover only prospective costs, not costs already incurred by the grantee) Restrict the grantees authority to carryover unobligated balances in the future Use the balance to reduce or offset funding for a future budget period 4. Use a combination of these actions **A decision about the disposition of the reported unobligated balance will be reflected in the Notice of Grant Award. The OGM and OAH offices have 30 days to process the request.
Once our review has been completed, we’ll response to the Carryover Request using the following actions: 1. Revise the NGA authorizing the grantee to spend the unobligated funds for approved purposes. (Carryover funds must be used to cover only prospective costs, not costs already incurred by the grantee) 2. Restrict the grantees authority to carryover unobligated balances in the future. 3. Use the unobligated balance to reduce or offset funding for a future budget period 4. Use a combination of these actions. The final decision will be reflected in the Notice of Grant Award.
The OGM and OAH offices have 30 days to process the request.
Office of Adolescent Health

26. Potential Pitfalls to Process
1. Delinquent or Incorrect Financial Status Reports – OGM must have a current FSR report on file which shows the budget period from 09/01/2010 through 08/31/2011 in order to review actual balances that are available for carryover. 2. Budget narrative justifications inadequate – not enough details to support each listed line item. 3. Submitting request to the OAH Program Office without sending it to the OGM Grants Management Specialist (or vice versa).
The following are pitfalls which slow our ability to process the carryover request:
Delinquent or Incorrect Financial Status Reports – OGM must have a current FSR report on file which show s the budget period from 09/01/2010 thru 08/31/2011 in order to review actual balances that are available for carryover.
Budget narrative justifications inadequate – not enough details to support each listed line item.
Submitting request to the OAH Program Office or Officer without sending it to the OGM Grants Management Specialist.
Office of Adolescent Health

27. Carryover Policy Citations
45 Code of Federal Regulations, Part 74 and HHS Grants Policy Statement; Rev. 01/2007; page II For additional questions regarding Carryover Requirements contact your assigned Grants Management Specialist
For additional information, you may reference the following:
45 Code of Federal Regulations, Part 74 and 92. HHS Grants Policy Statement
Or contact your assigned Grants Management Specialist
Office of Adolescent Health

28. Questions?
Office of Adolescent Health

29. Evaluation Progress Update
This is Jean Knab of the Eval TA Team. I know that many of you are getting out into the field in the next few weeks so thank you for taking the time to listen today.
[If at the end third of the call and this has not already been stated…] Evaluation progress reporting is required for Tier 1 C/D and Tier 2 grantees not participating in the federal evaluation. Tier 1 A/B grantees and Tier 1 C/D and Tier 2 grantees participating in the federal evaluation can exit the call at this time.
Hopefully you have all had a chance to review the annual reporting requirements regarding the progress of your evaluation. I will provide a brief overview of the requirements, some illustrative examples, and then we will have time for any questions you might have about the instructions.

30. What and Why? Assess two key aspects, now and at six-month intervals
Sample intake
Baseline equivalence
We will assess that data
Against the HHS evidence standards for attrition and equivalence
To look for areas in which evaluation implementation could be strengthened
Why start this now?
Documentation
Proactively identify potential issues
Target resources
To assess evaluation progress, we are requesting some specific details on two key aspects of the evaluation – sample intake and baseline equivalence. These are important aspects of the evaluation to monitor and are a bit too detailed for us to cover on our monthly phone calls. So we will use the annual and mid-year reporting process to collect this information from you. We will review the data and provide a written assessment of where the evaluation currently stands with regards to the HHS evidence standards, but we will also spend time discussing how to use the data to inform your evaluation implementation and data collection.
Sample intake and baseline equivalence are aspects of your evaluation that will be assessed for any paper or final report on your evaluation through the HHS evidence review. However it is also important to begin examining sample intake and baseline equivalence now for several reasons. First, documenting the initial sample intake process is critical for future reporting. Staff turnover and memory loss can make some of the minor details difficult to reconstruct several years down the line so putting this on paper now will be very beneficial. Documenting sample flow through the various stages of recruitment, assignment, consent, and data collection is also very valuable for pinpointing sample loss throughout the process. Understanding where you are losing your sample (not turning in consent forms, absent for surveys, or disproportionate loss from say one particular site or demographic group) is important for improving implementation processes for the current and/or future cohorts.
Finally examining baseline equivalence on a partial sample will allow you to determine whether they are baseline differences on key measures so large that would not allow you to attribute the impact to the intervention and the study would not meet HHS evidence standards. If you observe group differences early on, you can potentially target data collection efforts to rectify those differences (e.g. on demographic characteristics) by spending additional resources on targeted follow up with certain students.

31. General guidance Provide whatever data you have at this time
Provide data pooled across cohorts and sites
We recognize that some of you will have limited or no baseline data collected at this point, that’s fine. Provide whatever data you have. If you don’t have baseline data collected or processed yet, your documentation should reflect where you are in the process and the information on sample intake for the youth you are engaged with at various stages.
If you have enrolled multiple cohorts (this is unlikely for most grantees for this round of reporting but will be relevant for future reporting), please provide us data pooled across cohorts. However we encourage you to examine data separately by cohort and by site as well. You may find that your first cohort of youth is not baseline equivalent simply by chance. Knowing that early on would allow you to target your recruitment and data collection efforts for round 2 to attempt to rectify this.

32. Sample intake documentation for clustered random assignment designs
A paragraph describing the sample intake process
Definition of evaluation eligibility, clusters considered/recruited, outcome of recruitment effort, & prioritization of clusters for inclusion in the evaluation.
Number of clusters
Randomly assigned to each condition
Retained after random assignment - at each time point, program inception & each follow-up
Key information for youth
Are they also randomly assigned? If so, describe timing in the process.
If not randomly assigned, how are they assigned to cluster and what is the timing in the process?
Plus all information requested about youth, for clusters still participating
Clustered random assignment designs are those in which the unit of random assignment is a group such as a school, teacher, classroom, clinic, or community-based organization).
Paragraph is critical documentation on how the sample was formed and its representativeness and generalizability that can be difficult to reconstruct down the line so it is important to get that documented now.
This is the information needed for the clusters, now we go on to the information needed for the youth in these evaluations. When we calculate attrition of the youth, we start with the sample after cluster loss has been accounted for. So we ask in the next section that you only provide information for the youth sample once lost clusters have been removed. For instance, if you randomly assign 10 schools and one drops out after baseline data collection, do not include the students from that school in the youth sample documentation. That lost school should only be reflected when talking about the clusters. We will go through an example of this. And after we review an example of the documentation for cluster-level documentation, we will go through what should be reported for youth samples for all grantees.

33. Screen shot of what the first consort diagram should look like for a grantee with clustered random assignment. Grantee X has a cluster randomized controlled trial of a supplement to their regular health curriculum.
Let’s take a look at each of these pieces

34. Paragraph on intake process
The population of interest for this evaluation was high schools in two local counties. All high schools in the two counties were eligible (including those with a year round school calendar) with the exception of alternative high schools (n=4) and high schools with student populations less than 150 students (n=3). All of the 40 eligible schools were recruited and 32 consented to participate in the evaluation. Since the study only had resources to serve 20 schools in the evaluation, we grouped schools by size ( (n= 12) or 500+ (n=20)) and randomly selected 8 schools from the stratum of smaller schools and 12 schools from the stratum of larger schools.
First is the description of the sample – read.
Key items covered
Eligibility for the evaluation and exclusions made prior to random assignment
Timing of consent relative to random assignment
Sample loss prior to random assignment (as a result of non-consent)
Description of how the consented school were prioritized for inclusion in the sample
sample

35. Clusters – sample to date
Did not agree to be in study (n = 8)
Did not pass screening criteria
(n = 7)
Not randomly selected (n = 12)
Clusters Randomized (n = 20)
Date of Cluster Random Assignment 6/2011
Assigned to Treatment (n = 10)
Assigned to Comparison (n = 10)
Completed baseline data collection (n = 9)
Reason for cluster dropping out
One school had a leadership change in August 2011 and the new principal did not feel that school personnel could devote resources to supporting the evaluation at this time (n=1)
Completed baseline data collection
(n = 10)
Move into the traditional components of the consort diagram
At this point they have not completed any follow-up data collection so they stop here for the documentation at the cluster level. Then they move on to describing the youth sample.

36. Youth sample intake documentation for all designs
A paragraph describing the sample intake process for youth
Definition of evaluation eligibility, youth screened and determined eligible, reasons for screening out, & process for selecting pool to be evaluated among those eligible
Number of youth
Eligible to receive program
Consenting to the evaluation (by treatment/control)
Randomly assigned to each condition
With follow-up data at each time point, by condition
Key dates
Start and end dates for the program (and for comparison if applicable)
Start and end dates for each follow-up, by condition
Make sure documentation reflects the order in which the activities occurred if not apparent (e.g. random assignment of youth relative to consent of youth or assignment of youth to condition relative to random assignment of clusters).
While HHS evidence standards do not require attrition to be assessed for quasi-experimental designs, we ask that those using those designs provide this information as well. Important for determining whether there has been intervention-induced loss and at what stages in the process, which can be important when thinking about future implementation, and also the representativeness of your final analytic sample.

37. Again, screen shot of what the final consort diagram would look like for all grantees. Look at pieces

38. Youth enrollment process
All 9th graders enrolled in health class are eligible for the evaluation. (Note: not all 9th graders take health and there may be some 10th-12th graders enrolled in health classes that will not be included in the evaluation sample.) Students were assigned to health classes prior to random assignment of the schools. However student consent for the evaluation takes place after random assignment. The consent forms do not indicate the schools treatment status and staff have been instructed to keep parents and students unaware of the school’s condition. Of the 6,740 9th graders enrolled in health classes as of September 3rd in the 19 schools, 5,993 consented to the evaluation (2,987 of 3,315 treatment students and 3,006 of 3,425 control students).
Treatment students
Said did not want to participate (n = 78)
Did not return consent form (n = 250)
Control students
Said did not want to participate (n = 106)
Did not return consent form (n = 313)
Note: 340 T students and 412 C students were transferred out of 9th grade health class between random assignment of the schools and September 3rd. The schools indicated that these transfers were due to student mobility (62%), participation in activities that restricted student scheduling (23%), and changes in teacher schedules/capacity (15%).
Only for subsample (italicized)
Document eligibility for consent; document process used to ensure blind consent of students since random assignment of schools had already occurred, loss of students prior to September (b/n RA and student consent). Important to get reasons for sample loss of any kind (describe the whys) – account for all students potentially in the sample. These last two pieces are very important for a critical reader to determine whether the validity of the random assignment design has been upheld.

39. Youth – sample to date Program start date: 9/27/11
Youth eligible for evaluation in treatment schools (n=3,315)
Youth with parental consent for evaluation in treatment schools
(n = 2,987)
Youth eligible for evaluation in control schools (n=3,425)
Youth with parental consent for evaluation in control schools
(n = 3,006)
Completed baseline (n = 2,855)
Dates of data collection: 9/22-9/24
List reasons for non-completes
Absent (n=30)
Transferred out of class n=101)
Language barrier (n=1)
Completed baseline (n = 2,912)
Dates of data collection: 9/22-9/24
List reasons for non-completes
Absent (n=31)
Transferred out of class (n=56)
Student refusal (n=7)
Important that categories for non-completes are established prior to baseline data collection and are consistent across sites and conditions. Inform data collectors of how to code if a student refuses or what it means to have a language barrier.
Note program start and end dates (add comparable information if there is a program in place in control condition)
Program start date: 9/27/11
Program end date: 5/27/12

40. Baseline equivalence documentation
Focus on variables assessed under HHS evidence standards
Age, gender, race/ethnicity
Measures of sexual behavior
Template available on SharePoint
What to provide
Means, standard deviations, sample sizes for each measure
Any documentation of deviations from those basic statistics (e.g. you calculate statistical tests accounting for clustering)
Again, want pooled data across sites and cohorts

41. Required data processing
Construct race variable
Recode those selecting multiple races into a two or more races category
Construct dummy (binary) variables for yes/no survey items, gender, and Hispanicity
Construct full sample sexual behavior variables
E.g. youth that they did not have sex in the past 3 months should be coded as “no” or zero in the numerator and included in the denominator for the measure regarding sexual intercourse without a condom in past three months
Youth who skip out of certain questions b/c effectively we know their response because of a prior question should have their responses imputed.
Will discuss as we go through the example

42. Baseline equivalence excel template
Apologize that this is small. The numbers are not important, we are just going to review the structure and the pieces that require data entry.
This is the portion of the spreadsheet that examines baseline equivalence for demographic measures. All shaded cells are calculations performed by the spreadsheet.
Provide means and standard deviations for continuous measures and a t-test will be calculated on mean differences
Provide means of binary measures (please enter with decimals) and a t-test will be calculated on mean differences (using a formula for binary variables). No standard deviations are required for binary measures so we indicate with a dash.
For all of these we also ask that you provide the sample sizes for each measure because you may have item non-response (particularly for the behavioral measures) or you may not have processed all of the data you have collected and was reflected on the consort diagram.
Race variable is a bit different. Two highlighted areas are what you will data enter for race.
Enter counts in each sample size cell (Total and percents will be calculated for you). A chi-sq statistic will be generated based on the distribution of the counts in the race categories. If you have don’t have students of a particular racial group in either your treatment or control group, just enter zeros. The chi-sq statistic will not calculate, but that’s fine. Your TA Liaison will finalize the calculation.

43. Baseline equivalence excel template
Here is the section of the spreadsheet that looks at sexual behavior measures
Again, please enter binary data as decimals and no standard deviations are required for binary measures.
Walk through example of behavioral measure recoding
# of times measure
What is not displayed on these slides is an additional column for a p-value to be entered if it was calculated by you in a statistical package. For instance, if you have a clustered random assignment design and cluster the standard errors to account for that. In that case you would note your p-value and the adjustment made and test used (e.g. two-tailed t-test).
Again I just want to reiterate that we recognize that many of you will have very limited or potentially even no baseline data collected at this point. That’s fine. Provide what you have. And use this as a roadmap for the information you will be expected to provide in May for your continuation reporting and in subsequent years.

44. Questions?
Thank you again for your time. If you have any additional questions, you can follow up with your TA Liaison and your OAH Project Officer.
Office of Adolescent Health

45. Thank you for your time today!
Office of Adolescent Health