1. Review Administrator Vicky Johnson February 15, 2017
Pre-Review Orientation Conference Call Bureau of Primary Health Care HRSA National Training and Technical Assistance Cooperative Agreements (NCA’s) March 14-16, 2017
Welcome to the Pre-Review Orientation Conference Call for the Material Child Health Bureau HRSA National Training and Technical Assistance Cooperative Agreements (NCA’s) This review meeting will be held as a Teleconference review.
On behalf of HRSA, I would like thank you all for agreeing to serve as participants. Your participation is extremely important in the objective review process.
This objective review is a day ORC and we have ___24_____Applications and panels.
My name is Vicky Johnson and I am the assigned Review Administrator for this objective review. The purpose of this Pre-Review Orientation Conference call is to ensure accessibility to review material, provide detailed instructions concerning the objective review process, communicate information on specific reviewer responsibilities, and provide programmatic information and grant related guidelines, and answer your questions.
We have the following HRSA support for this review:
Review Administrator
Vicky Johnson
February 15, 2017

2. HRSA Support Staff
Program Staff: Kathleen Shannon, Bureau of Primary Health Care, Office of Policy and Program Development
Grants Management Staff: Brain Feldman, Department of Grants Management Operations
Contractor:
Lauren Tuite, Humanitas, Inc.
Read Slide

3. Roles of Participants DIR RA
To coordinate and oversee the review process and to ensure a fair, equitable and objective review is preformed for each eligible application.
Reviewers
To fairly, equitably and objectively review and evaluate each application.
Chairs
The role of the Chair is to preside over the ORC panel. Direct the order of review, lead the panel in the review process, clarifies and facilitates the progress of the review. Mediate questions and comments from panel to Program/Grants and vice versa. Assist in the preparation of review summary statement, certifies each application and ensures that each application is accorded a fair, equitable and objective review.
DIR RA
My role in the Division of Independent Review is to coordinate, guide and oversee the review process and to ensure that a fair, equitable and objective review is performed for each eligible application.
Reviewers
As reviewers, your role is to fairly, equitably and objectively review and evaluate each application. Please keep in mind you are expected to attend and fully participate on each day of this review.
Chairs
The role of the Chair is to preside over the ORC panel. Direct the order of review, lead the panel in the review process, clarifies and facilitates the progress of the review. Mediate questions and comments from panel to Program/Grants and vice versa. Assist in the preparation of review summary statement, certifies each application and ensures that each application is accorded a fair, equitable and objective review.
  The Chair will receive a copy of everything the Reviewers receive. While the Chair will not submit evaluations or scores, he or she must have a complete understanding of the application(s) assigned to their panel, and at a minimum read the abstract, program narrative and budget sections of each application, read the FOA in its entirety and other documents that the Reviewers will read and use for their evaluations.

4. Roles of Participants
Program
To ensure that program’s policies and guidance requirements are effectively conveyed and evaluated in accordance with the evaluation criteria and provide specific program policies & procedures and any legislative requirements today. Program will also be available to answer any questions or concerns you may have during the review.
Grants
To provide grant policy and budget development input applicable to assist in the budget review process.
DIR Contractor
The role of the Contractor is to provide logistical support to DIR such as coordinating hotel and travel arrangements, honoraria and reimbursements, transmission of review materials and any other special needs.
DIR SSO
The role of the SSO is to edit comments and capture discussions from the panel.
Program
Programs role is to ensure that program’s policies and guidance requirements are effectively conveyed and evaluated in accordance with the evaluation criteria. Program staff will provide you with specific program policies/procedures and any legislative requirements today. Program will also be available to answer any questions or concerns you may have during the review.
Grants
The role of the Grants Officer is to provide grant policy and budget development input applicable to assist you in the budget review process.
DIR Contractor
The role of the Contractor is to provide logistical support to DIR such as coordinating hotel and travel arrangements, honoraria and reimbursements, transmission of review materials and any other special needs.
DIR SSO
 The role of the SSO is to edit comments and capture discussions from the panel.

5. Conflict of Interest The HRSA objective review process must meet the highest ethical standards
A conflict of interest or the appearance of Conflict of Interest is prohibited!
An individual is considered to have a conflict of interest in an application if that person, or a close relative or professional associate of that person, actually has or has the appearance of having:
Received or expects to receive a direct financial benefit of any amount deriving from an application or proposal under review;
Received or expects to receive a financial benefit from the applicant institution, offer or principal investigator that in the aggregate exceeds $10,000 per year and for reviewers who are Federal employees the amount is $15,000 per year; this amount includes honoraria, fees, stock or other financial benefit, and additionally includes the current value of the reviewer’s already existing stock holdings, apart from any direct financial benefit deriving from an application or proposal under review: or
Any other interest or relationship in the application or proposal that is likely to bias the individual’s evaluation of that application or proposal.
Discuss Conflict of Interest Requirements (DIR)   
A close relative means a parent, spouse, sibling, son or daughter or domestic partner.
A professional associate means any colleague, scientific mentor, or student with whom the reviewer is currently conducting research or other significant professional activities or with whom the member has conducted such activities within one year of the date of the review.
Reviewers are asked to check the assigned applications for conflict of interest. You are to inform us whether your organization has submitted an application to be in this competition. If you have not already done so, contact us immediately following this call if you have or think you may have a conflict of interest not previously reported to us. Your conflict of interest and confidentiality certification form should be sent directly to the contractor. It is never too late to disclose a conflict of interest. If you discover that a conflict exists once your reviewing begins, you must notify me immediately.

6. Confidentiality safeguarding the applications
You must ensure that your participation in the review is conducted in a private non-public location to maintain confidentiality. You should minimize any potential distractions that may arise in and around the area in which you are participating in this objective review. You should ensure that the computer system used to access the applications and review materials is secured at all times. It is most important that you maintain confidentiality regarding participation in the review of the applications. You should not make any premature announcement, publication, or public discussion of your participation in the objective review process or the review in itself.

7. Confidentiality To be maintained before, during, and after the meeting
It is important to protect reviewers from the appearance of undue influence and to ensure a fair, ethical, and equitable review process.
As reviewers you must not discuss the applications, comments, recommendations, review scores, names of applicants, and/or their participation with anyone not involved in the review process.
You should not have direct contact with applicants or associated individuals.
Reviewers must not make copies or save application materials for the purpose of keeping them. Upon conclusion of the review, we expect review participants to destroy/delete all applications and review materials printed or saved on personal computers.
There should be no discussions of the review outside the actual committee meeting, such as on airplanes, in restaurants, in stores, etc.
Additionally, all communication, review materials, etc., that are printed/or viewed electronically and used during this entire objective review process is to be regarded as confidential information. Confidentiality is to be maintained before, during, and after the meeting.

8. How to Review
Read the entire FOA before reviewing your assigned applications
Applications should be evaluated and reviewed based on information and review criteria set forth in the FOA.
Individually read your assigned applications
Use the reviewer worksheet for each application.
Your reviewer worksheet template should match the review criteria to which applicants responded.
Please provide clear, accurate, useful and informative feedback that is supported by data and/or documentation provided within the application(s).

9. How To Review Do not compare applications
When independently and objectively reviewing applications you are not to compare one application to another. You are to independently review and score each application.  
Each application should be evaluated on its own merit.
Avoid comparing one application’s content, process, or budget to any other assigned application.
 No outside information should be included in your review of applications
You are not to consider or include outside information that is not included or stated in the application

10. How To Review
Scoring
All criterion scores must be justified with written strengths or weaknesses.
We are especially interested in the value of the statements, particularly in areas of weaknesses. There is nothing wrong with including a strong statement that is a “requirement” of the FOA.
When scoring use whole numbers and not fractions. In addition to scores, Reviewers enter budget recommendations. When Reviewers complete entering scores and budget recommendations in ARM, they must click on the “Submit to Chair” button.
 It is recommended that Reviewers read the entire application before entering their scores and comments in ARM.
Please note that once this process is completed, Reviewers are no longer able to modify their scores.

11. Pull Up HRSA-17-058 Webpage Lauren please pull up Hrsa-17-058 Webpage
Read from Script …..

12. WRITING Strengths and Weaknesses
A strength is defined as a response that clearly meets or substantially exceeds requirements set forth in a review criterion. A weakness is defined as a response that falls short of meeting requirements set forth in a review criterion. Use complete present tense sentences. When describing a weakness, always refer to the application’s weaknesses, not the applicant’s weaknesses. Use of the term “applicant organization” is also preferred over “applicant.” Avoid using phrases like “fails to” and other negative statements.
New Scoring Rubic

13. WRITING Strengths and Weaknesses
Please DO NOT use acronyms, when using acronyms, write out the entire name followed by the acronym in parenthesis.
Each application should be reviewed independently.
Avoid using phrases like “it would have, should have or could have…
Please see provide examples of undesirable and desirable ways of stating Strengths and Weaknesses in the Reviewer Instruction Guide
You can also refer to the Model Summary Statements provided for tips on composing your comments.

14. WRITING Strengths and Weaknesses
Please do not cut and paste words or information from the FOA.
No bullets points, discussion points, or incomplete sentences. - Your comments should be in final format when you upload them into ARM.
Do not compare applications.
No outside information should be included in your review of applications.

15. WRITING Strengths and Weaknesses
The applicant clearly describes prior practical experience related to assisting rural and frontier hospitals implement new services expansions.
Lastly – your sentences and statements should be entered into ARM separately.
Only one sentence per box please!

16. Please Note
It is imperative that Reviewers follow the DIR Reviewer Instruction Guide and the FOA and complete assigned tasks in accordance with established deadlines.
The potential integrity of the review process may be compromised if a Reviewer does not complete the assigned work or does not deliver the work on schedule.
Reviewers must make all necessary arrangements to be available to complete review assignments, attend and participate in the entire objective review.
If for some reason a Reviewer believes that they are unable to fulfill these duties, or if unforeseen circumstances prevent the Reviewer from completing their duties, they must immediately contact me or the Humanitas contractor.
Following this call, please review the contents and familiarize yourself with the website.
It will serve as a resource tool for the reminder of the review process.

17. Q&A
At the end of Program and Grants presentation we will open the call up to any questions that you might have for DIR, Program, Grants or the contractor.