1. Review Administrator Vicky Johnson April 18, 2017
Division of Independent Review Pre-Review Orientation Conference Call Bureau of Health Workforce Division of Health Careers and Financial Support HRSA Area Health Education Centers May 30 - June 1, 2017
Welcome to the Pre-Review Orientation Conference Call for Bureau of Health Workforce Division of Health Careers and Financial Support HRSA Area Health Education Centers This review meeting will be held as a Teleconference review.
On behalf of HRSA, I would like thank you all for agreeing to serve as participants. Your participation is extremely important in the objective review process.
This objective review is a day ORC and we have ________Applications and panels.
My name is Vicky Johnson and I am the assigned Review Administrator for this objective review. The purpose of this Pre-Review Orientation Conference call is to ensure accessibility to review material, provide detailed instructions concerning the objective review process, communicate information on specific reviewer responsibilities, and provide programmatic information and grant related guidelines, and answer your questions.
We have the following HRSA support for this review:
Review Administrator
Vicky Johnson
April 18, 2017

2. HRSA Support Staff
Program Staff: Lorener Brayboy and Kim Evans, Public Health Analyst, Health Careers Pipeline Branch
Grants Management Staff: Curtis Colston and David Treer, Department of Grants Management Operations
Contractor: Lauren Tuite, Humanitas, Inc.
Read Slide

3. Roles of Participants
DIR RA To coordinate and oversee the review process and to ensure a fair, equitable and objective review is preformed for each eligible application. Reviewers To fairly, equitably and objectively review and evaluate each application. Chairs The role of the Chair is to preside over the ORC panel. Direct the order of review, lead the panel in the review process, clarifies and facilitates the progress of the review. Mediate questions and comments from panel to Program/Grants and vice versa. Assist in the preparation of review summary statement, certifies each application and ensures that each application is accorded a fair, equitable and objective review.
DIR RA
My role in the Division of Independent Review is to coordinate, guide and oversee the review process and to ensure that a fair, equitable and objective review is performed for each eligible application.
Reviewers
As reviewers, your role is to fairly, equitably and objectively review and evaluate each application. Please keep in mind you are expected to attend and fully participate on each day of this review.
Chairs
The role of the Chair is to preside over the ORC panel. Direct the order of review, lead the panel in the review process, clarifies and facilitates the progress of the review. Mediate questions and comments from panel to Program/Grants and vice versa. Assist in the preparation of review summary statement, certifies each application and ensures that each application is accorded a fair, equitable and objective review.
  The Chair will receive a copy of everything the Reviewers receive. While the Chair will not submit evaluations or scores, he or she must have a complete understanding of the application(s) assigned to their panel, and at a minimum read the abstract, program narrative and budget sections of each application, read the FOA in its entirety and other documents that the Reviewers will read and use for their evaluations.

4. Roles of Participants Program
To ensure that program’s policies and guidance requirements are effectively conveyed and evaluated in accordance with the evaluation criteria and provide specific program policies & procedures and any legislative requirements today. Program will also be available to answer any questions or concerns you may have during the review.
Grants
To provide grant policy and budget development input applicable to assist in the budget review process.
DIR Contractor
The role of the Contractor is to provide logistical support to DIR such as honoraria and reimbursements, transmission of review materials and any other special needs.
DIR SSO
The role of the SSO is to edit comments and capture discussions from the panel.
Program
Programs role is to ensure that program’s policies and guidance requirements are effectively conveyed and evaluated in accordance with the evaluation criteria. Program staff will provide you with specific program policies/procedures and any legislative requirements today. Program will also be available to answer any questions or concerns you may have during the review.
Grants
The role of the Grants Officer is to provide grant policy and budget development input applicable to assist you in the budget review process.
DIR Contractor
The role of the Contractor is to provide logistical support to DIR such as coordinating hotel and travel arrangements, honoraria and reimbursements, transmission of review materials and any other special needs.
DIR SSO
 The role of the SSO is to edit comments and capture discussions from the panel.

5. Conflict of Interest The HRSA objective review process must meet the highest ethical standards
A conflict of interest or the appearance of Conflict of Interest is prohibited!
An individual is considered to have a conflict of interest in an application if that person, or a close relative or professional associate of that person, actually has or has the appearance of having:
Received or expects to receive a direct financial benefit of any amount deriving from an application or proposal under review;
Received or expects to receive a financial benefit from the applicant institution, offeror or principal investigator. This includes honoraria, fees, stock or other financial benefit, and additionally includes the current value of the reviewer’s already existing stock holdings apart from any direct financial benefit deriving from an application or proposal under review.
Any other interest or relationship in the application or proposal that is likely to bias the individual’s evaluation of that application or proposal.

6. Confidentiality safeguarding the applications
You must ensure that your participation in the review is conducted in a private non-public location to maintain confidentiality. You should minimize any potential distractions that may arise in and around the area in which you are participating in this objective review. You should ensure that the computer system used to access the applications and review materials is secured at all times. It is most important that you maintain confidentiality regarding participation in the review of the applications. You should not make any premature announcement, publication, or public discussion of your participation in the objective review process or the review in itself.
If you are using a different system at time of ORC please make sure you can access adobe and ARM.

7. Confidentiality To be maintained before, during, and after the meeting
It is important to protect reviewers from the appearance of undue influence and to ensure a fair, ethical, and equitable review process.
As reviewers you must not discuss the applications, comments, recommendations, review scores, names of applicants, and/or their participation with anyone not involved in the review process.
You should not have direct contact with applicants or associated individuals.
Reviewers must not make copies or save application materials for the purpose of keeping them. Upon conclusion of the review, we expect review participants to destroy/delete all applications and review materials printed or saved on personal computers.
There should be no discussions of the review outside the actual committee meeting, such as on airplanes, in restaurants, in stores, etc.
Additionally, all communication, review materials, etc., that are printed/or viewed electronically and used during this entire objective review process is to be regarded as confidential information. Confidentiality is to be maintained before, during, and after the meeting.

8. How to Review
New Reviewers – we recommend you start the process immediately. – Reviewer Guide/Sample Statements/FOA
Read the entire FOA before reviewing your assigned applications. –Learn the Criteria
Read your assigned applications separately- use the reviewer worksheet to list the required information for each application.
Applications should be evaluated and reviewed based on information and review criteria set forth in the FOA.
Your reviewer worksheet should be used during your review of each application.
Please provide clear, accurate, useful and informative feedback that is supported by data and/or documentation provided within the application(s).

9. How To Review
When independently and objectively reviewing applications you are not to compare one application to another. You are to independently review and score each application.  
Each application should be evaluated on its own merit.
Avoid comparing one application’s content, process, or budget to any other assigned application.
You are not to consider or include outside information that is not included or stated in the application

10. How To Review Scoring - Please use the Scoring Rubric
All criterion scores must be justified with written strengths or weaknesses.
We ask that you please make sure your weaknesses relate to a criterion.
When scoring use whole numbers and not fractions. In addition to scores, Reviewers enter budget recommendations. When Reviewers complete entering scores and budget recommendations in ARM, they must click on the “Submit to Chair” button. -New Reviewers ARM quick reference guide
 It is recommended that Reviewers read the entire application before entering their scores and comments in ARM.

11. WRITING Strengths and Weaknesses
A strength is defined as a response that clearly meets or substantially exceeds requirements set forth in a review criterion. A weakness is defined as a response that falls short of meeting requirements set forth in a review criterion. When describing a weakness, always refer to the application’s weaknesses, not the applicant’s weaknesses. Use of the term “applicant organization” is also preferred over “applicant.” Avoid using phrases like “fails to” and other negative statements. Use complete present tense sentences.
Please no long paragraphs or more then one sentence prior box please!

12. WRITING Strengths and Weaknesses
Please DO NOT use acronyms, when using acronyms, write out the entire name followed by the acronym in parenthesis.
Each application should be reviewed independently.
Avoid using phrases like “it would have, should have or could have…
Please see Reviewer Instruction Guide for examples of undesirable and desirable ways of stating Strengths and Weaknesses.
You can also refer to the Model Summary Statements provided for tips on composing your comments. -New Reviewers

13. WRITING Strengths and Weaknesses
Please do not cut and paste words or information from the FOA.
No bullets points, discussion points, or incomplete sentences. - Your comments should be in final format when you upload them into ARM.
Do not compare applications.
No outside information should be included in your comments.

14. ENTERING Strengths and Weaknesses
Lastly – your sentences and statements should be entered into ARM separately.
Only one sentence per box please!

15. Let’s Review the Reviewer Webpage HRSA-17-071
Lauren please pull up Hrsa Webpage
Read from Script …..

16. Prior To and Day of Review
Merged Summary Statements
What to have on Hand
Each Reviewer is expected to review the merged summary statements prior to the ORC for duplicative and contradictory comments, statements of fact, and/or personal opinions/biases.
Advanced review of the merged summary statements and identifying inconsistencies will help expedite the editing of the summary statements.
Merged Summary Statements
The Order of Review
The Participant list
Tele-Conference Instructions

17. Please Note
It is imperative that Reviewers follow the DIR Reviewer Instruction Guide and the FOA and complete assigned tasks in accordance with established deadlines.
The potential integrity of the review process may be compromised if a Reviewer does not complete the assigned work or does not deliver the work on schedule.
Reviewers must make all necessary arrangements to be available to complete review assignments, attend and participate in the entire objective review.
If for some reason a Reviewer believes that they are unable to fulfill these duties, or if unforeseen circumstances prevent the Reviewer from completing their duties, they must immediately contact me or the Humanitas contractor.
Following this call, please review the contents and familiarize yourself with the website.
It will serve as a resource tool for the reminder of the review process.

18. Q&A
At the end of Program and Grants presentation we will open the call up to any questions that you might have for DIR, Program, Grants or the contractor.