1 HRSA Division of Independent Review The Review Process Regional AIDS Education and Training Centers HRSA Toni Thomas, MPA Lead Review Administrator

2 DIR Mission To ensure a fair, ethical and objective review of each application

3 Conflict of Interest A conflict of interest occurs when a reviewer, close relative, or professional associate of the reviewer has a financial or other interest in an application that is likely to bias the reviewer’s evaluation of that application.

4 Conflict of Interest cont.  If it could be perceived that you have a conflict of interest then it is an apparent conflict of interest.  If you have questions as to whether you have a conflict of interest, consult DIR staff.

5 Confidentiality Before, during, and after the meeting. Before, during, and after the meeting. Results of deliberations are CONFIDENTIAL. Results of deliberations are CONFIDENTIAL. Discussions regarding applications should be confined to the meeting room. Discussions regarding applications should be confined to the meeting room. All review materials are to remain on-site after the meeting has concluded. All review materials are to remain on-site after the meeting has concluded. All electronic files must be permanently deleted from your electronic equipment. All electronic files must be permanently deleted from your electronic equipment.

6 Outside Information If information is not in the application the reviewer should review as though the information does not exist. If information is not in the application the reviewer should review as though the information does not exist. Applications should be reviewed and evaluated in accordance with the Review Criteria and the entire application. Applications should be reviewed and evaluated in accordance with the Review Criteria and the entire application. Applications should not be compared. Applications should not be compared. DO NOT discuss or mention any information that is not contained in the application itself. (includes web-site references). DO NOT discuss or mention any information that is not contained in the application itself. (includes web-site references).

7 Approach Avoid personal bias. Avoid personal bias. Independently review, evaluate and score each application based on the FOA and the Review Criteria. Independently review, evaluate and score each application based on the FOA and the Review Criteria. Consistently evaluate each application (i.e., scores should reflect the strengths and weaknesses noted). Consistently evaluate each application (i.e., scores should reflect the strengths and weaknesses noted). Participate in the panel discussion. Participate in the panel discussion.

8 Role of the Participants Chair: Review Facilitator: Directs the order of review, leads the panel in the review process, clarifies and facilitates the progress of the review, mediates questions and comments from panel to Program/Grants Management and vice versa, assists in the preparation of the review summary, and certifies the review process. Chair: Review Facilitator: Directs the order of review, leads the panel in the review process, clarifies and facilitates the progress of the review, mediates questions and comments from panel to Program/Grants Management and vice versa, assists in the preparation of the review summary, and certifies the review process. DIR: Review Administrator (RA): Guides, oversees, and verifies the integrity of the objective review process. DIR: Review Administrator (RA): Guides, oversees, and verifies the integrity of the objective review process.

9 Role of the Participants cont. Summary Statement Operator (SSO): Captures discussion from panel. Summary Statement Operator (SSO): Captures discussion from panel. Program: When requested, responds to specific technical and programmatic issues. Program: When requested, responds to specific technical and programmatic issues. Grants Management: When requested assists with budget issues. Grants Management: When requested assists with budget issues.

10 Steps in Conducting the Committee Review 1. INTRODUCTION: The Chair introduces the application and the application number to be reviewed. The Chair asks for any Conflicts of Interest. Those with conflicts leave the room for that application discussion.

11 Steps in Conducting the Committee Review cont. 2. OVERVIEW:  The Chair allows the reviewers two (2) minutes to quickly read the abstract of the application to be reviewed, if they have not prior to review.  The First Primary Reviewer gives a descriptive overview of the application that gives the “big picture” and the significant strength and weakness.  The Second and Third Reviewers give any additions, comments or clarifications to the First Reviewer’s overview as well as his/her additional significant strength and weakness.

12 Steps in Conducting the Committee Review cont. 3. INITIAL SCORING ANNOUNCEMENT: Each assigned Primary Reviewer provides his/her initial scores by criterion, as well as his/her overall total.

13 Steps in Conducting the Committee Review cont. 4.OPEN DISCUSSION:  The Chair invites discussion from all members of the Review Committee.  Other reviewers are expected to ask questions of the Primary Reviewers to promote a full discussion and understanding of the application  Approximately 20 minutes are allocated for Steps 2, 3 & 4.

14 Steps in Conducting the Committee Review cont. 5. FORMULATING AND EDITING THE SUMMARY STATEMENT:  The First Reviewer leads the Panel in editing of the projected summary statement.  The SSO makes the edits to the strengths and weaknesses of each criterion, based on the group discussion. (20 – 30 minutes).  Difference of opinion regarding strengths and weaknesses noted by the assigned Reviewers are to be discussed to derive group consensus. Contradictory statements are to be avoided.  Reviewers should focus on content, especially on accuracy and consistency as final editing of grammar, spelling, etc., will occur prior to mailing results.

15 Steps in Conducting the Committee Review cont. 6. SCORING APPLICATION:  The Chair asks each Reviewer to complete and sign a Final Score Sheet, including any revisions to initial scores, using a scale from 0-100, with 100 being the highest composite score.  Unless absent or recused, every Reviewer will score each application (by criterion) assigned to the committee. An individual criterion score may not exceed the maximum specified for that criterion.

16 Steps in Conducting the Committee Review cont. 7. BUDGET DISCUSSION:  The Primary Reviewers provide recommendations regarding the budget proposal.  Considering the assigned Reviewers’ recommendations, the panel discusses and determines a budget recommendation (10 minutes).  Reviewers cannot recommend an increase in the Applicant’s budget request. If a reduction in the budget request is recommended, Reviewers must provide a rationale.

Steps in Conducting the Committee Review cont. 8. RECOMMENDATIONS TO HRSA :  Reviewers may provide recommendations to HRSA at this time. The recommendations should reflect a consensus of the entire panel of reviewers 17

Summary Statement Development Please provide clear and useful feedback in the Summary Statement. Please provide clear and useful feedback in the Summary Statement. We are especially interested in the value of the statement, particularly in areas of weaknesses. We are especially interested in the value of the statement, particularly in areas of weaknesses. No contradictory statements! No contradictory statements! 18

Summary Statement Development, Continuation Each statement must be properly justified. Each statement must be properly justified. There is nothing wrong with leaving in a statement that is a “requirement” of the FOA, so long as it is not a contradiction with other statements within the summary. There is nothing wrong with leaving in a statement that is a “requirement” of the FOA, so long as it is not a contradiction with other statements within the summary. 19

20 Logistics Start time will vary each day based on panel progress Start time will vary each day based on panel progress AM/PM breaks provided. AM/PM breaks provided. No review of a new application will begin after 4:30 pm on the first day of review. No review of a new application will begin after 4:30 pm on the first day of review.

21 Post Review Process: Debriefing Conducted by the panel Chair and DIR after panel reviews are complete. Conducted by the panel Chair and DIR after panel reviews are complete. Encourage comments regarding program issues, pre- & post-review meeting process, instructions, etc. Encourage comments regarding program issues, pre- & post-review meeting process, instructions, etc. Please complete and submit the reviewer feedback form. Please complete and submit the reviewer feedback form.

22 Extra Notes This is a CLOSED Meeting. This is a CLOSED Meeting. Reviewers are not permitted to leave the call (meeting) room during consideration of an application except for emergencies. In such cases, the Chair must be consulted first. Reviewers are not permitted to leave the call (meeting) room during consideration of an application except for emergencies. In such cases, the Chair must be consulted first. It is the responsibility of the participants to safeguard the documents from the meeting. It is the responsibility of the participants to safeguard the documents from the meeting.

23 Extra Notes cont. To avoid distractions during the review, cell phones should be on vibrate or turned off. Disposition of review materials will be discussed after the review.

Time is PRECIOUS! Please be conscious of the time spent on each application. Please be conscious of the time spent on each application. Please be respectful of the times chosen to re-convene after breaks, lunch and start of day. Please be respectful of the times chosen to re-convene after breaks, lunch and start of day. 24

Removal of electronic Files in your PC/Lap Top Deletion of Files At the conclusion of the meeting, HRSA expects that you permanently delete all files from your electronic equipment. 25

26 Questions?  On behalf of HRSA, Thank You again for serving on this review panel!