1. U.S. Department of Health and Human Services Health Resources and Services Administration APPLICATION REVIEW MODULE (ARM) National Health Service Corps (NHSC) Scholarship Program 2012-13 Application Reviewers Manual

2. U.S. Department of Health and Human Services Health Resources and Services Administration A BOUT ARM 2 About ARM O nce you have read and considered each application, ARM is the web-based computer system you will use to record the results of your review evaluations. It provides a user-friendly environment to facilitate the 2012-13 NHSC SP application review process while allowing reviewers to participate from remote locations.  The users of ARM that are discussed in this material are the Application Reviewer and the Review Chair Person.

3. U.S. Department of Health and Human Services Health Resources and Services Administration A CCESSING ARM 3 1.Log-in under the “Reviewer Logon” tab at the top. 2.Click in or Tab to User Name and enter your username and then to Password and enter your password into the appropriate boxes. Then click the button. Note: Usernames and Passwords are case sensitive.  Each review in ARM has a unique Session ID. You may be a reviewer on more than one active review. Your Username and Password will provide you access to any active review to which you have been assigned. Accessing ARM You will receive an email prior to your review containing your login Username and Password as well as the link to the ARM website.

4. U.S. Department of Health and Human Services Health Resources and Services Administration S TARTING THE R EVIEW 4 Accessing Your Review  Find the Session ID or Session Name that matches the review you will be working on and click on the hyperlinked Session Name.  Click on the hyperlinked panel number to select your panel.

5. U.S. Department of Health and Human Services Health Resources and Services Administration S TARTING THE R EVIEW 5 Application Selection Screen  Click on the hyperlinked Application Name to advance to the Application Worksheet Screen and begin your review. Note: In the score column, if the score shown is -- you have not entered any scores for this application. If an “*” is present next to the score, you have not entered all of the individual scores required for this application. 1.Application ID 2.Status 3.Score 4.Application Name ** If there are no applications listed, please contact your NHSC SP Chairperson. This screen displays four major identifiers of the application.

6. U.S. Department of Health and Human Services Health Resources and Services Administration 6 View Application PDF File 6  Click on the Open PDF link to the right of the applicants name to open the PDF file. The file will open in a new window allowing you to view the Two Letters of Recommendation, the Curriculum Vitae (CV) and the three Essays.

7. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION S CREEN 7 This screen serves as the primary control panel for evaluating an application. All applications will be scored based on the individual criterion. Application Worksheet Screen  The scoring criteria are listed in the first column.  The middle column is where you will click to enter your scores. The score will automatically total as you move through the criteria.  The last column is where you will be able to click and enter comments after you have entered scores. )

8. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION S CORING 8 Application Scoring First, users must enter a score BEFORE any comments can be entered. Note: You will not be able to select a score greater than the maximum defined by the criterion.  In the Score column, click on the hyperlinked -- to begin scoring the application.  Using the drop-down arrow, select the appropriate score by clicking on the number.  Click. Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7.

9. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION S CORING 9 Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7. Entering Comments ONLY AFTER YOU ASSIGN A SCORE to an individual criterion can you include corresponding comments. Note: If you have not entered all required scores for an application, your Actual Score will be followed by an * indicating that your scoring is not complete.  Again, as seen in the image, the only activated Comment links are those with a corresponding score. The link will automatically appear once you have scored the criterion.

10. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION C OMMENTS Entering Comments Clicking the Comment link will open the Comments window. Click on the drop-down arrow to open the menu. Select whether you are describing a Strength or Weakness. Enter a page number or any page identifying information. Note: All fields in this screen are required. Enter your strength or weakness. You can type directly into this field or copy and paste from a word processing program. After entering the comment, it is strongly recommended that you Spell Check your work by clicking on the Spell Check button. Click t to save your comment as part of your evaluation. 10

11. U.S. Department of Health and Human Services Health Resources and Services Administration S AVING AND R ETRIEVING C OMMENTS 11 Saving and Retrieving Saved Comments Clicking on the SAVE button will allow you to save data into the ARM system. The data will remain even if you time out or you decide to enter additional text at a later time. It is important that you click on the “Submit” button to finalize the comment even if you have saved the comment. Clicking on the submit button will provide you the opportunity to continue adding additional comments.

12. U.S. Department of Health and Human Services Health Resources and Services Administration APPLICATION EVALUATION COMMENTS Entering Multiple Comments You can enter multiple strengths or weaknesses for each criterion.  To enter multiple comments, repeat steps from slide 10.  All previous comments will be visible toward the bottom of the page, below the “Add a New Comment” box.  Regardless of the order in which they were entered, comments will group by category (Strength or Weakness).

13. U.S. Department of Health and Human Services Health Resources and Services Administration APPLICATION EVALUATION COMMENTS Editing Comments You can edit your saved comments at any time. Deleting Comments  Click Delete to permanently remove a comment from your evaluation.  To edit your comments, click on the word Edit next to the comment number you would like to change.  Clicking on the Edit link will open the comment window and you will be able to edit your content. When complete click to save your changes.

14. U.S. Department of Health and Human Services Health Resources and Services Administration S ESSION T IME -O UTS AND P REVENTING D ATA L OSS Session Time Out  In order to provide maximum data security, the ARM system sessions are designed to “Time-Out” after 30 minutes of inactivity. After 25 minutes, if you are not clicking between ARM web pages, a warning message will appear. If no action is taken within 5 minutes, you will be timed out. Any unsaved work will be lost. It is Highly recommended that you take advantage of the SAVE button. 14

15. U.S. Department of Health and Human Services Health Resources and Services Administration 15 Saving and Retrieving Saved Comments  Clicking on the “Retrieve the Last Comment” link will load the comment that was saved most recently. In case of a time out, you should go to the respective criteria and click on the “Retrieve the Last Comment” link in order to retrieve the comment you were working on. A PPLICATION E VALUATION C OMMENTS

16. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION S CORING 16 Completed Comments Once you have finished entering all of your comments for a given criterion, you will need to return to the Application Worksheet Screen. Click on the hyperlinked word Evaluation in the bar at the top of your screen to return to the Application Worksheet Screen after you have included all comments for a given criterion.

17. U.S. Department of Health and Human Services Health Resources and Services Administration A PPLICATION E VALUATION S CORING 17 Entering Multiple Scores At Once  If you would like to enter all of your scores at one time, click on the hyperlinked column header Score.  The full Add Scores screen will open, and you will be able to click on the drop-down arrow for each criterion and assign a score.  When you have entered all of the scores, click. Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7.

18. U.S. Department of Health and Human Services Health Resources and Services Administration S CORING R EPORTS 18 Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7. Reviewing your Work  To view all of your strengths and weaknesses for each criterion, click. Click to view your entries for all other sections. Each of these views open in a new window and show information for that individual application, and each view can be printed as a report.  You can review your work at any time in the process. To view your scores by criterion, you can click the at any time. The view will open in a new window from which you can also print the view as a report.

19. U.S. Department of Health and Human Services Health Resources and Services Administration S UBMIT R EVIEW 19 Note: The Submit to Chairperson button will not be activated until ALL scores have been entered. Note: Once you submit your application evaluation to your Chairperson, further edits CANNOT be made. Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7. Submitting Your Review When you have completed scoring the application and are satisfied with your comments, you will need to submit your evaluation to your Chairperson.  Click the button to submit your application evaluation to your Chairperson.  Click OK in the confirmation box to finalize your submission.  The button will not be activated until all sections are completed.  To evaluate other assigned applications, click “Applications List” located on the blue tool bar at the top of the page.  When all applications have been evaluated and submitted to the chair, log-off of the system.

20. U.S. Department of Health and Human Services Health Resources and Services Administration S CORING AND D OCUMENTS R ULES 20  You can only see your own scores and comments.  You must enter a score for each criterion BEFORE proceeding to enter strengths and weaknesses. The Comments link will be automatically activated after you enter a value in the score field.  The button does not become active until the your review of an application is complete. You cannot submit your review to the Chairperson until you have scored all criteria and comments.  You cannot change or modify your scores, strengths and weaknesses once an application has been submitted to the Chairperson. You can, however, still view and print your own scores, comments and entries to other sections.  All Fields marked with an asterisk * are mandatory.  If the Back Arrow or the blue Navigation Bar are available, use one or the other to navigate within ARM. Only close a window when neither the Back Arrow or Navigation Bar are available.  Exit ARM by clicking on Logon on/off in the upper right corner or your screen. When you finish entering all your scores, strengths and weaknesses, you can click the button and that application’s status changes to Submitted to Chair. Application Worksheet Rules

21. U.S. Department of Health and Human Services Health Resources and Services Administration R EVISE A PPLICATION E VALUATION 21  To change or revise a criterion score, click the applicable Score link. When you have finished updating or entering, click to send your evaluation back to the Chairperson. Note: Reviewer will view Criteria pertinent to the NHSC Scholarship Program. The Criteria will be similar to the screenshot on page 7. Revising Your Evaluation Score Note: Once you re-submit, you will no longer have access to the Return History screen from this screen.

22. U.S. Department of Health and Human Services Health Resources and Services Administration E ND 22 Please follow the instructions in this manual to process all the applications (about 15 Applications) on your panel. If you have technical questions, please contact the HRSA ARM Technical Assistance Team at ArmTechAssistance@hrsa.govArmTechAssistance@hrsa.gov If you have questions on National Health Service Corps Scholarship Program, please email BCRSReview@hrsa.gov or call: Jestover Jones III, Management Health Analyst at: 301-443-5166BCRSReview@hrsa.gov