1. Longwood University Buffalo, NY October, 2017
Review of Phase I Pilot and Recommendations for Next Step in Scaling Up EvaluateUR
Longwood University
Buffalo, NY
October, 2017

2. Institution Profile
Longwood University, founded in 1839, is located in Farmville, VA
It is a primarily undergraduate public institution in Central Virginia
4,574 undergraduates, 522 graduate students

3. Summer Research Program
PRISM (Perspectives in Research in Science and Math)
Running since Summer 2013 (5 completed years)
10-11 Faculty Mentors work with Longwood undergraduate students
STEM-H fields (Biology, Environmental Science, Chemistry, Physics, Computer Science, Math, Psychology, Exercise Science)
Start research in spring semester prior to eight week summer program

4. UR Program Director Profile
Associate Professor of Biology
Director of PRISM since Fall 2016
As Director I am responsible for:
Selecting faculty mentors
Assisting faculty mentors with selecting student researchers
Running professional development sessions for students and faculty (monthly during spring semester, weekly during summer)
Program Assessment
Managing program budget

5. Phase I Participation Overview
In Summer 2017, we had:
10 faculty mentors and 14 students participate in PRISM and the Phase I Pilot
I modified Buffalo’s Dashboard for PRISM Program:
Upload Final Research Paper and Poster at Step 11
Change wording to fit Longwood events (present at Longwood Fall College Showcase, etc)

6. Progress Completing Dashboard Steps
Same Mentor

7. Orientation
All faculty mentors and students begin and end PRISM on the same days (May 22-July 14)
On the first day of PRISM, I held a mandatory orientation session (with breakfast) to explain the EvaluateUR system
Jill Singer joined us through a web conference and she explained the history, purpose, and main points of EvaluateUR
Then I explained the specific expectations for our program (due dates for various steps)
I also provided the explanations, expectations, and due dates on a hand-out

8. Tracking
To track progress of the teams, I checked the EvaluateUR site once a day during the times that steps of EvaluateUR were due (first two weeks, weeks 4-5, last week of program).
During the times steps were not due, I checked the EvaluateUR site approximately every three days to check in on people who were late completing steps.
I sent reminder s two days before a step was due and on the day the step was due.
If a step was more than one week late, I then ed a reminder to the individual who had not completed the task. I also cc’d the other member of the team (mentor or student) so they could also remind the member of the research team to complete the step.

9. What Worked Well
Faculty and students both stated that they liked the implicit nature of this evaluation program and that they were able to think critically about aspects of research that they would not have thought about on their own
Having the group orientation session helped the faculty and students understand the goals and expectations of the evaluation program

10. Your Top 4 Challenges
1) Ensuring the faculty mentors and students complete the EvaluateUR steps on time
2) Dedicating enough time at the end of the program to complete the final Research Assessment and Research Meeting
3) Timing of the EvaluateUR assessment- ours should start in January when the research begins
4) Including submission of the final poster and final paper both in Step 11(a timing oversight on my part)

11. Your Top 3 Lessons Learned
1) Faculty mentors and students do not necessarily follow deadlines, no matter how many times you them
As a solution, I recommend making a dedicated time to complete some of the assessment steps as a group (if possible).
In the future, I will ask students to complete the pre-research reflection and ask mentors to review the reflection before our first orientation session
As part of the initial orientation session, students and mentors can have their initial meeting and I can approve the reflection at the end of the session
This will allow the completion of Steps 1-4 by the end of the orientation session

12. Your Top 3 Lessons Learned Continued
Furthermore, since all of the research groups are on the same schedule, I plan on having a mid-assessment and final assessment lunch.
I will ask the teams to complete their mid/end-research assessment steps before this lunch. Then, at the end of the lunch, mentors and students can have the assessment meeting.

13. Your Top 3 Lessons Learned Continued
2) I am planning on changing the timing of the EvaluateUR program to better match the timing of our program
The Initial assessment steps will be completed in January when the student begins working with their mentor. The mid- program assessment will take place on the first day of PRISM in May and the final assessment will take place after the PRISM poster session in July.

14. Your Top 3 Lessons Learned Continued
3) I learned that students can not go back and edit a step they have already completed
I made a mistake and asked students to submit the final poster (due last Monday of program) and final paper (due last Friday of program) both at Step 11
I will need to make these two separate steps. Also, I learned that most students submit their final paper late, so I might take this step out of the EvaluateUR program or make it the very last step so it does not delay completion of the assessment.