RIS Research Consultation Outcome Survey Report to Public Services Council on the Pilot January 19, 2006 This report, the web questionnaire and related documents are posted on Data Farm | Shared File Depository [

RIS Research Consultation Outcome Survey Purposes of this pilot Procedures Commence survey process after mid-term, as students begin to hone in on research projects. Request student’s permission to do a follow up survey after the consultation. With consent, request address Include subjects’ s in Data Farm reports of sessions Two weeks (later one week) after sessions, start Data Farm process to pole eligible subjects via and web questionnaire Dump survey inputs into a table linked to Research/Instruction database (NB: The poling is a convenient way of collecting student feedback, not a scientific sampling of the student body.) Attempt to gauge the impact of research service on student research skills Measure student perception of service effectiveness Design/test a questionnaire and an automated procedure for delivering it

RIS Research Consultation Outcome Survey Each day, DF scans RI db for Van Pelt entries that are 14 days old Extract address, patron name, staff name and form_id from RI db Insert names and survey url into text and send Van Pelt staff record session, with into DF using consultation form Data Farm RI db Survey Process Data Farm Survey db PK form_id Students complete survey on the web

RIS Research Consultation Outcome Survey Dear Jane, In an effort to learn if the Penn Library is doing its best to help students with their research, I am writing to ask if you would be willing to complete a brief (4 Question) survey about your recent appointment with Joe Holub, on October 19,2005. We are interested to know if you found the session useful and if the information you learned will be helpful for future projects. Please click on this link to reach the questionnaire: Any additional comments you add to the survey will be appreciated. If you have any questions, or if I can be of any further assistance, please feel free to be in touch. With thanks on behalf of the Penn Library, Marjorie Hassen Assistant Director,Research and Instructional Services Van Pelt Library (215) Invitation

RIS Research Consultation Outcome Survey Web Survey

RIS Research Consultation Outcome Survey What We Asked 2. We would like to know if the session provided you knowledge or abilities that could be useful in the future. As a result of the session do you feel better prepared to… Identify resources relevant to a field of study Construct a search to find reseach material on a topic Evaluate information I use in my resarch Develop and produce a bibliography for a project Carry out future research effectively on my own Scheduling was convenient? 1. We would like to know if the information covered in the session (for example, research tools, search strategies or specific resources) helped you with your research project? Do you feel the session was... 1) very helpful 2) helpful 3) not too helpful 4) of no value If the session was of little or no help, please tell us why, in a few words. If none of the preceding cases apply, tell us about skills you might have acquired. 3. Was the scheduling of the session convenient? 4. How can our research services better meet student needs?

RIS Research Consultation Outcome Survey Student Response (from Nov.2 – Dec. 20) 127 Persons Surveyed 57 Responses [45%]

RIS Research Consultation Outcome Survey Evaluation of Service Ugrad n=39 Grad n=18 All n=57 Q. How helpful was the session? Avr. Score [Very helpful (1)…of no value (4)] Q. Was the scheduling convenient?” pct. who said yes 95% 100% 97% Perceived Outcome of the Service All n=57 Q. Do you feel better prepared to… pct. who checked a. Identify resources relevant to a field of study 81% b. Construct a search to find research material on a topic 86% c. Evaluate information I use in my research 23% d. Develop and produce a bibliography for a project 28% e. Carry out future research effectively on my own 72% What the Students Said

RIS Research Consultation Outcome Survey Identify resources relevant to a field of study Construct a search to find research material on a topic Evaluate information I use in my research Develop and produce a bibliography for a project Carry out future research effectively on my own Outcome: “I feel better prepared to…” Pct. who checked Undergrad (39) Grad (18)

RIS Research Consultation Outcome Survey Gauging Impact

RIS Research Consultation Outcome Survey Summary Conclusions Response was higher than expected (45% rate, during the busiest weeks of the term) No “dissatisfied” customers answered the survey. Service was highly rated as helpful Everyone who answered perceived at least one positive outcome from the session About 90% of the respondents identified more than one outcome from the session “I’m better prepared to construct searches” was the most frequently identified outcome “I’m better prepared to evaluate information used in research” was the least frequent outcome Tests validated the reliability of the automated survey procedure, notwithstanding potential problems with (e.g. spam-filtering) Twenty-two surveys (39%) contained comments—mostly praise and some suggestions for improvements

RIS Research Consultation Outcome Survey Comments They [research services] are wonderful as is. Very prompt response to original . Helpful, personalized service. I think that you do a good job, thanks! I still feel a overwhelmed by accessing many of the resources and knowing what the library provides, but Patty helped me figure out where to start with my searches and some helpful resources to use for my area of research. A lot of the sources Mr.Holub and I found were not available at Penn, so we had to place orders on inter-library loan. I ended up receiving only ONE of those orders before my paper was due. Give a sheet of codes for searching the databases I would just try and publicize the service more, I had no idea it existed. I would like to see more of your research services advertised to the student body. If possible, electronic appointment scheduling might be helpful. Longer sessions as needed. Let student do the work while reference expert watches. Work through the research process more slowly with the student Cookies?

RIS Research Consultation Outcome Survey Next Steps