Say what?! Getting and Responding to Feedback in the Pre-award Office Joseph McNicholas, Ph.D. Director Office for Research and Sponsored Projects Loyola Marymount University Cynthia Carr, M.A. Assistant Director Office for Research and Sponsored Projects Loyola Marymount University

Loyola Marymount University (LMU) A great university in the Jesuit tradition founded in Mid-sized, private, predominantly undergraduate, some master’s degrees. Teaching focused, inwardly directed. About $4-6 million per year in grants and contracts.

How can the ORSP best serve this university? The ORSP mission is to facilitate the submission of grant proposals and contracts, to enhance LMU’s ability to win grants and contracts, and to provide outstanding customer service. But....

What is LMU’s Mission? TEACHING SERVICE Seeking External Support RESEARCH For some at LMU, the answer could be depicted like this.

TEACHING SERVICE RESEARCH We would like to move to something more like this. Seeking External Support What is LMU’s Mission?

So we needed more feedback from faculty Importance of feedback to office functioning Our office’s specific mission and goals Involving our core support and engaging those whose support we need Current, Happy Clients Current Unhappy Clients Future Clients Never will be clients How do we best serve our clients and what would help us convert nonclients to our cause?

Squeaky Wheels Some faculty were complaining: Grant efforts are not recognized in the tenure process! The ORSP does not provide enough assistance! The ORSP only supports certain disciplines! Grants do not help in teaching!

They are still complaining. …but we thought we’d conduct a little research study. We wanted to find out what our faculty thought about: 1.Research and Creative Activity 2.The Tenure Process 3.ORSP Services

Was it that easy? No! First we developed the questionnaire. Over 6 months In between deadlines, awards, new hires, etc. Countless, endless, ongoing revisions Pilot testing! Friendly advice! Concerns of folks up the food chain!

And then what happened? There were so many reasons we could not conduct the survey in Fall: End of the semester rush Faculty survey fatigue Cranky Research Council

There were so many more reasons we could not conduct the survey in Spring… But we did, just ducking under new “guidance” on surveys! …and the questionnaire was deployed.

Survey Conditions ORSP “ clients ” – those who have worked with us during the last 2 years – were invited. 100 other tenureline faculty members were selected at random from the remaining population.

Client, Nonclient Responses Clients8758% Nonclients4933% Nonfaculty89% Total Responses %

1) 1)Research, Scholarly and Creative Activity (RSC) All faculty participants rated RSC as important to: Quality of Instruction Excellence in Teaching Staying Current in Discipline

Surprise! Nonclient ratings were higher than the total population. Even faculty who are not submitting grant proposals recognize grant activity as important to teaching and staying on top of their areas.

Research, Scholarly, and Creative Activity (RSC) Agree: 91% Agree: 77% Agree: 98% Quality of Excellence Remain Current Instruction in Teaching in Discipline

Nonclient rating of RSC Quality of Excellence in Remain Current Instruction Teaching in Discipline 95% Agree 84% Agree 100% Agree 91% Agree 77% Agree 98% Agree (Nonclient rating) (Total population)

2) The Tenure Process Most faculty participants (53%) felt that writing grants is recognized in the tenure process. Nonclients agreed more (86%) and disagreed less (12%).

Surprise! So, nongrant writing faculty DO understand the importance of the pursuit of external funding to the tenure process. They are just not writing grants.

Pursuit of External Funding and the Tenure Process Total Agree: 53% Total Disagree: 33% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Client-Nonclient: External Funding & the Tenure Process Client Agree: 58% Client Disagree: 29% Nonclient Agree: 86% Nonclient Disagree: 12% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

More to the Point: 58% of faculty participants agreed that writing grants is important to career advancement. Only 47% of nonclients agreed, and 42% of nonclients disagreed.

An important difference So, although grant writing may be recognized in the tenure process, other issues of career advancement mitigate this importance for some faculty members.

Grants and Career Advancement Agree: 58% Disagree: 29% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Client vs. Nonclient: Grants and Career Advancement Client Agree: 58% Client Disagree: 29% Nonclient Agree: 47% Nonclient Disagree: 42% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

3) The ORSP We were happy… Because the faculty said they know who to call if they need help with a grant. …we are pretty sure that means us!

Who do they call? Agree: 90% Disagree: 6% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

We were kind of sad… Because the faculty barely seemed to notice our amazing, fantastic, redesigned website.

Helpfulness of Website Agree: 38% I don’t know: 44% Disagree: 18% No significant difference between clients and nonclients. Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

66% look forward to working with us again… Agree: 66% I don’t know: 17% Disagree: 16% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Perceptions of Service 70% of faculty described working with LMU ORSP staff as excellent. But why? What is it we do that makes them think we are excellent?

The Pursuit of Excellence Was it our clear submission procedures? (67% agreed they are clear) Was it the way we answered questions? (80% said we did answer well)

Still Pursuing Excellence Was it the way we helped them prepare grants? (73% said we were helpful) What about professional courtesy? (90% think we are nice)

Perceptions of ORSP Service Agree: 69.9% I don’t know: 8.6% Disagree: 21.5% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Clear Submission Procedures Agree: 67.1% I Don’t Know: 13.2% Disagree: 19.8% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Questions Were Answered Agree: 80% I don’t know: 5% Disagree: 15% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Grant Preparation Agree: 73% I don’t know: 11% Disagree: 16% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Professional Courtesy Agree: 90% I don’t know: 4% Disagree: 5% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Analytical Question #1 What contributes towards a faculty member feeling that they had an excellent experience with the LMU ORSP? We borrowed a statistician to help us determine the correlations. Three were found to be significant: Clear submission procedures Staff answered questions Staff helped in submission

QuestionSignificanceVariance ORSP Website.565 Knowledge of ORSP Assistance.803 Clear Submission Procedures % Staff Answered Questions % Staff Helpful in Preparation % Staff Were Courteous.944 Dependant Variable: Excellent Experience Working with ORSP *Results are significant at.05, highly significant at.01.

Analytical Question #2 What would make faculty recommend us to other faculty? Active behaviors: Staff answered questions Staff were helpful in grant preparation

Recommend the ORSP Agree: 73% I don’t know: 13% Disagree: 14% Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

QuestionSignificanceVariance Knowledge of ORSP.185 ORSP Website.123 Clear Submission Procedures.225 Staff Answered Questions % Staff Helpful in Preparation % Staff Were Courteous.213 Dependant Variable: Recommend ORSP *Results are significant at.05, highly significant at.01.

Analytic Question #3 How do these responses vary from college to college at LMU? Faculty in the college of liberal arts and the college of science and engineering are more clear on tenure expectations Other colleges were not as clear, including: Education Film and Television Communication and Fine Arts

College Level Analysis 1 = Strongly Agree 5 = Strongly Disagree Bellarmine and Seaver are both under 2 (Agree), while other Colleges are just under 3 (I Don’t Know). Tenure expectations in Bellarmine and Seaver are more clearly understood by faculty.

And furthermore: Faculty from the liberal arts college and the other colleges are less likely to think that external funding is important for tenure. The scientists and engineers see external funding as important, however.

Pursuit of External Funding Recognized in Tenure Process 1 = Strongly Agree 5 = Strongly Disagree Seaver faculty (under 2) most strongly note that pursuit of external funding is recognized in tenure process. Bellarmine and Other Colleges faculty rate 3 (I Don’t Know). Seaver recognizes external funding more effectively in tenure process.

Our liberal arts faculty are most likely to see research, scholarly and creative activity as important for teaching.

RSC and Excellence in Teaching 1 = Strongly Agree 5 = Strongly Disagree Bellarmine, Seaver not significantly different from Other Colleges. Bellarmine significantly different from Seaver. Bellarmine faculty most interested in RSC as promoting excellence in teaching.

More data, more fun The next revision can be conducted at your school for comparison. Data will be provided to you Analyze it! Discover your own surprises! Participate in a Research Administrator research study! Please give Cynthia your card if you would like to discuss participation.