Say what?! Getting and Responding to Feedback in the Pre-award Office Joseph McNicholas, Ph.D. Director Office for Research and Sponsored Projects Loyola

Say what?! Getting and Responding to Feedback in

the Pre-award Office

Joseph McNicholas, Ph.D.DirectorOffice for Research and Sponsored ProjectsLoyola Marymount [email protected]

Cynthia Carr, M.A.Assistant DirectorOffice for Research and Sponsored ProjectsLoyola Marymount [email protected]

Loyola Marymount University (LMU)

• A great university in the Jesuit tradition founded in 1911.

• Mid-sized, private, predominantly undergraduate, some master’s degrees.

• Teaching focused, inwardly directed.• About $4-6 million per year in grants

and contracts.

How can the ORSP best serve this university?

The ORSP mission is• to facilitate the submission of grant

proposals and contracts, • to enhance LMU’s ability to win

grants and contracts, and • to provide outstanding customer

service.But. . . .

What is LMU’s Mission?

TEACHING

SERVICE

Seeking External Support

RESEARCH

For some at LMU, the answer could be

depicted like this.

TEACHING

SERVICE RESEARCH

We would like to move to something

more like this.

Seeking External Support

What is LMU’s Mission?

So we needed more feedback from faculty

• Importance of feedback to office functioning

• Our office’s specific mission and goals

• Involving our core support and engaging those whose support we need

Never will be clients

Future Clients

Current Unhappy Clients

Current, Happy Clients

•How do we best serve our clients and what would help us convert nonclients to our cause?

Squeaky Wheels

Some faculty were complaining:

• Grant efforts are not recognized in the tenure process!

• The ORSP does not provide enough assistance!

• The ORSP only supports certain disciplines!

• Grants do not help in teaching!

They are still complaining.

…but we thought we’d conduct a little research study.

We wanted to find out what our faculty thought about:

1. Research and Creative Activity2. The Tenure Process 3. ORSP Services

Was it that easy?

No!

First we developed the questionnaire.• Over 6 months• In between deadlines, awards, new hires,

etc. • Countless, endless, ongoing revisions• Pilot testing! Friendly advice! Concerns of

folks up the food chain!

And then what happened?

There were so many reasons we could not conduct the survey in Fall:

• End of the semester rush• Faculty survey fatigue• Cranky Research Council

There were so many more reasons we could not conduct the survey in Spring…

But we did, just ducking under new “guidance” on surveys!

…and the questionnaire was deployed.

Survey ConditionsSurvey Conditions

• ORSP “clients” – those who have worked with us during the last 2 years – were invited.

• 100 other tenureline faculty members were selected at random from the remaining population.

Client, Nonclient Client, Nonclient ResponsesResponses

Clients 87 58%

Nonclients 49 33%

Nonfaculty 8 9%

Total Responses

144 100%

1) Research, Scholarly and Creative Activity (RSC)

All faculty participants rated RSC as important to:

• Quality of Instruction• Excellence in Teaching• Staying Current in Discipline

Surprise!

Nonclient ratings were higher than the total population.

Even faculty who are not submitting grant proposals recognize grant activity as important to teaching and staying on top of their areas.

Research, Scholarly, and Research, Scholarly, and Creative Activity (RSC)Creative Activity (RSC)

Agree: 91% Agree: 77% Agree: 98%

Quality of Excellence Remain CurrentInstruction in Teaching in Discipline

Nonclient rating of RSCNonclient rating of RSC

Quality of Excellence in Remain CurrentInstruction Teaching in Discipline95% Agree 84% Agree 100% Agree

91% Agree 77% Agree 98% Agree

(Nonclient rating)

(Total population)

2) The Tenure Process

Most faculty participants (53%) felt that writing grants is recognized in the tenure process.

Nonclients agreed more (86%) and disagreed less (12%).

Surprise!

So, nongrant writing faculty DO understand the importance of the pursuit of external funding to the tenure process.

They are just not writing grants.

Pursuit of External Funding Pursuit of External Funding and the Tenure Processand the Tenure Process

Total Agree: 53% Total Disagree: 33%

Strongly Agree Agree I Don’t Know Disagree Strongly Disagree

Client-Nonclient: External Client-Nonclient: External Funding & the Tenure ProcessFunding & the Tenure Process

Client Agree: 58% Client Disagree: 29%Nonclient Agree: 86% Nonclient Disagree: 12%


More to the Point:

• 58% of faculty participants agreed that writing grants is important to career advancement.

• Only 47% of nonclients agreed, and 42% of nonclients disagreed.

An important difference

So, although grant writing may be recognized in the tenure process, other issues of career advancement mitigate this importance for some faculty members.

Grants and Grants and Career AdvancementCareer Advancement

Agree: 58% Disagree: 29%


Client vs. Nonclient: Grants Client vs. Nonclient: Grants and Career Advancementand Career Advancement

Client Agree: 58% Client Disagree: 29% Nonclient Agree: 47% Nonclient Disagree: 42%


3) The ORSP

We were happy…

Because the faculty said they know who to call if they need help with a grant.

…we are pretty sure that means us!

Who do they call? Who do they call?

Agree: 90% Disagree: 6%


We were kind of sad…

Because the faculty barely seemed to notice our amazing, fantastic, redesigned website.

Helpfulness of WebsiteHelpfulness of Website

Agree: 38% I don’t know: 44% Disagree: 18%No significant difference between clients and nonclients.


66% look forward to 66% look forward to working with us again…working with us again…

Agree: 66% I don’t know: 17% Disagree: 16%


Perceptions of Service

70% of faculty described working with LMU ORSP staff as excellent.

But why?

What is it we do that makes them think we are excellent?

The Pursuit of Excellence

Was it our clear submission procedures? (67% agreed they are clear)

Was it the way we answered questions? (80% said we did answer well)

Still Pursuing Excellence

Was it the way we helped them prepare grants? (73% said we were helpful)

What about professional courtesy? (90% think we are nice)

Perceptions of ORSP Perceptions of ORSP ServiceService

Agree: 69.9% I don’t know: 8.6% Disagree: 21.5%


Clear Submission Clear Submission ProceduresProcedures

Agree: 67.1% I Don’t Know: 13.2% Disagree: 19.8%


Questions Were Questions Were AnsweredAnswered



Grant PreparationGrant Preparation



Professional CourtesyProfessional Courtesy



Analytical Question #1

What contributes towards a faculty member feeling that they had an excellent experience with the LMU ORSP?

We borrowed a statistician to help us determine the correlations.

Three were found to be significant:• Clear submission procedures• Staff answered questions• Staff helped in submission

Question Significance Variance

ORSP Website .565

Knowledge of ORSP Assistance

.803

Clear Submission Procedures .000 4.84%

Staff Answered Questions .007 1.79%

Staff Helpful in Preparation .000 4.04%

Staff Were Courteous .944

Dependant Variable: Excellent Experience Working with ORSP

*Results are significant at .05, highly significant at .01.

Analytical Question #2

What would make faculty recommend us to other faculty?

Active behaviors:

• Staff answered questions• Staff were helpful in grant

preparation

Recommend the ORSPRecommend the ORSP



Question Significance Variance

Knowledge of ORSP .185

ORSP Website .123

Clear Submission Procedures .225

Staff Answered Questions .003 2.79%

Staff Helpful in Preparation .001 3.69%

Staff Were Courteous .213

Dependant Variable: Recommend ORSP

*Results are significant at .05, highly significant at .01.

Analytic Question #3

How do these responses vary from college to college at LMU?

Faculty in the college of liberal arts and the college of science and engineering are more clear on tenure expectations

Other colleges were not as clear, including:

• Education• Film and Television

• Communication and Fine Arts

College Level AnalysisCollege Level Analysis1 = Strongly Agree5 = Strongly Disagree

Bellarmine and Seaver are both under 2 (Agree), while other Colleges are just under 3 (I Don’t Know).

Tenure expectations in Bellarmine and Seaver are more clearly understood by faculty.

And furthermore:

Faculty from the liberal arts college and the other colleges are less likely to think that external funding is important for tenure.

The scientists and engineers see external funding as important, however.

Pursuit of External Funding Pursuit of External Funding Recognized in Tenure ProcessRecognized in Tenure Process

1 = Strongly Agree5 = Strongly Disagree

Seaver faculty (under 2) most strongly note that pursuit of external funding is recognized in tenure process.

Bellarmine and Other Colleges faculty rate 3 (I Don’t Know).

Seaver recognizes external funding more effectively in tenure process.

Our liberal arts faculty are most likely to see research, scholarly and creative activity as important for teaching.

RSC and Excellence RSC and Excellence in Teachingin Teaching

1 = Strongly Agree5 = Strongly Disagree

Bellarmine, Seaver not significantly different from Other Colleges.

Bellarmine significantly different from Seaver.

Bellarmine faculty most interested in RSC as promoting excellence in teaching.

More data, more fun

The next revision can be conducted at your school for comparison.

• Data will be provided to you • Analyze it! • Discover your own surprises!• Participate in a Research Administrator research study!

Please give Cynthia your card if you would like to discuss participation.

Documents

Say what?! Getting and Responding to Feedback in the Pre-award Office Joseph McNicholas, Ph.D. Director Office for Research and Sponsored Projects Loyola