SoTL: Scholarship of Teaching and

LearningData Collection and Analysis

Dr. Staci Trekles, atrekles@pnw.edu

Review: SoTL Basics

• Reflective process, similar to research in any disciplinary field

• Goals typically relate to improving student learning and experiences

• Five primary steps (Bishop-Clark & Dietz-Uhler, 2012):1. Generate the research question and do literature review2. Design the study3. Collect data4. Analyze data5. Present and publish

Types of Data to Collect• Mainly you will have either

qualitative or quantitative data, or both (in mixed methods)

• Quantitative data usually involves some statistical work (such as results from surveys)

• Qualitative data will involve looking through and finding commonalities or themes

Tools for Data Collection• Qualtrics: Great choice as it gathers data anonymously and does common

statistics on surveys• Cameras/microphones/iPad/smartphone: Great for interviews or focus

groups, recordings are easy to transfer or listen to again and transcribe• BlackBoard: good place to get assessment data from within your courses,

can do basic statistics on test scores, rubrics• SPSS: University has a site license available to use SPSS to run statistical

analysis on quantitative data• NVIVO and HyperRESEARCH: Common tools in qualitative analysis (not

free) but there are others – see https://en.wikipedia.org/wiki/Computer-assisted_qualitative_data_analysis_software

• University Institutional Research Office: http://www.pnw.edu/institutional-research/

Qualtrics Overview

• http://purdue.qualtrics.com• Log in with your Purdue career account• They just updated their interface! You now have a choice

between classic and new views• Detailed and helpful tutorials:

• New “Insight Platform”: http://support.qualtrics.com/explore-the-new-qualtrics

• Classic platform: http://www.qualtrics.com/university/

Differences in Interfaces

Let’s Explore Qualtrics

• Great for anonymous surveys, research, feedback, and voting

• A link is provided that you can send to anyone to complete the survey

• Reports are detailed and can be analyzed in a number of ways

• More help and manuals: http://centers.pnw.edu/teaching/qualtrics/

The Basics• Use the Create Project button at

the top right to create a new survey• Once you’re editing a survey you

have a toolbar of options• Survey: Content and design• Distribution: Get your link and other

settings• Data & Analysis: Analyze your data

collected• Reports: Download reports of results

Qualtrics Survey Questions

• Qualtrics offers many different types of questions, everything from basic multiple choice to matrixes, ranking, and more

• Additional options are available for timing, custom validation, and skip logic

Coding and Categorizing• Your data will start to line up into

categories based on repetition, comparisons

• Some people like tools like NVIVO which help with coding digitally

• Others will use colors (markers or digital) to show where themes intersect visually, or make a table

• Use the method that makes sense for you, and realize that it may take several passes through the data

• Resources for coding methods:• https://researchrundowns.co

m/qual/qualitative-coding-analysis/

• http://programeval.ucdavis.edu/documents/Tips_Tools_18_2012.pdf

• http://onlineqda.hud.ac.uk/Intro_QDA/how_what_to_code.php

Analyzing Data and Making Conclusions

• You’ll never be able to say with 100% certainty that you proved something in research

• But, you can usually say that there is good evidence for a conclusion or recommendation based on what you see so far

• If it’s still unclear, get more data from other sources and triangulate the findings

Digging Deeper• Remember that it’s difficult to

generalize most SoTL research conclusions

• Recommendations, though, can lead to further research or to new literature out there to review

• Often the process is cyclical and you’ll find yourself going back, adding new data sets, and confirming or altering your conclusions over and over again

Your Turn!

• Where are you at so far with your project?• Where do you need more help (more advice, more data,

more questions)?• What’s confounding you so far in the process?• What questions do you have about analyzing data, getting

tools to help, and so on?

Places to Publish

• Annual SoTL Conferences: http://www.washington.edu/teaching/sotl-annual-conferences/

• Great list of interdisciplinary and teaching journals: http://www.fctl.ucf.edu/researchandscholarship/sotl/journals/

• Directory of Teaching Journals: http://cetl.kennesaw.edu/teaching-journals-directory

• Directory of Teaching Conferences: http://cetl.kennesaw.edu/teaching-conferences-directory

More Resources

• Vanderbilt SOTL “getting started” guide: https://my.vanderbilt.edu/sotl/doing-sotl/getting-started/

• Guidebook to SOTL – thinking of a problem and the questions: https://my.vanderbilt.edu/sotl/files/2013/09/1SoTLProblem4.pdf

• Faculty Center for Teaching and Learning at Univ. of Central Florida: http://www.fctl.ucf.edu/ResearchAndScholarship/SoTL/

• Center for Engaged Learning at Elon University videos: https://www.youtube.com/channel/UCNIm8Apo1feU73SPyxEXXgg

References• Bishop-Clark, C, & Dietz-Uhler, B (2012). Engaging in the scholarship of teaching

and learning. Sterling, VA: Stylus. • Boyer, E.L. (1990). Scholarship reconsidered: Priorities of the professorate. San

Francisco, CA: Jossey-Bass.• Felten, P. (2013). Principles of good practice in SoTL. Teaching and Learning

Inquiry: The ISSOTL Journal, 1(1), 121-125.• Hutchings, P. (2000). Opening lines: Approaches to the scholarship of teaching and

learning. Menlo Park, CA: Carnegie.• Nelson, C. (2003). Doing it: Examples of several of the different genres of the

scholarship of teaching and learning. Journal of Excellence in College Teaching, 14(2), 85-94.