Upload
asa-ingerson
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
Keyek-Franssen & Briggs
Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support
Deborah Keyek-Franssen
University of Colorado at Boulder
Charlotte Briggs
Loyola University Chicago
Copyright Deborah Keyek-Franssen and Charlotte Briggs, 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational
purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written
permission from the author.
Keyek-Franssen & Briggs
Take-aways
• Motivation to think strategically about research and decision making
• Understanding of the breadth of data collection methods
• Some data, just for fun
Keyek-Franssen & Briggs
Strategic Research
• Timing is everything
• Need to think holistically
• Consider teaching and learning needs first, technology second
Keyek-Franssen & Briggs
Backward Marketing
• ‘Backward’ Market Research by Alan Andreasen (1985)
• Know first what decisions you need to make before you begin your research, instead of beginning with a research problem.
• Focus first on why you need to know, and then on what you want to know.
Keyek-Franssen & Briggs
Backward Marketing vs. Traditional Research
Standard research Backward marketing
Define research problem Determine key decisions
Check secondary sources Determine information needs
Determine primary resources
Determine questions to be asked
Research & survey Which questions already answered
Analysis & reporting Research & survey
Analysis & reporting
Keyek-Franssen & Briggs
Defining research problem vs. determining key decisions
“Curiosity” questions
How do we invest our limited resources in instructional technology to have the greatest impact on the teaching &
learning of the greatest number of faculty and students?
Keyek-Franssen & Briggs
Embedded Principles
How do we invest our limited resources in instructional technology to have the greatest impact on the teaching & learning of the greatest number of faculty and students?
1. Focus first on teaching and learning, and then on technology as a tool to support both
Keyek-Franssen & Briggs
Embedded PrinciplesHow do we invest our limited resources in instructional technology to
have the greatest impact on the teaching & learning of the greatest number of faculty and students?
2. Aim for an impact on Roger’s Early Adopters and Early and only slightly Late Majority
Adapted From Sdmagazine.com
Keyek-Franssen & Briggs
Embedded PrinciplesHow do we invest our limited resources in instructional technology to
have the greatest impact on the teaching & learning of the greatest number of faculty and students?
3. Emphasis on undergraduate education4. We don’t know what we don’t know
Keyek-Franssen & Briggs
Less than PerfectBackward Marketing
How will we know what data will result in what decisions?
1. Number of responses?
2. Passion of responses?
3. Differences among schools and colleges?
4. Political position of responses?
Keyek-Franssen & Briggs
Determining information needed
• Still worked within framework of existing technologies: learning management systems, classroom technologies
• For student data, also included communication technologies and administrative applications
• Did *not* ask about video, overhead projectors, slide projectors--because that information wouldn’t influence our decisions.
Keyek-Franssen & Briggs
Sorta Curiosity Questions
• Who’s using what?
• How’s it working?
• What current use and support could be improved?
Keyek-Franssen & Briggs
Strategic Questions
• What would you like to use but aren’t?
• What would enable that use?
• How can technology improve teaching and learning?
Keyek-Franssen & Briggs
Research Partnerships
• Institutional Research
• IT Services
• IT Governance Groups
Keyek-Franssen & Briggs
Why IR?• Expertise• Good citizens• Understanding of survey cycle• Ability to sample• Sweet survey application• Non-IT branding• Staff• Expertise!
Keyek-Franssen & Briggs
Why ITS?• Empirical data• Potential impact to unit
Keyek-Franssen & Briggs
Why IT Governance Groups?• Political vetting• Audience expertise• A nudge toward the strategic
Keyek-Franssen & Briggs
Good Methods, Good Data
• Data mining
• Surveys
• Focus groups
Keyek-Franssen & Briggs
Questions already answered: data mining
• Number of students served by WebCT? (over 23,000)
• How many courses use “clicker” technology? (about 60, with approximately 4000 students total)
• What instructional technology questions come through the help desk? (WebCT password problems and urgent classroom technology glitches)
• How many “smart” classrooms are managed centrally (just over 100)
Keyek-Franssen & Briggs
Survey Design• Faculty survey
– Spring 2005
– About 40% of the faculty (statistical validity & good citizenship), only 33% response rate, lower on certain questions
– Stratified by school & college
• All-faculty survey– Fall 2005
– Somewhat shorter version
– Opportunity for all faculty to respond
Keyek-Franssen & Briggs
Student Survey• Fall 2005• Stratified by year & declared major• 2000 undergraduates (just under 10%)
and 1000 graduates (about 20%)• Low response rate (25%)• Pivot questions to allow additional
information to be gathered about laptop owners, graduate students, and teaching assistants
Keyek-Franssen & Briggs
Student Survey
• Important points– Ownership not as crucial as use (How often do you
bring your laptop to class? For more than taking notes?)
– Want to understand if there is a gap between faculty use (even intended) and student expectations or preferences (student expectations can be a driver for change)
– Teaching assistants asked both student and faculty questions
Keyek-Franssen & Briggs
Effective Practices & Caveats
• Online surveying• Sampling• Short survey• Incentives• Good survey• Pilot, pilot, pilot• Combine questions
• Second bias• Not all voices heard• But, but, but…• Don’t always work• No caveat here!• It’ll cost ya• Increases complexity
Keyek-Franssen & Briggs
Combining Questions
Keyek-Franssen & Briggs
Focus Groups & Interviews
• Cutting edge
• “Normal Joe & Josephine”
• Distributed Academic Technology Coordinators
• Research computing
• Digital Asset Management
Keyek-Franssen & Briggs
Effective Practices & Caveats: Focus Groups
• Conversation• Informal• Focused
• Tentative• Objectivity difficult• Hard to generalize
Keyek-Franssen & Briggs
In-Class Use: Frequency and Importance
Tech use Frequency Important
Laptop as chalk replacement
60%all or regularly
73%
Laptop beyond chalk 33% all or regularly
72%
Feedback systems 15% all or regularly
45%
Keyek-Franssen & Briggs
Online components: Frequency and Importance
Component Frequency Important
Syllabus 60% 65%
Readings 55% 64%
Interactive simulations
14% (24% more would like to use)
33%
Keyek-Franssen & Briggs
Enablers to Using Technology
Enabling factor Would it help?
More time 40%
Training 56%
Money 31%
Staff 30%
Tenure & promotion 27%
Keyek-Franssen & Briggs
Open-ended comments
• Hey! What about VHS, PAL, DVDs, slides, overheads, demonstrations, chalk, and rocks?
• “I still use chalk…nothing beats chalk’s flexibility, at least nothing I’ve seen yet.”
Keyek-Franssen & Briggs
The clicker debate…
• “Clickers…are nothing but a heavy-handed way to force attendance. [They] measure outcome and not process.
• “It is time for the university to realize that this is an important technology and get ITS to support it.”
Keyek-Franssen & Briggs
…morphs into musings about pedagogy.
• “I am not at all persuaded that clickers work better…as the kinds of interactive learning we already use, which do not require technological assistance.”
• “I am not comfortable with pre-prepared presentations that seem to thwart questions that lead to important diversions.”
Keyek-Franssen & Briggs
But all roads lead to laments...
• “I would use the smart classroom stuff more if the equipment worked better.”
• “I’d like tech assistance and repair services available all the time.”
• “[We need] better maintenance of the equipment available in the classrooms.”
• “Updated facilities!”
Keyek-Franssen & Briggs
…and comments to keep us honest.
• “I prefer to spend my time learning the subject matter better. I find instructional technology pretty boring actually.”
• “As with any technologies, these are just tools and it looks like that has been forgotten.”
Keyek-Franssen & Briggs
Work yet to be done:
• In-depth analysis (including open-ended)
• Compare faculty use and desires with student expectations
• All-faculty survey
• Focus groups
• Communication
• Plug-in to the decision-making process
Keyek-Franssen & Briggs
Effective Practices To Go• Know where you’re going (backward
marketing can help)• Make good use of strategic partnerships
on campus• Employ a mix of data collection methods• Aspire to high research standards• Don’t forget the grain of salt.• You’ll find flaws afterwards.• Expect unexpected consequences.
Keyek-Franssen & Briggs
Beyond Chalk: Determining Faculty Needs for Instructional Technologies and Support
Deborah Keyek-Franssen
Charlotte Briggs