A Dissertation Study

ONLINE PEDAGOGICAL QUALITY QUESTIONED: PROBING INSTRUCTIONAL DESIGNERS’

PERCEPTIONS OF LEADERSHIP COMPETENCIES CRITICAL TO PRACTICE

ByMarcia L. AshbaughCapella UniversityDoctoral ResearcherMentor: Dr. Rod Sims

Context of Study- Observations

Naidu (2003) found “a schedule, brief outline of the course content, PowerPoint slides of the lecturer’s notes, and sometimes, sample examination papers” (p. 349)

and not much more.

Inferior Online Course Quality

New Mode of

Learning

Common Issues

• Economic shortages

• Team motivation

• Staying current

Context of Study - Environment

Expert Designers

Academics

Online

Research Focus

Quality of OnlineLearning Course DesignPedagogies Explored for…

…Influence of Leadership

Competencies on Quality

Key Research - Quality

QualityPhipps & Merisotis (1999); Russell (2001); Cavanaugh (1999); Warren and Holloman (2005) > No significant difference

Clark (1994); Merrill (2005) > Computers only tools

Dede et al. (2007) > Positive effect in online communities

Der-thanq, Hung, and Wang (2007) > Traditional pedagogies = low course effectiveness

NSSE (2008) > Learning sought on deeper level; increased collaboration

Allen & Seaman (2006); Seaman (2009) > Faculty consider online inferior

Bernard et al. (2009) > Aligned strategies/learning context

Key Research - Competency

ID CompetencyIBSTPI (2000); Spector et al. (2006) ~ 23 ID competencies

Leadership CompetencyDooley and Lindner (2007) ~ Increased leadership competencies

Scott, Coates, and Anderson (2008) ~ Poorly understood

Fullan and Scott (2009) ~ Leadership lacking

Key Research - Leadership

Leadership in PracticeCampbell, Schwier, and Kenny (2009) ~ Perceptions of change agent, leader

Kim, Baylor, and Group (2006); Velletsianos and Miller (2008) ~ Leadership exhibited in contemporary onlinelearning tools

Gessick and Derry (2010) ~ Leadership competencies revealed during asynchronous activities meaning

DeBlois (2005) ~ Leadership competencies linked to success for advancing ID field

Theoretical Construct

Central Research Phenomena:

Quality of online

academic pedagogies.

Instructional Design Competency

Leadership Quality

Theoretical Framework

RESEARCH PROBLEM AND PURPOSE

THE PROBLEM

Lack of online course quality

THE PURPOSE

◦Investigate current quality status

◦Relationship to ID leadership competencies

◦Effect on learners

Research Questions

What instructional design leadership competencies are identified as critical to creating quality online learning designs?

What are the characteristics of courses created by participants who have identified critical leadership competencies?

How do students evaluate the quality of courses by those using the identified critical leadership competencies?

Methodology-Qualitative

Population/Sample◦Instructional designers

◦AECT members

◦6 experts

◦Higher academic roles

◦US and Canada

Data Collection◦In-depth interviews

(Masterman et al., 2009)

◦Course designs◦Student evaluations

Instrumentation◦Work samples◦Researcher/

interviewer

• Bracketing (Kaplan, 2004; Moustakas, 1994)

Methodology-Process

Data Analysis

Interviews◦NVivo9 software

◦Constant comparison (Berelson, 1952; Gelo et al., 2008; Richey & Klein, 2007)

◦Themes and categories (Masterman et al., 2009)

• Documents• Content analysis• Quality Matters™ rubric• O/A/S/A alignment • (Sims, in press)• Student engagement (NSSE, 2008)• Student satisfaction• Perceived course quality

• Journals• Reflexivity (Kaplan,

2004; Moustakas, 1994)

• Research (Seidman, 2006)

Competencies for Practice

Competencies for Designs

Strategy

Vision

Personality

Productivity

Student Engagement ◦ Learner-Centered◦ Personal Knowledge

ControlStudent Satisfaction

◦ Perception of Quality◦ OASA Alignment

QUALITY

SUMMARY OF FINDINGS

RQ1 Analysis“What instructional design leadership competencies are identified as critical to creating quality online learning

designs?”

Themes and Categories

Vision

Prod-uct-iv-

ity

Strategy

Competencies

Values

Personality

Emo-tional Strength

Attributes

Mentor-ing

Duties

Response

Participant Words on Leadership:P3 ~Strategy - knowledge of current theoriesProductivity - best practicesP1 ~Personality – collaboration; clear

communicationVision – convey purposeP2 ~Mindset – ready for change

Implications

StrategiesContextualized learningEngagementResearch-based strategiesEthical practicesCollaboration, connectedness

VisionStewards, change

agentsGlobal thinking,

collaborationOpen-source LO’sWikis, Blogs

PersonalityCommunicationWithout it, leaders failVital to quality

ProductivityAssumes responsibilityLeaders as producerImpacts the learner

Themes and Categories

RESULTS of DATA ANALYSIS FOR RESEARCH QUESTION 2

RQ2 Analysis“What are the characteristics of courses created by participants who have identified critical leadership

competencies? ”

10%15%

10%65%

Competencies in Participant Designs

ActivitiesAssessmentsObjectives/OutcomesInstructional Strategies

31%

31%

15%

8%

8%8%

Instructional Strategies by Cat-egory

Authentic TasksInteractionLearner-control TasksProblem-solvingTheory-based Instruc-tionValues-based Instruc-tion

Response

Course Design Results by Quality Matters

Category Points achieved/Points possible

Percentage standards met

Learning objectives

8.2 our of 9 91.1% 

Assessment and measurement

11.4 out of 13 87.7% 

Learning engagement

8.6 out of 10 86.0% 

Total 28.2 out of 32 88.1% 

Implications

Objectives

Strategies

Activities

AssessmentsQUALITY

Themes and Categories

RESULTS of DATA ANALYSIS FOR RESEARCH QUESTION 3

RQ3 Analysis“How do students evaluate the quality of courses by those

using the identified critical leadership competencies?”

Student Engagement = 90%

Student Satisfaction = 87%

High Course Quality = 90%

Response

Student evaluations – are they about the teacher or the student?

P6 ~“The purpose for the instructional design

must be the learningand [the] change of the learners”

Implications

Quality

Design decisions

Engagement

Learner-control

Satisfaction

Authentic tasks

Future Research

Research Ideas

Leadership principles in action

Case studies, ethnographies of designers

Impact of principles on learning

Action research in course rooms

Leadership in design process to validate models, metrics

Design and development research

The 7 Ps of Leadership Model

Principles of Leadership(to Possess or Engage In)

Prescience (Vision)Preventive/Proactive Thinking

(Strategy)Provision for the Unexpected

(Strategy)Personality (Interpersonal Skills)Productivity (Productivity)Psychological Toughness

(Decision-making)Personal Convictions (Values)

Kowch (2009) ~“New leaders need new ways to address these issues beyond the instructional leadership literature, which was found lacking due to a classical focus on the supervision of instructors/teachers more than with the leading of the instructional process (design and development, as we know it).”

Significance to Instructional Design Field

Leadership

Online Registrations Globe Unraveling Mobile Learning