Young 2003 - Enhancing Learning Outcomes- The Effects of Instructional Technology, Learning Styles, Instructional Methods and Studen Behavior

7/30/2019 Young 2003 - Enhancing Learning Outcomes- The Effects of Instructional Technology, Learning Styles, Instructional

1/14

http://jmd.sagepub.com/Journal of Marketing Education

http://jmd.sagepub.com/content/25/2/130The online version of this article can be found at:

DOI: 10.1177/0273475303254004

2003 25: 130Journal of Marketing EducationMark R. Young, Bruce R. Klemz and J. William Murphy

Methods, and Student Behaviornhancing Learning Outcomes: The Effects of Instructional Technology, Learning Styles, Instructi

Published by:

http://www.sagepublications.com

can be found at:Journal of Marketing EducationAdditional services and information for

http://jmd.sagepub.com/cgi/alertsEmail Alerts:

http://jmd.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://jmd.sagepub.com/content/25/2/130.refs.htmlCitations:

What is This?

- Aug 1, 2003Version of Record>>

at UNIV DE LOS ANDES on April 30, 2013jmd.sagepub.comDownloaded from
http://jmd.sagepub.com/http://jmd.sagepub.com/http://jmd.sagepub.com/http://jmd.sagepub.com/content/25/2/130http://jmd.sagepub.com/content/25/2/130http://www.sagepublications.com/http://jmd.sagepub.com/cgi/alertshttp://jmd.sagepub.com/cgi/alertshttp://jmd.sagepub.com/subscriptionshttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsPermissions.navhttp://jmd.sagepub.com/content/25/2/130.refs.htmlhttp://jmd.sagepub.com/content/25/2/130.refs.htmlhttp://jmd.sagepub.com/content/25/2/130.refs.htmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://jmd.sagepub.com/content/25/2/130.full.pdfhttp://jmd.sagepub.com/http://jmd.sagepub.com/http://jmd.sagepub.com/http://online.sagepub.com/site/sphelp/vorhelp.xhtmlhttp://jmd.sagepub.com/content/25/2/130.full.pdfhttp://jmd.sagepub.com/content/25/2/130.refs.htmlhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsReprints.navhttp://jmd.sagepub.com/subscriptionshttp://jmd.sagepub.com/cgi/alertshttp://www.sagepublications.com/http://jmd.sagepub.com/content/25/2/130http://jmd.sagepub.com/


2/14

ARTICLEAUGUST 2003JOURNAL OF MARKETING EDUCATION

Enhancing Learning Outcomes:

The Effects of Instructional Technology,

Learning Styles, Instructional Methods,and Student Behavior

Mark R. Young, Bruce R. Klemz, and J. William Murphy

Thedeliveryof marketing educationseemstobe rapidly shift-

ing toward pedagogy rich in experiential learning and

strongly supported with educational technology. This studyintegrates and extends previous research efforts and investi-

gates the simultaneous effects of multiple influences of tech-

nology and nontechnology factors on learning outcomes.

Responses wereobtained acrossa marketingcurriculum with

technology-accustomed students. The findings suggest that

the use of preferred instructional methods will enhance each

of the three different measures of learning outcomes, while

encouraging supportive class behaviors can increase self-

report performance and course grade. Regardless of the

dependent outcome measure, only one of the five instruc-

tional technology variables proved significant, suggesting

that in contrast to previous studies that examined technology

in isolation, when analyzed relative to other learning factors,technologys influence is secondary. Implications are dis-

cussed with practical suggestions for the classroom and

direction for further investigation.

Keywords: pedagogy; instructional technology; learning

styles; student behavior; learning outcomes

Online media-rich e-books, Internet-enhanced cases, chatrooms, electronic bulletin boards, CD-ROMs, electronic

libraries, laptop computers, and an ever-expanding array ofinstructional technologies promising to engage and motivate

students, accelerate learning, and increase the economic

worthof students soundsenticing, but does it work?Certainly

thepractice of marketingcompanies andentire industries has

been transformed in effectiveness and efficiencies by the

deployment of information technology; will the same be true

in marketing education? The answer to both these questions

rests on the scholarly investigation of the impact that various

educational tools, pedagogies, and other learning-related fac-

tors have on learning outcomes.

Initial scholarly investigationhas producedsomeinforma-tive guidance on factors that influence the selection of

instructional technology resources (Strauss and Frost 1999),

recommendations on technologytools to achievespecificstu-

dent outcomes (Clarke, Flaherty, andMottner 2001), types of

student behaviors that affect performance (Brokaw and Merz

2000), preferredlearningstyles of marketing majors (Stewart

and Felicetti 1992), and how pedagogical preference affects

attitudes toward the major (Davis, Misra, and Van Auken

2000). Many of these current studies attempt to identify how

specific typesof instructional technologyor pedagogical fac-

tors affect learning. However, the reality of most classroom

environments is that there is a multitude of instructional fac-

tors that produce a joint effect on learning, thereby limitingthe usefulness of the reported effects of a specific instruc-

tional technology examined in isolation.

Further limitations of previous research include single-

item measures, lack of comparisons to nontechnology

pedagogies, measuring only attitudes and not performance,

sampling from a single technology-based course, and exam-

ining a narrow set of predictors of performance. The purpose

of this study is to provide an exploratory next step in this

evolving research by extending and integrating these previ-

ous research efforts on the investigation of the impact of

instructional technologies, learning styles, instructional

methods, and student behaviors on learning outcomes as pre-

sented in the conceptual framework in Figure 1. Specifically,

130

Mark R. Young is a professor of marketing, Bruce R. Klemz is an associate

professor of marketing, and J. William Murphy is a professor of business

education, all in the Department of Marketing at Winona State University,

Winona, Minnesota.

Journal of Marketing Education, Vol. 25 No. 2, August 2003 130-142DOI: 10.1177/0273475303254004 2003 Sage Publications

http://jmd.sagepub.com/http://jmd.sagepub.com/http://jmd.sagepub.com/http://jmd.sagepub.com/


3/14

these impacts are examined across the marketing curriculum

rather than a single class, multi-item measures of both affect

and learningperformance areemployed,both technologyand

nontechnology pedagogies are included, as well as student

behavior and preferred learning styles.

DEFINITIONS OF LEARNING OUTCOMES

The use of multiple outcome variables in an educational

setting is recommended to help ensure that themultiple goals

and the multiple dimensions of outcomes in the classroom

environment are represented (Marks 2000; Williams 1992).Many measures of learning outcomes have been used in edu-

cational research including course grade (Brokaw and Merz

2000; Devadoss and Foltz 1996; Romer 1993), student per-

ceptions of overall learning, ability to get a job and expected

performance on the job(Clarke, Flaherty, andMottner 2001),

task performance and goal achievement (Deeter-Schmelz,

Kennedy, and Ramsey 2002), overall course value percep-

tions (Marks 2000), and exam scores (Hamer 2000; Ritchie

and Volkl 2000).

However, from a theoretical standpoint, learning may be

viewed as knowledge acquisition through cognitive process-

ing of information acquired both from being part of society

and from individual thought processes (Bandura 1986). In

addition, performance can be defined as a multidimensional

construct involving the behaviors or actions that are relevant

to the goals of the course with three primary determinants of

relative variance: (1) declarative knowledge and procedures

that are prerequisites for successful task performance, (2)

procedural knowledge and skills, and (3) volitional choice or

effort expended (McCloy, Campbell, and Cudeck 1994).

Therefore, combining the two conceptual definitions of

learning and performance provides an outcome variable

called learning performance, which will be defined as stu-

dents self-assessment of their overall knowledge gained,

their skills and abilities developed, and the effort they

expended in a particular class relative to other classes.

In addition, favorable attitudes or affect have been shown

tobe theresult of using instructional methods that arecongru-

ent with preferred learning styles (Goodwin 1996; Davis,

Misra, and Van Auken 2000) and have been correlated to

other measures of course achievement (Dunn et al. 1990).

Therefore, learning outcomes in our study were represented

with the two self-report outcome variables learning perfor-mance and pedagogical affect in addition to the commonly

used course grade outcome variable. Each of the two self-

report variableshas appearedin priormarketing educationlit-

erature and has been shown to have exhibited sound

psychometric properties involving multi-item scales (Davis,

Misra, and Van Auken 2000; Young 2001).

HYPOTHESES DEVELOPMENT:ANTECEDENTS TO LEARNING OUTCOMES

Learning Styles

The manner and process in which knowledge is acquired,

skills developed, and abilities refined distinctly vary among

individuals, producing a typology of learning styles. Kolbs

(1984) experiential learning theory describes a four-stage

sequential process for creating knowledge through the trans-

formation of experience. A persons preference for which

stage of the learning cycle he or she prefers and which stages

he or she tends to avoid creates a four-category learning style

typology (Convergers, Assimilators, Accommodators, and

JOURNAL OF MARKETING EDUCATION 131

Antecedents Outcomes

LearningStyles

Instructional

Technology

Instructional

Methods

Student

Behaviors

Learning Outcomes

Learning Performance

Pedagogical Affect

Course Grade

FIGURE 1: Conceptual Framework of Factors Affecting Learning Outcomes



4/14

Divergers). Petkus (2000) and Young (2002) provide recent

overviews of the experiential learning cycle, along with

examples of its application in marketing education.

Kolb (1988) suggests that students with similar learning

styles prefer academic disciplines and teachers with methods

of teaching that are most congruent with their learning style.

In addition, there is empirical evidencethat learningstyles are

also highly related to work preference (Lashinger and Boss1984), educational involvement, motivation and learning

(Honey and Mumford 1992), and student performance

(Brokaw and Merz 2000; Holley and Jenkins 1993;

Okebukola 1986; Roach, Johnston, and Hair 1993). Hence,

we propose the following hypotheses; however, given the

exploratory nature of this study, we do not hypothesize about

particular learning styles and particular preferred methods or

technologies. Instead, post hoc analyses will be undertaken if

support for the general hypotheses is provided.

Hypothesis 1a: Learning style will account for variation in pre-

ferred instructional technologies.

Hypothesis 1b: Learning style will account for variation in pre-

ferred instructional methods.

Hypothesis 1c: Learningstyle willaccount forvariation in learn-

ing outcomes.

Instructional Technology

As technological capabilities expand, academicsand busi-

nesses are rapidly integrating technology into their class-

rooms and operations to provide a competitive edge. The

study of individual reactions to computer technology and

Internet usage in business has been researched from a variety

of theoretical perspectives, including rate of adoption (Hill,

Smith, and Mann 1987), diffusion of innovations (Compeau

and Meister 1997), theory of reasoned action (Webster and

Martocchio 1992), and social cognitive theory (Compeau and

Higgins 1995). Reactions to integrating technology into the

classroom have been primarily anecdotal or at an aggregate

level of performance. John Schacter (1999) provides a com-

prehensive review of research regarding the impact of tech-

nology on student learning. Evidence from Schacters review

indicates that both positive and negative outcomes can be

realized when technology is integrated into the learning

environment.

More recently in themarketingeducation literature,a pos-

itive relationship was found between self-reported overall

learning and 9 of 14 educational tools (instructor home page,Internet project, online homework assignments, online lec-

ture outlines, online syllabus, online student roster age,

online student grade page, Web project page, and technology

lectures) (Clarke, Flaherty, and Mottner 2001). In addition,

Stttinger and Schlegelmilch (2002) reported that students

perceive instructional technologies to be advantageous based

on their perceptions of the course and career-related benefits

of using the technology and the amount of exposure the stu-

dents have had to the technology. The use of instructional

technologies has the potential to more actively involve and

motivate students, thereby enhancing student learning out-

comes. Consequently, we hypothesize the following:

Hypothesis 2: When student preferred instructional technolo-

gies are used, student learning outcomes will increase.

Instructional Methods

A preponderance of marketing education literature sug-

gests a shift from passive, knowledge-transfer instructional

methods to interactive, experiential learning (Frontczak

1998). Empirical evidence supports that business students

prefer pedagogies that are active and concrete (Nulty and

Barrett 1996), prefer learning with other students (Matthews

1994), and prefer instructional pedagogies that are stimulat-

ing and real-world oriented (Karns 1993).

Numerous specific instructionalmethods have been inves-

tigated such as the use of in-class exercises, cases, and lec-

tures that produced a favorable global attitude toward the

marketing major (Davis, Misra, and Van Auken 2000); com-

bining writing and electronic media (McNeilly and Ranney

1998); group research projects (Bridges 1999); group pro-

jects and teamwork (McCorkle et al. 1999), and the effect of

class activities on student learning (Hamer 2000). Evidence

also suggests that favorable attitude toward teaching style

leads to higher achievement (Johnson 1996) and that match-

ing instructional methods with learning styles results in

greater learning (Dunn et al. 1990). Therefore, we hypothe-

size the following:

Hypothesis 3: Whenstudent-preferred instructional methodsare

used, student learning outcomes will increase.

Student Behavior

Learning outcome, typically measured by course grade,

has been directly related to supportive-type class behaviors

such as class attendance (Devadoss and Foltz 1996; Romer

1993), in addition to the number of hours spent studying per

week, lectures attended, reading the textbook, and taking

optional exams (Brokaw and Merz 2000). On the other hand,

the amount of competing time activities such as number of

hours worked, the hours spent socializing or in sports, and

total credithourstaken duringthe term were found tobenega-

tively related to learning outcomes (Brokaw and Merz 2000;

Erekson 1992). Each of these studies has found that class-room-related student behaviors can be empirically related to

learning outcomes, suggesting the following hypotheses:

Hypothesis 4a: Studentbehaviors thatare course supportivewill

be positively related to student learning outcomes.

Hypothesis 4b: Studentbehaviors thatare competing timeactivi-

ties will be negatively related to student learning outcomes.

132 AUGUST 2003



5/14

METHOD

The data werecollected at the end of fall semester 2001 by

administering an in-class survey to each section of Principles

of Marketing, Market Analysis, Marketing Planning, and

Marketing Management, the required core courses in the

Marketing curriculum at a midwestern 4-year public univer-

sity. The university requires all students to lease laptop com-puters and provides complementary computer projectionand

communication technology for most classrooms. The pri-

mary useof instructional technology in thePrinciples of Mar-

keting courseis toassistthe instructor in lecture presentations

and communication of assignments and grades to the stu-

dents.Students arenot required to bring their laptops to class;

however, many of the students do use their laptops to com-

plete homeworkassignments outside of class. In contrast, the

other three marketing classes require laptops in the class-

room, and class activities typically are based on computer

usage. Examples of computer applications range from statis-

tical analysis, Internet searches, presentation creation, and

computer simulations. The sequence of courses is alsodesigned to systematically expose students to a variety of

instructional methods. Market analysis is structured around

group research projects, marketing planning uses Internet

research to analyze cases, and marketing management is

structured around decision making based on computer simu-

lations; in addition, all classes require written communica-

tions andoralpresentations.In summary, thecurriculumdoes

expose students to each of the instructional methods and

instructional technologies being investigated in this study.

A typical absenteeism rate on the day of the survey pro-

duced a response rate of approximately 78%, yielding an

effective sample of 207. The distribution of the completed

sampleacrossclasses wasPrinciples of Marketing(threesec-

tions), n = 122(59%);Market Analysis (two sections), n = 39

(19%); Marketing Planning, n = 29 (14%); and Marketing

Management, n = 17 (8%). Demographically, the sample can

be describedas traditional undergraduates, 42% female, 31%

marketing majors, and 16% marketing minors; in addition,

the Principles of Marketing students closely mirrored the

College of Businesss distribution of majors (accounting

19%, business administration 42%, marketing 18%, and

other business 21%).

VARIABLES

Learning Outcomes: Dependent Variables

Learning performance. Learning performance was

operationalized usingsix items (knowledge yougained, skills

you developed, effort you expended, your ability to apply the

material, your desire to learn more about this subject, your

understanding of this subject) measured with 6-point scales

verballyanchored with extremely high (a level rarelyattained

in other classes) to very low (much below that of other

classes), which is a modification of a performance scale

reported by Young (2001).

Pedagogical affect. Affect representsthepositive thoughts

or feelings toward the instructional methods deployed in a

particular class. The statement Overall, in this class, the

methods of instruction were . . . was responded to with foursemantic differential-type items measured on a 7-point scale.

The four scales (effective/ineffective, useful/useless, satis-

factory/unsatisfactory, good/bad) for evaluating this overall

affect were created from a scale developed by Mitchell and

Olsen (1981) and then adopted by Davis, Misra, and Van

Auken (2000) to measure the overall affect of marketing

majors toward instructional effectiveness and program

quality.

Course grade. The instructor-assigned grade in the course

is also used as a measure of learning outcome. Following the

definition and scaling of course grades used by Brokaw and

Merz(2000),grades had a range of0 (an F)to 4 (an A)andare

treated as a metric variable.

Independent Variables

Learning styles. TheKolb Learning Style Inventory (Kolb

1984) measures studentslearning style preference by having

the students rank four statements for each of the 12 items

making up the inventory. Twoprimarydimensions arecreated

from the four stages, ACCE is the dimension created by sub-

tracting the scores for the Concrete Experience (CE) scale

from the Abstract Conceptualization (AC) scores, while the

AERO dimension represents the difference between the

Active Experimentation (AE) scores and the Reflective

Observation (RO) scores. The four quadrants created by thetwo dimensions represent the four types of learning styles:

Convergers (high ACCE and high AERO scores), Assim-

ilators (high ACCE and low AERO scores), Accommodators

(low ACCE and high AERO scores), and Divergers (low

ACCE and low AERO scores).

Instructional technology. Instructional technology covers

a broad spectrum of options ranging from videotapes to

sophisticated computer-based instructional programs. Five

instructional technologies (e-mail, Internet access,

PowerPoint presentation, Blackboard course management

software, and laptop computers) listed in Grasha and

Yangarber-Hicks (2000) and that had been deployed acrossthe courses sampled in this study were rated on a 7-point

effective/ineffective semantic differential scale. The five

instructional technologies were evaluated based on the ques-

tion Ingeneral,for any class,whichtechnologiesdo youfind

most effective in helping you learn?

Instructional methods. Nine commonly used teaching

methods (Davis, Misra,andVan Auken 2000) were rated on a




6/14

7-point effective/ineffective semantic differential scale as to

the statement In general, for any class, which methods of

instruction do you find most effective in helping you learn?

The nine instructional methods are instructor lectures, cases,

computer simulations, group projects, individual projects,

exams, class discussions, in-class exercises, and written

assignments.

Student behavior. Two supportive student behaviors (aver-

age class attendance, hours studied for this class) and two

competing time behaviors (hours involved with social or

sports organizations, hours worked) were measured with fill-

in-the-blankresponses asused byBrokaw andMerz(2000).

RESULTS AND DISCUSSION

Before the overall model displayed in Figure 1 was esti-

mated, we investigated the potential preference for instruc-

tional methods and instructional technologies based on the

four underlying learning styles. Each student was classified

into one of the four learning styles based on Kolbs Learning

Inventory method with a graphic overview of the samplepre-

sented in Figure 2. Each of the four learning styles is ade-

quately represented, ranging from 19%to 36%of thesample.

One-way analysis of variancewas used to test Hypotheses 1aand 1b, and the results are displayed in Table 1.

Differences between learning styles did not significantly

(.05 level of significance)account for variation in preferences

for instructional technology. Therefore, we cannot accept

Hypothesis 1a and conclude that preference for different

instructional technologies is not dependenton a students pre-

ferred learning style. The lack of a significant relationship

between learning style and instructional technologies may

suggest students view the technology simply as a tool that is

involved in implementing the instructional method. In addi-

tion, a particular instructional technology can be employed

with great variation within different instructional methods.

For example, PowerPoint may be used to assist an instructor

with the traditional lecture, or it may be used by student

groups to present findings from their experiential learning

activities. Whereas the literature seems to be lacking in the

investigation of learning style and instructional technology

preference, this study suggests that a students preference for

instructional technology is not inherently based on funda-

mental learning style.Three instructional methods (lectures, exams, and written

assignments) had significant differences based on learning

styles, supporting Hypothesis 1b. In particular, accommo-

dators (prefer concrete experiences and active experimenta-

tion) rated lectures and exams lower than the students with

other preferred learning styles. Interestingly, Brokaw and

Merz (2000) suggest that accommodators tend to prefer mar-

keting careers and, with the trend in marketing education

toward experiential learning, these findings may be inter-

preted as support for the direction marketing education has

taken.

In addition, writing assignments were evaluated highest

by assimilators (high abstract conceptualization and reflec-tive observation). Writing assignments can encourage stu-

dents to explore and incorporate abstract concepts into their

learning and are typically the basis for reflection-type

activities.

These findings arecongruentwith the literature and, given

the results (three of the nine learning methods), we find sup-

port for Hypothesis 1b that different learning styles can

account for different preferences for instructional methods.

134 AUGUST 2003

3020100-10-20-30

30

20

10

0

-10

-20

-30

Assimilating Converging

AccommodatingDiverging

Active Experimentation Reflective Observation Scale

AbstractConceptualization-Con

creteExperienceScale

FIGURE 2: Distribution of Kolbs Learning Styles (N= 207)



7/14

Knowinga students learningstyle mayassist the instructor in

selecting appropriate teaching methods (Brokaw and Merz

2000) or recognize that multiple instructional methods must

be incorporated into classes with wide distributions of learn-

ingstyles. This maybe particularlyrelevant with courses that

involve students with different cultural backgrounds in thatJaju, Kwak, and Zinkhan (2002) reported significant differ-

ences in learning styles between different cultures.

The conceptual model displayed in Figure 1 was

operationalized with three metric criterion variables (learn-

ing performance, pedagogical affect, and course grade) that

were analyzed with a set of predictor variables composed of

three metric covariates (instructional technology, instruc-

tional methods, andstudent behavior) andonenonmetric fac-

tor (learning styles) having four levels. Regression analysis

was used to reveal which instructional technologies, instruc-

tional methods, student behaviors, and learning styles covary

with learning outcomes and can reveal which explanatory

variables are most determinant of learning outcomes. Giventhat the two self-report criterion variables are correlated (r=

.45,p = .000), performing separate regression analyses would

not incorporate the information providedby the interrelation-

ship among these criterion variables and would defeat the

purpose of having multiple criterion measures. Therefore,

multivariate multiple regression analysis was performed

using the general linear model multivariate procedure in the

Statistical Package for the Social Sciences software.

First, the two self-report outcome variables, pedagogical

affect and learning performance, were assessed for internal

consistence and reliability. The results, presented in Table 2,

indicate a Cronbachs alpha of .89 and .80, suggesting robust

scales as compared with Nunnallys (1978) recommendation

of at least a .70 level. The factor loadings present evidence ofthe dimensionality of the two constructs. Two factors were

extracted using principle components analysis and varimax

rotation.The totalvarianceexplained was62%,andeach item

did load on the expected dimension with all but two loadings

above Fornells (1982) recommendation of .70 or higher for

retainingitems, since they explainalmost50% of thevariance

ina particular construct. In summary, thereliabilityof thetwo

outcome scales seems satisfactory.

These two dependent variables were also examined for

departures from multivariate normality by performing

Kolmogorov-Smirnovs (Lilliefors significant correction)

test of normality and by examining normal Q-Q plots. The

results (learning performance, p = .001; pedagogical affect,p = .000) of these tests suggest no departures from normality.

In addition, Boxs test of equality of covariance matrices of

the dependent variables across groups (p = .258) and

Levenes test of equality of error variances across groups for

each of the dependent variables (performance, p = .242;

affect,p = .445) could not be rejected; therefore, it seems rea-

sonable toproceed with themultivariate analysis.The follow-

inganalyses were performedboth with the twooutcome vari-


TABLE 1

LEARNING STYLE DIFFERENCES: ONE-WAY ANOVA RESULTS (N= 207)

Learning Stylesa

Variable Accommodator Diverger Assimilator Converger F-Value

Instructional technology

E-mail 5.08 4.51 4.64 4.65 1.10

Internet access 5.97 5.83 5.88 5.88 0.81PowerPoint presentation 5.62 5.71 5.57 5.31 0.78

Blackboard software 4.51 4.54 4.31 3.69 2.17

Laptop computer 5.62 5.27 4.96 4.92 1.59

Instructional methods

Instructor lectures 4.26 5.41 5.21 4.98 6.50*

Cases 4.82 5.10 4.77 5.12 0.89

Computer simulations 4.72 5.05 4.64 4.50 0.98

Group projects 5.21 5.29 4.73 4.96 1.47

Individual projects 4.92 4.98 5.00 5.10 0.17

Exams 4.10 4.59 5.04 4.73 3.31*

Class discussions 5.72 6.02 5.59 5.67 1.89

In-class exercises 5.72 5.98 5.61 5.79 0.87

Written assignments 4.41 4.68 5.01 4.48 2.47*

Learning outcomes

Learning performance 4.52 4.26 4.17 4.29 2.39Pedagogical affect 5.54 5.75 5.47 5.58 .79

Course grade 3.35 2.93 3.40 3.17 2.34

a. Means. Degrees of freedom: between groups 3, within groups 203, except for course grade (n= 93).*p< .05.



8/14

ables factor scores and with the variables represented as an

average of the items for each scale. Virtually identical results

were obtained, and therefore, the simpler and more intuitive

averaging of items to represent the outcome variables is

presented.

Next, an examination of the predictor variables correla-

tion matrix was performed and revealed several moderately

high correlations (e-mail and Web access, r= .62; lectures

and exams, r= .40; simulations and cases, r= .48; individual

projects and writing assignments, r= .41; class exercises and

class discussions, r = .44), suggesting the potential of

multicollinearity and requiring caution in the interpretation

of the regression results. It seems intuitive that the correlatedpairsof technologies andpedagogiesare mostlikelyavailable

and used in combinations in the classroom, and therefore,

would be expected to be correlated. These variables could be

reduced through factor analysis to solve the statistical prob-

lem of multicollinearity; however, the explicit relationships

among the variables would be lost. The robustness of regres-

sion analysis to multicollinearity for variables with correla-

tions below .50 is typically accepted (Tull and Hawkins

1990). However, a resultingconsequence of examiningmany

variables simultaneously that exhibit multicollinearity is that

the standard errors of the regression coefficients will tend to

be large, thereby artificially lowering their t-values (Dillion

andGoldstein1984). Whereas specificsignificance levels arereported in the tables, we provide the following interpreta-

tions based on a .10 level of significance to compensate for

the inflated standard errors. Given the exploratory nature of

this study, examining these multiple factors simultaneously

mayproduce results that provide valuable insights despite the

more lenient interpretation of statistical assumptions and

significance.

The results of the multivariate regression analysis are pre-

sented in Tables 3 and 4,1 and according to the results, learn-

ing performance is driven (R 2 = .18)2 by project-oriented

instructional methods (both group and individual project

coefficientsare significant, supportingHypothesis3), the use

of Blackboard course management software (supporting

Hypothesis 2), and the amount of time students spend study-

ing (positive relationship, supporting Hypothesis 4a) and

working (negative relationship, supporting Hypothesis 4b).

Recall that performance was defined to have three primary

dimensions: volitional choice or effort (e.g., hours spent

studying and working), ability to apply knowledge (e.g., pro-

jects for pedagogy), and knowledge gained (e.g., feedback ontests and assignments using Blackboard), which seems con-

sistent with these results.

Selecting student-preferred, project-based pedagogies

enhances involvement andmotivationfor learning(Stttinger

and Schlegelmilch 2002), which supports learning perfor-

mance. The significant coefficients for group and individual

project-type instructional methods suggests that the trend in

marketing education toward the application of marketing

knowledge (Karns 1993) and experiential learning (Front-

czak 1998) is appropriate.

The use of course management software Blackboard

offers the ability to provide online syllabi, readings, outlines,

assignments, grade information, and students rosters, all ofwhich have shown to be related to students perceptions of

overall learning (Clarke, Flaherty, andMottner 2001) andcan

be very effective in providing timely feedback on perfor-

mance also shown to enhance learning (Bransford, Brown,

and Cocking 1999). Thus, course management software

seems to be effective in enhancingself-report learningperfor-

mance by communicating direction, expectations, and status

of performance.

136 AUGUST 2003

TABLE 2

PEDAGOGICAL AFFECT AND LEARNING PERFORMANCE SCALE DESCRIPTIONS (N= 207)

Mean Standard Deviation Pedagogical Affecta Learning Performancea

Overall, in this class the methods of instruction were . . .

effective/ineffective. 5.44 1.08 .86

useful/useless. 5.49 0.99 .77

satisfactory/unsatisfactory. 5.61 1.06 .88good/bad. 5.72 1.05 .87

Evaluate this class on . . .

the knowledge you gained. 4.39 0.79 .71

the skills you developed. 4.11 0.84 .71

the effort you expended. 4.06 1.20 .70

your ability to apply the material. 4.35 0.91 .70

your desire to learn more about this subject. 4.30 1.10 .72

your understanding of this subject. 4.52 0.83 .61

% of variance explained (eigenvalue) 45.31 (4.53) 16.92 (1.69)

of subscales .89 .80

a. Principal components analysis, Varimax rotation with Kaiser normalization.



9/14

Consistent with the literature, these results also indicate

the importance of student behavioral determinants on perfor-

mance even when preferred instructional methods and tech-

nology are provided. It is clear that students must have the

ability (time available forstudying) and thewillingness (time

spent studying) to raise their learning performance. The abil-

ity to devote time to studying may be influenced through

advising on what constitutes a reasonable course load, work

schedule, and extra curricula commitments in addition to

clearly specifying expectation on the time commitment


TABLE 3

MULTIVARIATE MULTIPLE REGRESSION ANALYSIS PARAMETER ESTIMATES (N= 206)

Dependent Variable Parameter B SE t Significance

Learning performance (R2

= .18) Intercept 2.802 .428 6.552 .000

LECTURE 6.104E-02 .042 1.468 .144

CASES 4.970E-03 .038 .131 .896

SIMULATION 4.460E-02 .035 1.275 .204GROUPPROJECT 6.421E-02 .033 1.973 .050*

INDIVPROJECT 8.423E-02 .046 1.849 .066*

EXAMS 9.732E-03 .033 .297 .767

CLASSDISCUSSION 1.305E-02 .045 .291 .771

CLASSEXERCISE 4.508E-02 .046 .989 .324

WRITINGASSIG 2.492E-02 .043 .580 .563

EMAIL 2.879E-02 .034 .835 .405

WEBACCESS 2.266E-02 .045 .498 .619

POWERPOINT 2.278E-02 .041 .557 .578

BLACKBOARD 6.137E-02 .027 2.253 .025*

LAPTOP 1.702E-02 .028 .602 .548

WORKHOURS 8.740E-03 .004 2.424 .016

STUDYHOURS 2.662E-02 .009 3.109 .002*

PARTYHOURS 3.388E-03 .005 .714 .476

CLASSATTEND 6.426E-04 .003 .229 .819ACCOMMODATING .197 .141 1.398 .164

DIVERGING .172 .136 1.267 .207

ASSIMILATING .181 .120 1.509 .133

CONVERGING 0a

Pedagogical affect (R2

= .28) Intercept 1.691 .539 3.139 .002

LECTURE 9.932E-02 .052 1.896 .060*

CASES 7.900E-02 .048 1.656 .099*

SIMULATION 3.430E-02 .044 .778 .438

GROUPPROJECT 9.468E-02 .041 2.309 .022*

INDIVPROJECT 8.526E-02 .057 1.485 .139

EXAMS 5.628E-02 .041 1.364 .174

CLASSDISCUSSION 6.493E-02 .056 1.150 .252

CLASSEXERCISE 9.923E-02 .057 1.727 .086*

WRITINGASSIG 2.725E-02 .054 -.503 .615

EMAIL 1.657E-02 .043 .381 .703

WEBACCESS 6.315E-02 .057 1.102 .272

POWERPOINT .112 .051 2.182 .030*

BLACKBOARD 4.166E-02 .034 1.214 .226

LAPTOP 3.634E-02 .036 -1.020 .309

WORKHOURS 5.105E-03 .005 1.124 .263

STUDYHOURS 3.757E-03 .011 .348 .728

PARTYHOURS 5.347E-03 .006 .895 .372

CLASSATTEND 3.340E-04 .004 .094 .925

ACCOMMODATING 3.485E-02 .177 .197 .844

DIVERGING 1.088E-02 .171 .064 .949

ASSIMILATING .132 .151 .872 .384

CONVERGING 0a

NOTE: Estimation method: general linear model multivariate procedure (Statistical Package for the Social Sciences).

a. This parameter is set to zero because it is redundant.* Significant at .10.



10/14

required for a particular course. The time spent studying was

onlycorrelated withinstructionalmethods thatinvolvedcom-

puter simulations and exercises. Apparently, certain instruc-

tional methods either require more study time or providemore motivation to study; however, whether because of or in

spite of instructional pedagogies, student behaviors must be

accounted for in explaining learning performance.

Pedagogical affect is primarily explained (R 2 = .28) by

preferences for different types of instructional methods (lec-

ture, cases, group projects, and class exercises all having sig-

nificant coefficients). It should be noted that the

nonsignificant instructional methodology variables had high

pairwise correlation with the significant variables, and there-

fore their coefficients may be the result of multicollinearity,

suggesting that thewholemixof instructional methods drives

pedagogical affect. The only instructional technology vari-

able that provided a significant coefficient was PowerPoint,which was used to support instructor lecturing and aid in stu-

dent presentations. It must be noted that these results are

based on responses from students who have been acclimated

to various instructional technologies throughout their college

education, which reduces potential Hawthorne Effects of

one course or onetime exposure to new technology or peda-

gogy. These results seem in contrast to findings of Stttinger

and Schlegelmilch (2002) that positive attitudes toward

instructional technologyare stronglycorrelated with technol-

ogy exposure. This suggests that this relationship may not be

a simple linear relationship but instead an inverted U-shape,meaning that at some point, with very high exposure, stu-

dents perceptions of the benefits of technology diminish. A

corollary explanation may be that instructional methods are

themost important factor and that instructional technology is

simply a tool to carry out instructional methods, thereby

reducing its influence when examined relative to instruc-

tional methods. In summary, pedagogical affect seems to be

primarily driven by instructional methods (supporting

Hypothesis3) with secondaryeffects of technology(support-

ingHypothesis 2) and is not significantly influenced by other

nonpedagogicalaspects (student behaviorsor learningstyles)

of the class.

Parameter estimates obtained from the multivariateregression analysis for course grade are presented in Table 4.

Student respondents were given the option of including their

technical identification number for reasons of anonymity,

which resulted in a subsample of 93 students whose grades

were able tobe matched to therest of thevariables.Thedistri-

bution of grades for this subsample was 1%, Ds; 7%, Cs;

58%, Bs; and 34%, As. Correlations between grades and

138 AUGUST 2003

TABLE 4

COURSE GRADE REGRESSION ANALYSIS PARAMETER ESTIMATES (N= 93)

Dependent Variable Parameter B SE t Significance

Course grade (R2

= .14) Intercept 3.392 .883 3.842 .000

LECTURE 6.041E-02 .065 .925 .358

CASES 6.021E-02 .058 -1.046 .299

SIMULATION 3.619E-03 .052 .069 .945GROUPPROJECT .175 .055 -3.174 .002*

INDIVPROJECT .133 .072 1.841 .070*

EXAMS 2.291E-02 .049 .465 .643

CLASSDISCUSSION 5.556E-03 .056 .100 .921

CLASSEXERCISE 8.322E-02 .062 1.346 .183

WRITINGASSIG 9.851E-02 .076 1.288 .202

EMAIL 1.149E-02 .054 .214 .831

WEBACCESS 6.787E-02 .063 1.071 .288

POWERPOINT 5.703E-02 .061 .936 .352

BLACKBOARD 9.711E-03 .043 .226 .822

LAPTOP 2.267E-02 .050 -.455 .650

WORKHOURS 5.137E-03 .005 .959 .341

STUDYHOURS 2.065E-02 .011 1.871 .065*

PARTYHOURS 5.502E-03 .007 .747 .457

CLASSATTEND 6.620E-03 .007 -.900 .371ACCOMMODATING .310 .194 1.599 .114

DIVERGING .103 .207 .497 .621

ASSIMILATING .232 .174 1.332 .187

CONVERGING 0a

. . .

NOTE: Estimation method: general linear model multivariate procedure (Statistical Package for the Social Sciences).a. This parameter is set to zero because it is redundant.* Significant at .10.



11/14

learning performance and pedagogical affect were insignifi-

cant. This lack of correlation may be due to the sampled

courses being upper-leveland, as canbe seen in thegradedis-

tribution, the variable is highly skewed toward As and Bs,

whereas the other two dependent variables have normal dis-

tributions. In addition, the upper-level courses, where grades

were reported,were team taughtby twoor three faculty mem-

bers, which probably further reduced students ability to

accurately estimatecoursegrades. If onewere to assumeself-

report learning performance represents a students judgment

forhis or herexpectedgrade,we couldexpect poorerstudents

to overestimate their performance and better students to

underestimate their performance (Kennedy, Lawton, and

Plumlee 2002), which, when range restricted to As and Bs,

would produce insignificant or nonmeaningful correlations.

To keep consistent in reporting the results, a general linear

model that incorporated the three dependent variables in the

model was formulated. The analysis produced the same

parameter estimates for course grade as a univariate regres-

sion, given the lack of significant correlations among coursegrade and learning performance and pedagogical affect.

Similar to the learning performance results, the instruc-

tional methods that were significant are group and individual

projects.In addition,numberof hours spent studying wassig-

nificant, whilenoneof the instructional technologies or learn-

ing styles produced significant coefficients. The results make

intuitive sense given this sampleandtheheavy useof project-

based pedagogies and the substantial out-of-class effort

required to complete the projects. Interestingly, course grade

seems independent of students preference for different

instructional technologies when examined relative to other

antecedents of learning outcomes. Overall, these results add

further support for Hypotheses 3 (instructional methods) and

4a (supportive behaviors).

Noteworthy, the factor learning styles was found not to be

significant in explaining learning performance, pedagogical

affect, or course grade. Even analyzing (ANOVA) learning

performance, pedagogical affect, and course grade without

thecovariates andonlyusing thefactor learningstyles, nosig-

nificant variation was accounted for in the outcomes. On the

basis of both ANOVA and multivariate regression results, we

do not find support for Hypothesis 1c that learning styles will

account forvariation in learning outcomes and, given thepre-

vious conclusions for Hypotheses 1a and 1b learning styles,

seemslacking in itsability inpredicting learningoutcomes.Apossible explanation, for this particular sample, may be that

much effort has been made by the faculty to incorporate

aspects from each of the four stages of the experiential learn-

ing cycle specifically trying to providing opportunities for

each leaning style. With ample opportunity for students to

learn in their own preferredstyle andby exposingall students

toallfour stagesof thelearningcycle, thelearningstyles vari-

able may simply wash out in this particular sample.

CONCLUSIONS AND IMPLICATIONS

As a whole, the combined analyses using three different

measures of learningoutcomes imply that theuse of preferred

instructional methods will enhance each of thedifferent mea-

sures of learning outcomes, while encouraging supportive

class behaviors and limiting competing time activities can

enhance self-report performance and course grades. Regard-less of theperformanceoutcomemeasure,only oneof thefive

instructional technology variables proved significant, sug-

gesting that in contrast to previous studies that examined

technologyin isolation, whenanalyzedrelative tootherlearn-

ing factors, technologys influence is secondary. Also in con-

trast to theliteraturewas thelackof influenceof learningstyle

on learning outcomes; however, once again, the issue of ana-

lyzing a single factor relative to multiple influences may

account for these findings. Preliminary insight from this

exploratory synthesisandextensionof previousresearchsug-

gests that caution should be used in interpreting findings

based on technology tools or other antecedents of learning

examined in isolation.Although more evidence is needed to draw a definitive

conclusion, we believe these results indicate that learning is a

two-way street where the primary contribution from the

instructor is appropriate instructional methods and the pri-

mary contribution from the student is study time. Note that

study time was significant in both performance-type out-

comes. The role of technology is probably a moderator that

can assist or distract from the instructional methods and the

time students spend studying primary course concepts.

Understanding learning styles can help instructors design

appropriate instructional methods, while technology profi-

ciency can leverage students study time.

From a marketing educators perspective, the results lead

to the following teaching implications. First, we recommend

that project-based instructional methods be used to enhance

involvement and motivation leading to enhanced perfor-

mance. In particular, our results suggest that group projects

weresignificant in enhancingaffect, self-report performance,

and course grade. Most experiential learning techniques

incorporate some form of projects, suggesting that the

reported trendtoward experiential learning in marketingedu-

cation seems appropriate. Group project-based learning also

encourages collaborative learning and can change the role of

theinstructor from a formalauthority role tomore of an infor-

mal coach, which facilitates student-faculty interaction.Second, the importance of student behavior, particularly

study time, shouldbe recognized, andefforts to createproper

expectations of time on task and study habits should be a pri-

mary consideration in course design. In this study, we found

that the use of computer simulations increased the number of

hours students reported studying fora class. It maybe that the

simulations provided motivation for increased studying or

simply required more out-of-class work.




12/14

Third, we recommendusinginstructional technologysuch

as Blackboard software that will assist in communicating

high expectations andcan provide promptfeedbackand mon-

itoring of performance. The adage that what gets measured

gets attention also seems to hold true in education.

We believe that the lack of significant coefficients for

learningstylesin this study is a researchartifactgiven ourfac-

ultys specific efforts to systematically incorporate a range ofteaching methods within courses and across the curriculum

that addresses the variety of learning styles. Therefore, our

final recommendation based on the education literature is to

design courses with a variety of pedagogical approaches to

teach to thediversity of learning stylesand to expose students

to all four stages of Kolbs learning cycle for major concepts.

The above recommendations provide a student-centered

learning environment that incorporates Chickering and

Gamsons (1987) sevenprinciples for good practice in under-

graduate education.

Although this article provides a first empirical attempt to

incorporate multiple influences on learning outcomes, fur-

therresearchis clearly needed. Overcoming thepotential lim-

itations of this study provides guidance for further research.

First, this study was based on a sample from one university

that has relatively high exposure to instructional technology.

Student samples with limited technology exposure may real-

ize Hawthorne effects and skew the results either artificially

high because of their technology focus or artificially low

because of their perceived problems of technology adoption.

Replicating this study in different educational environments

with different levels of technology and instructional methods

is needed to assess the generalizability of our findings. Stu-

dent exposure or familiarity with the technology should be

explicitly accounted for in future studies.Second, directextensions of this study would be the inclu-

sion of additional antecedent variables and the refinement of

the measurement of existing variables. In particular, refine-

ment of how a particular instructional technology is deployed

and its interaction with the instructional methods should be

developed. The choice of the dependent variable as an affect

construct, a self-report performance construct, and as the

instructor-assigned course grade does provide different

insight into the effect of various technologies, instructional

methods, andbehaviors. We expected to find learning perfor-

mance to be significantly related to pedagogical affect; how-

ever, the insignificant relationship between instructor-

assigned course grade and both self-report learning perfor-mance and pedagogical affect was unexpected. Whereas this

study was not intended to address this particular issue, it does

point out the sensitivity of the results to the selection of the

dependent variable and the necessity for further investigation

into appropriateness of particular dependent variables for

specific research questions.

Third and perhaps the most critical event to guide this

stream of research is the adoption or formulation of a broad-

based learning theory to direct systematic investigation and

provide assistance in the interpretationof findings.Psycholo-

gists have suggested a variety of theories to understand and

explain how people learn. Basic theoretical perspectives of

learning include behaviorists theories, developmental theo-

ries, and cognitive theories. In particular, social cognitive the-

ory provides a conceptual framework for clarifying the psy-

chological mechanisms through which social-structuralfactors are linked to performance (Bandura 1986). Behavior,

personal factors, and cognitions, as well as environmental

events interact bidirectionally so that people are both prod-

uctsand producersof theirenvironment. Social cognitive the-

ory provides not only explanatory and predictive power but

also explicit guidelines about how to equip people with com-

petencies and the sense of efficacy that will enable them to

enhance their accomplishments (Wood and Bandura 1989).

Thequestfor enhancingour ability to teacheffectivelyand

increase student learning is gaining importance as an identifi-

able research stream and is evolving in its academic scholar-

ship. Clearly definedoutcomesandexaminingmultipleinflu-

ences simultaneously seem to be critical in advancing our

understanding of technology and other educational

pedagogies. As the conceptual rationale for technology and

teaching pedagogies continues to develop and the empirical

evidence grows, our understanding of their effects on learn-

ing and teaching will help prepare both faculty and students

for their respective careers.

NOTES

1. The analysis was also conducted using only the Principles of Mar-keting students to determine if the findings and conclusions would differgiven varying levels of exposure to the explanatory variables. Consistent

with the total sample results, learning styles were insignificant for both per-formance and affect outcomes (not supporting Hypothesis 1), instructionaltechnologies were insignificantfor bothoutcomes (notsupporting Hypothe-sis 2), instructional methods (group projects) were significant for both out-comes (supporting Hypothesis 3), and student behaviors (work hours nega-tively related to performance and study hours positively related to pedagogyaffect) results supporting Hypothesis 4. Our conclusion that appropriateinstructional methods and student behaviors are the primary determinantswith technology as a probably moderator does not change.

2.Thereportedsummaryofoverallmodel fitis theadjustedcoefficientofdetermination (R

2). This fit statistic not only represents the proportion of

variability in the response variable that is accounted for by the regressionmodel but it takes into account the number of predictors presented in themodel. Whereas the multiple coefficient of determination (R

2) can be artifi-

cially increased by adding explanatory variables, the adjusted R2

will onlyincrease if the t-value of thenewly added variable is greater than one(Dillionand Goldstein 1984). The magnitude of the reported R

2s should be expected

to be relatively low given these models incorporate more than 20 variableswith the majority oft-values less than 1. The intent of this analysis was tosimultaneously examine the predictors and not to build a parsimoniousmodel with a highR

2. In context, Davis, Misra, andVan Auken (2000)report

R2s ranging from .24 to .38 after stepwise variable reduction in predicting

pedagogical preference;Deeter-Schmelz,Kennedy, andRamsey (2002) esti-mated an R

2= .07 for teamworks prediction of performance; Adrian and

Palmer (1999) used three variables to explain grades with an R2

= .59; andNonis andSwift (1998) reportR

2s ranging from .06to .35in examiningclass-

roombehavior.Thus, themagnitudeof ourreportedR2s isconsistentwiththe

literature.

140 AUGUST 2003



13/14


14/14

Stewart,Karen L., andLindaA. Felicetti. 1992. Learning styles of marketing

majors. Educational Research Quarterly 15 (2): 15-23.

Stttinger,Barbara,and BodoB. Schlegelmilch.2002. Information andcom-

munication technologies in tertiary education: A customer perspec-

tive. Marketing Education Review 12 (2): 63-72.

Strauss, Denise T., and Raymond D. Frost. 1999. Selecting instructional

technology media for the marketing classroom. Marketing Education

Review 9 (1): 11-20.

Tull, Donald S., and Del I. Hawkins. 1990. Marketing research: Measure-ment and method. 5th ed. New York: Macmillan.

Webster, J., and J. Martocchio. 1992. Microcomputer playfulness: Develop-

ment of a measure with workplace implications. MIS Quarterly 16 (2):

201-26.

Williams, Tim. 1992. Validating a degree: Subject relevance and assessment

issues in the development of a new degree. Education and Training 34

(3): 31-33.

Wood, Robert, and Albert Bandura. 1989. Social cognitive theory of organi-

zationalmanagement.Academy of ManagementReview 14(3):361-384.

Young, Mark R. 2001. Windowed, wired and webbedNow what? Journalof Marketing Education 23 (1): 45-54.

. 2002. Experientiallearning= hands-on+ minds-on.MarketingEdu-

cation Review 12 (1): 43-52.

142 AUGUST 2003

Documents

Young 2003 - Enhancing Learning Outcomes- The Effects of Instructional Technology, Learning Styles, Instructional Methods and Studen Behavior