The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana An Empirical Question: How Students Learn Economics? Some Evidence

The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana

An Empirical Question: How Students Learn Economics?

Some Evidence from The JEE (1969 to 2011)

Dr. Nürşen Albayrak Zanca Associate Professor of Economics

University of the Incarnate WordDepartment of EconomicsSan Antonio, Texas, 78209

[email protected]

OVERVIEW

PROBLEM STATEMENT: Motivation & Objective EMPIRICAL EVIDENCE ON CLASSROOM

EXPERIMENTS: Do They Improve Learning? Literature from 1970s; 1980s; 1990s; and 2000s

A META-ANALYSIS OF LECTURE-LESS TEACHING APPROACH (The Journal of Economic Education, 1969-2011)

CONCLUDING REMARKS & COMMENTS



PROBLEM STATEMENT: Motivation & Objective

The higher education in the U.S. has become a mass process.

The U.S. Population # of Enrollment

152 million (1950) 2.3 million (1950)

> 300 million (2010) >16.9 million (2005) Financially, the reliance on large classes as the

major teaching device is very important to the universities.


As a result of this mass process, lecture classes have become larger to a point where there can be no interaction between student and teacher. In other words, students become an entirely passive subject in the learning process.


Lecture-less vs. Lecture Education specialists agree that instructors

should use active learning.

An important issue in economics education is that the predominant method of teaching style is traditional lecture method which provides little opportunity for active learning.

Lecture = “One-Way Talk” = “Chalk & Talk”




An increasingly popular alternative to the lecture-oriented ‘Chalk-and-Talk’ approach is ‘Classroom Experiments’

Classroom experiments engages students in active learning (away from passive learning).

Some economists questioned whether the choices that instructors make about how to teach introductory economics make a difference in learning of economics.

A Wide Range of Other EXPERIENTIAL Teaching Tools

duration/length of the course classroom size student characteristics (cognitive skills, gender, seating, math skills, note taking skills) absenteeism use of algebra and graph use of newspaper articles use of sports; use of case studies Use of CAI (computer assisted instruction) use of computer technology and integrating various Internet tools use of music use of literature and drama, motion picture use of humor, using the Simpsons /animated cartoons use of acceptance speeches by the Nobel Laureates in economics and, and so on.


METHODOLOGY

Most frequently, researchers have conducted controlled tests, where the relative effectiveness of any of the above listed experiential learning style against traditional lecture was analyzed within a multiple linear regression model.

The dependent variable (a change measured as the difference in post- and pre course TUCE scores) was controlled with a variety of student- and instructor-level characteristics.

Often, the independent variables included experimental group vs. control group (captured with a dummy variable), cumulative GPA (representing students aptitude), academic experience (credit hours earned), business major, other student demographic characteristics, such as sex, ethnicity, age, and so on.



METHODOLOGY

The most common technique (Multiple Linear Regression)

Y = f ( S, PRE, X, )

Y adjusted post-TUCE examination score

S teaching style 1 for the Experimental section

0 for the Control (lecture) section

PRE pre-TUCE examination score

X a vector of student characteristics

(Cumulative GPA, Academic Experience, Major, Sex, etc)

error term A test of differences between means (t-test)


Scope & Limitations of the Paper

The primary aim of this paper is to provide a literature review on classroom experiments.

Articles focused on describing the actual experiments are out of the scope of this paper.

Articles in which instructors describe their experiences using classroom experiments are considered.

The scope of the META-ANALYSIS is restricted to literature review from the JEE (1969-2011)


Objective of the Paper

1st : The primary purpose is to determine to what degree classroom experiments contribute to the learning of economics. Is there any systematic evidence for classroom

experiments in learning of economics? Can students learn economics with classroom

experiments better than in the traditional lecture?

2nd :To provide a literature review and a meta-analysis on classroom experiments from the JEE since its inception (Vol. 1, Fall 1969) up to date.


Classroom Experiments: Background

Simply, teaching economics through the use of experiments

A pedagogy that is derived from the field of experimental economics.

Chamberlin used experimental economics to discover Monopolistic Competition (1948)

Vernon L. Smith won Nobel Prize in Economics for developing Experimental Economics (2002)

Vernon Smith is considered by many to be the father of experimental economics.


Trends in the number of publications in experimental economics field (1948-1998)

Books Textbooks WebPages Internet

Newsletter Classroom Experiments

Academic journals

Conference Papers

Reports

Unpublished papers


Empirical Evidence on Classroom Experiments: Do They Improve Learning?

While there are many advocates of classroom experiments [Post (1985); Millerd & Alastair (1987); Spencer & Van Eynde (1986); Walker (1987); Friedman & Sunder (1994); Frank (1997); Gremmen & Potters (1997); Holt (1999); Carlson & Skaggs (2000); Dickie (2000); Truscott et al. (2000); Wilson & Nelson (2000); Barnett & Kriesel (2003); Jensen & Owen (2003); McGuire & Zanca (2003); Emerson & Taylor (2004), Zanca & McGuire (2005), and Dickie (2006)], there are also those who claim classroom experiments have not been empirically any superior to the chalk-and-talk method [McConnell & Lamphear (1969); Paden & Moyer (1969); Danielsen & Stauffer (1972); Luker et al. (1975); Sharp, et al. (1975); Wentford & Lewis (1975); Seigfried & Fels (1979); Frass (1982); De Young (1993); Fels (1993); Williams & Walker (1993); Cardell et al. (1996), Laury (1999); Yandell (1999); and Maxwell et al. (2005)]


NEGATIVE Empirical Evidence

No systematic evidence exists to support the hypothesis that students participating in experiments learn economics better than students getting lecture treatment.

Undergraduate instruction in economics is largely a passive experience. Any opportunity that replaces lectures with active participation would benefit students.

However, efficacy of classroom market experiments are drawn from anecdotal evidence (positive remarks made by students) and subjective analysis in supporting the effectiveness of the experimental approach.


POSITIVE Empirical Evidence

Students exposed to the classroom experiments have higher gains in the TUCE than their traditional lecture-oriented (non-experimental) cohorts.

The results were statistically significant.

In conclusion, indeed, using classroom experiment does increase student learning.

Back to Question #1: Is there any systematic evidence for classroom experiments in learning of economics?

Anecdotal Evidence “Anything is better than a lecture!” “If it is good for me, it is good!” “This is what I do, and I like it!

So do my Students!” Empirical Evidence (i.e, statistical significance)

Some say YES, and some say NO!


What is a META-ANALYSIS?

It is well acknowledged that it is not proper to recommend an intervention or a treatment based on a single study.

Instead, it is advisable to examine the cumulative effect of numerous studies before application. How to we learn about the cumulative effects?

The goal of the meta-analysis is accumulating knowledge.

Meta-analysis is the rubric used to describe quantitative methods for combining evidence across studies.


HOW TO CONDUCT A META-ANALYSIS?

STEP#1: Literature Search The scope of this study is restricted to literature review from the

JEE. The JEE frequently offer original articles on innovative teaching techniques in economics. (The JEE, Fall 1969, Volume 1, Number 1).

STEP #2: Inclusion Criteria Lecture-less vs. Traditional Lecture

STEP #3: Describing, Classifying and Coding Research Studies

STEP #4: Computation and Interpreting of Effect Sizes ()


A META-ANALYSIS OF CLASSROOM EXPERIMENTS (The JEE, 1969-2007)

An effect size is defined as the strength of the relationship between the independent variable and the dependent variable. For example, if one is comparing an intervention group to a control group, the most informative and straight forward measure of experimental effect size is the mean difference between experimental and control groups divided by pooled within-group standard deviation.


XCE SXX /)(ˆ

2/11 222 CECCEEP nnSnSnS

2/111 CEP nnt

A META-ANALYSIS OF CLASSROOM EXPERIMENTS (The JEE, 1969-2007)

How should (i.e., effect size) be interpreted?

small effect = 0.2

medium (moderate) effect = 0.5

large effect = 0.8


RESULTS OF THE LITERATURE REVIEW

&

THE META-ANALYSIS

OF CLASSROOM EXPERIMENTS

(The JEE, 1969-2007)The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana


RESULTS Economists have been active in the improvement of

their discipline. A great deal of work has been done in the

improvement of learning of economics. The effect of classroom experiments on student

learning has been widely investigated, but the results have been inconclusive, and often contradictory.

While the empirical evidence shows some support for the classroom experiments over classical lecturing methods, there is no consensus on this matter.

RESULTS The current paper provides some cumulative evidence

on the impact of lecture-less teaching on student learning in economics.

The results of the meta-analysis indicate that the average effect size for comparisons of lecture-less (experiment group) vs. traditional lecture (control group) instruction is less than 0.2.

In other words, there is NO significant difference between lecture-less vs. lecture style teaching on students’ learning.


RESULTS Evidence of this paper indicated that lecture-less

vs. lecture format of teaching does not have any statistically significant difference in student learning.

In other words, students do not learn economics better in experimental group vs. control group (i.e, old-traditional-boring lecture!)

As an advocate of classroom experiments, the results of the meta-analysis was surprising for the author!


RESULTS The current paper provides some support

for classroom experiments being an effective teaching tool, however, not necessarily an effective learning method.

Despite the inconclusive result on learning of economics with lecture-less class, there is a pattern for student as well as instructor satisfaction with classroom experiments.



RESULTS A byproduct of classroom experiments is that it

renews instructors commitment to teaching and students participation in active learning.

The vast majority of students enrolled in principles of microeconomics are not economics majors, they generally have a limited aptitude for microeconomic analysis.

Such innovative, interesting techniques can enhance the recruitment of economics majors both to sustain our discipline, and to increase the analytical skills of college students.

RESULTS The reason to adopt of experiential teaching

techniques may vary from Making students active learners Attracting students into ECON

classes/discipline Making class less boring, less abstract and

more applicable Improve student evaluations

Improving students’ learning is NOT the case!


Lecture = “One-Way Talk” = “Chalk & Talk”

Some researchers argue may be students feel more comfortable with the traditional lecture style, where the instructor feeds them the information that they needed to know for the test.

Some argue that the lecture method’s comparative advantages lie in transmitting information.

Lectures are often less effective than other teaching methods in promoting independent thought, developing critical thinking skills in students, and changing student attitudes.

If one wants to achieve these latter objectives, one should be aware of the limitations of one-way talk (and chalk), and modify the lectures accordingly and/or supplement them with other teaching devices.


FUTURE RESEARCH Extend the Meta-Analysis to

Other Journals Conference papers Other form of discussion/academic papers

Submit the findings of this paper to the JEE

TITLE: “Lecture vs. What? A Comparison of Outcomes in Economics Learning?”


FUTURE RESEARCH Many articles on this topic appear regularly in most

mainstream economics journals. Among those journals are American Economic Review,

Journal of Economic Education, Journal of Economic Inquiry, Journal of Economic Perspective, Southern Economics Journal, and American Economist, and many others.

Others, including conferences sponsored by the AEA, and others (just like this one, the GCEA!) have sessions on teaching of economics where innovative papers are presented in this field.


FUTURE RESEARCH Overall, the level of student interest and active learning

involvement component of classroom experiments cannot be denied.

There has not been enough evidence that these games are improving students’ economics understanding.

Economics educators should continue to search the effect of experiential pedagogy on student learning.

The author of this paper hopes that the paper has created an appetite to the reader to learn more about classroom experiments.



PERSONAL ANECDOTAL EVIDENCE(Positive remarks made by students) n= 47 students, ECON 2302 MICRO

Expernomics made economics much easier. It was very efficient getting the main principle across the

class. It helped me to better understand and master the economic

concepts. It helps learn better. My only suggestion is to HAVE MORE games/experiments. Other classes should adopt it, too. Everyone participated b/c it involved food and money! We tend to remember easier.

On the first day of school, a teacher was glancing over the roll when she noticed a number after each student’s name, such as 154, 136, or 142.

“Wow! Look at these IQs,” she said to herself. “What a terrific class!”


Anecdotal Evidence….


The teacher promptly determined to work harder with this class than with any other she ever had.

Throughout the year, she came up with innovative lessons that she thought would challenge the students, because she did not want them to get bored with work that was too easy.


Her plan worked!

The class outperformed all the other classes that she taught in the usual way.

Then, during the last quarter of the year, she discovered what those numbers after the students’ name really were: their locker numbers.




COMMENTS &

QUESTIONS

Thank You!

Documents

The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana An Empirical Question: How Students Learn Economics? Some Evidence