CALICO 2004“CALL: Focusing on the Learner”

Carnegie Mellon University

Bridging Theory and Practice:Research-Based Listening Tasks for Video Comprehension

Luba Iskold, Ed. D.

Statistical analysisGreg Cicconetti, Ph.D.

Muhlenberg CollegeAllentown, PA

Statement of the Problem

• How do students develop listening skills by using video materials?

• Do they learn best by mere “exposure to …comprehensible input?”

• Should we assist students in comprehending a videotext?

• Do listening tasks performed by students while they are watching a video episode help them concentrate on important information in that video?

• Which tasks leave the learner with higher levels of video comprehension?

The Purpose of the Study

• To investigate if activities performed by learners during the listening stage enhance comprehension of a video episode.

• Two conditions of video viewing: ‘exposure only ‘and using Video Guides specifically designed for the present investigation.

• Thus, the purpose of this empirical study was to examine:

1. the effect of listening tasks performed during video viewing on the comprehension of second-semester Russian undergraduate college students of video episodes, as measured by Immediate Recall Protocols written in English, and

2. the effect of a Video Guide condition on students’ recall, recognition, and application of the content, vocabulary, structures, and cultural information from “Начало” video, as measured by immediate and delayed tests completed in Russian.

Previous Research

Theoretical perspectives on language learning

– Comprehension-based approaches– Cognitive-theoretical views– Sociocultural perspectives

Research Related to Listening

– Research on listening and reading comprehension– Factors that affect listening comprehension– Research on listener characteristics– Authentic materials in listening research– Video in listening research

How does one teach listening?

• Constructing tasks for listening comprehension

• Assessment of listening

Constructing Tasks for Listening

Richards (1983) suggested manipulation of two variables as a means to develop listening micro-skills:the input and the task

INPUT MICRO-SKILLS TASKS

His taxonomy includes 33 micro-skills for listening to conversations 18 additional skills for academic listening:

Joiner (1990) recommended listening/viewing guides which students use while watching the video episode

Assessment of Listening Comprehension

How does one determine which listening tasks work best for learners?

• Recall protocols, frequently used in L1 research.

• In 1986-87, Bernhardt and James applied recall protocols to assess reading and listening comprehension in L2.

• Glisan (1988) advocated using comprehension checks in the native language, therefore avoiding the mixture of skills.

Materials designed for this study:

• Interactive Viewing Guides

• Recall Protocols Scoring Sheets

• Immediate and Delayed Quizzes(recall, recognition, application)

• Exit Survey

• Participant Information Survey

• Baseline Test

The Viewing Guides Constructed for the Study

• The purpose:

To facilitate understanding of the video episodes. To help students focus on specific important aspects of each videotext, NOT to assess their comprehension of the video

• Features:

The tasks were explained before students start watching the video. The reading load of the tasks was kept to a minimum. The items were presented in the true/false format

• Sections: Instructions Main Ideas: characters, places and events Details: vocabulary, phrases, cognates, idioms Cultural similarities/differences Space for note taking

The Study

Setting

An undergraduate, four-year college with 1, 800 students.

Maintains a foreign language requirement.

Languages: French, German, Hebrew, Italian, Russian, Spanish, Greek, and Latin.

An interdisciplinary major/minor in Russian Studies.

Elementary Language Level I and II are first and second semester courses of a target language sequence offered by the college.

Placement is dependent upon experience or a placement test.

Population and Sample

The study took place in the spring of 2004. Sixteen students enrolled in one section of a fifteen-week,

second semester Elementary Russian II participated in the study:

12 freshman, 4 sophomores; 7 men, 9 women; all students took Elementary Russian I;13 students never took Russian in high school; 3 participants had 2 years of Russian;11 were planning to major/minor; 5 were fulfilling the College’s FL requirement;14 students were native speakers of English; 2 were heritage speakers of Polish.

Treatments

An intact class of sixteen Elementary Russian IITaught by one faculty member participated The class was divided into two groups

Group homogeneity in composition:

Students were ranked by prior achievement in Elementary Russian I and then randomly assigned to Group 1 (G1, n1 = 8) and Group 2 (G2, n2 = 8).

• Baseline Test on listening comprehension.

• Both groups followed the same syllabus:

» received the same instruction» completed the same assignments» took the same quizzes» on the same dates» in the same setting.

Watching the Video The participants watched four «Начало» video episodes on CD-ROM

at the Language Learning Center, under two different conditions:

• A control group: ‘exposure only’ to the videotext in the commercially available format.

• An experimental group: the same videos in conjunction with the on-line Video Guides designed for the present investigation.

• Group One (G1) served as the control while Group Two (G2) served as the experimental group for video episodes one and three.

• Group One (G1) served as the experimental group, while Group Two (G2) served as the control for video episodes two and four.

Table 1Allocation of Groups to Video Formats

Video Episode 1 Episode 2 Episode 3 Episode 4

Group I Control Experimental Control Experimental

Group II Experimental Control Experimental Control

Variables:

Controlled variables: • Groups (control and experimental)

• Treatments, conditions for video viewing.

Dependent variables:1. the amount of text material (or the number of scored points)

recalled in four Immediate Recall Protocols;

2. the scores in listening comprehension achievement, as measured by four sets of immediate and delayed quizzes, and

3. participants’ opinions regarding the effectiveness of treatments, as measured by the Exit Survey.

Materials

TextsThe videotexts employed in the study were taken from

«Начало» (Lubensky et. al)|

– a video-driven package for beginning students of Russian– integrates the teaching of language and culture– provides a balance between structure and communicative

activities. – sixteen 2.5-3.5- video episodes, connected by a story line;

filmed in Moscow

Simulated Authentic discourse

Testing and Scoring Procedures|Sample Recall Protocol Scoring Sheet Начало # 5: Бизнес по-московски

4 --most important event 1 --least important eventIdea Units/Sentences: 15 Total Points: 48

Points Possible Idea Units Points Scored

2 What a nightmare! How terrible!

4 The house is new, but there is no asphalt.

4 Lena cannot find rubber boots. They don’t have her size.

4 All neighbors have a big problem.

4 Viktor understands their problem and is ready to help.

4 Viktor has a business. He has rubber boots of all sizes, but he does not sell them.

3 Every morning he waits for the neighbors in the hallway and hands out rubber boots to them.

3 Viktor’s friend is at the bus stop. He waits for the neighbors and takes the boots.

3 The neighbors pay 500 rubles.

3 They think it’s expensive.

4 Viktor says that 500 is not expensive. For this money they get clean boots and good mood.

2 Viktor says that pretty girls get the boots for free.

2 Sasha thinks, its discrimination.

2 Victor says it’s a joke.

4 Viktor thinks that Lena is a very pretty girl.

Sample Quiz (Immediate)sections 2 & 3

Findings

Table 2

Summary of Nested Factorial Statistical Analysis of the Immediate Tests

The results presented in Table 2 suggest that:

• There were no significant differences among scores from Recall (1) that may be attributed to either episode version, or treatment;

• Although the scores from Recognition (2), Application (3) and Total scores do exhibit differences, the treatment participation of a student (Control or Experimental) does not appear to affect his/ her scores.

Quiz Group N

Recall (1) P = 0.087 P = 0.370 62

Recognition (2) P < 0.0005 P = 0.585 62

Application (3) P = 0.002 P = 0.444 62

Total Score (4) P = 0.002 P = 0.354 62

Immediate Recall Protocols (5) P < 0.0005 P = 0.022 58

Statistically significant differences were found among the treatment groups and video episodes on the Immediate Recall Protocol:

The Experimental group consistently performed better on Immediate Recall Protocols.

The findings strongly suggest that the Video Guides were effective.

Table 3Mean Scores and Standard Deviations on Immediate Recall Protocols

Video Episode Treatment Group Mean Standard Deviation N

1 Control 83.67 18.82 7

Experimental 88.69 20.98 8

2 Control 82.74 21.84 7

Experimental 98.62 2.88 8

3 Control 78.29 20.51 7

Experimental 96.57 4.86 7

4 Control 52.75 10.35 7

Experimental 71.98 20.32 7

Table 4Mean Scores and Standard Deviations on the Exit Survey

Question Mean Standard Deviation

Using Video Guides helped me to stay focused during video viewing. 2.05 0.42

Using Video Guides helped me to remember the video episodes better. 2.00 0.61

In my opinion, using an interactive Video Guide may help me learn more from the video episodes.

2.00 0.50

I would like to continue using a Viewing Guide while watching «Начало» episodes next year.

1.82 0.52

I liked the design of the current Video Guide. 2.17 0.80

The Video Guide distracted me from watching the video. 3.82 0.95

Limitations

• The study was conducted at a small liberal arts undergraduate institution.

• The nature of the population may present questions about the role of attitude, interest, and motivation in FL video viewing.

• Because all students were in the same class, there was no random selection of participants.

• The sample size was limited to 16 students. Randomizing the sample and broadening its size might enhance the validity of the study.

• In the future, incentives may be incorporated to decrease the number of missing observations.

• The findings are based on a particular kind of videotext, a simulated authentic discourse based on a fictional narrative, that might not be applicable to other types of videotext (e.g., news broadcasts or interviews).

• The results may not generalize to studies that employ different assessment instruments.

• A longitudinal study over several semesters may be better suited for the purpose.

Contact Information:

Dr. Luba Iskold

2400 Chew StreetMuhlenberg College,

Languages, Literatures and Cultures,Allentown, PA 18104

Phone: 484-664-3516Fax: 484-664-3722

E-mail: iskold@muhlenberg.eduhttp://www.muhlenberg.edu/depts/forlang/LLC/iskold_home/index.htm