Are lecturers' and students' needs different? A needs

analysis for reading tasks in Flemish higher education

Elke Peters & Tine Van HoutvenLessius University College, Antwerp

elke.peters@lessius.eu & tine.vanhoutven@lessius.eu

TBLT 2009 2

Outline

• Project description• Background• Aim and research questions• Methodology• Results and interpretation• Conclusion

TBLT 2009 3

Introduction

• Language plays a key role in education.• Mastery of academic language is crucial.

– But research has shown that many students, non-native as well as native speakers of Dutch, struggle with academic language upon entering Flemish university colleges.

– poor command of Dutch and of academic Dutch in particular

Þ Projects centering around the theme of (L1) language support

Þ Project focusing on text competence/reading skills

TBLT 2009 4

Project

• Aim of our project is to provide an answer to this problem by

• Determining required level of text competence• Carrying out a descriptive study into first year students’

reading skills and text competence• Comparing students’ existent level of reading/text

competence with the required level• developing reading materials for four courses in four

different curricula

• In order to facilitate first-year students’ chances of achieving academic success

TBLT 2009 5

Project

• How?– Not “one-size-fits-all-approach”– Necessity of a large scale needs analysis in four

different curricula. • “the language learning needs of particular groups of

learners or individuals (…) are learner- or group-specific, (…) are tied to local contexts and may change over time” (Van Avermaet & Gysen, 2006: 19)

• What? – NA findings used in design and development of

task-based reading support materials.

TBLT 2009 6

Background: Long (2005)

• Long (2005)– a number of methodological issues that need to

be considered in learner needs analysis in terms of sources, methods, and source x method combinations.

– The aim should be to obtain reliable, valid, and usable data about the tasks students need to carry out to be successful.

• Van Avermaet & Gysen (2006)– Take into account both subjective and objective

needs.

TBLT 2009 7

Background: Long (2005)

• A task-based needs analysis• Possible sources for a needs analysis:

– Literature, learners, teachers/applied linguists, domain experts, and triangulation.

– Needs analysis should involve insiders/domain experts– Use of multiple sources: add breadth/depth to the analysis

• Possible methods for a needs analysis:– intuitions, (un)structured interviews, questionnaires,

observation, tests, diaries, role plays etc. .– Unstructured interviews.– Questionnaires: ascertain existing views, not creating new

views; often over-rated.– Use of multiple methods of data collection

• A needs analysis = time-consuming

TBLT 2009 8

Aim and research questions

• Which reading tasks do we need to develop for first year students from four different curricula?– First-year students of four different curricula & university

colleges clearly-defined domain = academic language proficiency• What is the required level of text competence?• What is the actual of first year students’ text

competence?• Is there a difference between the two?

– Practical RQ in order to develop the reading materials

TBLT 2009 9

Aim and research questions

• Which source(s) or method(s) or source x method-combinations is/are the most reliable and informative?– As compared in four case studies (four different curricula

and university colleges)– Methodological/evaluative RQ in order to +/- corroborate

Long’s hypotheses

TBLT 2009 10

Methodology

• Four sources• Four methods• Triangulation of sources and methods• Same methodology in four case studies

TBLT 2009 11

Sources

• PTHO (= Profiel Taalvaardigheid Hoger Onderwijs (Language Proficiency Higher Education))– Description of tasks students need to be able to

carry out at the start of their academic career• Determining expected level of text competence

• Students from 4 different curricula:– First-year students– Third-year students– Convenient and purposive sample

• Lecturers from 4 different curricula (= domain experts)

• Language experts methodological advice

TBLT 2009 12

Methods

• Literature survey• Reading test• Questionnaire• Interview• Triangulation by sources & methods

TBLT 2009 13

Method 1: Reading test PTHO

• Target group: Dutch as a foreign language• Based on needs analysis typical tasks a student

needs to be able to carry out• N = 176 (L1 Dutch = 165; L2 Dutch = 9)

• Part 1: multiple choice questions– Questions = “reading-the-lines” level (Alderson, 2000) or

descriptive level (Bogaert et al., 2008)• Part 2: summary

– Read three texts on same topic– Write one summary = “reading-between-the lines” level

(Alderson) or upper-textual level (Bogaert et al.)

TBLT 2009 14

Method 1: Reading test PTHO: results

• Part 1 (multiple choice questions): high scores– Ceiling effect

• Part 2 (summary)– 1/3 of students = problematic

• Difficulty with information processing functional reading• Wrong/incomplete account of information

– Large differences in terms of educational program in secundary education/preparatory training

• General > technical > vocational secundary education– Problem areas were identified: vocabulary, text cohesion

and synthesis

• Answer to RQ1 in terms of problem areas for each curriculum

TBLT 2009 15

Method 2: Questionnaire

• Questionnaire tapped into– Types of reading texts– Strategy use

• Orientation and planning (e.g. reading title/images/…)• Monitoring reading process (e.g. looking up unknown

words)• Evaluating reading process (e.g. how difficult do you

find … linked to activities of different levels of information processing)

– Possible, useful reading tasks• Closed questions with pre-specified response

categories + 1 open question• Questionnaire was piloted

TBLT 2009 16

Method 2: Questionnaire - example

Arrange in order of difficulty.

- Visualize the structure (e.g. highlighting, annotating)

- Detect the topic sentence in a section- Interpret charts and diagrams- Attain a high level of comprehension- Make comparisons and connections - Represent information schematically* setting: course or handbook

TBLT 2009 17

Method 2: Questionnaire - results

• Sources/participants:– Students: N = 455 what do you think/do?– Lecturers: N = 97 what do students do/think?

• Tasks with increasing text competence• Reading tasks were perceived more difficult by

lecturers compared to the students• Answer to RQ1 in terms of problem areas, students’

strategy use, and useful tasks for each curriculum.

TBLT 2009 18

Method 3: Interview

• Semi-structured interview– Partially based on results reading test– Partially based on data of questionnaire

• One-hour audio-taped interviews with – Lecturers in four different curricula– Students in four different curricula

• 1st and 3rd year students in one curriculum• Nine interviews in total• All interviews were transcribed

TBLT 2009 19

Method 3: Interview - results

Information obtained about …• Target reading tasks and implementation methods

were identified• Students modified students’ answers supplied in

questionnaires more in line with lecturers’ opinions

• Students contribute to ‘means analysis’ (they provide useful information on learning styles, likes and dislikes, etc.)

TBLT 2009 20

Discussion

Four case studies

• Reading test identify problem areas

• Questionnaire identify problem areas, students’ strategy use, useful tasks

• Interview indentify target reading tasks and implementation methods

• Design support materials

TBLT 2009 21

Discussion: RQ1

• Differences in target reading tasks and implementation methods between the four curricula needs vary greatly– one-size-fits-all approach doesn’t work– NA = prerequisite for effective design of support materials– taking into account specificities of each course and curriculum – beneficial for both students’ and lecturers’ motivation + gain

an insight into their attitudes (what they think and do) self-knowledge ; level of awareness

• combining and balancing needs of students, lecturers and language experts– Students tended to overestimate themselves in the

questionnaires but counterbalanced in the interviews

TBLT 2009 22

Discussion: RQ2

• Evidence of four case studies– Use of several methods and sources obtain

more reliable data– Sources: triangulation of sources

• Lecturers (domain experts)– Methods: triangulation of methods

• Interviews semi-structured interview– BUT only because of the results of the reading test and

questionnaire– Interview alone would not have sufficed

• Our results tend to corroborate Long’s findings but with regard to the method there is an “if”.

TBLT 2009 23

Discussion: RQ2Methods + -

reading test PTHO

- Language Proficiency Higher Education relevant tasks

- small sample- unfeasible to test all relevant tasks

questionnaires - large sample- normalized, quantifiable data

issues of validity:- overestimation of oneself- pre-specified anwering categories limit variety of responses- interpretation differences between lecturers and students

semi-structured interviews

- thorough coverage of the matter- target reading tasks and methodologies were identified

-time-consuming

TBLT 2009 24

Conclusion

NA• time-consuming undertaking, but prerequisite for design of support

materials• multiple sources and methods should be carefully sequenced

Sources• insiders/domain experts: informative source• 1st year students: can’t be the sole or principal source because they

lack experience and understanding of present/future needs

Methods• interviews yield important information, but only because results of

test and questionnaires could be used• questionnaires effective for ascertaining existing beliefs, not for

creating new views

TBLT 2009 25

References

• Alderson, J.C. (2000). Assessing Reading. Cambridge: Cambridge University Press

• Bogaert, N., Devlieghere, J., Hacquebord, H., Rijkers, J., Timmermans, S. & Verhallen, M. (2008). Aan het werk! Adviezen ter verbetering van functionele leesvaardigheid in het onderwijs. Den Haag: Nederlandse Taalunie  Den

• Long, M. (Ed.) (2005). Second Language Needs Analysis. Cambridge: Cambridge University Press

• Profiel Taalvaardigheid Hoger Onderwijs van het Certificaat Nederlands als Vreemde Taal. Downloaded from http://www.cnavt.org/files/Profielbeschrijving%20Profiel%20Taalvaardigheid%20Hoger%20Onderwijs.pdf on September 5 2008

• Van Avermaet, P. & Gysen, S. (2006). From needs to tasks: Language learning needs in a task-based approach. In K. Van den Branden (Ed.), Task-Based Language Education (pp.17-46). Cambridge: Cambridge University Press

TBLT 2009 26

Acknowledgements

• OOF-comittee of the Association K.U.Leuven• Projectpartners

– Katholieke Hogeschool Limburg, Katholieke Hogeschool Kempen, Katholieke Universiteit Leuven, Katholieke Hogeschool Leuven, GroepT, KATHO, Katholieke Hogeschool Mechelen,Katholieke Hogeschool Brugge-Oostende, Katholieke Hogeschool Sint-Lieven, Hogeschool Universiteit Brussel

• If you have any questions, you can always send us an e-mail:– Elke.peters@lessius.eu– Tine.vanhoutven@lessius.eu