Test de Français Laval-Montreal: Does It Measure What It ... · suggesting elements of response (Commission des états généraux sur l’avenir de la langue française au Québec,

TESL CANADA JOURNAL/REVUE TESL DU CANADA 87VOLUmE 32, SpECiAL iSSUE 9, 2015

Test de Français Laval-Montreal: Does It Measure What It Should Measure?

Romain Schmitt & Shahrzad Saif

This article reports on a study conducted as part of a larger investigation of the predictive validity of the Test de Français Laval-Montreal (TFLM), a high-stakes French language test used for admission and placement purposes for Teacher-Training Programs (TTPs) in major francophone universities in Canada (Schmitt, 2015). The objective of this study is to examine the validity of TFLM tasks for measuring language abilities required by tasks common to the Target Language Use (TLU; Bachman & Palmer, 2010) domains in which preservice teachers are expected to function. Adopting Messick’s conception of construct validity (1989) and Bachman & Palmer’s Framework of Task Characteristics (2010), the study features a comprehensive task analysis detailing the characteristics of TFLM tasks in contrast to those of three major TLU academic and instructional contexts linked to the test. The results of the study are discussed in terms of the standards of validity (Messick, 1996) and qualities of usefulness (Bachman & Palmer, 1996). Findings suggest that TFLM tasks and constructs do not represent those of the TLU contexts and do not address the language needs of preservice teachers as identified by the Ministère de l’Éducation, du Loisir et du Sport (MELS). The implications for the consequential aspect of TFLM validity and the potential nega-tive consequences of TFLM use as an admission test are discussed.

Cet article présente une partie d’une étude plus complète sur la validité prédictive du Test de Français Laval-Montréal (TFLM), test de langue française à enjeux critiques utilisé comme test d’admission et de placement dans les programmes de formation initiale en enseignement d’importantes universités francophones au Canada (Schmitt, 2015). Le but de cette étude est d’analyser la validité des tâches du TFLM à des fins d’évaluation des compétences linguistiques exigées dans les tâches communes aux domaines d’utilisation de la langue cible dans lesquels les enseignants en formation doivent fonctionner (Target Language Use (TLU); Bachman & Palmer, 2010). Basée sur la conception de la validité conceptuelle de Messick (1989) et le cadre d’analyse des caractéristiques des tâches de Bachman & Palmer (2010), l’étude compare de manière détaillée les tâches du TFLM à celles de trois contextes académiques et pédagogiques d’emploi de la langue cible. Les résultats de cette analyse sont évalués en termes de validité (Messick, 1996) et des qualités des tests (Bachman & Palmer, 1996). Les résultats indiquent que les tâches du TFLM et les construits qu’il est sensé évaluer ne correspondent pas à ceux des contextes d’emploi de la langue cible et ne répondent pas aux besoins des ensei-gnants en formation tels qu’identifiés par le Ministère de l’Éducation, du Loisir et du Sport (MELS). La validité du TFLM, les conséquences ainsi que les aspects potentiellement négatifs de son utilisation comme test d’admission sont discutés.

88 ROmAiN SChmiTT & ShAhRzAD SAif

Background to the Study

This study examines the task characteristics of a high-stakes French language test whose score is used as an admission requirement to the teacher train-ing programs (TTPs) in francophone universities in the province of Quebec. Previous studies examining the predictive validity of admission tests have mainly focused on standardized English language tests as measures of L2 proficiency (Elder, 1993; Morris & Cobb, 2004). Few have investigated the validity of admission tests measuring candidates’ L1 (Blais, 2001; Romain-ville, 1997), and none, to our knowledge, has investigated a French language test as a measure of L1 proficiency in Quebec, the sole officially unilingual French province in Canada and the stronghold of the French language in North America.

The status of the French language in Quebec as well as Quebecers’ percep-tion of French is rooted in history, the language being part of Quebec identity. It is therefore not surprising that measures have been taken by successive governments of all allegiances to highlight and protect this linguistic iden-tity of Quebec. In 1977, for example, Quebec legislators passed into law Bill 101, the Charter of the French Language, in an attempt to ensure that French remains the language of the majority and continues to grow in Quebec. Bill 101 legally restricts access to English school boards, requires immigrants to be schooled in French, and mandates businesses to offer services primarily in French. Although disputed and polemical, Bill 101 is considered to be the cornerstone of Quebec language policy and is revered by most as a piece of legislation that preserves the French language from the overwhelming domi-nance of English spoken in the rest of Canada.

The preoccupation with the quality of the French language is therefore paramount at all levels in the Quebec educational system. Teachers, in par-ticular, are expected to act as linguistic models for their students and to pos-sess a comprehensive command of French, both linguistically and culturally. This priority was underlined in 2001 during the États Généraux de la langue française au Québec, a province-wide symposium whose purpose was to iden-tify issues with the quality of French in Quebec and find possible ways to improve it in all layers of society and for all users. The result was a compre-hensive and lengthy report (commonly named the Larose Report, after the commissioner who presided over the symposium) identifying the issues and suggesting elements of response (Commission des états généraux sur l’avenir de la langue française au Québec, 2001). One particular issue addressed in the report was the quality of French among teachers identified in response to a request by the Association Québécoise des Professeures et professeurs de Français (AQPF, 2001; Association of Quebec French Teachers) demanding that the government set minimal requirements for French, both linguistically and cul-turally, for the admission of candidates into TTPs in Quebec universities. This demand coincided with a referential document (Ministère de l’Éducation,


du Loisir et du Sport [MELS], 2001) that elaborated the requirements for the teaching profession in the form of 12 professional competencies (PCs), two of which specifically address the issues raised by the AQPF: the cultural knowl-edge (PC1) and the linguistic competency (PC2) expected from all preservice and inservice education professionals:

PC1: To act as a professional who is inheritor, critic, and interpreter of knowl-edge or culture when teaching students. (p. 57)

PC2: To communicate clearly in the language of instruction, both orally and in writing, using correct grammar, in various contexts related to teaching. (p. 63)

In addition, the practicums that were part of the TTPs’ curriculum underwent a major reform in 2001, the implementation of which necessitated major fi-nancial commitments on the part of the universities offering these programs. The new practicum guidelines required extra training, hiring, sometimes creating whole departments/offices for the sole purpose of organizing place-ments, evaluating students’ performance during their practicums, and setting up administrative and information technology (IT) platforms necessary for these activities.

These governmental orientations were, in large part, in response to a gen-eral and societal demand voiced by the popular media targeting teachers’ poor language skills and calling for school teachers to be, first and foremost, the linguistic role models for Quebec youth (Commission des États généraux sur l’éducation, 1996; Gagnon, 2005; Gauvreau, 2003). This popular and me-dia-led criticism highlighted the need for screening the candidates at the time of admission as a way of ensuring that the graduating teacher candidates en-tering the workplace would be qualified for teaching jobs. At the same time, for faculties faced with the financial burden of the government-imposed ad-justments to the TTPs’ curriculum, the need to select only the students most likely to succeed in their program increased dramatically. Quebec’s two major francophone universities, Université Laval and Université de Montréal, there-fore, embarked on the development of a screening device, Test de Français Laval Montréal (TFLM), as an admission requirement to all TTPs, including the English as a Second Language (ESL) teacher-training program for which the language of instruction is English. It is important to note that the appli-cants to TTPs in Quebec are overwhelmingly francophone, hence the status of TFLM as a measure of candidates’ L1. TFLM scores are therefore highly consequential in Quebec’s teacher-training domain. The question, however, remains as to whether TFLM tasks adequately measure the language abilities required for the successful completion of the language-related tasks in such real-life contexts as undergraduate TTPs or school contexts. In the following section, the content and the constructs measured by TFLM are introduced and discussed in some detail.


Test de Français Laval-Montreal: Tasks and Constructs

TFLM is a paper-and-pencil test of linguistic code used as an admission re-quirement for many programs, including TTPs at Université Laval (Appendix A) and Université de Montréal (and its affiliates, such as École Polytechnique de Montréal). The test, administered in 90 minutes, is composed of 65 (Uni-versité Laval) or 66 (Université de Montréal) multiple-choice questions, di-vided into 5 sections: lexical spelling (4 items), grammar spelling (24 items), morphology (5 items), syntax (13 items), and vocabulary (19 items). Different sections of the TFLM, with sample items, are summarized in Table 1.

Table 1 TFLM Sections and Sample Questions at Université Laval, Quebec

SectionSample itemInstruction: Rayez l’énoncé incorrect

Parts

of T

FLM

Lexical spelling4 items

A : Je marche rapidement.B : Le chien aboie méchament.C : Mes parents m’encouragent allègrement. D : Tu parles trop vite.E : Tous les énoncés sont corrects.

Grammar spelling24 items

A : Marie a envoyé des fleurs à sa mère.B : Les amis à qui Jean a parlés l’ont rappelé.C : Les amis à qui Marie a parlé l’ont rappelée.D : Mon chien a dévoré la perruche du voisin.E : Tous les énoncés sont corrects.

Morphology (Genders, singular-plural, conjugation)5 items

A : Le loup hurle à la lune.B : Les amis de Jean lui envoyent des souhaits

chaque année.C : Je regrette, mais cela ne sera pas possible.D : Vous connaissez cette personne?E : Tous les énoncés sont corrects.

Syntax (use of auxiliary verbs, modes and tenses, use of rela-tive pronoun, structure of sen-tences, negation, punctuation)13 items

A : Je suis monté à l’étage.B : J’ai monté la télévision à l’étage.C : J’ai descendu à la cave.D : J’ai descendu les escaliers.E : Tous les énoncés sont corrects.

Vocabulary (Anglicism, use of prepositions, conjunctions, knowledge of lexicon)19 items

A : Le chanteur a un rhume ; cependant, il a annulé son spectacle.

B : Le chanteur a un rhume ; il a malgré cela annulé son spectacle.

C : Le chanteur a un rhume ; il a donc annulé son spectacle.

D : Le chanteur a un rhume ; nonobstant, il a annulé son spectacle.

E : Tous les énoncés sont corrects.


At Université de Montréal, the cut-off score is 60% for all programs, in-cluding TTPs, whereas at Université Laval, the context of this study, the pass-ing score is 75% for all TTPs, with the exception of the Teaching English as a Second Language program, for which the passing score is 60%. The cut-off scores seem to have been arbitrarily assigned by the two universities, as there is no systematic study justifying the differences. It is also worth mention-ing that despite a lower passing score, Université de Montréal requires the students who do not score 60% to take fewer remedial French courses (i.e., one course) than Université Laval (see below). At Université Laval, the test score is used by the admission office to group incoming students into three proficiency groups: high-performing students (HIGH), whose TFLM score is above 75%; average-performing students (MID), whose TFLM score is be-tween 60 and 74%; and low-performing students (LOW), whose TFLM score is below 60%. Extra French courses are imposed on students placed in the MID (1 course) and the LOW (2 courses) groups. To ensure the successful completion of the program, the university requires the students to complete all remedial courses within the first two years following admission to TTPs. TFLM is therefore considered to be a high-stakes test, by the test-takers and the test users alike, as TTPs are the only compulsory path for anyone wanting to become a teacher in the province of Quebec.

As for the courses within the TTPs, they present two different dimensions: specialized courses that are specific to each program (such as science, history, French as a first or second language, etc.) and common didactic courses (so-cial aspects of education, laws, ethical concerns, etc.) that focus on pedagogi-cal aspects of the teaching profession. To these two dimensions is added the quality of language, which involves cultural knowledge and the fundamen-tals of the profession (MELS’ 12 professional competencies, 2001). As a result, during their training program, preservice teachers have to perform a variety of language-related tasks in different academic and real-life settings, namely the content-specific and didactic courses, French language courses (to which they will be assigned based on their TFLM score), and practicums in public schools. A language admission test to TTPs should, therefore, reflect the com-plexity of tasks required by these various settings. To fully understand the validity of TFLM as a measure of candidates’ linguistic ability for function-ing in these target language use (TLU) contexts, this study will present an analysis of the test task characteristics in relation to those of the TLU contexts.

The Problem

As an admission and placement criterion for TTPs at major francophone universities in Quebec, the TFLM score is consequential to test-takers and score users alike. TFLM is the gatekeeper for TTPs, which constitute the only path to the teaching profession in the province of Quebec. At the same time, extra French language courses imposed on candidates with lower scores on TFLM are costly and could prolong their studies. As well, the lo-


gistics of the test administration and the subsequent placement of students in remedial courses are costly and time-consuming for the universities. Given this high-stakes status of TFLM, it is of utmost importance to ensure that the test score is valid for the purposes to which it is put: admitting can-didates that possess the language abilities fundamental to success in TTPs as well as real-life instructional contexts, and placing students in different remedial courses linked to the test. The validity of the TFLM score (or lack thereof) could have important positive or negative consequences for the candidates, for the teacher training programs, and, eventually, for Quebec education.

Despite claims to the contrary (see Appendix A), to our knowledge, TFLM has not to date been subject to a systematic validity study. The present study, as part of a larger investigation of the predictive validity of TFLM (Schmitt, 2015), explores the validity of TFLM tasks for measuring language abilities required by tasks common to the TLU (Bachman & Palmer, 2010) domains in which preservice teachers are expected to function. The study addresses the following research questions (RQs):1. What are the task characteristics of TFLM?2. Is there a correspondence between TFLM tasks and those of the content

courses (TLU1)?3. Is there a correspondence between TFLM tasks and those of the remedial

courses (TLU2) linked to it?4. Is there a correspondence between TFLM tasks and those of the real-life

teaching context (TLU3)?

Conceptual Framework

The testing instrument under investigation in this study and in our context, TFLM, is a high-stakes test of French language proficiency whose score is used for making important admission and placement decisions that are consequen-tial to the test-takers and score users alike. In other words, the TFLM scores are not simply used as tools for descriptive purposes; they are used as a basis for further actions or decisions by francophone universities in Quebec. As a result, the suitability of the TFLM scores for admission and for making dif-ferential placement decisions crucially depends on how well it can predict the students’ performance in the programs and future remedial courses linked to it. The relationship between the abilities measured by the test and later success in the program is therefore central to the meaning of the construct (Shepard, 1997), and the test users need to ensure that valid inferences are made with respect to the candidates’ preparedness for TTPs and/or their placement in remedial courses based on TFLM scores. As Cronbach (1988) puts it, “the bot-tom line is that validators have an obligation to review whether a practice has appropriate consequences for individuals and institutions, and especially to guard against adverse consequences” (p. 6). Given this, the concept of validity adopted in this study is the unitary concept of construct validity defined by


Messick (1989) as the procedures underlying the inferences made with respect to the meaningfulness of the test scores. Fundamental to this unitary concep-tion of validity is that the social values and social consequences of the test use have to be taken into consideration in a discussion of validity (Messick, 1980, 1989) and that the validation process “must link concepts, evidence, social and personal consequences, and values” (Cronbach, 1988, p. 4).

In the field of language testing, Bachman (1990) acknowledges the sig-nificance of the value implications of test interpretation as well as the social consequences of test use for the development and use of language tests by pointing out that “tests are not developed and used in a value-free psycho-metric test-tube, they are virtually always intended to serve the needs of an educational system or of society at large” (p. 279). Subsequently, building on Messick’s view of validity, Bachman and Palmer (1996) propose the model of test usefulness with six qualities that serve as essential bases for quality con-trol over the entire test development process. In their model, in addition to test practicality, Bachman and Palmer identify test use consequences—which they name impact—along with reliability and construct validity, as qualities of test “use” and test “scores.” To this, they add authenticity and interactiveness, the qualities of test tasks that directly contribute to the construct validity of any language test. They define authenticity as the correspondence between the test tasks and those of real-life or target language use tasks. To enhance authenticity, test tasks should resemble, as closely as possible, the TLU tasks and measure the same constructs as those of the TLU tasks. Interactiveness refers to the correspondence between the test tasks and test-takers’ charac-teristics, such as language ability or topical knowledge. To achieve interac-tiveness, then, the test tasks should engage test-takers in such activities from which their language abilities can be inferred.

Bachman and Palmer (1996, p. 22), therefore, define construct validity in relation to two aspects of score interpretation: (a) the extent to which inter-pretations made on the basis of the test scores are indicative of the “language ability” in question, and (b) the generalizability of score interpretations to other relevant language use contexts.

Given the crucial role of test tasks for measuring language abilities in a test, Bachman and Palmer propose a detailed framework of Language Task Characteristics (2010, pp. 66–68), to be used in the process of test develop-ment, for describing the characteristics of the TLU tasks, and for compar-ing them with those of the test task. The present study, therefore, uses this framework as a guiding principle for analyzing the characteristics of the TLU tasks the applicants to TTPs need to perform, and compares them with those of the TFLM test.

The Study: Task Analysis ProcedureThis study is part of a larger research study on the predictive validity of TFLM (Schmitt, 2015) conducted in three phases using a convergent parallel


mixed-method design (Creswell & Plano-Clark, 2011). In the present article, we report on the qualitative study conducted in Phase 1 of that research—that is, the analysis of the test task characteristics as well as a comparison between the test tasks and those of the three TLU domains within which the applicants to TTPs should function. The analysis of the test task characteristics was con-ducted using the following official documents:

• a version of TFLM; • course descriptions and syllabi from content courses in a TTP at Univer-

sité Laval; • detailed course descriptions and syllabi for the 3 French language reme-

dial courses (FRN-1902, FRN-1903, and FRN-1904) imposed on students based on TFLM score at the time of admission (Appendix B);

• official description of PC2 (MELS, 2001).

The above documents are either public-domain documents, accessible through Université Laval’s website, or course documents published on indi-vidual course portals.

Using Bachman and Palmer’s model of Task Characteristics (2010), a com-plete description of the characteristics of TFLM tasks and those of the 3 TLU contexts was conducted. The model of Language Knowledge, proposed by Bachman and Palmer (2010), was used to delineate the constructs involved in the input and expected response required by tasks in each TLU context. To determine the authenticity and interactiveness of the test tasks, the charac-teristics of the test tasks were then detailed and compared with those of the TLU contexts for possible differences and/or correspondences between them.

Results

As explained above, TFLM tasks measure test-takers’ recognition ability. The test only measures grammatical knowledge out of context using a multiple-choice format. However, a survey of the documents related to the remedial courses linked to the test revealed communicative writing tasks—that is, “process-oriented writing” and “revision”—as the tasks practiced and evalu-ated in such courses. Similarly, the mandatory content courses and practi-cums required by the program necessitate such language-related academic tasks as the comprehension of oral/written material, report/summary/essay writing, and the comprehension/production of speech.

Given this, as a first step, we proceeded to the analysis of the objectives of the test tasks as compared to those common to the TLU contexts; in particu-lar, the inferences made based on TFLM scores, the use to which those scores are put, and the impact that the test is expected to have on different players in the educational context. Table 2 reflects a detailed and, at the same time, an instant picture of the objectives of the test tasks and those of the TLU contexts.


Tabl

e 2

The

Obj

ectiv

es o

f TFL

M T

asks

vs.

TLU

Tas

ks C

omm

on to

TTP

sTy

pe o

f an

alys

is

TFLM

TLU

Tas

ks 1

: C

onte

nt c

ours

es

TLU

Tas

ks 2

: Rem

edia

l Fr

ench

cou

rses

TL

U T

asks

3: P

ract

icum

/real

-life

teac

hing

situ

atio

n (2

001

MEL

S’ P

rofe

ssio

nal C

ompe

tenc

y 2)

INFE

REN

CE

Lang

uage

ab

ilitie

s required for

the

com

ple-

tion

of th

e ta

sk(s

)

• Recognition of the

writ

ten

mat

eria

l inv

olv-

ing

orga

niza

tiona

l kn

owle

dge:

• G

rammatical (spell-

ing,

mor

phol

ogic

al,

synt

actic

, voc

abul

ary

know

ledg

e)• Textual (conjunc

-tio

ns)

• Com

prehension/

prod

uctio

n of

the

written/oral material

in s

ubje

ct-re

late

d or

no

n-su

bjec

t-rel

ated

co

urse

s of

TTP

s (li

ngui

stic

s, li

tera

ture

, pe

dago

gy, a

nthr

opol

-og

y, co

mm

unic

atio

n,

etc.

) inv

olvi

ng to

pica

l kn

owle

dge

and

all

area

s of

lang

uage

kn

owle

dge

• Interactional con

-texts require the use

of m

etac

ogni

tive

stra

tegi

es

Writ

ing

lingu

istic

ally

acc

urat

e an

alyt

ical

and

arg

umen

tativ

e te

xts

base

d on

the

com

pre-

hens

ion

of c

onte

nt-b

ased

w

ritte

n la

ngua

ge m

ater

ials

in

volv

ing

topi

cal k

now

ledg

e,

and

area

s of

org

aniz

atio

nal

know

ledg

e, a

nd te

xtua

l kn

owle

dge

“To

com

mun

icat

e cl

early

in th

e la

ngua

ge o

f ins

truct

ion,

bo

th o

rally

and

in w

ritin

g, u

sing

cor

rect

gra

mm

ar, i

n va

rious

con

text

s re

late

d to

teac

hing

.… U

ses

appr

opri-

ate

lang

uage

whe

n sp

eaki

ng to

stu

dent

s, p

aren

ts, a

nd

peer

s.…

Obs

erve

s ru

les

of g

ram

mar

and

sty

listic

s w

hen

writ

ing

text

s in

tend

ed fo

r stu

dent

s, p

aren

ts, o

r pe

ers.

… Is

abl

e to

take

up

a po

sitio

n, s

uppo

rt hi

s or

he

r ide

as, a

nd a

rgue

his

or h

er s

ubje

ct m

atte

r in

a co

nsis

tent

, effe

ctiv

e, c

onst

ruct

ive,

and

resp

ectfu

l way

du

ring

disc

ussi

ons.

… C

omm

unic

ates

idea

s co

ncis

ely

usin

g pr

ecis

e vo

cabu

lary

and

cor

rect

syn

tax.

… C

or-

rect

s th

e m

ista

kes

stud

ents

mak

e w

hen

spea

king

and

w

ritin

g.…

Con

stan

tly s

trive

s to

impr

ove

his

or h

er o

wn

oral

and

writ

ten

lang

uage

ski

lls.…

Lev

el o

f mas

tery

ex-

pect

ed: M

aste

r the

rule

s of

ora

l and

writ

ten

expr

essi

on

so a

s to

be

unde

rsto

od b

y th

e lin

guis

tic c

omm

unity

; Ex

pres

s hi

mse

lf or

her

self

with

the

ease

, pre

cisi

on, e

ffi-

cien

cy, a

nd a

ccur

acy

expe

cted

by

soci

ety

of a

teac

hing

pr

ofes

sion

al.”

(200

1, p

p. 6

3–67

)-Involving all areas of language know

ledge including

area

s of

met

acog

nitiv

e st

rate

gy u

se

(con

tinue

d on

nex

t pag

e)


Type

of

anal

ysis

TF

LMTL

U T

asks

1:

Con

tent

cou

rses

TL

U T

asks

2: R

emed

ial

Fren

ch c

ours

es

TLU

Tas

ks 3

: Pra

ctic

um/re

al-li

fe te

achi

ng s

ituat

ion

(200

1 M

ELS’

Pro

fess

iona

l Com

pete

ncy

2)

USA

GE

Dec

isio

ns

mad

e on

th

e ba

sis

of

the

resu

lts

obta

ined

• Adm

ission to TTP

s • 1

, 2, or no extra

rem

edia

l cou

rse

im-

pose

d on

stu

dent

s in

thei

r pro

gram

of

choi

ce

Pass/fail decisions

for c

onte

nt c

ours

es.

Afte

r 2 fa

ilure

s fo

r th

e sa

me

cour

se,

stud

ents

may

be

expe

lled

from

thei

r pr

ogra

m o

r put

und

er

prob

atio

n (in

kee

p-in

g w

ith th

e st

udie

s’

polic

ies

and

regu

la-

tions

). Sc

ores

affe

ct

stud

ents

’ GPA

.

Pass/fail decisions. Students

who

do

not d

emon

stra

te

a sufficient m

astery of the

cour

se c

onte

nt a

s ev

alua

ted

by te

ache

rs o

f eac

h co

urse

w

ill ha

ve to

atte

nd th

e co

urse

ag

ain.

Afte

r 2 fa

ilure

s fo

r the

sa

me

cour

se, s

tude

nts

may

be

exp

elle

d fro

m th

eir p

ro-

gram

or p

ut u

nder

pro

batio

n (in

kee

ping

with

the

stud

ies

polic

ies

and

regu

latio

ns).

Results for each course

coun

t tow

ards

stu

dent

’s G

PA,

whi

ch m

ust k

eep

a ce

rtain

le

vel i

n ea

ch c

ycle

so

the

stu-

dent

can

gra

duat

e.

• During internships, PC2 is part of evaluation and is a

disqualifying criterion

• Students may not be hired by school system if PC2 is

deem

ed to

o lo

w

INTE

ND

ED IM

PAC

T

On

stu-

dent

sTe

st-ta

kers

are

pla

ced

into proficiency groups

acco

rdin

g to

thei

r TF

LM scores. It is ex-

pect

ed th

at th

is p

lace

-m

ent w

ill he

lp s

tude

nts

impr

ove

thei

r lev

el o

f proficiency and their

perfo

rman

ce in

con

-te

nt c

ours

es

Subj

ect-r

elat

ed

cour

ses

are

expe

cted

to

offe

r con

tent

nec

-es

sary

for s

tude

nts

to

func

tion

in th

eir f

utur

e ca

reer

and

in th

e sc

hool

env

ironm

ent

durin

g th

e co

mpu

l-so

ry in

tern

ship

s

Expe

cted

to p

rovi

de th

e ap

prop

riate

hel

p to

wea

ker

stud

ents

pla

ced

in c

ompe

-te

ncy

leve

ls b

ased

on

TFLM

sc

ores

. Thi

s su

ppor

t aim

s at

im

prov

ing

resu

lts o

f wea

ker

stud

ents

in T

TPs,

inte

rnsh

ips,

an

d th

eir f

utur

e ca

reer

The

inte

rnsh

ips

are

expe

cted

to h

elp

stud

ents

con

-solidate the know

ledge acquired through university

cour

ses.

The

MEL

S PC

s ar

e ex

pect

ed to

faci

litat

e th

e ap

plic

atio

n of

the

abst

ract

con

cept

s in

real

-life

situ

-at

ions

and

thus

ens

ure

the

inte

grat

ion

of s

tude

nts

in

thei

r fut

ure

care

er

(con

tinue

d on

nex

t pag

e)


Type

of

anal

ysis

TF

LMTL

U T

asks

1:

Con

tent

cou

rses

TL

U T

asks

2: R

emed

ial

Fren

ch c

ours

es

TLU

Tas

ks 3

: Pra

ctic

um/re

al-li

fe te

achi

ng s

ituat

ion

(200

1 M

ELS’

Pro

fess

iona

l Com

pete

ncy

2)

On

the

teac

hing

st

aff

NA

NA

Uni

form

cou

rse

plan

is e

x-pe

cted

to m

otiv

ate

teac

hers

to

ada

pt th

eir t

each

ing

activ

i-tie

s to

the

cour

se o

bjec

tives

Real-life teaching situation draw

s on theoretical con

-ce

pts

taug

ht th

roug

h un

iver

sity

cou

rses

. Exp

ecte

d to

m

otiv

ate

the

teac

hing

sta

ff to

focu

s on

the

appl

icat

ion

of th

e ab

stra

ct c

once

pts

On

the

educ

atio

nal

syst

em

• Select only the most

prep

ared

stu

dent

s• Identify weaker stu

-de

nts

so th

ey re

ceiv

e th

e su

ppor

t the

y ne

ed

to s

ucce

ed in

TTP

s• A

ccurately discrim

i-na

te a

nd p

lace

the

low-proficiency stu-

dent

s in

app

ropr

iate

re

med

ial c

ours

es

• Provide subject

mat

ter n

eede

d by

fu

ture

teac

hers

for

func

tioni

ng in

inst

ruc-

tiona

l con

text

s• Target the acquisi

-tio

n of

the

conc

epts

in

line

with

the

cour

se

obje

ctiv

es

• Help the students placed

in lower proficiency levels

catc

h up

with

the

adva

nced

st

uden

ts• E

nhance the success rate

in T

TPs

• Improve the enrolment by

help

ing

the

stud

ents

with

lo

wer

sco

res

on T

FLM

to

meet the language require

-m

ents

for t

he p

rogr

ams

• Ensure the future teachers have the linguistic ability

to tr

ansf

er k

now

ledg

e w

ith e

ase

• Facilitate the integration of teachers in educational

cont

exts

On

soci

ety

A va

lid la

ngua

ge

test

use

d fo

r scr

een-

ing

the

cand

idat

es

to T

TPs

will

ensu

re

futu

re te

ache

rs w

ill ef

fect

ivel

y us

e th

e la

n-gu

age

of in

stru

ctio

n in

ed

ucat

iona

l con

text

s

Appr

opria

te c

onte

nt

cour

ses

inte

gral

to

the

train

ing

of c

om-

pete

nt a

nd e

ffect

ive

teac

hers

Teac

hers

’ com

pete

nt u

se o

f th

e la

ngua

ge o

f ins

truct

ion

lead

s to

bet

ter l

earn

ing

on

the

part

of th

e st

uden

ts

Teac

hers

are

bet

ter p

repa

red

to re

al-li

fe te

achi

ng s

itu-

ations when they experience first-hand teaching situa-

tions

dur

ing

thei

r tra

inin

g

Tabl

e 2

(con

tinue

d)


As for the areas of language knowledge required to complete the TFLM tasks and the tasks of the TLU contexts, Table 3 presents the ability areas tapped by TFLM and the tasks in the three TLU contexts.

TFLM tasks and the 3 TLU contexts’ tasks were also analyzed in terms of the characteristics of the context of administration, characteristics of the administration and directives of tasks, specific characteristics of input, spe-cific characteristics of the expected responses (output), and the relationship between content of TLU tasks and the expected output in the tasks (Table 4).

This side-by-side examination of the test tasks and those of the TLU con-texts allows us to discuss whether TFLM tasks foreshadow the tasks and language needs featured in the remedial French courses, in undergraduate content courses, and in the future teaching contexts. In the next section, we will discuss whether it is logical that TFLM be used to determine the level of competency students should have in order to succeed in their program of choice. We will then draw conclusions as to the validity of TFLM as a mea-sure of language ability and as an admission criterion for TTPs in Quebec francophone universities.

DiscussionThe information presented in Tables 2, 3, and 4 reflects the characteristic fea-tures of the TFLM tasks compared to those of the three TLU tasks. This type of evidence has implications for the validity of the test tasks in general, and the extent to which the characteristics of the test tasks correspond to those of the three major TLU tasks (authenticity) and those of the intended test-takers (interactiveness) in particular.

With respect to the authenticity of TFLM tasks, the evidence suggests fun-damental differences between the test tasks and those of the TLU contexts. As discussed previously, TFLM scores are used by universities to make ad-mission and placement decisions for their programs. As such, the test scores are expected to provide a measure of the candidates’ linguistic prepared-ness for undergraduate studies and predict their chance of success in such programs. TFLM tasks, however, as illustrated in Tables 2 and 3, measure only test-takers’ knowledge of the linguistic code of French. The constructs are limited to the components of the “organizational knowledge” measured in an indirect way, out of context, and by means of a multiple choice format (RQ1). However, the tasks of the TLU contexts (content courses, remedial French courses, and real-life teaching contexts in practicums, as determined by MELS’s 2001 PC2) are radically different from the indirect recognition tasks included in TFLM and the completion of them requires the mastery of a much broader range of constructs than those measured by TFLM (RQs 2, 3, 4). Content courses (TLU 1), for example, require the students to fully function in interactional and linguistically complex instructional settings (i.e., consult written materials, understand spoken language, produce written texts, and express academic concepts orally) necessitating mastery of all areas of lan-guage knowledge (Table 3) at an advanced level.


Table 3Language Abilities Required by TFLM Tasks vs. TLU Tasks Common to TTPs

Type of analysis TFLM

TLU task 1: content courses

TLU task 2: Remedial French courses

TLU tasks 3: practicum/ real-life teaching situation (2001 MELS’ Professional

Competency 2)ORGANIZATIONAL KNOWLEDGE

Grammatical knowledgeKnowledge of vo-cabulary

Knowledge of syntax

Knowledge of phonology

Textual knowledgeKnowledge of co-hesion

Knowledge of conversational organization

PRAGMATIC KNOWLEDGEFunctional knowledge

Objectives of communication intentions

Sociolinguistic knowledgeKnowledge of dialects

Knowledge of reg-isters

Knowledge of nat-ural or idiomatic expressions

Knowledge of cul-tural references

METACOGNITIVE STRATEGY USEAreas of metacog-nitive strategy use


Tabl

e 4

TFLM

vs.

TLU

Tas

k C

hara

cter

istic

sTy

pe o

f an

alys

isTF

LMTL

U T

ask

1: C

onte

nt

cour

ses

TLU

Tas

k 2:

Rem

edia

l Fr

ench

cou

rses

TLU

Tas

k 3:

pra

ctic

um/re

al-li

fe te

achi

ng s

itua-

tion

(200

1 M

ELS’

Pro

fess

iona

l Com

pete

ncy

2)C

HA

RA

CTE

RIS

TIC

S O

F TH

E A

DM

INIS

TRAT

ION

CO

NTE

XTLo

gist

ical

ch

arac

ter-

istic

s: te

st-

take

rs, t

ime,

du

ratio

n

Loca

tion:

cla

ssro

om

envi

ronm

ent (

lum

i-no

sity,

tem

pera

ture

, hu

mid

ity, e

tc.)

Dur

atio

n: 9

0 m

inut

esD

ate:

var

iabl

eN

umbe

r of t

est-t

aker

s:

varia

ble

Loca

tion:

cla

ssro

om e

nvi-

ronm

ent (

lum

inos

ity, t

em-

pera

ture

, hum

idity

, etc

.)D

urat

ion:

var

iabl

eTi

me:

var

iabl

eN

umbe

r of s

tude

nts:

var

i-ab

le

Sam

e as

TLU

1Pr

imar

y, se

cond

ary,

or a

dult

educ

atio

n cl

ass-

room

env

ironm

ent

In Quebec, Canada (im

portant variation de

-pe

ndin

g on

sch

ools

)

CH

AR

AC

TER

ISTI

CS

OF

THE

TASK

S’ A

DM

INIS

TRAT

ION

AN

D D

IREC

TIVE

SD

irect

ives

In French. Clear and

conc

ise

• French instructions, but

poss

ibilit

ies

of d

irect

ives

in

Engl

ish

• Variety of tasks used (re

-po

rts, r

eadi

ngs,

pre

sent

a-tio

ns, M

CQ

, etc

.)

Fren

ch in

stru

ctio

ns, b

ut p

ossi

-bi

litie

s of

dire

ctiv

es in

Eng

lish.

• variety of w

ritten tasks (e.g.,

reports on readings/presen-

tatio

ns, M

CQ

, etc

.) PL

US

com

pute

r-bas

ed e

xerc

ises

.

All t

asks

rela

ted

to te

achi

ng c

onte

xt a

nd

situ

atio

ns. D

irect

ives

are

var

iabl

e de

pend

ing

on th

e tu

torin

g-te

ache

r. Ad

min

istra

tive

task

s ad

ded

Task

s’

mod

aliti

esM

CQ

in a

sep

arat

e bo

okle

t with

«bl

acke

n th

e co

rrect

ans

wer

» an

swer

she

et

• All types of tasks available

(repo

rts, r

eadi

ngs,

pre

sen-

tatio

ns, M

CQ

, etc

.).• A

ccess to spellcheck

softw

are

and

all r

esou

rces

av

aila

ble.

• French instructions

• Mostly linguistic code exer

-ci

ses

(pap

er a

nd p

enci

l) pl

us

com

pute

r-bas

ed e

xerc

ises

All t

asks

rela

ted

to te

achi

ng c

onte

xt a

nd s

itu-

atio

ns. D

irect

ives

are

var

iabl

e de

pend

ing

on

the

tuto

ring-

teac

her.

Log-

book

. Adm

inis

trativ

e ta

sks

adde

d

Tim

e al

lotte

d90

min

utes

Varia

ble

Varia

ble

Varia

ble

Eval

uatio

n pr

oced

ure

Opt

ical

read

erVa

riabl

e de

pend

ing

on ty

pe

of ta

skVa

riabl

e de

pend

ing

on ty

pe

of ta

skLo

gboo

k; p

ortfo

lio, r

epor

t; se

lf-ev

alua

tion;

liv

e or

dis

tanc

e su

perv

isio

n

(con

tinue

d on

nex

t pag

e)


Type

of

anal

ysis

TFLM

TLU

Tas

k 1:

Con

tent

co

urse

sTL

U T

ask

2: R

emed

ial

Fren

ch c

ours

esTL

U T

ask

3: p

ract

icum

/real

-life

teac

hing

situ

a-tio

n (2

001

MEL

S’ P

rofe

ssio

nal C

ompe

tenc

y 2)

SPEC

IFIC

CH

AR

AC

TER

ISTI

CS

OF

INPU

TFo

rmat

of

pres

enta

tion

Mul

tiple

cho

ice

item

s w

ith 4

opt

ions

; no

con-

text

pro

vide

d

Varied, scientific special-

ized articles/presentations

• Oral/w

ritten prom

pts

• Workshops

• Problem

resolution exercises

• Com

puter-b

ased exercises

Varia

ble,

dep

endi

ng o

n pr

ogra

m, a

ge o

f le

arne

rs, s

choo

l boa

rd, p

rovi

nce,

cou

ntry

SPEC

IFIC

CH

AR

AC

TER

ISTI

CS

OF

EXPE

CTE

D R

ESPO

NSE

S (O

UTP

UT)

Form

at o

f pe

rform

ance

Recognition of the cor-

rect

ans

wer

• Report

• Short and long answers

• Erro

r analysis; writing tasks

• Grammar knowledge ex

-er

cise

s• S

elf-correction exercises

• Com

puter-b

ased exercises

(MC

Q)

Oral/w

ritten communication, interactional, in

vario

us c

onte

xt a

nd w

ith v

ario

us p

urpo

ses

(com

mun

icat

ion

to s

tude

nts,

par

ents

, pro

fes-

sion

als,

col

leag

ues)

Ling

uist

ic

cont

ent o

f ex

pect

ed

resp

onse

(le

ngth

, or-

gani

zatio

nal,

and

prag

-m

atic

cha

r-ac

teris

tics)

No

prod

uctio

n on

the

part

of th

e te

st-ta

kers

(M

CQ

)

Varia

ble

depe

ndin

g on

the

task

at h

and;

resp

onse

ra

ngin

g fro

m n

one

(MC

Q)

to full-length oral/w

ritten

text

s

Varia

ble

depe

ndin

g on

the

task

at h

and.

Fro

m n

one

(MC

Q) t

o fu

ll-le

ngth

writ

ing

task

s

• Teachers are expected to use language pro

-fe

ssio

nally

. (M

ELS,

200

1)• T

he teacher’s linguistic com

petency need

not b

e of

the

sam

e le

vel a

s th

at o

f a s

peci

al-

ist,

but i

t mus

t be

grea

ter t

han

that

of t

he

ordinary citizen and include specific usages

dist

ingu

ishi

ng it

from

that

of o

ther

pro

fess

ions

. Introducing youngsters to language and cul

-ture is a specialized task unique to the work of

teac

hers

. (M

ELS,

200

1)

REL

ATIO

NSH

IP B

ETW

EEN

CO

NTE

NT

OF

TLU

TA

SKS

AN

D E

XPEC

TED

OU

TPU

TReactivity

Non

reci

proc

alReciprocal/N

onreciprocal

Non

reci

proc

alReciprocal or adaptative

Scop

e of

re

latio

nshi

pVe

ry li

mite

d Va

riabl

e Va

riabl

eVa

riabl

e

Dire

ctne

ss

of re

latio

n-sh

ip

Indirect

Dire

ctD

irect

Dire

ct

Tabl

e 4

(con

tinue

d)


Even more paradoxical is the case of the content of the remedial French courses linked to TFLM (TLU 2) at Université Laval. Based on their TFLM score, the applicants to TTPs are placed in French language courses that are supposed to address their linguistic shortcomings as identified by TFLM. However, the tasks and constructs developed in these courses, detailed in the Results section above, are quite the opposite of those measured by TFLM (Appendix B). All three courses aim to develop the production of academic written texts based on the comprehension of written language mechanisms and using a direct communicative approach. The constructs promoted in these courses cover far beyond the organizational knowledge measured by TFLM. Note that TFLM does not include any comprehension task, reading is not evaluated, and the prompts for multiple-choice ques-tions are extremely brief and limited. It is not clear why the students are placed in remedial language courses based on the scores from a test whose tasks and content are so different from the tasks and contents of the lan-guage courses linked to it. Similarly, TFLM tasks bear no resemblance to the complex interactional and communicative tasks required by real-life teach-ing contexts (TLU 3) that draw upon the topical knowledge as well as all areas of language ability. TFLM tasks, therefore, lack authenticity, as they seriously underrepresent the constructs required by TLU tasks (Messick, 1996) that teacher-training students have to perform during and after the completion of their programs.

This narrow range of language abilities measured by TFLM tasks is in sharp contrast with the range of language abilities required by the TLU tasks common to TTPs. Aside from the fact that production and comprehension abilities are not measured, the test tasks do not presuppose the appropriate area or level of topical knowledge critical to academic disciplines. In addi-tion, the indirect nature of TFLM tasks makes it impossible to engage the test-takers in performances other than simple recognition of the grammatical forms. Test-takers are not required to process input and formulate a response through a set of interdependent tasks from which valid inferences can be made with respect to their language knowledge and strategy use, the abilities fundamental to the TLU tasks identified above. TFLM tasks, therefore, clearly lack interactiveness.

Conclusion

As discussed earlier in this article, the extent to which test tasks meet the qualities of authenticity and interactiveness has direct implications for the construct validity of the test scores and the interpretations made based on them (Bachman & Palmer, 1996, 2010). The above findings, however, make it abundantly clear that the constructs measured by TFLM do not adequately reflect the test’s purposes and that the test scores are not relevant to or ap-propriate for the decisions based on them.


Given the high-stakes status of TFLM and the consequences of the de-cisions based on its scores for the applicants to TTPs, for the educational institutions, and for the society at large, this absence of construct validity is disconcerting. The obvious mismatch between the constructs measured by TFLM and those of the corresponding TLU contexts not only renders the intended consequences of the test score (Table 2) unachievable but is also likely to create negative consequences such as the admission of candidates who do not meet the linguistic demands of the teaching profession; wrong-ful placement of candidates in remedial courses; imposing unnecessary lan-guage courses, thus prolonging certain students’ education while at the same time neglecting the language problem of others; loss of time, money, and energy administering a test that does not discriminate validly; and, more importantly, risking the quality of public education in Quebec by compromis-ing the quality of the teacher education programs. The findings of this study therefore confirm that TFLM scores do not help the test users (i.e., univer-sity admission offices) to make the desired interpretations about test-takers’ language abilities or make the right admission and placement decisions. An important implication of these findings for the validation of the high-stakes tests is that the decisions made on the basis of the test scores should be con-sidered as part of the validity evidence:

A decision is a choice between courses of action. The college admits or rejects a prospective student.… Testing is intended to reduce the number of incorrect predictions and hence the number of deci-sions that will be regretted later. When validating a decision-making process, the concern is with the question: What is the payoff when decisions are made in the proposed way, and how does this compare with the payoff resulting when decisions are made without these data? (Cronbach, 1971, p. 448)

Given the use of TFLM scores for making high-stakes admission and placement decisions in Quebec universities, the next phase of this study (to be reported in a separate article) focuses on examining the predictive power of TFLM as evidenced by the students’ scores during their four-year under-graduate studies in TTPs.

The AuthorsRomain Schmitt is coordinator of the International office and internships and placement services at Cégep Limoilou, Quebec. He holds a Master’s degree in applied linguistics from Université Laval with a specialization in language testing. His research interests include validity of high-stakes language tests and teacher training programs.

Shahrzad Saif is a professor of Language Testing and Assessment at the Departément de langues, linguistique et traduction, Université Laval (Québec). She conducts research on the impact of high-stakes tests on teaching and learning practices, test consequences, language test develop-ment and validation, classroom assessment, and needs assessment.


ReferencesAssociation Québécoise des professeures et professeurs de Français (AQPF). (2001). Mémoire

déposé pendant les États généraux de la langue française au Québec. http://www.spl.gouv.qc.ca/fileadmin/medias/pdf/memoires/220_Ass_queb_prof_fr.pdf

Bachman, L. (1990). Fundamental considerations in language testing. Oxford, UK: Oxford University Press.

Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford, UK: Oxford University Press.

Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford, UK: Oxford University Press.

Blais, F. (2001). Le Canadian Achievement Test utilisé comme indicateur de réussite scolaire (Unpublished master’s thesis, Université d’Alberta, Campus Saint-Jean, Edmonton).

Centre d’évaluation, Faculté des Lettres, École de langues, Université Laval. (2013). Le TFLM (Test de Français Laval-Montréal), Épreuve de français langue maternelle. Retrieved from http://www.elul.ulaval.ca/fileadmin/elul/fichiers/tests-de-classement/sinscrire-a-un-test/TFLM_Description.pdf

Commission des états généraux sur l’éducation. (1996). Exposé de la situation. Gouvernement du Québec, Ministère de l’éducation.

Commission des états généraux sur l’avenir de la langue française au Québec. (2011). Le fran-çaise, une langue pour tout le monde (Rapport Larose). Quebec, QC: Gouvernement du Québec, Ministère de l’éducation.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Los Angeles, CA: Sage.

Cronbach, L. J. (1971). Validity. In R. L. Thorndike (Ed.), Educational measurement (pp. 443–507). Washington, DC: American Council on Education.

Cronbach, L. J. (1988). Five perspectives on validation argument. In H. Wainer & H. Braun (Eds.), Test validity (pp. 3–17). Hillside, NJ: Erlbaum.

Elder, C. (1993). Language proficiency as a predictor of performance in teacher education. Melbourne Papers in Language Testing, 2(1), 68–85.

Gagnon, K. (2005, November 28). À l’école des cancres. La Presse. Montréal.Gauvreau, C. (2003). Renforcer la maîtrise du français : une priorité! Actualités UQAM, 3(7), 3. Messick, S. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–

1027.Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). New York:

Macmillan.Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256.Ministère de l’éducation, du Loisir et du Sport (MELS). (2001). Teacher training: orientations and

professional competencies. Retrieved from http://www.education.gouv.qc.ca/fileadmin/site_web/documents/dpse/formation_ens_a.pdf

Morris, L., & Cobb, T. (2004). Vocabulary profiles as predictors of the academic performance of teaching English as a second language trainees. System, 32(1), 75–87.

Romainville, M. (1997). Peut-on prédire la réussite d’une première année universitaire? Revue Française de Pédagogie, 119, 81–90.

Schmitt, R. (2015). Analyse de la validité prédictive d’une épreuve standardisée de langue française chez des étudiants locuteurs natifs francophones inscrits dans des programmes de formation en ensei-gnement : le cas du TFLM (Unpublished master’s thesis, Université Laval, Québec, Québec).

Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16(2), 5–8, 13, 24.


Appendix A. TFLM Descriptive Document Sent to Candidates, Université Laval, Fall 2013 (Centre d’évaluation, Faculté des Lettres, 2013)Centre d’évaluation École de langues de l’université Laval

TFLM (Test de français Laval-Montréal), épreuve de français langue

maternelle

Le TFLM est un test objectif de 65 questions à choix multiple. Il a été conçu par l’Université de Montréal et l’Université Laval et ses six versions ont fait l’objet d’une rigoureuse validation scientifique. Pour s’y préparer, vous pouvez consulter le manuel suivant : CLAMAGERAN S., I. CLERC, M. GRENIER et R.-L. ROY. 2015, Le français apprivoisé, 4e édition, Montréal, Modulo.

Description

L’orthographe lexicale (4 questions)

L’orthographe grammaticale (24 questions) : Homonymes Accords de l’adjectif, des indéfinis, du verbe, du participe passé La morphologie (5 questions) : Le genre et le pluriel de certains noms La conjugaison

La syntaxe (13 questions) : Emploi des auxiliaires, des modes et des temps Emploi du pronom relatif et de la conjonction de subordination Structure de la phrase, négation, interrogation Ponctuation

Le vocabulaire (19 questions) : Anglicismes Mots-liens Emploi de la préposition Connaissance du vocabulaire

Temps alloué : 90 minutes. Aucun ouvrage de référence n’est permis.


Appendix B. Course Plans for FRN-1902, FRN-1903, and FRN-1904, Université Laval, Fall 2013

Appendix 2

Course plans for FRN-1902, FRN-1903 et FRN-1904, Université Laval, Fall 2013

2


2


4


5


6

7


7

8


8

Documents

Test de Français Laval-Montreal: Does It Measure What It ... · suggesting elements of response (Commission des états généraux sur l’avenir de la langue française au Québec,