Introduction Precepts Method Model 1 Model 2 Conclusion

Prescriptivism versus colloquialization:the rise of relative that

Benedikt Szmrecsanyi

KU LeuvenQuantitative Lexicology and Variational Linguistics

10th UKLVC conference, York, September 2015

Introduction Precepts Method Model 1 Model 2 Conclusion

slides @ http://www.benszm.net/UKLVC.pdf

Introduction Precepts Method Model 1 Model 2 Conclusion

Collaborative work

Hinrichs, Lars, Benedikt Szmrecsanyi & Axel Bohmann (to appear).Which-hunting and the Standard English relative clause.Language 91(4). (http://tinyurl.com/qz6tp9w)

Lars HinrichsUT Austin

Axel BohmannUT Austin

Introduction Precepts Method Model 1 Model 2 Conclusion

Introduction

Introduction Precepts Method Model 1 Model 2 Conclusion

Relativization options in StE RRCs

subject position

(1) a. Tom saw the car that caught fireb. Tom saw the car which caught fire

object position

(2) a. Tom saw the car that Mary had soldb. Tom saw the car which Mary had soldc. Tom saw the car Mary had sold

Introduction Precepts Method Model 1 Model 2 Conclusion

This presentation in one slide

• alternation between which, that and zero as restrictiverelativizers in StE undergoing massive shift from which tothat

• in the Brown corpora (1960s & 1990s AmE/BrE), AmEspearheads this change

• study 16,868 RRCs and annotate for additional areas ofvariation regulated by prescriptivism

• that-shift is a case ofinstitutionally backed colloquialization

Introduction Precepts Method Model 1 Model 2 Conclusion

The Brown family of corpora

Four corpora with (near-)identical design sampling writtenStandard English (1 million words each):

R E C E N T C H A N G E S I N F U N C T I O N A N D F R E Q U E N C Y O F

S TA N DA R D E N G L I S H G E N I T I V E S

443

Figure 2. The Brown quartet of matching corpora of written Standard English

abstract, ‘informationally’ oriented variant, a multivariate analysis such as ours will

help to distinguish between those aspects of genitive variation that can actually be

ascribed to colloquialization, and those which might be better explained as, for example,

economization strategies (see our discussion of this aspect in section 7.3 below).

In short, our research objectives in this article are:

(i) to determine the hierarchy of factors that influence genitive choice in journalistic

language, based on the analysis of all four corpora of StE;

(ii) to explore, and account for, differences in genitive choice between BrE and AmE;

(iii) to model the ongoing shift from of- to s-genitives in press language in terms

of changing weights associated with noncategorical constraints in a probabilistic

grammar framework (cf., for instance, Bresnan, Cueni, Nikitina & Baayen

forthcoming; Manning 2003).

On the methodological plane, it follows naturally from the above that we will adopt a

variationist approach to genitive variation, in the spirit of, for example, Labov (1969)

and Weiner & Labov (1983). In this connection, we will seek to demonstrate the

value of part-of-speech-tagged (POS-tagged) corpora in combination with multivariate

variationist methodology.

2 The data

Our choice of data is press material (sections A and B) in the Brown family of corpora, a

set of four corpora of written StE documenting two varieties of English at two different

points in time: British English and American English in the 1960s and 1990s (see

figure 2). All corpora were compiled according to the design of the first corpus, Brown,

(see Hinrichs et al. 2010)

Introduction Precepts Method Model 1 Model 2 Conclusion

The Brown family of corpora

R E C E N T C H A N G E S I N F U N C T I O N A N D F R E Q U E N C Y O F

S TA N DA R D E N G L I S H G E N I T I V E S

471

Appendix

Table A. Text categories in the Brown family of matching 1-million-wordcorpora of written StE

Genre group Category Content of category No. of texts

Press (88) A Reportage 44

B Editorial 27

C Review 17

General Prose (206) D Religion 17

E Skills, trades and hobbies 36

F Popular lore 48

G Belles lettres, biographies, essays 75

H Miscellaneous 30

Learned (80) J Science 80

Fiction (126) K General fiction 29

L Mystery and detective fiction 24

M Science fiction 6

N Adventure and Western 29

P Romance and love story 29

R Humor 9

TOTAL 500

References

Allen, C. L. 2003. Deflexion and the development of the genitive in English. English Language

and Linguistics 7, 1–28.

Altenberg, B. 1982. The genitive v. the of-construction: A study of syntactic variation in 17th

century English. Malmo: CWK Gleerup.

Barber, C. 1964. Linguistic change in present-day English. London and Edinburgh: Oliver and

Boyd.

Behaghel, O. 1909/10. Beziehungen zwischen Umfang und Reihenfolge von Satzgliedern.

Indogermanische Forschungen 25.

Biber, D. 1988. Variation across speech and writing. Cambridge: Cambridge University Press.

Biber, D. 2003. Compressed noun-phrase structure in newspaper discourse: the competing

demands of popularization vs. economy. In J. Aitchison & D. M. Lewis (eds.), New media

language, 169–81. London and New York: Longman.

Biber, D. & E. Finegan. 1989. Drift and the evolution of English style: a history of three

genres. Language 65, 487–517.

Biber, D. & E. Finegan. 2001. Diachronic relations among speech-based and written registers

in English. In S. Conrad & D. Biber (eds.), Variation in English: Multidimensional studies,

66–83. London: Longman.

Biber, D., S. Johansson, G. Leech, S. Conrad & E. Finegan. 1999. Longman grammar of

spoken and written English. Harlow: Longman.

Bock, K. 1986. Syntactic persistence in language production. Cognitive Psychology 18,

355–87.

Bresnan, J., A. Cueni, T. Nikitina & H. Baayen. Forthcoming. Predicting the dative alternation.

In G. Boume, I. Kraemer & J. Zwarts (eds.), Cognitive foundations of interpretation.

Amsterdam: Royal Netherlands Academy of Science.

Introduction Precepts Method Model 1 Model 2 Conclusion

Relativizer rates in the Brown corpora

subject + object contexts (N = 16, 868)

Introduction Precepts Method Model 1 Model 2 Conclusion

Relativizer rates in the Brown corpora

subject + object contexts (N = 16, 868)

Introduction Precepts Method Model 1 Model 2 Conclusion

Relativizer rates in the Brown corpora

subject + object contexts (N = 16, 868)

Introduction Precepts Method Model 1 Model 2 Conclusion

Prescriptivism

• the extent to which prescriptivism influences actuallanguage use is a matter of current debate(see e.g. Woolard and Schieffelin 1994; Cameron 1995; Auer 2006;

Busse and Schroder 2006; Poplack and Dion 2009; Anderwald 2014;

Poplack et al. 2015 . . . )

• the decline of which seems in tune with recentrecommendations in style guides

Strunk and White (1999: 59),The Elements of Style

Careful writers [. . . ] gowhich-hunting, remove thedefining whiches, and by sodoing improve their work

Introduction Precepts Method Model 1 Model 2 Conclusion

A competing account: colloquialization

[. . . ] increasingly so since the 1960s and 1970s, an egalitarianand informal communicative culture has been promoted in thepublic domain which has brought the norms of writingcloser to the norms of spoken usage. In grammatical terms,this has favored the rapid disappearance of archaisms [. . . ], andled to a decrease in the popularity of typical markers of formaland written style such as the passive voice. On the other hand,it has facilitated the spread of informal grammaticaloptions such as contractions [. . . ]

(Mair 2006: 88; emphases mine)

bookish which, colloquial that

that is theinformal & vernacular

variant(e.g. Tagliamonte et al. 2005,

Biber et al. 1999: 610)

Introduction Precepts Method Model 1 Model 2 Conclusion

Research question

• Is this a prescriptivism-driven change?

• or is it a change fueled bycolloquialization?

Introduction Precepts Method Model 1 Model 2 Conclusion

A history of prescriptivist thought on

relativizer usage

Introduction Precepts Method Model 1 Model 2 Conclusion

The historical facts

• since Middle English, competition between that and which(see, e.g., Fischer 1992: 296)

(3) Let us now go even unto Bethlehem, and see thisthing which is come to pass.(King James Bible, Luke 2:15)

(4) And God created great whales, and every livingcreature that moveth(King James Bible, Genesis 1:21)

• that-shift ahistorical

Introduction Precepts Method Model 1 Model 2 Conclusion

That versus which in restrictive relative clauses

precept recommending that has been part of 20th-centuryprescriptive literature since the publication of the first editionof Fowler’s Modern English Usage

if writers would agree to regard that as the defining relativepronoun, & which as the non-defining, there would be muchgain both in lucidity & in ease.(Fowler and Crystal 2009: 635)

Introduction Precepts Method Model 1 Model 2 Conclusion

Strunk and White (1999), The Elements of Style

• written by Strunk in 1918and first published by hisstudent E.B. White in1959

• college students in NorthAmerica are stronglyencouraged to purchase acopy

• sales figures are very, veryhigh.

51yQb9-P84L._SL500_AA300_.jpg (JPEG Image, 300 × 300 pixels)file:///C:/Dokumente und Einstellungen/Ben/Desktop/51yQb9-P84...

1 of 1 4/24/2012 3:10 PM

Introduction Precepts Method Model 1 Model 2 Conclusion

Strunk and White (1999: 59) on which-hunting

The use of which for that is common in written and spokenlanguage (“Let us now go even unto Bethlehem, and see thisthing which is come to pass.”). Occasionally, which seemspreferable to that, as in the sentence from the Bible. But itwould be a convenience to all if these two pronouns were usedwith precision. Careful writers, watchful for smallconveniences, go which-hunting, remove the definingwhiches, and by so doing improve their work.

(emphasis mine)

Introduction Precepts Method Model 1 Model 2 Conclusion

A critical appraisal of S&W

Geoffrey K. Pullum (2009), “50 Years of Stupid Grammar Advice”,

The Chronicle of Higher Education, April 17, volume 55, issue 32,

sec. The Chronicle Review.

• “advice ranges from limp platitudes toinconsistent nonsense”

• “[the book’s] enormous influence hasnot improved American students’ graspof English grammar; it has significantlydegraded it”

• “both authors were grammaticalincompetents”

Introduction Precepts Method Model 1 Model 2 Conclusion

Other style guides

1. Oxford Journals Copy-editing Style Guide(http://www.oxfordjournals.org/our journals/french/

for authors/style guide.pdf): “Relative clauses: use ‘that’ inrestrictive clauses and ‘which’ in non-restrictive clauses”

2. The University of Edinburgh’s Editorial Style Guide for Print(http://www.ed.ac.uk/polopoly fs/1.16486!/fileManager/

Style-Guide.pdf): “As a general rule, in restrictive relativeclauses[. . . ] ‘that’ (preferable) or ‘which’ can be used”

3. A handbook for authors and translators in the EuropeanCommission(http://ec.europa.eu/translation/english/guidelines/documents/styleguide english dgt en.pdf): “the use of‘which’ in defining relative clauses is often considered to bestilted and overly formal. ‘That’ reads more naturally. It alsohelps make the meaning clearer”

Introduction Precepts Method Model 1 Model 2 Conclusion

Microsoft Word

Introduction Precepts Method Model 1 Model 2 Conclusion

Method

Introduction Precepts Method Model 1 Model 2 Conclusion

The problem

A causal connection between linguistic precepts and observedusage (frequencies) is easy to assume but hard to prove.

Introduction Precepts Method Model 1 Model 2 Conclusion

The method

1. identify variable relativizer tokens in the Brown corpora

2. create a richly annotated dataset – considering not onlylanguage-internal and external predictors but alsoprescriptivism-related predictors

3. use regression modeling to explore factors that conditionrelativizer choice

4. check extent to which choice of that correlates withuptake of other precepts

Introduction Precepts Method Model 1 Model 2 Conclusion

The variable context

• focus on restrictive subject and object RRCs withinanimate antecedents introduced by that, which, andzero:

(5) a. Tom saw the car that caught fireb. Tom saw the car which caught fire

(6) a. Tom saw the car that Mary had soldb. Tom saw the car which Mary had soldc. Tom saw the car Mary had sold

• N = 16, 868 RRCs

Introduction Precepts Method Model 1 Model 2 Conclusion

Extracting zero

• the UT Austin team utilized a supervised machinelearning algorithm to identify candidates for zerorelativization in the dataset (motto: optimize recall,eliminate false positives by hand)

• the algorithm employs a conditional random field (CRF)framework (Lafferty et al. 2001) and was trained on thePenn Treebank

• recall: 85%, precision (before manual screening): 60%

Introduction Precepts Method Model 1 Model 2 Conclusion

Annotation

in total n > 30 predictor variables:

1. internal predictors(e.g. relative clause length)

2. external & stylistic predictors(e.g. genre, real time)

3. prescriptivism-related predictors

Introduction Precepts Method Model 1 Model 2 Conclusion

Prescriptism-related predictors (text level)

• passives: fraction of passive constructions over active lexicalverbs in a given corpus text (log-converted)(The motion was tabled)

• stranding: proportion (in %) of stranded prepositions out ofall prepositions in a given corpus text(The house which I looked at)

• splitInf: frequency (per 10, 000 words) in a given corpus textof split infinitives(To boldly go where no one has gone before)

• shallWill: ratio between tokens of modal verbs will and shall(I will/shall discuss this issue in what will follow)

Hypothesis

if the that-shift is aprescriptivism-fueled change, writersand editors who go which-hunting

should also comply with othercanonical precepts

(no passives, no stranding, etc.)

Introduction Precepts Method Model 1 Model 2 Conclusion

“Use the active voice”

The active voice is usually more direct and vigorous than thepassive:

I shall always remember my first visit to Boston.

This is much better than

My first visit to Boston will always be remembered by me.

The latter sentence is less direct, less bold, and less concise.

[. . . ] The habitual use of the active voice, however, makes for

forcible writing.

(Strunk and White 1999: 18)

Introduction Precepts Method Model 1 Model 2 Conclusion

Preposition stranding is bad grammar

Only the writer whose ear is reliable is in a position to use bad

grammar deliberately [. . . ] Years ago, students were warned

not to end a sentence with a preposition; time, of course, has

softened that rigid decree. Not only is the preposition

acceptable at the end, sometimes it is more effective in that

spot than anywhere else. “A claw hammer, not an ax, was the

tool he murdered her with.” This is preferable to “A claw

hammer, not an ax, was the tool with which he murdered her.”

Why? Because it sounds more violent, more like murder. A

matter of ear.

(Strunk and White 1999: 77-78; emphasis mine)

Introduction Precepts Method Model 1 Model 2 Conclusion

Split infinitives and shall

• split infinitives: “the construction should be avoided”(Strunk and White 1999: 58)

• shall versus will : “the future tense requires shall for thefirst person, will for the second and third.”(Strunk and White 1999: 58)

Introduction Precepts Method Model 1 Model 2 Conclusion

Analysis

• mixed-effects logistic regression analysis using R librarylme4

random effects: file (intercept), category (intercept), corpus (slope)

• model 1: zero versus that/which in nonsubject RRCs – isthe choice of zero related at all to stylistic or prescriptivistconsiderations?correctly predicted: 78.7% (baseline: 64.1%), C = 0.87, κ = 7.7

• model 2: that versus which in subject and nonsubjectRRCs – to what extent does the choice between that andwhich correlate with the uptake of other precepts?correctly predicted: 83.4% (baseline: 55.8%), C = 0.92, κ = 17.6

Introduction Precepts Method Model 1 Model 2 Conclusion

Model 1: choice between zero and overt

relativizers

Introduction Precepts Method Model 1 Model 2 Conclusion

Variable

(7) some kind of trick Budd had thought up(Brown-N01)

versus

(8) some kind of trick that / which Budd had thought up

Introduction Precepts Method Model 1 Model 2 Conclusion

Dataset

all object RRCs (inanimate)

N = 5, 738

Introduction Precepts Method Model 1 Model 2 Conclusion

Significant predictors

• PrecedingRelativizer: preceding overt relativizers disfavorzero

• antecedentLength: long antecedents disfavor zero

• RClength: long RCs disfavor zero

• Variety: AmE favors zero

• TTR: lexical density favors zero

• SubordinatingConjunctions: frequent subordinationdisfavors zero

• PersonalPronouns: frequent personal pronouns favor zero

not significant:

chronological orprescriptivism-related

predictors

Introduction Precepts Method Model 1 Model 2 Conclusion

Model 1: summary

• no evidence whatsoever for any statistical correlation(negative, positive) between choice of zero and any of theprescriptivist variables

• variation also remarkably insensitive to external factors

• choice of zero instead mainly governed byprocessing-related factors(note that zero plays no role in the prescriptivistliterature)

Introduction Precepts Method Model 1 Model 2 Conclusion

Interpretation

• language users make a primary choice between an overtand a deleted relativizer

• this choice is constrained by language-internal factors

• once zero is ruled out, a selection between that andwhich is made.

Introduction Precepts Method Model 1 Model 2 Conclusion

Model 2: choice between THAT and WHICH

Introduction Precepts Method Model 1 Model 2 Conclusion

Variable

(9) It was a question that required no answer.(F-LOB-L16)

versus

(10) the answer to a question which had been in theirminds(F-LOB-N13)

Introduction Precepts Method Model 1 Model 2 Conclusion

Dataset

all subject and object RRCs (inanimate)using either that or which

(zero excluded)

N = 13, 192

Introduction Precepts Method Model 1 Model 2 Conclusion

Significant language-internal predictors

• precRel: preceding that favors that, preceding whichdisfavors that

• relativizerFunction: subject function favors that

• antecedentPOS: non-nominal antecedents favor that

• antecendentLength: long antecedents disfavor that

• RClength: long RCs disfavor that

Introduction Precepts Method Model 1 Model 2 Conclusion

Significant external and stylistic predictors

• time: 1990s favor that

• variety: AmE favors that

• genre: fiction favors that, general prose & learned disfavor

Introduction Precepts Method Model 1 Model 2 Conclusion

Significant prescriptivism-related predictors

• passives: as the number of passive constructions relativeto active constructions increase in a text, the choice ofthat becomes less likely4 consistent with prescriptivism account

• stranding: those text samples that have higherpercentages of prepositions in stranded position favor thatas restrictive relativizer8 not consistent with prescriptivism account

Introduction Precepts Method Model 1 Model 2 Conclusion

So . . . ?

neat alignment in terms of formality:

• more active voice (informal) ê more that (informal)

• more stranding (informal) ê more that (informal)

writers act according to a dominant logic of formality!

Introduction Precepts Method Model 1 Model 2 Conclusion

Significant interaction effects . . .

. . . improve the model but don’t change the story.

Introduction Precepts Method Model 1 Model 2 Conclusion

Model 2: summary & interpretation

• StE writers’ preference for that . . .

• increases over time, led by AmE• correlates with two of the prescriptivism-related factors• style (formal vs. informal) is key!

• which tends to be used in more complex contexts

Introduction Precepts Method Model 1 Model 2 Conclusion

Discussion & Conclusion

Introduction Precepts Method Model 1 Model 2 Conclusion

Summary

• relative that is on the rise(Leech et al. 2009)

• why? mere discussion of usage frequencies cannotconclusively link the that-shift to the stylistic advice thatprescriptive grammar givesê study of the conditioniong of that

• key result: prescriptivism alone cannot account for thethat-shift

Introduction Precepts Method Model 1 Model 2 Conclusion

Interpretation

• formality-informality continuum interferes with“adherence to prescriptivism” as a grammatical logic toexplain usage change

• can prescriptivism change language use?– Yes, but it needs the help of larger stylistic drifts suchas colloquialization

• the overall change in the StE RRC can be described ascase of institutionally backed colloquialization.

Introduction Precepts Method Model 1 Model 2 Conclusion

Future directions

• spoken English?

• perception?

• towards an aggregate measure of prescriptive correctness

• coherence of multiple linguistic variables(Guy 2013)

Introduction Precepts Method Model 1 Model 2 Conclusion

Thank you!

benszm@kuleuven.be

http://www.benszm.net/

Literatur

Appendix

Literatur

RRC tokens by relativizer form and syntactic

function

Literatur

Model selection procedure

• customary steps to construct minimal adequate models(see, e.g., Gorman and Johnson 2013)

• stepwise reduction of a maximal model by removingfactors that lack explanatory power (α = 0.01)

• minimal adequate model only contains indispensablepredictors

Literatur

That versus which: genre effects

Literatur

Interaction terms in model 2

• variety × genre: genre effects different in BrE and AmE

• time × variety: the that-shift is considerable morerobust in AmE than in BrE

• stranding × time: over the 30-year time span covered bythe corpora, writers adjust their practice towards lessdivergent handling of the stranding rule and the that-rule

Literatur

References I

Algeo, J. (1991). Sweet are the usages of diversity. Word 42(1), 1–17.

Anderwald, L. (2014). Measuring the success of prescriptivism: Quantitativegrammaticography, corpus linguistics and the progressive passive. EnglishLanguage and Linguistics 18(01), 1–21.

Auer, A. (2006). Precept and practice: The influence of prescriptivism on the EnglishSubjunctive. In C. Dalton-Puffer, D. Kastovsky, and H. Schendl (Eds.), Syntax,style and grammatical norms: English from 1500-2000, Number 39 in Studies inLanguage and Communication, pp. 35–53. Berne: Lang.

Ball, C. N. (1996). A diachronic study of relative markers in spoken and writtenEnglish. Language Variation and Change 8(02), 227–258.

Biber, D., S. Johansson, G. Lech, S. Conrad, and E. Finegan (1999). The LongmanGrammar of Spoken and Written English. London: Longman.

Bohmann, A. and P. Schultz (2011). Sacred that and wicked which: Prescriptivismand change in the use of English relativizers. In Proceedings of the NineteenthAnnual Symposium About Language and Society–Austin.

Busse, U. and A. Schroder (2006). From prescriptivism to descriptivism? 140 years ofEnglish usage guides: Some old and new controversies. In C. Houswitschka,G. Knappe, and A. Muller (Eds.), Anglistentag 2005 Bamberg. Proceedings, pp.457–473. Trier: WVT.

Cameron, D. (1995). Verbal hygiene: The politics of language. London: Routledge.

Literatur

References IICurzan, A. (2014). Fixing English: Prescriptivism and language change. Cambridge:

Cambridge University Press.

Fischer, O. (1992). Syntax. In R. Hogg and N. Blake (Eds.), The Cambridge historyof the English language. Volume II: 1066-1476, Volume 2, pp. 207–408.

Fowler, H. W. and D. Crystal (2009). A dictionary of modern English usage. Oxford;New York: Oxford University Press.

Gorman, K. and D. E. Johnson (2013). Quantitative analysis. In R. Bayley,R. Cameron, and C. Lucas (Eds.), The Oxford handbook of sociolinguistics, pp.214–240. Oxford: Oxford University Press.

Guy, G. R. (2013, June). The cognitive coherence of sociolects: How do speakershandle multiple sociolinguistic variables? Journal of Pragmatics 52, 63–71.

Guy, G. R. and R. Bayley (1995). On the choice of relative pronouns in English.American Speech 70(2), 148–162.

Haussamen, B. (1997). Revising the rules: Traditional grammar and modernlinguistics. Kendall/Hunt.

Hinrichs, L., N. Smith, and B. Waibel (2010). Manual of information for thepart-of-speech-tagged, post-edited ”Brown”corpora. ICAME Journal 34, 189–231.

Hundt, M. and C. Mair (1999). ’Agile’ and ’uptight’ genres: The corpus-basedapproach to language change in progress. International Journal of CorpusLinguistics 4, 221–242.

Literatur

References IIIJaeger, T. F. (2006). Redundancy and Syntactic Reduction in Spontaneous Speech.

Ph. D. thesis, Stanford University.

Labov, W. (1972). Sociolinguistic patterns. Philadelphia: University of Philadelphiapress.

Lafferty, J., A. McCallum, and F. Pereira (2001). Conditional random fields:Probabilistic models for segmenting and labeling sequence data. In Proceedings ofICML-01, pp. 282–289.

Leech, G., M. Hundt, C. Mair, and N. Smith (2009). Change in contemporary English:A grammatical study. Cambridge: CUP.

Mair, C. (2006). Twentieth-century English: History, variation, and standardization.Cambridge: CUP.

Poplack, S. and N. Dion (2009). Prescription vs. praxis: The evolution of futuretemporal reference in French. Language 85(3), 557–587.

Poplack, S., L.-G. Jarmasz, N. Dion, and N. Rosen (2015, January). Searching forstandard French: The construction and mining of the Recueil historique desgrammaires du francais. Journal of Historical Sociolinguistics 1(1).

Pullum, G. K. (2009, April). 50 Years of Stupid Grammar Advice. The Chronicle ofHigher Education, B15.

Rohdenburg, G. (1996). Cognitive complexity and increased grammatical explicitnessin English. Cognitive Linguistics 7, 149–182.

Literatur

References IV

Strunk, W. and E. B. White (1999, September). The Elements of Style (4th ed.).Longman.

Tagliamonte, S., J. Smith, and H. Lawrence (2005). No taming the vernacular!Insights from the relatives in northern Britain. Language Variation andChange 17(1), 75–112.

Woolard, K. A. and B. B. Schieffelin (1994, October). Language Ideology. AnnualReview of Anthropology 23(1), 55–82.