ETFrcTS OF A TU.E\LY M,ILL GMrcNET{T IN IIATIffIA'TICS NSMI.trTIC.I

By Robert F. Orgel

ABSTRACT

This study, conducted in an introductory-IeveI nathcourse at the University of Kansasr cctrpares the per-fornance of students using the Fluency II LearningSystem with traditionbl course Presentation and studynrethods. To simulate ttreal worldrr conditions the Fluency IILearning Syst€rn was inplerrnnted under less than optirnalconditions. Even so, students using Fluency II scoredan ayerage of 29 points higher than the control groupson three separate examiations. Not only nras the vari-ability significantly reduced, but the size of the dif-ference increased on each successive exanination. Thegroup exposed to the Fluency II Learning Systenr perfornedtrnre than two tinps better on Beneralization probes con-sisting of untrained word problenrs. Differences of thismatnitlde appear nowtrere eise in the exPerinnntal, edu-cat ional or t raining I i terature.

Theoretical Background. Earlier studies by the investigator suSSestthat terminal perforrnances in wtrat are Senerally regarded as rrcogni-tiverrtasks might be signif icantly irnproved by strengthening eachntaskil to both an accuracy criterion (conventionally nrasured by per-cent correqt) and a f luency criterion (neasured by nrrr$er correctper minute). In a previous study, the investigator denpnstrated thatstudents trained to perform at high terminal frequencies retained 3 to 5

tines npre than lour frequency students, and two tinns rpre than middlefrequqncy students on a task where they were equally accurate, and thatthese iltfferences 1r€re durable and s igni f icant. For an introductory-level ntsth course, l\dath 000, an atternPt was nnde to incorporate thegeneralization technology suggested by Stokes and Baer and the frann andiask analysis suttested by lvlarkle. Inportant concepts were trained forreflexivity, synnetry, and transitivity as suggested by Sidnan, and trainedboth serially and concurrently as suggested by Schroeder and Baer, andLindsley, et. al.

lbthod. An introductory-level Mathernatics course at the university ofKansas was used as a test site for this procedure for generating andpresent ing curr lculun. The Fal t l98l sect ion becrrr a control condit ionior all subsequent nEasurernents and procedures. All ccrponents of thecourse, the exsninations, the instructor, the book, etc. renrained constantbetween the Fall and Spring serEsters with the exception of the Fluency IIdrill corrponent, u,hich was added to the Spring senester Presentation ofthe course. Thb follo,ing analysis will display an inportant dif ferencebetween the tuio senesteri that can confidently be attributed to the addedFluency II drill comPonent. [hile a confoundnray exist due to Potentialdifferinces between lhe students for the Spring and Fall sanesters, previous

analysis by others sugtest that Spring senester students tend' on the aYeraget

io rior. tiss urell iffi the Fall ienpiter students in renedial neth courses'i; ;iliii"n,-th;-irorp ot students vtro independently.elected not to use theaiirii piorided ai uiit" dinnnsion of carpai'ison, ahd thus helo account forthis potential sou.ci of vari"iion.-(r'bt;I-Tha ;u"ter of students (N) in thei;;;r;;-oitiu...d belour decreased-in atl gIoIP:.?cros: tests due to dropoutsanl students choosing not to use Fluency II Drill cards')

f,,ere so large that they posed no threat to the believability oIt could still be true, however, that other variables, eitherSpring 1982 class was in sonr way suPeselection threaten the validity of the

r ior , or that an ar t i facstudy. These threats to external valid-

he s tudy.t thef self-

n cases in wtrich randcrn selection is not possible. Appliedwtich deals vith hrrrsn subjects in quasi-experinental de-

his question with m.rltiple-baseline designs in wtrich as nnasured but no intervention is applied until control hasor reversal designs, in which an experinental treatrent whichful, is rernoved.

ftthato

ity always exist iBehavior Analys i s,s i tos r addre sses ttrntched behaY ior ibeen establ i shed,has proved success

Iitt

The currieulun of the introductory level l\[athcnatici class lent itself uni-quely to the use of a reversal design to denpnstrate that the underlyingpopulations for both Spring and Fall groups were the sarrc. For a reversal toLe- appropriate, the behavi6r should n6t UL able to naintain itself, andshouii nbt be a crrnrlative task *trere forner practice might hinder inter-pretatlon. Chapter 10, hmrever, consisted of Gecrrtry, and part of Chapter,6tonsisted of a subset of Analytic Gecrntry. Both of these curricula were dis-cont inuous :wi it -

it i-rtgiuri-piiuont*J in ct "piiiu- i-i

'

-.na-rruia-it ur provlde ' .'a unique opportunity to perform a reversal. ":'

t rni''decided that no Fluency II Dritl cards rrculd berprovided for theset:nrs; and that the performance of the Fall and Spring grouPs on a controlledask could bc observed. lf these in fsct rePre3ented the sixrE populations,hen test scores ,for these itg1ns ahould be ielativety similar.

The last histogran in Figure 5 shmrs the results for this Study. The Falllgt? (A - contiol) group-had a score of 69$ on these itenrs wtrile the SpringlgEz (D - Experinpnial) group scored 68%. The trouP of problrns frqn whichthese data cirre fran repieseirt 70.5 out of l2l-possible points (5t.25, wtrilethe Algebra carponent oi the Final for wlrich Fluglcy II drill cards vere pro-vided iepresentld 50.5 out.of 121 pointsr or 41 .7_X. Thlt, this carparisonwas rade on a substantial enough subsct of Test 3 to reiect the'hypothesisof different underlying populations with confidence. In additionr the factthat the two trouPs of-Fluincy I.I drills distributed to the class coveredonly *.2% of tf,e niterial covered on the exsnination nakes the differencesalriady noted betveen the cxperinental and control conditions for this testthat m,rch npre inpressive.

dec is iswhicthe

;ml nu

ded to enphasize training of the first and second skills, with cnPha-placed on the second. A wide variety of probl€m tyPes ucre presentedh the student would have to recoBnize alnpst instantaneously to npetcriterion of one correct ansuer -very 3.25 seconds (35 correct in turotes ) .

Results for this portion of the experinent are consistent and convincing.In every case, for all six problenrsr the Fall and Spring groups of non-users (A and B) unre significantly loc,er than the Spring 1982 Group Cwtrich used Fluency II Drill cards and dermnstrated criterion-level nasteryin both fluency and accuracy. Taken as a trouPr the Fall l9El Group A, rfiowere Don-usGrsr average 41.48 for these six problerns. Thc Spring 1982 non-users, Group B, scored 47.t% as a group. And the Spring 1982 groupr rfiichused and nastered the Fluency II drill scored an average of 8r.8r%r betterthan tvice the score of the control troup. Again there is a nearly ccrpletefunctional nntching betrrcen Groups A and B, with Group B iust slithtly

that un arc dealing irith a porcrful eiperinental intlrvention that signifi-cantly inproves thE ability of student to suicessfully do *prd problds innathenatics it extrcncly clear and convincing.

Ccncralizetior Probe - Tect ,, Finat Eranrination (Figurc 5). The-exPerinnntinitiated on Test 2 was continued on the Final Exanination (Test 3). fui attenPtwas nade to cxtend the reliability of the findlngs by replicating with fouradditional similar s,ord problerns, problenrs 37 through 40. Tlte rrran perfornnncefor the Fall lgtl Controi Group (Group A) was 4r.75. Scores for Group Br Spring1982, Functional &ntrol Group for the ssrn problenrs rras 49.25 %. Scores forGroup C, the Spring 1982 Expeiinpntal Group, ruhich darpnstrated nastery of the,Fluency II drill presented at a rate of 17.5 per minute averaged 83.fi.

Results for the generalization probe on Test 3 validates the generalizationprobe for Test 2. In all four cises the ordering effect seen on Test 2 is seenbn Test 3. lrlonusers (no exposure) scored the lorcst; users with a limited ex-

Ttrose denpnstrating fluent nastery utre aPProximately tvice as accurl!e onstandard examinations as either of the other trro Sroups. The probability ofseeing both this regularity in the order and the nagnitude of the results oc-curritg by change if less itran .0001. In only one oI the ten probllTt- (Prob-tenr 46-Telt 2) diA the ayerage score for'the Experinental Group fall beneatht0% correct. fhis problcnr, houreverr was aPParently the npst diff icult in theeet, with icores lir Groui ,t ot ZtX, and ltoreq lor GrouP B of z{:^Pntrastedwith thgle scores, Group C with an averagc of 56sr rcPretentr a 167S irprove-npntr a factor of x 2.67, ccrpared with Groups A and B, the largcst inprovarrntindex recorded for the l0 problern tet. Thc rran inprovcncnt factor for the gen-cralizatlon prohes taken tlgether ras 109$, or x Z.Og, with a standard-devia-tion of .40, indicating not-only a highty cucccssful, but an Yery stable'reliable, and lacrful eiperinental result.

cant I oved the pe rforrrBnce of the students wtrobe po out that the expe r imen ta Ilwest bsundary of expected ef f ect i venesof 35 cor rec t responres pcr ttro rni nutes

DISCtr.IISI(I{. There can be no doubt that Flucncy II

te

: the

Iluent nastery. Other studies urc have done denpnstrate definitivelythat a criteria of 40-60 correct resPonses per minute produced m.rchrDre durable learning as rEasured by retention and generalization.Another unfavorable Experinrntal condition was that the contingencies forrnstery of the Fluency-II DrilI cards represented only 15%of the grade,rtrich was too urcak a contingency to adopt and perfect a totally new rePer-toire of study behavior. Is- it possible' then, that the hi6her elilt scorescan be accounted for by self-selection? C-ould it be rnrely that better stu-dents were nrade better? Or that by adding an additional task, another wayto predict the rmre rtrptivatedrr students was found?

The experirrpnt was designed to control for these possibilities in a nrlrDerol wayi. First, there ia the strong relationship betuee! -the Fall Group(A) aira the Spiing l.lonusers (B) npntioned earlilr (see Table l). Table Isurrnarizes th; data fron this study. Groups A and B are rerrarkably similaron all these cxaninations in shape, disperslon, and central tendency, Bod.sre uniformly different frqn Groirp C (the exPerinnntal condition). Then the

:'

distributioni are notrlralized for i ize by c(mParing percentates instead.ofraw frequencies (Figures l, 2, and 3 - lF and lG, 2E and ZFr 3E and 3F)r

To get the Functional Control Groups (lB, 2Br 38) to natch the True Contr.olGroips (11, ?Ar 3A) and Etill have-a significant effect of thc kind notedfor ihe Exfcrinrntal Group ule ctould have had to pre-select a tlnglion{control grbup of students-r*ro xcre one grade poin.t higher initially. Thenvith the-IwLr end of the distribution ienrved, the rensinder would aPpear :' :

m.rch like the control SrouPa This is an unlikely explanat-ion because of theclose natching for shafe, is telt as central tendency and dispersion on allthree dl str ibf t ions. The-probabi I i ty of tett ing repeatable s imi lar i t ies ofthe type noted is highty irnlikely ii und6rlyin[ populations are different'In adilition, the "go6d itudentn alternative hypothesis is. refuted becausethe sane reiative [ercentsSe of Ats and B's'rviie obtained in the. SpIingtlonusers. There ari tuo ot6er areas of ccrparison, hovever, urhich allow us torcfute this threat to interpretation with inpunity.

The generalization probes Provide another view of the relative perfornsnce

not;d atrosi ait ten probes for the A and B trouPs-significaltll increasesour confidence that tirey are frcn the sarm populatioils. The fact that thesedata shoc, the BGroup hla slightty higher scores than the AGroup-rePresentsno threat to the vatiOity of itre itua! becausc of the snBll relative-rnagnituderytren icrparea wi th thc Exper inpnta I Piocedure (9) . Tlut , i f -ths1c. 9r is ted

. an

a priori'dif ference in tht under lying PoPulat ions it is of slight irrPortancc;trffiiiia with the armunt of ifrrpr6vlmint in perfornnnce accountcd for bythe Flucircy II Drill carponent.

There is another hypothesis uhichr in Iight of the sunrntive resultst isrxlre tenable - ttrai'there ms sare Fluency II Drilt usate by B trouP, --and that tha inprovenrnt noted is a function of armunt of drilt' In addition'the npre c*posui'u to the sheiis would tend to serYe the ssrr 'function as dis-i:iiU-uiion-si a-noti-tfii"t *ricrr hiShlighted the inportant itcrn to be studiedfor a rlyen tcction. Thit ".uiJ t"ia tE upgrade thi ltudy tiT" o{.thorc croused oidinary tcchniques, end sould account adcqualely for,ghc ltigh:-ElProve-ild; ;il;;e;e. naeiiinn.ity, thc fast that the rirrrter or;Ffp3p-eu,II:.Drill uscrr''&;;.;;S.';;;h i;;-;;;;"ii'6r:iitr"'ni"ns thatl lor Ttst 2, th;re tGre a'mnfier

of students who had previously used the drills for Test l, but did not forTest 2. The ssrr is true to an even greater extent for Test 3, rtrere only l4students denpnstrated fluent rnastery on the turo chapter drill sets ccrparedwith 39 for the'first test, and 25 for the second. If, as previous studiesindicater the inprovernnt provided by the usage of this procedure is durable,ule would expect it to show up in terrns of a slight inprovenrnt on the generali-zation tasks for these students.

A final carparison nakes this last hypothesis the rmre plausible of the two.The ccrparison for both serrnsters on the Crccrrtry section of the Final Ex-amination suggests that if anything, the Spring group had a slightly less na-ture nathenatical ability. lrlot only did the Spring troup score less, but 5 tol0 of their ntrrber nade their ov,rn cards and drilled thernselves. Thus, a truecontrolled ccrparison would probably have yielded a dilference of frqn 4-8%Iess for the Spring group, enough to yield a statistically significant dif-ference confirming that the Spring group x,Bs slightly inferior to begin with.

The evidence clearly suggests, even without this carparison, that the underlyingpopulations urcre not different in any inport4nt xray prior to the introductionof the experinpntal procedures usedr ard that the differences noted are certainly

, a function of those procedures. The data and analysis presented provide incon-troyertible support for the conclusions reached.

conrcLuslor\{s

l. Even given a srcakened version of Fluency II Drill procedure, a nEaninprovernent of 27,t, or alrnst 3 letter grades ras noted in thisstudy.

2. In all cases the variability of scores'in the experinnntal troupuras significantly lo*rrr than the control trouPs for the areasFluency II was used, shmring a high degree of experircntal control.

3. On thc npst diff icult troup of problerrs for this population, thcexperinrental procedure reliably inproved perfornance by a factor ofrmre than tx,o.

+. Vithout the benefit of the Fluency II Drill procedure, the perfornanceol the Spring group rmuld have been identical to the control grouP.All differcnces noted in the study are ctearly a function of theprocedures u$ed.

L The FluencyproYides s i

IHPLICATIONS

II Learning Systern oPerating at only partial efficiencygnif icant and inportant inprovenrnts in learning tech-actual inf ornat ion.

?.

3 a

4

The high positive correlation between increases in fluency and in-creases in generalization suggests that'optinnl Ievels of fluencyincrease generalization by 3 to 5 tinns that of traditional train-ing technology.

The size and reliability of gains recorded with use of the Fluency IILearning Systen suggest that it currently rePresents the rDst advancedtraining technology available for developing a broad range of skillsincluding conplex concept fornation and problern solving skiIls. Itwill provide irrpressive results for any training progratn to dtich itis appl ied.

,*

GRCLffi $.Lf,IARIZED BY $hDITION AF{) SH,ESTER

TABLE I

FALL 19tI (A) SPR.II-.IGr\lcl\t-usER (cNrrcL] ts{-usER (

TEST .1 N=65

M = 77.64SD = 16.0

27

aQ t-e t *G

-?'- --- - - -

e

-- -+

* *

-

*F---

------- --c----

*-

--

eI--

- -t -aaif

*-

NM

SD"95

(B)trNlm-)

SPR.ITG L}E? (C}E(PERIMENTAL

=23= 93.34= 3.55

N=29iM= 77

SD=14

TEST 2N= 57M = 74,4

3D = 26.56

rI=M=

5D=

37772e

.85

.65

N=25M =100.38

5D = 10.3E

34671E

=E--* --E- ---I-- -----e-e-

g

-

G-----

-5 ----***t*-QaQC*a- p

- - -

*

- - -- - - - - --- - -

E

- -- - -- - - - -- ---- ---,-- - --* -

*

TEST 3N= t*5

M = 6E.5SD = IE .07

t' Received Fluency Drill Training for only 2 out

* N = t4.8 M = 96.5*,2A SD = g.EZ

--t--t----------------e*-E-----A--*- ----- ---------- ----

*----F--E--

NM

SD

of 3 Units Tested.

I

II

I

I

I

i'

F IGt"R.E {T L

TEST# T

(by percentegr)TEST II L

(by nrrr$er of students )

Fall l9E1: atl non-user6

i ,rtor? ol tr r t 1

Spring 1982: all non-users

TA

IB

IC

1D

Fall t9tll all non-u$crg

{te$iG ofi tGrt}

Spring l9E2: all non-users

i itlr,' ,ro t? i: ;

ji,:rtre *t' :e"I;

{

lc

II

4,..rF 5.t .rr. l:

$pring l9t2: I set of FluencY Drill Spring l9E2: I tet oI Fluency Drill

'. tcore Jn it5'" i

:iiqlart ffi ter:,

Spr i ng I 9E2: ent i re c lass

t;(rii ?r: :fa:;

Spr ing l9E2: 2 sets of Fluency Dr i I I

.: ::: :

Spring 1982; entire class IE

IF

Spr ing t9E ?: 2 tets of Fluency Drill

arf r atirrtrt{tnr{l

+r:irr e .. l!.bldrr ia I r rrrr- lh.1]

T3

TAST fttbf nrntcr of r tudentr )

TEST'2thr

l?tll rll non-urcrt

Pc rccntatc

tco?? on tcrt l

tt..;?re n:ert!

)

Fall Ittlr ell non-uicr3

$pr iq t ?ll: rlo{!-uiGf t

' Xpr ing ltltr {u I I Flu;ncy Tra in iag

Frl I 2E

l0

2S Sr'Iry ttlrt lmo*tlsart

,0

TOrl :) *: l'; 3i ?i '.','. i:: :::

nrlrtcr ol rtudcntrr' J7trera[c tcrrtcir0.Drtlndrrd d*r irt ion.r6. t5

*r*tr cf +tud*fit1,'25rr€l$*s rrsrtr"l08.lEr tril{r$r rt dcr l*l i sn * I l. 3E

cfi ittt

{ lcorr on rcrt i

'{rrar; on ttrt}

2A

IF

rcrc *pr i6 ttltr fut t ftucncY Trrining

l0f

2t} tprirq l?ltr Gntlrc rlrrr

t_

TEST tr(f ipal)(by nurtcr of itudcntr)

TESTTT(finet)(by pcrccntage)

Fal I llEl: al I non-uicrs }A Frl I l93l: el I non'uicrl

,0

,E

,r

,c( rcorr an lGtt I

Spr int lrtz: non-uiGrr

i, 2, ll :rl, !'j 6li rcorc on tcst i

tgotc or IGrt

Sprint t?t2r full flutncy Training

"lB fPr.lng lrt2s non'ulcrl

rtorr st! tGrt,

rc Ipring l?tXr full Flurncy Training

*rr&r:r of *tudrntr*l&rYf rag,a *t:ertrfl*. ]*tfrrdard o*r irt ion:,9, tl

m*rr al *1s4["611*llt"rrrtt trsr+rlf.llrtradrrd *+rirt tsn*10. I t

tn mlt l

3frIry ltltr Gntiru cl***

}

,G

forln3 t lt! t tr rr g{*rr *r

im*n

FIGI.RE #{: PERCENI (nrRECf Cr.l SIX \ICRD FROBLEIVIS (TEST#2)

Group A: Fall lgEl (no Fluency Drill)Group B: Spring l9E2 (no Fluency Drilt) '

Group C: Spring lgEZ (wittr Fluency Drill t 17.5 corrcctrtni

x 2,35x2 x 2.2

x ?,lt \

x 2.67

0(

6

00t

prob I enr #47 prob I enr #4E

roft

ffi

x 1.35

prob I enr # 19 prob I srr #?A

FICTfIE ll 5 t PEIICENT (f,IUfECT CN FIML(UffiD PRcsLEtrIS AIrJ ToTAL CEo\,IETRY}

Croup Ar Fatl tgEl (no Fluency Drill)Gioup B: Spring t9E2 (no Fluency Dritt)Group C.: Spr ing l9t2 (wittr Ftuency Dri t l: 17.5 cocrcctir

I .91 x I .657

7

,0

all gesrrstrY Probfrnn?rnl fnr {rrnttn di

emsfprpnrrtl

IIIIr tttIIIIrltlIITIIIIItrrrrIIIttlrI:llIrlrrIITrITI

prob lqn lt37 problem #3E problenr /139 problem ll40f

ro

ffiI a a a a a a a a a a a o- a a r Ilr a a t a a a a a a a a a a t a .Il a a l a a ra a a. ta aa a tll. t a a a a a a a a a a a a a . rII ra a t a a a taa atlaa allr a a a a a a a a D r a a a a a .Il. a a a a a a r o r a t a a.aI

-ffi

-,F

,. r.l

.*

*!.:i,.$,ffi

'q+"

t"\

\

,

:'

|l0