4..

"NPRDC TR 84-29 FEBRUARY 1984

COMPUTER-MANAGED INSTRUCTION: STABILITY OFCOGNITIVE COMPONENTS

APPROVED FOR PUBLIC RELEASE,DiSTRIBUTION UNLIMITED 1

ELECTE,

AP~R 0 9 984

NAVY PERSONNEL RESEARCHAND

DEVELOPMENT CENTERSan Diego, California 92152

84 04 05 035

NPRDC TR 84-29 February 1984

COMPUTER-MANAGED INSTRUCTION: STABILITY OFCOGNITIVE COMPONENTS

Pat-Ar thony Fecferico

Reviewed byE. G. Aiken

Approved by3. S. McMichael

- For

A Released by

J.W. RenardCaptain, U.S. Navy

Commanding Officer

Navy Personnel Research and Development Center

San Diego, California 92152

a ________________________________

UNCLASSIFIEDSECURITi CLASSIFICATION OF THIS PAGE Men.. Data Entereio

jBREPORT DOCUMENTATION PAGE . DCOMMZrMG nOM'1. REPORT NUMUER 2. GOVT ACCESSION NO. . RECIPIENT S CATALOG NUMN t

NPRDC TR 84-29 r -_-______,_

4. TITLE (.-d Subtitle) 5. TYPE oF REPORT & PE.IOO COVEREo

COMPUTER-MANAGED INSTRUCTION: STABILITY Interim ReportOF COGNITIVE COMPONENTS Oct 1982-Sep 1983

S. PERFORMING ORO. REPORT NUMSER

51-84-17. AUTmOR(a) ' . CONTRACT OR ORANT 'WUeMO,)

Pat-Anthony Federico

9. PERFORMING ORGANIZATION NAME AND ADDRES o,.RMRLB&T PR+ RT, ANavy Personnel Research and Development Center 63720NSan Diego, California 92152 RF63-522-801-013-03.04

11 CONTROLLING OFIFICE NAME AND ADDRES$S 12I. REPORT DATE i

Navy Personnel Research and Development C'.nter February 1984

San Diego, California 92152 21. uM§FROF PAGE

14 MONITORING AGENCY NANE a AOORESSQIf different itoi Centroilin Office) 1S. SECURITY CLASS. (of Iboe '"at)

UNCLASSIFIED

Ila. OECM. AW0ICATIoN/OOWN ORADINGSCNE OULE

I1. DISTRI9UTION STATEMENT (of this Report)

Approved for public release; distribution unlimited.

17. DISTRIOUTION STATEMENT (of Ihe obetact entered In Block 20, II difIfrent lrem Rot)

IS. SUPPLEMENTARY NOTES

IS KEY WORDS (Continue on reverse slde If nec.aary aind Identify by block number)

Computer-managed instruction Cognitive componentsMastery learning Aptitude-treatment-interaction

Individual differences Crystallized and fluid intelligence

20. ABSTRACT (Continue on revere slde It neceosaer a" Identllf by block miber)

To ascertain changes in cognitive correlates of learning as students advance throughhierarchical instruction, 24 individual difference measures were obtained from 166Navy trainees who had completed a computer-managed course in electricity andelectronics. Principal component analysis and varimax rotation were computed forcognitive characteristics, producing factor scores that were used in multiple regression

D SON 1473 EDTION Of I NOV IS 0O5tLED ETN O I LUNCLASSIFIEDS/N 0102- LF- 014. 6601 S41OP60SECURITY CAIS.AtlICATIOMN OF THISl PAGE (11ban W

UNCLASSIFIED

sWeumNv CLAW IICAIkg OF THIS PAGE (31 DOS. NO.,

analyses to predict achievement in I modules of instruction. During acquisition ofcourse content, cognitive components sampled shifted noticeably in importancethroughout the curriculum. The results have implications for aptitude-treatment-interaction (ATI) research, transition from novice to expert, crystallized and fluidintelligence, task demands of instruction, and computer-managed mastery learning.

,:I'.'

RI

LS/N O 102-LF- 014- 6601UNCLASSTPTRD

____ ~SECURITY CLASSIPSCATION Of THIS PAGE(Whoeu Data Entered)

FOREWORD

This research was performed under exploratory development work unit RF63-522-801-013-03.04 (Testing Strateg. !s for Operational Computer-based Training) under thesponsorship of the Chief of Naval Material (Office of Naval Technology). The generalgoal of this work unit is to evaluate the impact of different computer-based testingstrategies for operational training.

The results of this study are primarily intended for the Department of Defensetraining and testing research and development community.

3. W. RENARD JAMES W. TWEEDDALECaptain, U.S. Navy Technical DirectcrCommanding Officer

I..o

L%W,o

SUMMARY

Problem

Very little research has examined the nature of the relationship of aptitudes toachievement asstudents progress through computer-managed instruction (CMI). Data arerequired to help establish whether aptitude-aciievement relationships are stable enougnto warrant consideration by instructional researchers and developers as well as trainingmpnagers.

Objective

The objective of this research was to determine the nature and extent of changes inthe cognitive correlates of learning as students advance through hierarchical modules ofCMI.

Approach

Twenty-four measures of cognitive attributes were obtained from 166 Navy traineesas they completed a computer-managed mastery Lourse in electricity and electronics.Principal ,omponent analysis and varimax rotation were computed for student individualdifference measures, producing factor sco.'es that were used in .riultiple regressionanalyses to predict achievement in 11 moM- - -s of instruction.

Results

Within limits, student proficiency throughout the modules could be predicted by usingmeasures of these cognitive components. Changes in the proportion of variance in studentperformance throughout the modules accounted for by certain cognitive componentsrepresented shif s in their emphasis during the process of acquiring the course content.These shifts in predictor patterns of cogni+ve components appeared to be related towhether a module required students to rememLýr or use facts, concepts, princiyles, and/orrules. Different cognitive components seemed to contribute more or less to studentachievement at distinct modules or stages of learning.

Discussion

Considerable changes occurred in the cognitive predictors of achievement as studentsprogressed through the sequential modules of instruction. During the acquisition of coursecontent, the importance of the cognitive components sampled shifted noticeably through-out the curriculum. Different components appeared to contribute variance at earlier andat later phases of mastery. After progressing through hierarachical modules, individualdifferences in learning depend more on certain cognitive components and less on othersthan they did when beginning to acquire the course contents. The use of specificcomponents is minimal in early stages of training, but prerequisite for later acquisition.Yet, other cognitive components may remain rather unchanging during the mastery of thecomplete curriculum.

Conclusions and Recommendations

It was highly likely that the cognitive processing involved In the initial phases oflearning differed from the processing at terminal phases of acquisition. This suggestedthe requircment for protocol analyses of the cognitive processing involved in early,

vii

intermnediary, and late±r phases oi learning. It must be determined whether particularaptitudes that contribute to learning at distinct stages reflect the presence of pre-requisite copnitive cornpeTencies, schemata, knowleage, and learning sets required toacquire the subject matter at particular phases of mastery. The insta.bility of therelationships of some cognitive components across distin.t stages of learning suggestedthe impcrtance of concentrating mn the process of change from ignorance to competencz.

Viii

CONTENTS

"Page

INTRODUCTION........................................................1

ProbiemDUC ................ .............................................. 1" O bjective . ....... ...... ...... ....................................... 1

A.PPROb ACH ..................... ....................................... I

Subjects ................................................................ I"% Cognitive Characteristic Measures ....................................... 2

Instructional Treatment ................................................... 2"Computer-managed Instruction ........................................ 2-Mastery Learning .................................... .............. 4M dule Booklets ................................................... 4Subject-matter Content ....................................... ......... 4Achievement Measures .. ".... .. ... .. .. .. . ... . .. . .

Statistical Analyses .............. .................................... 5

RESULTS ................................................................. 6

Descriptive Statistics ..................................................... 6Component Structure of Cognitive Characteristics ........................... 6

- Cogrative Characteristics and Module Achievement Relations .................. 7

DISCUS•1CN ........................................................... 16

"CONCLUSlUNS AND RECOMMENDATIONS ................................. 20

REFERENCES ............................................................. 21

DISTRIBUTION LIST ....................... ........................... 23

LIST OF TABLES

1. Cognitive Characateristic Measures ............................... 3

2. Means, Standard Deviations, and Intercorrelations for the 24Cognitive Characteristics and '1 Module Achievement Scores .............. 9

"3. Initial .lr-ncipal-component Solution, Communalities (h2 ),

Associated Eigenvalues, and Percent Variance for the CognitiveChara, teristics ......................... ............................ 12

4. Terminal Varimax Solution for the Cognitive Characteristics ............... 13

. Simple Correlation and Stanr1.,riized Partial Regression CoefficientsBetween Cognitive Characteristic Ccmponents and Module AchievementScores .............................................................. 14

ix

INTRODUCTION

Problem

Several studies have shown that aptitudes demanded by p -rceptual-motor tasksusually shift as students improve their performance (Ferguson, 1935; Fleishman, 1972;Fleishman & Bartlett, 1969). Other investigations have extended this result to morecognitive learning tasks (Bunderson, 1967; Dunham, Guilford, & Hoepfner, 1968;Frederiksen, 1969; Gagne & Paradise, 1961; Hultsch, Nesselroade, & Plemons, 1976;Labouvie, Frohring, Baltes, & Goulet, 1973; Roberts, King, & Kropp, 1969). If aptitudesrequired at one stage of mastery are not always similar to those needed at later stages,th.-n some established aptitude-learning relationships may be m'sleading for certaininstructional treatments. They would represent the importance ot specific aptitudes onlyat rhose specific times when achievement was assessed. This could make the results ofaptitud=e-treatment-interaction (ATI) studies (Cronbach & Snow, 1977) more difficult tointerpret and contribute to the conflicting findings of many of these investigations. Mostof the research upon which this speculation is based involved the practice of laboratorylearning tasks--not academic instruction.

Very little research has examined the nature of the relationship of aptitudes toachievement as students progress through scholastic instruction. The results of Burns'study (1980) suggested that certain aptitude-learning relationships are not stablethroughout instruction. These findings did not support an important assumption made 'inATI research--that the relationship of aptitudes to achievement is stable during thecourse of learning for a specific ;nstructional treatment. The curricular materials used inthe Burns' investigation were hierarchical learning units for an imaginary science, whichwere taught and tested within a l-day time frame. Neither the subject matter nor theduration of learning appeared to represent with sufficient fidelity real-world, school-based, instructional situations. Consequently, the results of this study are at bestsuggestive.

Additional data are required to 'ielp establish whether aptitude-achievement relation-ships are stable enough to warrant consideration by ATI researchers.

Objective

The objective of this rese,-ch was to determine the nature and extent of :hanges in

the cognitive correlates of learning as students advance through hierarchical modui!es ofcomputer- managed instruction (CM:).

APPROACH

Subjects

The subjects were 340 individuals who graduated from recruit training at the NavalTraining Center (NTC), San Diego and were scheduled for instruction at the BasicElectricity and Electronics (BE/E) School at NTC San Diego. Before beginning the BE/Eorientation, the subjects were administered 12 tests--6 designed to measure theircognitive styles; and 6, their abilities. Test data were discarded for 20 subjects who didnot follow directions and/or completed less than 9 of the 12 tests and 40 who did notgraduate (35 for academic and 5 for nonacademic reasons). Thus, test data were availablefor 280 BE/E graduates.

Aptitudes of all individuals entering the Navy are measured by theik scores on the 12subtests of the Armed Services Vocational Aptitude Battery (ASVAB). However, theASVAB scores for 108 subjects of this study were either incomplete or missing. For 6additional graduates, the module test sccre data usually maintained by the CMI system forall BE/E students were missing or incomplete. Thus, the final sample consisted of 166BE/E graduates.

Cognitive Characteristic Measures

The three types of cognitive characteristic measures used in the study were tests ofcognitive styles, abilities, and aptitudes. Cognitive styles are the dominant modes ofinformation processing that individuals typically employ ,.hen perceiving, lear1 ling, orproblem solving. Abilities are the intellectual capabilities of individuals that are generaland pervasive to the performance of many tasks. Aptitudes are indices used to selectpersonnel to perform tasks that demand specific skills and to find the right person for acertain job or school.

The six tests designed to measure cognitive styles were chosen because of theirimplications for adaptive instruction (Kogan, 1971); and the six tests design'-d to measureabilities, because they represent various types of information-processing tasks (Carroll,1974, 1976) and are relevant to the BE/E subject matter. The ASVAB subtests wereselected as measures of aptitudes because they are typically readily available for Navypersonnel and the basis for assigning personnel to different Navy schools. Also, the abilityand aptitude measures were included in the investigation to reflect crystallize2 (G ) and

fluid (Gf) intelligence (Cattell, 1971; Horn, 1976; Snow, 1980; Cronbach & Snow, 1977).

Tests that are allegedly indicative of traditional educational achievement (i.e., verbal,quantitative, and reasoning abilities) usually index C . Tests that are alleged.y indicative

of adaptation in novel learning situations (i.e., abstract, nonverbal, spatial, and figuralreasoning abilities) usually index Gf. All of the tests are (1) relatively independent, (2)

moderate to high in reliability, (3) paper and pencil in nature, and (4) fairly short induration.

Table I presents the 24 cognitive characteristic tests used in this study.

Instructional Treatment

The instructional treatment consisted of the first 11 modules of the BE/E schoolcurriculum, course file 69. This involved CMI to implement the mastery learning of thesubject matter of the modules.

Computer-managed Instruction

In CMI, students self-study and self-pace themselves through off-line lesson modules(i.e., they do not interact directly with the system while !earning). This differs fromcomputer-assisted instruction where students interact in real time with course contentsand tests stored in the computer via on-line terminals. Also in CMI, the computer via itsdistributed terminals (1) scores criterion-referenced multiple-choice tests students takeoff-line, (2) interprets test results and provides the students with feedback regarding theirperformance, (3) advises students to learn the next or alternative lesson or to remediatemastery modules, and (4) manages student records, instructional resources, andadministrative data (Baker, 1978: Orlansky & String, 1979).

2

0 c

V) - 0

f4 0

rd -C4)

C 0 0 < urC r 04 -0 -

Cc -> o >q,, x4)~i 4) <4) <)IJ

m ID - Cn 0 .0= L;Oý .

.4- U, btY -- -

4.0 u- C4 a c -t Z-

- 0 U 4~ 0 ;), <.- .-8 M , 0 C 0 1

0 - L 4 )mCuo

I) C 0 C

-a v - . .- - c

r-) o- 0- j >:' 4)Cf U I

(a-= W- )t 22 I

UEO -0..C

a) I' tic 0L - 0.u0 E'V v- (a 4)C cu - (D

to 0-- 0c CC0 E ut o C0cM - -E I- 0 0r

C 0 <U Z .0z - <V)<

C, 0 r -

C. .2 4)

0 ac4E C

C0 v X

0 c 0 W o 0- I

0 J Ow 0 >0 EziU~0 uJ.'I- c%.LJ

I- 0 c u

-. 74)u-0 l

Z- >. 4lC" C .

U,~ 0)~0 0,

4) 0. U C 00 to E 4) Eu c E 0 U

*n IC- 0. rU - 00

r- V'Ce W* r c) 4

ind) >4) V04 tic

U: 0L C 0( C w~~U,II E E

4) 4) C.L L u

0~ J cc~ z z C) ul 0 Uo0z . < <r,=

i- ý 0 i U z < Z L 0 L 3

Mastery Learning

Mastery learning has many major features (Block, 1974; Bloom, 1974, 1976):

1. Mastery is explained relative to the specific instructional objectives everystudent is required to achieve.

* .*. 2. The instruction itself is structured into clearly defined learning units or modules,

3. Every student must master each module completely before proceeding to thenext module.

4. A diagnostic objective-referenced test is administered to every student at theend of each module to provide feedback on the adequacy of the student's learning.

V 5. Based upon the diagnostic information, 3 student's original instruction is remedi-ated and/or supplemented so that he/she can successfully master the nodule.

6. Time to complete each module is used as the means of indivicualizing instructionand thus promoting mastery of the material.

.J.• Module Booklets

"The individualized learning materials used in this study were a set of 11 hierarchical

learning modules designed and developed to teach basic facts, concepts, principles, and. rules regarding electricity and e!ectronics. These modules were selected for this research

since students from all electronics-related Na,.y ratings must masz" them successfullybefore proceeding to more specialized trai ilig. Each module was prese.•ced as a self-study booklet consisting of three to seven less...•s.

To learn a lesson within a booklet, students could choose, based upon their experienceand preference, a narrative presenta-ion, programmed instruction, and/or straightforwardsummary. These alternative lesson training treatments could be complemented byenrichment material or the instructor if the students so desired. Students wereencouraged to use as many of the instructional resources as necessary to master themodule material. Many schematics, circuic diagrams, photographs of meters, andalgebraic expressions supplemented the descriptive prose in each of the booklets.Typically, the presentation of the many facts, concepts, principles, and rules was followedby appropriate examples.

Subject-matter Content

The subject-matter content of the ,I modules' le_.sons was as follows:

I. Electrical current--electricity and the electron, electron movement, currentflow, measurement of current, and the ammeter.

":.:-.: 2. Voltage-electromotive force from chemical action, magnetism, electromagneticinduction, AC voltage, uses of AC and DC, and measuring voltage.

3. Resistance-characteristics of resistance, r.-sistors, resistor values, and ohm-meters.

L '4

4. Current and voltaae in series circuits-- measuring current and voltage in a seriescircuit and using the multimeter as a voltmeter.

5. Relationships of current, voltage, 3nd resistance--voltage, resistance, and cur-rent Ohm's law formula, power, internal resistance, and troubleshooting series circuits.

6. Parallel circuits-.-rules for voltage and current, rules for reitneand power,

variational analysis, and troubleshooting parallel circuits.

7. Combination circuits and voltage dividers--solving complex circuits, voltagereference, and voltage dividers.

8. Induction--electromagnetism, inductors and flux density, inducing voltage, andinductance and induction.

9. Relationships of current, counter electromotive force, and voltage in in-ductance-resistance circuits--rise and decay of current and voltage, inductance-resistancetime constant, using the universal time constant chart, inductive reactance, relationshipsin inductive circuits, and phase relationships.

10. Transformers-transformer construction, transformer theory and operation, turnsand voltage ratios, power and current, transfocrmer efficiency, and semiconductorrectifiers.

11. Capacitance--the capacitor, theory of capacitance, total capacitance,resistance-capacitance time constant, capacitive reactance, phase and power relationhips,and capacity design considerations.

Achievement Measures

The achievement test scc.e for each of the.e sequential modules was the number ofitems correct on a student's first attempt at ta.ing a mastery quiz. These end-of-moduletests consisted of from 10 to 45 four-alternative multiple-choice items that werecongruent with instructional objectives. The number of contact hours each studentrequired to master the instructional material of each module wa., retrieved from the CMIsystc.r:. The means and standard deviations (given in parentheses; of the times, in hours,required by the students to complete the 11 modules were 5.56 (3.59), 6.93 (3.45), 6.34(2.77), 8.05 (4.68), 14.27 (7.72), 9.18 (4.89), 19.83 (9.60), 6.43 (3.41), 9.58 (4.51), 6.98(4.06), and 8.55 (4.08), respectively. The average total number of contact hours forstudents to complete this curriculum was 101.70, which represents 16.95 course days of 6hours of instruction each.

Statistical Analyses

A p-incipal components analysis with no iterations (Hotelling, 1933) was computed forthe 24 x 24 intercorrelation matrix of the cognitive characteristics to obtain a smallerand more manageable number of variables. The minimum eigenvalue criterion wasemployed to establish the number of significant principal components to be rotated forthe terminal solution (i.e., only components with eigenvalues greater than or equal to onewere retained. Kaiser's (1958) varimax procedure was used to rotate orthogonally theinitial component solution. Derived component scores were computed for subjects andused as predictors in performing I I multiple regression analyses. These determined theamount of variance that the cognitive components accounted for in the module achieve-ment scores; that is, the terminal rotated solution also yielded orthogonal dimensions

5

resulting in independent component scores. Consequently, it was feasible to ascertain

the relative contributions of the cognitive components to module achievement.

RESULTS

"Descriptive Statistics

The means, standard deviations, and intercorrelatiors for the 24 cognitive character-istics ana I I module achievement scores are presented in Table 2.1 Some of these

.% .' correlations are noteworthy. The diagonal correlations between successive module scoresdid not increase monotonically, which suggests that the performance in each module didnot correlate the strongest with the performance in the modules immediately preceding orfollowing it. As the sequential separation between modules increased (i.e., moving acrossthe rows of the matrix to the right or down tae columns), the correlations did notdecrease. This implies that the remoteness of a module from another module did not"necessarily lessen the relationship between them. Most of the cognitive styles measureswere not significantly correlated with ability and aptitude measures. The primaryexception was FILDINDP, which was related to GENLREAS, ASSOFLUN, MATHKNOL,ELECINFO, and MECHCOMP. Some of the strongest relationships existed amongmeasures of WORDKNOL, ELECINFO, MECHCOMP, GENSCIE, and SHOPINFO. Themany significant correlations between cognitive attributes and module scores were not as"strong as expected.

Component Structure of Cognitive Characteristics

Table 3 presents the initial principal-component solution for the cognitive character-istics and its accompanying communalities, associated eigenvalues, and percent varianceaccounted. Aptitudc and ability measures are the prime contributors to the initialprincipal component. The seven components accounted for 60.1 percent of the variance"of these measures.

Table 4 presents the terminal varimax solution for the cognitive attributes. Con-"sidering only those characteristics with loadings equal to or greater than .3 and discussingthe measures in order of the magn.tude of their weights, the derived components wereinterpreted as follows:

1. The first component was defined by MECHCOMP, SHOPINFO, AUTOINFO,ELECINFO, GENSCIE, WORDKNOL, SPACPERC, GENLINFO, ARTHREAS, AND VERB-

. - COMP. The ten tests loading this component were diverse in content and probablyindicative of undifferentiated general intelligence, G.

2. The tests contributing to the second component were NUMROPER, MATHKNOL,"ATTNDETL, ARTHREAS, TOLRAMBQ, and GENLREAS. All of these measures seemedrelevant to scholastic mathematical achievement. Consequently, this component waslabeled crystallized mathematical intelligence, G

c m

3. The third component was dominated by four tests of verbal educational achieve-ment; namely, ASSOFLUN, IDEAFLUN, VERBCOMP, and WORDKNOL. Therefore, this

W- component was called crystallized verbal intelligence, Gcv

'Because of the large numb of tables in this sectic., relative to the amount of text,

the tables (and figure) are placed dt the end of the section, commencing on page 9.

6

4. Four tests identified the fourth component: FILDINDP, INDiY-iCN, SPA,-

. PERC, and CATEWIDH. These primarily nonverbal reasoning tests )f .ýpaci i. ad figur Iprocessing were thought to represent fluid intelligence, Gf.

5. The fifth component was chiefly defined by four tests of conventi'uiai educ,ýtional achievement in reasoning: GENLREAS, LOGIREAS, TOLRAIIbQ, and VERBCOMP.It seemed suitable to label this component, crystallized reasoning int.ligence, G c r

6. Two tests primarily loaded the sixth component: REFLIMPL ani COGCOMPX.This component seemed tz symbolize simplistic processing, Ps, because the first test

loading this component was keyed for impulsivity and the second one was negativelyweighted.

7. The seventh component was dominated by CATEWIDH, CONCSTYL, and TOLR-XMBQ. Since the last two tests were negatively loaded, it appeared reasonable to callthis component global processing, Pg.

Cognitive Characteristics and Module Achievement Relations

Simple and multiple correlation as well as standardized partial regression coefficientsbetween cognitive characteristic components and module achievement scores are tabu-lated in Table 5. The multiple correlation coefficients indicate the relationships betweenall cognitive components and the achievement of each module; 10 of 11 multiplecorrelations were significant. For modules I and 4 through 9, these correlations weresomewhat stable, ranging from .31 to .37. For modules 2, 10, and 11, the multiplecorrelations were larger, ranging from .43 to .47. Figure 1 represents these relationshipsindirectly by depicting the amount of variance of the achievement of each moduleaccounted for by the cognitive components. As can be seen in Figure 1, the 10 significantmultiple correlations indicated tCat the cognitive crmponents explained 10 to 22 percentof the variance of module achievement.

Focusing on the individual cognitive components' contributions to the achievement ofeach module (presented in Table 5 and portrayed in Figure 1), a different pictureappeared. The components differed in their importance regarding module achievement.None of the cognitive components manifested a stable contribution to achievement acrossall the modules. In fact, two of the components, Ps and Pg, only made trivial or random

contributions to achievement. The G component accounted for a significant share ofachievement in only three modules: 2, 10, and 11. G demonstrated no relationship to

cmachievement in the first ten modules, but did contribute to module 11. The GC

component was an important influence on achievement in two modules: 2 and 5. Gf

explained portions of achievement for five modules: 2, 6, 8, 9, and 10. These significantcontributions were somewhat stable for ju: four of the five modules, ranging from .23 to.28. Lastly, +h• G component manifested significant relationships to the achievement

crof eight modules: 1, 2, 4, 5, 6, 7, 10, and 11. These varied from a low of .16 to a high of.31, with six being rather stable, ranging from .20 to .27. In terms of the number of

7

significant regression coefficient the most important components contributing -oachievement through the modules were Cc and Gf respectively.

Cr f

Across the modliles, the number of significant components contributing to achieve-ment ranged from one to four, with the change occurring according to no discerniblepattern or obvious trend. The relative importance of the components in terms of theamount of variance accounted for, or the magnitude of the regression coefficients, variednotably throughout the modules. Various combinations of cognitive components predictedthe achievement of specific modules. Different modules drew on different components todifferent degrees.

Classifying the subject matter of each of the II modules according to the task-content matrix of the instructional quality inventory (Ellis, Wulfeck, & Fredericks, 1979),revealed that the first five modules primarily required the students to remember facts,concepts, principle, and/or rules; and the last six modules, to use concepts, principles,and/or rules. The results of the multiple regression analyses suggested that, in a relatlvesense, Gc was more important for remembering facts, concepts, principles, and/or rules

rand Gf was more important for using concepts, principles, and/or rules.

To some extent, the cognitive characteristics the students possessed prior tobeginning their training determined their achievement. Within limits, student proficiencythroughout the modules could be predicted by using measures of these cognitivecomponents. Changes in the proportion of variance in student performance throughoutthe modules accounted for by certain cognitive componen.t. represented shifts in theiremphasis during the process of acquiring the course content. These shifts in predictorpatterns of cognitive components appeared to be related to whether a module required thestudents to remember or use facts, concepts, principles, and/or rules. Differentcognitive components seemed tc contribute more or less to student achievement atdistinct modules or stages of learning.

8

-'1N " 0'-'00NON LON-

00

-W

tr "9m P 0a rM brI L Y Da1

C,, C)c

.c . . . . .. nIf. ..to. . ..r

S0V 00 0 ~N 01 0'NON 'DNI, r0100'N

.!0 P4 %D'N 'N 00 .N-OOI&O .O-C-NS199 CIýc .99I9 C9 9 v

-'I 1!..-a. o r4Do m 0N00N00 '@ or0*'000yNN 'N'-00N~

- Crr - -0 N N -0 0 -00 N -t0000P00 2 N - W

10 o

cc C D <O = LWý- 0 Zu .IL-I5

U K ov Nf L

-L -a. - r ~

aI,

C0 g - 0 '

-N NlC C 0 00-0000

Wi

q , o . . . .

Cii

7

C! Cý

I i

S4I t. If I

~4t.

o Ii

S.o 0 C 0' 0 9. .90 . 9 N °Z 0 .9 9 N 0

I I I I ,I

"I 'C z0 0 00 o 00 00'N

U C

'1.~~jr -2 C) (LO''-C O04

-,ooo I,.,. . . .o .•, °. . .. ..

f-5 <4.25U 0F

'8 79 7i j'"''' "S=-Z-wZ

um- >Q< < tEo-- '-- 0-OO10

o- . --N --O -OO-OOO-0-OzU z

01

0.

A-

0

C4

.Jtit

I IV

!2a--

7 I -

-'L

U)

.- . .. L07 ' 0 f'~ *' >-1

17

.

m U -- --

cOw z ..

4' v4) ;v 7 7mual-7 m.7w1 Wv)< Z

Table 3

Initial Principal-component Solution, Communalities (h2), AssociatedEigenvalues, and Percent Variance for the Cognitive Characteristics

-4,e

Cognitive Initial SolutionCharacteristic 1 2 3 4 5 6 7 h2

FILDINDP .24 .20 -. 05 .54 .40 -. 07 -. 07 .56CONCSTYL .10 .21 .07 .38 -. 10 -. 24 -. 44 .46REFLIMPL -. 10 -. 15 .22 -. 49 .58 .11 -. 06 .66TOLRAMBQ .09 -. 01 .41 .18 .01 -. 54 .12 .51CATEWIDH .20 .1t, .03 .02 .41 .45 .42 .60COGCOMPX -. 07 -. 02 -. 17 .26 -. 58 .25 .08 .51VERBCOMP .56 .18 .51 -. 02 -. 04 -. 05 .10 .61GENLREAS .46 .42 -. 01 .08 .14 -. 32 .32 .63ASSOFLUN .32 .29 .59 .12 -. 10 .24 -. 03 .63LOGIREAS .37 .20 -. 09 .12 -. 03 -. 20 .55 .55"INDUCTON .20 .39 -. 02 .43 .12 .45 -. 13 .60IDEAFLUN .28 .37 .34 -. 21 -. 22 .33 -. 00 .53GENLINFO .56 -.16 .21 -. 24 .05 -. 01 -. 02 .44

2....:: NUMROPER .39 .49 -. 34 -. 32 -. 20 -. 03 -. 01 .65ATTNDETL .03 .45 -. 36 -. 25 -. 09 .16 .00 .43WORDKNOL .70 -. 04 .27 -. 24 -. 14 .04 -. 27 .71ARTHREAS .69 .i8 -. 28 -. 24 .07 -. 19 -. 1' .70SPACPERC .51 -. 24 -. 30 .15 .37 .05 -. 16 .59MATHKNOL .70 .33 -. 28 !0 .04 -. 18 -. 16 .7b"ELECINFO .67 -. 29 -. 02 23 .00 .13 -. 07 .ý'1MECHCCMP .73 -. 25 -. 16 .12 .11 .21 -. 09 .70"GENSCIE .73 -. 18 .09 -. 06 -. 08 -. 09 -. 17 .62SHOPINFO .59 -. 49 -.11 -. 09 -. 20 .02 .22 .69AUTOINFO .61 -.43 -. 06 .14 -. 21 .08 .27 .71

AssociatedEigenvalue 5.42 2.02 1.64 1.50 1.42 1.31 1. 14

Percent

Variance 22.6 8.4 6.9 6.2 5.9 5.4 4.7

Notes.

1. Only factors with associated eigenvalues greater than or equal to 1.0 aretabulated. This minimum eigenvalue criterion ensures that only factors accounting for atleast the amount of total variance of a single variable are significant.

2. Cognitive characteristics are defined in Table 1.

12

Table 4

Terminal Varimax Solution for the Cognitive Characteristics

Cognitive Terminal SolutionChcracteristic 2 3 4 5 6 7

FILDINDP .10 -. 05 -. 06 .68 .26 .09 -. I 1CONCSTYL -. 02 .02 .09 .33 .02 -. 10 -. 58REFLIMPL -. 04 -. 08 .04 -. 11 -. 21 .73 .28TOLRAMBQ -. 01 -. 32 .13 -. 08 .50 .13 -. 34CATEWIDH .07 .03 .16 .32 .14 .14 .66COGCOMPX .01 .01 .02 -. 04 -. 13 -. 70 .02VERBCOMP .30 -. 00 .62 .03 .35 .12 -. 06GENLREAS .11 .30 .13 .19 .68 .09 -. 00ASSOFLUN .07 - .08 .75 .20 .09 -. 03 - .02LOGIREAS .15 .16 .02 .04 .66 -. 15 .20INDUCTON -. 01 .14 .27 .68 -.10 -. 18 .11IDEAFLUN .01 .24 .67 -. 03 -. 04 -. 06 .11GENTINFO .52 .06 .29 -. 13 .09 .24 .00NVMROPER .09 .77 .13 -. 05 .15 -. 07 -. 02ATTNDETL -. 17 .61 .01 .03 -. 05 -. 10 .14WORDKNOL .59 .20 .50 -. 09 -. 04 .15 -. 20ARTHREAS .50 .59 .03 .05 .22 .20 -. 13SPACPERC .59 .09 -. 24 .37 -. 02 .20 .03MATHKNOL .43 .63 .08 .20 .25 .11 -. 20ELECINFO .73 -. 05 .10 .23 .04 -. 09 -. 01MECHCOMP .78 .12 .05 .27 -. 02 .00 n9GENSCIE .70 .13 .24 -. 02 .09 .08 -. 21SHOPINFO .75 -. 02 -. 03 -. 26 .13 -. 15 .15AUTOINFO .74 -. 110 .03 -. 08 .20 -. 28 .16

Note. Cognitive characteribtics are defined in Table i.

ft1

'p

• i 13

CL'C

Ic uc C

C:

* *0

r 0 "D m - R 0

-u 0 L)

C -

"C t' -C Cc C'c,~

*: I' r- Nc t -0 - EV

CC:> C~ 0 u'iO

t, 4) * 0

I I 0

Co C- 0 C a C

0' * C'

0.

C>,0)1

4L) 0 C I I

m- - W- j

-0 0,- > -C " ' NC

- Cc C 2 ~' 0

"1 0)0

C- (U

~C V. ' c' U .

a C CE C' -0 a; V)

CL> C 1 ~c

-t C N2 C- N- C'-:0

1.> >u r-

-~~ ~ A 0 d

K0

-- 4 40

-o

E0

>E

cc-a4-

0 u

c4

is, E0

0 0I

CU4)

C4d

00~~~ icvm-40 mO 0L(4 C4 C4 C C4

30NVIUV IN4)4)

DISCUSSION

The results clearly indicated that considerable changes occurred in the cognitivepredicators of achievement as students progressed through the sequential modules ofinstruction. During the acquisition of course content, the cognitive components sampledshifted noticeably in importance throughout the curriculum. This was manifested byvariations in the regression coefficients of particular cognitive predictors at distinctstages of learning. Different componet. t s appeared to contribute variance at earlier andlater phases of mastery. G c seemed more important during the first half of the 11

modules; whereas Gf seemed more important during hte second half. Gc, however,r

appeared equally important during both phases of acquisition, since its contributionthroughout most of the modules was relatively stable. These results suggest that, at earlystages of acquiring complex subject matter, the C component played somewhat of a

cvmajor role. As learning continued through the modules, this cognitive component becameless important as a determiner of individua' differences in mastery. At the same time, Gfincreased in importance as ,earning continued until it was one of two components that

entered to a significant extent a:-4 number at the latter stages of mastery. As thestudents progressed through the entire curriculum, the contribution G c made to their

acquisition of the course content nevertheless remained somewhat stable. It was impliedthat, after progressing through hierarchical modules, individual differences in .'earningdepend more on certain cognitive components and less on others than they did whenbeginning to acquire the course contents. Earlier in learning, the use of specificcomponents is minimal but prerequisite for later acquisition. Yet, other cognitivecomponents may remain rather unchang:ng during the mastery of the complete cur-riculum.

"With respect to conducting ATI research, changes in the component predictor patternover the course of learning underscored the necessity of ascertaining what aptitudes arecontributing to acquisition at distinct stages. It was highly likely that the cognitiveprocessing involved in the initial phases of learning differed from the processing atterminal phases of acquisition. This suggested the requirement for protocol analyses of

I-: the cognitive processing involved in early, intermediary, and later phases of learning. Itseemed possible that particular aptitudes contributing to learning at distinct stagesreflect the presence of prerequisite cognitive competencies, schemata, knowledge, andlearning sets required to acquire the subject matter at particular phases of mastery.

The instability of the relationships of some cognitive components across distinctstages of learning suggested the importance of conce.itrating on the process of changefrom ignorance to competence. Glaser (1976) has aptly explained the transformation fromnovice to expert during the course of mastering complex subject matter or irltricate skillas follows:

(a) Variable, awkward, and crude performance changes to perfor-mance that is consistent, relatively fast, and precise. Unitary actschange into larger response integrations and overall strategies.

(b) The contexts of performance change from simple stimulus pat-terns with a great deal of clarity to complex patterns in whichinformation must be abstracted from a context of events that are notall relevant.

16

(z) Performance becomes increasingly symbolic, covert, and auto-matic. The learner responds increasir'gly to internal representationsof an event, to internalized stan-4 -rds, and to internalized strategiesfor thinking and problem solving.

(d) The behavior of tne competent individual becomes increasinglyself-sustaining in terms of skillful employment of the rules when theyare applicable and subtle bending of the rules in appropriate situa-tions. (p. 9)

"Burns (1980) mentioned that alterations in aptitude demands through learning mayform the basis of the transition from novice to expert performance. In attempting toaccount for the nature of such shifts, he hypothesized that instructional treatments arecomposed of two distinct origins of aptitude requirements: those related to the method ofinstruction and those related to the content of instruction. Each of these sourcesdemands a specific type of aptitude. It was postulated that the method of instructionprimarily requires the Gc aptitude; and the content, the Gf aptitude.

According to Snow (1980), Gc, Cattell's (1971) crystallized ability, represents a general

dimension of measures that are good predicators of conventional educational achievementor scholastic ability (e.g., verbal, quantitative, vocabulary, reading comprehension,

• .information, mathematical, and prior scholastic achievement). Gf, Cattell's (1971) fluid

ability represents another general dimension of measures and probably represents theassembly and control processes necessary to structure adaptive strategies for solvingnovel and immediate problems (e.g., abstract, spatial, figural, and nonverbal reasoningtests).

in attempting to answer why G c measures are often better predictors of learning

outcome than are Gf measures, Snow (1980) speculated:

One reason may be that Gc represents the !ong-term accumulation of

knowledge and skills, organized into functional cognitive systems byprior learning, that are in some sense crystallized as units for use infurture learning. Because these are products of past education, andbecause education is in large part accumulative, transfer relationsbetween past and future learning are assured. The transfer need notbe primarily of specific knowledge but rather of organized academiclearning skills. Thus Gc may represent prior assemblies of perfor-

mance processes retrieved as a system and applied anew in instruc-tional situations not unlike those experienced in the past, whereas Gfmay represent new assemblies of performance processes needed inmore extreme adaptat'ons to novel situations. The distinction, then,is between long-term assembly for transfer to familiar new situationsversus short-term assembly for transfer to unfamiliar new situations.(p. 37)

iL This distinction between G c and Gf led Burns to theorize that, since Gc confers

pervasive learning skills--not specific knowledge--then Gc itself transcends the particular

17

content of instruction. To the extent that this type of instruction had been experiencedby students previously, Gc would inculcate a general learning set to process and interpretthis kind of instruction. Consequently, Burns posited that Gc could manifest a nearly

cstable relationship over the course of learning if there were no pronounced changes in themethcd of instruction. This could not be so with G But it was possible, speculated

Burns, that the content of the instruction perixlically and differentially requires theprocessing reflected by Gf. Because the content of instruction usually changes during a

course, it was reasonable to sDeculate that the Gf learning relationships demonstrate

instability as students progressed through a particular curriculum from novices to experts.Bum.; hypothesized that, for these reasons, G typically manifests more consistent ATI

cresults than specialized aptitudes like Gf (Cronbach & Snow, 1977).

In this study, both Gf and Gcr demonstrated somewhat stable relationships to

achievement. However, Gf was rather stable oniv for the second half of instruction; and

Gc , primarily for approximately the first two-thirds of instruction. The content of ther

first five modules principally required students to remember facts, concepts, principles.I eand/or rules; and the last six modules, to use concepts, principles, and/or rules. These

results seem to imply that the content of instruction and the task demanded of thestudents as they progress through the course determine the nature of the relationship ofGf to achievement. The content of instruction changed from one module to the next

throughout the entire course. Nevertheless, Gf exhibited a fairly stable relationship to

mastery for the second part of the curriculum. The chang'ng course content had noapparent impact ppon the stability of Gf. The type of processing task required of the

students seemed to be more important though regarding Gf. If the cognitive task demands

remained more or less constant, Gf's relationship to achievement would remain more or

less stable. Possibly, to perform a particular processing task, students would have toresc . to the prerequisite cognitive competencies, schemata, structures, knowledge andrepresentational systems that are assessed by specific aptitudes. If the task demands donot change, then the aptitude requirements do not change. Conversely, if the taskrequirements are altered, then the aptitude demands are altered. This interpretationregarding Gf differs from Burns' speculations.

Insofar as G c is concerned, the data obtained in this study seem to suggest that Gc is

independent of course content, while dependent upon task demands and the method ofinstruction. Gc exhibited a fairly staEte relationship to achievement throughout the

rearlier part of the curriculum, which involved primarily remembering facts, concepts,principles, and/or rules. The method of instruction remained the same while the contentsof each module varied through the entire curriculum. During the final phase ofinstruction, when the students were using concepts, principles, and/or rules, the relation-ship of GC to learning became unstable. With the method of instruction and taskcr

demands constant, while the content changed during the initial two-thirds of the course,

18

t'-...

the relationship of G to achievement was somewhat stable. In the final third of theCr

curriculum, the method of instruction was still constant while the content and taskrequirements changed. The introduction of change in task demands was associated withthe unstable relationship of G to learning. This imp. -d to some extent that the

Crassociation of G to achievement was contingent upon task requirements and the method

c r

of instruction while being unsubordinated to course contents. This speculation differedfrom Burns' suppositions.

The CMI system used to implement mastery learning of the BE/E curriculum probablycan be considered a "new" learning situation in Snow's (1980) scheme of things:

What constitutes a "new" learning situation is not really clear. Butone can predict that as an instructional situation involves combina-tions of new technology (e.g., computerized instruction, or tele-vision), new symbol systems (e.g., computer graphics or artisticexpressions), new content (e.g., topological mathematics or astro-physics), and/or -ew contexts (e.g., independent learning, col-labora-tive teamwork in simulation games), Gf should become moreimportant and Gc less important. (p. 59)

CMI can be viewed as a relatively new technology. The comprehension of many circuitschematics and the solution of numerous algebraic equations can be thought of as newsymbol systems. The perception of several relationships among voltage, resistance, andcurrent, as well as the reduction of complicated circuits to simpler ones, can representnew content. Lastly, self-study and self-pacing together with mastery learning can beregarded as new contexts. According 'o Snow, the relationship of Gc to achievement

should be stronger in ordinary educational environments. This has been established inmuch of the AT- research (Cronbach & Snow, 1977).

If the typical instructional treatment is altered as in computer-managed masterylearning, then the strength of the association of G to learning is lessened and an ATI will

likely appear. Consequently, students who lack well-developed, conventional, academicaptitudes will benefit from the unorthodox, educational treatment, while those whop!::-•s these skills may not be able to apply them. Computer-managed mastery learningis individualized instruction based on carefully defined objectives, hierarchical in itscontent, modular in its presentation and assessment, with diagnostic achievement testsand immediate feedback on student progress; it structures, segments, and directs learningfor lower-aptitude students--doing for them what they cannot do well for themselves.Snow maintained that this unconventional instructional treatment is probably dysfunc-tional for more apt studentb--those who can organize and control their own learningbecause of the nature of the cognitive processing required and acquired previously byconventional, educational experiences. Therefore, G aptitude is probably of no

particular advantage in novel instructional situations like computer-managed masterylearning Within this context, Snow expected that Gf would be associated with achieve-ment in if-novative instructional situations- -different from those the students experiencedin the past. In these novel educational environments, Gc will likely be irrelevant; and Gf,

relevant. The data obtained herein, however, demonstrated that not only is Gf pertinent

19

to learning in new instructional situations but so is G (i.e., its components). Thisc

suggested that some unconventional educational settings are not necessarily dysfunctionalfor abler students. In these situations, they can just as easily exercise and capitalize uponthose skills developed and applicable in more traditional instructional situations.

CONCLUSIONS AND RECOMMENDATIONS

It was highly likely that the cognitive processing involved in the initial phases oflearning differed from the processing at terminal phases of acquisition. This suggestedthe requirement for protocol analyses of the cognitive processing involved in eatiý,intermediary, and later phases of learning. It must be determired whether particularaptitudes that contribute to learning at distinct stages reflect the presence of pre-requisite cognitive competencies, schemata, knowledge, and learning sets required toacquire the subject matter at particular phases of mastery. The instability of therelationships of some cognitive components across distinct stages of learning suggestedthe importance of concentrating on the process of change from ignorance to competence.

20

REFERENCES

Baker, F. Computer-managed instruction: Theory and practice. Englewood Cliffs, NJ:Educational Technology Publications, 1978.

Bieri, J., Atkins, A., Briar, S., Leaman, R., Miller, H., & Tripodi, T. Clinical and socialjudgment: The discrimination of behavioral information. New York: John Wiley &Sons, 1966.

Block, J. (Ed.). Schools, society, and mastery learning. Holt, Rinehart, & Winston, 1974.

Bloom, B. An introduction to mastery learning theory. In J. Block (Ed.), Schools, society,and mastery learning. New York: Holt, Rinehart, & Winston, 1974.

Bloom, B. Human characteristics and school learning. New York: McGraw-Hill, 1Q7 6 .

Burns, R. Relation of aptitudes to learning at different points in time during instruction.Journal of Educational Psychology, 1980, 72, 785-795.

Bunderson, C. Transfer of mental abilities at different stages of practice in the solutionof concept problems (Research Bulletin 67-20). Princeton, NJ: Educational Testing-Service, 1967.

Carroll, J. Psychometric tests as cognitive tasks: A new "structure of intellect"(Research Bulletin 74-16). Princeton, NJ: Educational Testing Service, 1974.

Carroll, J. Psychometric tests as cognitive tasks: A new "structure of intellect." In L.Resnick (Ed.). The nature of intelligence. Hillsdale, NJ: Lawrence ErlbaumAssociates, 1976.

"Cattell, R. Abilities: Their structure, growth, and action. Boston: Houghton Mifflin,1971.

Clayton, M., & Jackson, D. Equivalence range, acquiescence, and over-generalization.Educational and Psychological Measurement, 1961, 21, 371-382.

Cronbach, L., & Snow, R. Aptitudes and instructional methods: A handbook for researchon interactions. New York: Irvington Publishers, 1977.

Dunham, J., Guilford, J., & Hoepfner, R. Multivariate approaches to discovering theintellectual components of concept learning. Psychological Review, 1968, 75, 206-221.

Ekstrom, R., French, J., Harman, H., & Derman, D. Manual for kit oi factor-referencedcognitive tests. Princeton, NJ: Educational Testing Service, 1976.

Ellis, J., Wulfeck, W., & Fredericks, P. The instructional quality inventory: II. User's* manual (Spec. Rep. 79-24). San Diego: Navy Personnel Research and Development

Center, 1979. (AD-A083 678)

Ferguson, G. Human abilities. Annual Review of Psychology, 1965, 16, 39-62.

Fleishman, E. On the relation between abilities, learning, and human experience.*9, American Psychologist, 1972, ?7, 1017-1032.

Fleishman, E., & Bartlett, C. Human abilities. Annual Review of Psychology, 1969, 20,349-379.

21

Frederiksen, C. Abilities, transfer, and information retrieval in verbal learning. Multi-variate Behavioral Research Monographs, 1969, (No. 69-2).

Gagne, R., & Paradise, N. Abilities and learning sets in knowledge acquisition.Psychological Monographs, 1961, 75(14, Whole No. 518).

Glaser, R. Components of a psychology of instruction: Toward a science of design.Review of Educational Research, 1976, 46, 1-24.

Horn, 3. Human abilities: A review of research and theory in the early 1970's. AnnualReview of Psychology, 1976, 27, 437-485.

Hotelling, H. Analyses of complex statistical variables into principal components.Journal of Educational Psychology, 1933, 24, 417-441.

Hultsch, D., Nesselroade, J., & Piemons, 3. Learning-ability relations in adulthood.Human Development, 1976, 19, 234-247.

Jackson, D. Personality research form manual. Goshen, NY: R, larch Psychologist Press,Inc., 1974.

Kaiser, H. The varimax criterion for analytic rotation in factor analysis. Psvchometrika,1958, 239 187-200.

Kogan, N. Educational implications of cogniti-,e styles. In G. S. Lesser (Ed.). Psychologyand educational practice. Glenview, IL: Scott Foresman, 1971.

Labouvie, G., Frohring, W., Baltes, P., & Goulet, L. Changing relationship between recallperformance and abilities as a function of stage of learning and timing of recall.Journal of Educational Psychology, 1973, 64, 191-198.

Orlansky, 3., & String, 3. Cost-effectiveness of computer-based instruction (IDA PaperP- 1375). Arlington, VA: Institute for Defense Analyses, 1979.

Pettigrew, T. F. The measurement and correlates of category width as a cognitivevariable. Journal of Personality, 1958, 26, 532-544.

Roberts, D., King, F., & Kropp, R. An empirical investigation of Ferguson's theory ofhuman abilities. Canadian Journal of Psychology, 1969, 23, 254-267.

Rydell, S., & Rosen, E. Measurement and some correlates of need-cognition. Psycho-logical Reports, 1966, 19(I-V19), 139-165. (Monograph Supplement)

Snow. R. Aptitude processes. In R. Snow, P -A. Federico, & W. Montague (Eds.). Aptitude,learning, and instruction. Volume 1: Cognitive process analyses of aptitude. Hillsdale,NJ: Lawrence Erlbaum Associates, 1980. Also published as NPRDC/ONR TR 81-5, SanDiego: Navy Personnel Research and Development Center, 1981. (AD-A096 209)

Wallach, M., & Kogan, N. Modes of thinking in young children. New York: Holt,Rinehart, & Winston, 1965.

Werdelin, I., & Stjernberg, G. The relationship between difficulty and factor loadings ofsome visual-perceptual tests. Scandanavian Journal of Psychology, 1971, 12, 21-28.

22

DISTRIBUTION LIST

Chief of Naval Operations (01B7), (OP- 135C4), (OP-987H)Chief of Naval Material (NMAT 0722)Chief of Naval Research (Code 270), (Code 440) (3), (Code 442), (Code 442PT)Chief of Naval Education and Training (00A), (N-21)Chief of Naval Technical Training (00), (N-6)Commander Navy Recruiting Command (Code 20)Commandant of the Marine Corps (MPI-20)Commanding Officer, Naval Aerospace Medical Institute (Library Code 12), (2)

• Commanding Officer, Naval Technical Training Center, Corry Station (Code 101B)Commanding Officer, Naval Education and Training Program Development Center (Tech-

nical Library) (2)Commanding Officer, Naval Trainin, Equipment Center (Technical Library) (5), (Code N-

1)Commanding Officer, Office of Naval Research Branch Office, Chicago (Coordinator for

Psychological Sciences)Superintendent, Naval Postgraduate SchoolCommander, Army Research Institute for the Behavioral and Social Sciences, Alexandria

(PERI-ASL), (PERI-ZT), (PERI-SZ)Commander, Air Force Human Resources Laboratory, Brooks Air Force Base (Manpower

and Personnel Division), (Scientific and Technical Information Office)Commander, Air Force Human Resources Laboratory, Williams Air Force Base

(AFHRL/OT), (CNET Liaison Office AFHRL/OTLN)Commander, Air Force Human Resources Laboratory, Wright-Patterson Air Force Base

(AFHRL.'LR)Commander, Headquarters AFMTC/SR, Lackland Air Force BaseCommander, Air Force Hun an Resources Laboratory, Lowry Air Force Base (Technical

Training Branch)Chief, Army Research Institute Field Unit-USAREUR (Library)Chief, Army Research Institute Field Unit, Fort KnoxCommanding Officer, U.S. Coast Guard Research and Development Center, Avery PointDirector, Naval Education and Training Program Development Center Detachment,

MemphisInstitute for Defense Analyses, Science and Technology DivisionDefense Technical Information Center (DDA) (12)

23

.7