DR. T. EUGENE AVERY*

University of IllinoisUrbana, Ill. 61801

HAROLD B RKHART tUniversity of Georgia

Athens, Georgia 30601

Screening Tests forRating Photo InterpretersThe reliability of correct selection can probablybe increased from about 41 % (without testing)to about 73%.

(Abs/ract on page 478)

THE PROBLEM

I F ASKED TO LIST the essential qualificationsfor persons to be trained as photo inter­

preters, one might logically include goodstereoscopic vision, normal color perception, asound educational background, and above­average mental ability or I.Q. rating. Butwould the use of these cri teria alone consis­tently separate good potential photo inter­preters from poor ones? Probably not! Thus,to supplement the accepted testing proce­dures for evaluating these qualities, we alsoneed some measure of an individual's motiva­tion, judgmen t, and logical reasoni ng' abili­ties. To state the problem in another way, canwe develop a screening process for selectingthe 50 best potential interpreters from an ap­parently homogeneous group of 300 or moreapplicants?

The need for a battery of screening tests forrating trainees prior to specialized photointerpretation (P. I.) instruction was outlinedin 1956 by Sims and Hall. In 1960, Meyer andMiller reported that photo interpretation per­formance, based on recogni tion and assess­ment 01 industrial features, could be predictedwith reasonable success from scores on apti­tude tests (problem solving and spatial rela­tions) and visual perception examinations.Following a similar study, it was concludedby Sadacca, Martinek, and Schwartz (1962)that the task of image interpretation is highlycognitive, and if basic visual standards are

* Head, Department of Forestry.t Doctoral candidate in the School of Forestry.

met, the ability to make necessary complexdecisions and judgments will depend mostlyon the mental capacity, interest, and experi­ence of the individual.

In 1965, Avery reported on the initiation ofa research study aimed at determining (1)whether the elusive qualities of motivation,judgment, imagination, and logical reasoningcould be determined by various screeningtests, and (2) whether scores on such screen­ing tests could be correlated with interpreta­tion performance at the termination of aspecialized P. I. training course. This reportsummarizes the results of that study.

DIL T. EUGENE AVERY

476

SCREENING TESTS FOR RATING PHOTO INTERPRETERS

FIG. I. Examples of stereograms from the Stereo Performance Test. The Ilumbers "3"and" I I" correspond to the questions and allswers listed in the text of this article.

477

SCREENING TESTS

A total of 99 male college studen ts at theUniversity of Georgia participated as traineesin the testing experiment. Most of the menwere 20 to 25 years old, and all had goodstereoscopic vision. These studen ts were en­rolled in beginning classes in photo inter­pretation during the 1964-65 and 1965-66academic years; none had received previouscivilian or military training in aerial photointerpretation.* At the beginning of classes,

* To obtain the highest possible degree ofhomogeneity in the trainee group, results of femalestudents were not included in the analysis. Simi­larly, students with poor stereoscopic acuity andthose with previous P.1. training were excluded.Participation in the experiment was voluntary,and results had no bearing on an individual'sfinal college grade in the course. The course in­cluded about 75 hours of classroom and laboratoryinstruction in elementary photogrammetry, photointerpretation, and mapping techniques.

following a brief introduction to stereoscopicviewing, a battery 01 seven screening tests wasadministered to each trainee. The tests weregiven in the following order:

1. S'imutated jJhotographic 'images test (Avery,1964). This test requires identifIcation of 39sketches tha t depict overhead views or shadowsof common urban, rural, or physiographicfeatures.

2. Aerial photoidel1tificatiol1 test (Avery, 1964).Twenty-four common urban and rural featuresmust be identified on vertical photographs.

3. Picture integration puzzle (Avery, 196-!).This type of test requires that examineesassemble cut-out portions of an aerial photo­graph into a composite and complete picture.Scoring is based on the time required to completethe puzzle.

4. Mental ability or I.Q. test (Otis, 1954).The Otis test is a group-administered intelli­gence quotient exam consisting of 80 questionson vocabulary, word relationships, progressions,mathematics, and figure analogies.

5. Object visualization test (M iller, 1955).This test consists of 44 drawings; geometric

478 PHOTOGRAMMETRIC ENGINEERING

solids are presented as flat or unfolded patterns.Examinees must visualize the geometric shapesthat will be formed when the flat patterns arecorrectly folded or rolled together.

6. Standard progressive rnatrices (Raven, 1958).Sixty figure analogies are utilized to measure anindividual's capacity for observation, clearthinking, and logical reasoning.

7. Stereo practice quiz (Avery, 1964). Consist­ing of 25 stereograms, this unpublished testrequires the student to make image identifica­tions, object counts, height rankings, and onelength measurement.

Except for the picture in tegration puzzleand men tal abili ty test, scores were recordedas a percen I. of correct responses. Scori ng ofthe picture integration puzzle test was basedon the time required to complete the puzzleand computed as 1,000 divided by the puzzleassembly time in minutes. Mental ability wasscored as the intelligence quotient indicatedby the Otis test.

trainees, and the fact that the students weremajoring in such diverse subject-matter fieldsas geography, geology, forestry, landscapearchi tecture, and physics.

In spite of the foregoing difficulties, twoperformance tests were devised. The first, aVisual Search Test, is designed to evaluatephoto scanning ability. Sometimes referred toas a spatial orientation or grid location test,this examination requires the student to de­termine the exact coordinat.e positions ofsmall photo cut-outs on a complete verticalprint of the same area. In addition, one partof the test requires the trainee to make cor­rect translations from map locations to anaerial photo index of the same area. Ex­aminees were rated on a percentage scaleaccording to the time required to list the cor­rect grid coordinates on a special answer form.

The second part of the performance rati ng

ABSTRACT: Can individuals be rated according to photo interpretation abilityprior to specialized training? Results of a two-year study involving 99 universitystudents indicate that some measure of photo interpretation performance maybe reliably predicted by scores on two specialized screening tests administered atthe onset of training. One required the identification of 39 sketches that depictoverhead views of common urban, rural, and physiographic features. The othertest consisted of 25 image identifications and measurements made on preparedstereograms. In combination, scores on these screening examinations accountedfor about one-hi lf of the variability in final photo interpretation performanceratings.

THE ELUSIVE MEASURE OF

PERFORMANCE

To determine the relative values of thescreening tests administered at the onset oftraining, an objective measure of performancewas required at the end of the photo inter­pretation course. The performance criteriamost frequently employed in the other studiesinvestigated was the composite course gradeor overall proficiency record of each indi­vidual at the termination of a training period.Scholastic grades were deemed unsuitable inthis study, however, because these compositescores were based on written examinationsand map compilations as well as on photo in­terpretat.ion abilities. What was desired was afinal performance rating based on visualscanning ability and on the reliability ofphoto identifications. Devising such per­formance tests was complicated by two non­controllable elements: the inevitable differ­ences in cultural backgrounds of the 99

was based on identifications of cultural fea­tures or signposts of civilization as depicted on25 specially prepared vertical stereograms.Insofar as available photographs permitted,the test items were chosen to evaluate anindividual's abilities in deductive reasoningrather than requiring mere recognition byrote or memorization. Because of the par­ticular associations of features pictured, thisexamination proved much more difficult thanthe analogous stereo-practice quiz used as ascreening test. Nevertheless, the verticalviews illustrated features which studentswould have quickly recognized from closeground observation; the less' familiar over­head views t.hus served to separate the think­ing interpret.ers from their less perceptiveclassmates.

A single composite performance rating foreach student was derived from a weightedaverage of scores received on the VisualSearch Test (weight 1/4) and the Stereo Per-

SCREENING TESTS FOR RATING PHOTO INTERPRETERS 479

whereX, = Simulated Photo I mage Test ScoreX 2 = Stereo Practice Quiz ScoreWhen applying this prediction eq uation,

the independent variables (Xl and X 2) mustbe entered as percent of correct responses onthe respective tests. This will provide a pre­dicted performance rating in percent. Simplelinear correlation coefficients between the twoscreening tests used in the prediction equationand the final performance rating are shown inthe tabulation that follows. These correlationcoefficients substantiate the highly significantassociation between screening test scores andfinal performance ratings (Table 1).

DISCUSSION AND CONCLUSIONS

Assuming that the proposed method of rat­ing interpreter perf01 mance is valid, the morepromising candidates for P.1. training may beselected on the basis of their scores on twoscreening tests. The prediction equation ac­counts for 48 percent of the variability thatwill exist in a trainee's final performancescores. Thus, the equation permits the rank­ing of individuals as an aid in selection fortraining. With their predicted performancescores arranged in rank order, the requirednumber of P.I. trainees could be chosen bystarting with the highest ranking individualand working downward. Accuracy of rankingwill depend on the size of the group and rela­tive aptitudes of individuals making up thegroup. As a practical example, if a group of 50P.I. candidates are to be selected for training,it would be desirable to screen them from agroup of 300 or more individuals rather thanfrom a group of only 100 persons.

To demonstrate further the validity of theselection criteria proposed here, performanceratings for the 99 students participating inthis study were com pu ted by su bsti tu ti ngtheir screening test scores in the predictionequation. The predicted performance ratingswere then checked against actual performancescores for each man. A performance rating of80 percent or greater was arbitrarily classedas satisfactory; ratings of 79 percent andlower were deemed unsatisfactory. Compari-

TABLE 1.

** Significant at the .01 level.

0.62**0.60**

Corretation coefficientwith performance ratingScreening test

Simulated Photo lmagesStereo Practice Quiz

formance Test (weight 3/4). It seemed thatthese relative weights for photo scanningability and image identification provided arealistic measure of the type of performanceoften expected of both military and civilianinterpreters.

AN ALYSIS OF TESTING RESULTS

As outlined earlier, the primary objectiveof this study was to determine whether scoreson one or more of the preliminary screeningtests could be used to predict the final per­formance rating of trainees. Accordingly,multiple regression techniques were employedto analyze relationships between the sevenscreening test scores and the final perfor­mance rating for each of the 99 participatingindividuals.

The data were analyzed at the Universityof Georgia Computer Center using a Bio­medical computer program designated asBMD02R. This program carries out theregression analysis in a stepwise fashion asdescribed by Efroymsen (1960). In additionto the regression analysis, the program pro­vides the mean and standard deviation foreach of the independen t and dependen t vari­ables, a covariance matrix, and a correlationmatrix. Plottings of residual sums of squareson each independent variable are also carriedout. In all cases, there was a random scatterof points about the zero line, further support­ing the validity of a linear statistical modelassumption.

The analysis indicated that the SimulatedPhoto I mages Test (Screening Test No.1)was the best single predictor of performance,and it accounted for 38 percent of the vari­ability in performance scores. The StereoPractice Quiz (Screening Test No.7) was re­moved on the second step, and, combinedwith the Simulated Photo Images Test score,boosted the amount of performance variabil­ity accounted for to 48 percent. The StandardProgressive Matrices Test was the third in­dependent variable removed, but this only in­creased the predictive abili ty by abou t 2 per­cent. Consequently, this test was omittedfrom the prediction equation. The remainingscreening tests, when entered in the predic­tion equation, each added 1 percent or less inaccounting for the total amount of variabilityin performance scores.

A stepwise regression analysis using gradesfrom the Simulated Photo Images Test andthe Stereo Performance Quiz yielded the fol­lowing prediction equation:

Predicted Performance Rating= 19.16108+0.41728 X 1 +0.36619 X 2,

480 l'HOTOGRAMMETIUC ENGINEERING

TABLE 2.

STEREO PRACTICE QUIz-REQUIRED

IDENTIFICAnONS

I. (a) Gas station (b) church (c) shoppingcenter

2. Contour plowing, or terraced cropland

sons of predicted and actual ratings were asshown in Table 2.

Of the 3S persons predicted as satisfactory,25 actually proved to be satisfactory, while 10individ uals had unsatisfactory performanceratings. Furthermore, of the 64 persons pre­dicted to be unsatisfactory, 48 actuallyproved to be unsatisfactory, although 16 didachieve satisfactory performance rati ngs.Thus, by regarding 80 percen t on performancescores as an acceptable screening level, itwould have been possible to categorize cor­rectly 73 of the 99 men tested here as satis­factory or unsatisfactory in advance of P.I.training.

AVAILABILITY OF TESTS

Because of costs involved, it has not beenfeasible to reproduce here the screening andperformance tests employed in this study.However, sample questions from the variousscreening tests are illustrated in Avery (1965)cited at the end of this report. Copies of theScreening Tests 1 and 2 and the Visual SearchTest are available at nominal cost from thesenior au thor.

The Stereo Practice Quiz and the StereoPerformance Test consisted of 25 black-and­white stereograms each, assembled and anno­tated by the senior au thor. As these stereo­grams are in unpublished form, a listing ofcorrect responses for each question is includedhere to indicate the range of imagery and in­formation required. Both tests utilized pan­chromatic photographs at a scale of 660 feetper inch. Stereograms were arranged onlaboratory tables, and trainees were allowedthree minutes per stereogram or a total of 7Sminutes for each test (Figures 1, 2).

1. Identify group of cattle in pasture neardecaying foundation of old residentialbuilding

2. Determine time of day photography fromshadow patterns

3. Determine use of sand being hauled frompit (used for sand traps on nearby golfcourse).

4. Determine reason for land clearing (newhousing development).

5. Identify (a) former sites of oil tanks, (b)sand and gravel extraction.

6. Identify private estate for training racehorses.

7. (a) Explain pattern (piles of dumped sandfrom trucks). (b) Identify sand hoppers forloading dump trucks.

8. Identify oil drilling and pumping station.9. Identify city bus terminal and bus turn­

around facility.10. Deduce meaning of numbers painted on a

building roof (longitude and latitude ofcity).

11. Identify state prison and farm.12. Identify oil refinery and petroleum storage

facili ties.13. Identify cemetery and adjacent city water

purification plant.14. Deduce the principal raw materials (coal,

water) and product (electricity) of an in­dustry (electrical power plant).

15. Identify stacks of lumber, poles, and otherwood products in an industrial area.

16. Identify and explain function of railroadturntable and roundhouse.

17. Explain purpose of cleared strip throughforest cover (electric power transmissionline).

18. Identify products being hauled in trucks(petroleum products).

STEREO PERFORMANCE TEST-REQUIRED

IDENTIFICA nONS

3. (a) Golf course (b) tennis court (c) swimmingpool

4. Drive-in theatre5. Light manufacturing industry6. Recognition of cedar tree from shadow

pattern7. Orchard of fruit trees8. Grade school9. Graded dirt road

10. (a) School busses (b) football field (c) watertank

11. Bridge construction for main thoroughfare12. College or university campus13. (a) Auto junkyard (b) small stream or creek14. Auto garages for residences15. (a) steel electrical tower (b) river (c) con-

crete highway bridge16. Trucking terminal17. Cleared rights-of-way18. Concrete dam19. Mobile homes20. Baseball stadium21. Count types of railroad cars in a train22. Count number of tracks at railroad grade

crossing23. Rank several building roofs in height order24. Read name on building roof25. Determine length of a building

99

3564

TotaLs

41

2516

ActuaL peljonnance rating

58

1048

Unsatis-factory Satisfactory

Predicted/)erformance

rating

Totals

SatisfactoryUnsatisfactory

SCREENI G TESTS FOR RATING PHOTO [NTERPRETERS

FIG. 2. Examples of stereograms from the Stereo Performallce Test. The Ilumbers "5"and "21" correspond to the questions and answers in the text of this article.

481

19. Identify empty automobile carriers in ware­house parking lot.

20. Explain (a) activity being carried out onship (refueling or discharge of petroleum)(b) construction material of bridge (steelgrating).

21. Iden ti ty part of an au tomobile test rack.22. Identity steel manufacturing plant from ex­

posed raw materials.23. Identify hospital and large school in close

proximity.24. Identify (a) railroad passenger terminal (b)

railroad bridge construction.25. Identify a tunnel under street for pedestrians

and a covered outside walk.

REFERE TeES

Avery, T. Eugene. 1965. Evaluating the potentialof photo interpreters. Photogrammetric Engineer­ing, 31(6): 1051-1059, illus.

Colwell, Robert N. 1961. Some recent develop-

ments of basic significance for photo interpreta­tion. Paper presented at Third Regional Carto­graphic Conference of the United Nations forAsia and the Far East, Bangkok, Thailand, Oct.27-Nov. 10, 27 pp., ill us.

Dixon, W. J., Editor. 1964. BMD BiomedicalComputer Programs. Health Sciences ComputingFacility, Dept. of Preventative Medicine andPublic Health, School of Medicine, Univ. of Cal.,Los Angeles, Cal., 585 pp.

Efroymsen, M. A. 1960. 1I1athematical Methods forDigital Computers, multiple regression analysis.Edited by A. Ralston and H. S. Wi If, JohnWiley and Sons, Inc., New York, N. Y., pp.191-203.

Meyer, John K. and Miller, Robert E. 1960.Validity of photo interpreter predictors for testand training criteria. \\fright Air Dev. Div.,U.S.A.F., Lackland Air Force Base, Texas.Task 7719, Project 17108, 5 pp.

Miller, Daniel R. 1955. Survey of object visualiza-

482 PHOTOGRAMMETRIC ENGINEERING

tion. California Test Bureau, lVT onterey, Cal.4 pp., illus.

Otis, Arthur S. 1954. Otis quick-scoring mentalability test: Gamma test, form FlVT. Harcourt,Brace and World, Inc., New York, N. Y., 6 pp.

Raven, J. C. 1958. Standard progressive matrices:Sets A, B, C, D and E. H. K. Lewis and Co.,London, 60 pp., ill us.

Sadacca, Robert, Martinek, Harold, and Schwartz,Alfred J. 1962. Image interpretation task­status report, V. S. Army Personnel ResearchOffice, Washington, D. C. Tech. Research Report1129, 25 pp., illus.

Sadacca, Robert. 1963. Human factors in imageinterpretation. Photogrammetric Engineering,29(6): 978-988.

Sims, W. G. and Hall, Norman. 1956. The testingof candidates for training as air photo inter­preters. Commonwealth of Australia, Forestryand Timber Bureau, Canberra, A.C.T. 12 pp.,illus.

Steel, Robert G. D. and Torrie, James H. 1960.Principles and Procedures of Statistics. McGraw­Hill Book Co., Inc., New York, N. Y. 481 pp.,illus.

PUBLICATIONS COMMITTEECHAIRMAN-Kalvero N. Maki

Assoc. CHAIRMAN-Leon J. Koso/sky

PHOTOGRAMMETRIC ENGINEERING

EDITOR-G. C. TewinkeltADVERTISING MGR.-Mrs. Barbara N. Muir*

TECHNICAL REVIEw-Bernard J. CoinerSPECIAL ASSIGNMENTs-John T. Gerlach

COVER EDITOR-James R. Shepard

NEWSLETTER

EDITOR-William D. Lynn

t 11612 Michale Ct., Silver Spring, Md. 20904. Tel. 301-496-8140.* 2618 Spencer Road, Chevy Chase. Md. 20015 Tel. 301-JV 7-6107.

NATIONWIDE-INTERNATIONAL

Quality • Service • Accuracy

San Juan, Puerto Rico1105 Fernandez Juncos

Engineering

Santa Barbara, Calif.5760 Dawson Avenue

Minneapolis, Minn. 55426345 Pennsylvania Ave. So.

• Aerial Photography

• Topographic Mapping

• Planimetric Mapping

• Photomosaics