Deconstructing mental processes: Principles and

Deconstructing Mental Processes: Principles and Applications of the Quantified Process Approach

Amir Poreh, PhD Professor, Department of Psychology

Cleveland State University Clinical Associate Professor of Psychiatry

Case Western Reserve University School of Medicine

Cleveland State University College of Sciences and Health Professions

www.csuohio.ed

Historical Background

• American Pragmatic Empiricist Approach – Samuel Kohs

– Yerkes Point Scale Method

– The Wechsler Tests

• Dr. Safford Ackerly and the cases of PA and JP – Arthur Benton - Iowa-Benton Flexible Approach

– Ward Halstead – The Fixed Battery Approach

• The Migration of European Phenomenological Approach – Kurt Goldstein and Heinz Werner

– Edith Kaplan and the Boston Process Approach

– Baxter vs . Temple (2005)

• The Rise of Corporate Tests

• The Quantified Process Approach

American Pragmatic Empiricist Approach

• Ellis Island Psychological Tests

Samuel C. Kohs “success, speed and accuracy each has its own diagnostic importance and in order to make the tests most effective all should, and must, be taken into account in the final score summation”. … Yerkes (1921) claimed that the use of multiple indexes “ introduce a number of difficulties in administration and scoring which may be desirably avoided”.

American Pragmatic Empiricist Approach Yerkes & Bridges Point Scale (1914)

Robert M. Yerkes

• Empirical and a-theoretical

• Consistent with the anti-localization tradition

The Case of PA Dr. Safford Ackerly 1933

• A 37-year old woman with a slowly growing tumor

PA’s Psychological Assessment • Following bilateral frontal

resection – Performance on intelligence

and memory measures remained intact.

– She had difficulties in analyzing visual spatial information

– She “stopped showing respect for her husband’s authority”.

The Case of JP Safford Ackerly and Arthur Benton, 1948

The authors followed this patient since he was 19 y/o until he reached fifty. They observed that his life was “a semblance of normality.”

The Case of JP (continued) • JP was tested by both Arthur

Benton and Ward Halstead.

– His performance on measures of general intelligence and memory remained intact. The Ball and Field

Test (Terman 1911) The Porteus Maze Test

The two measures that were sensitive to JP’s deficits

The Case of JP and the Emergence of the Modern American Neuropsychological Tradition

Ward Halstead Arthur Benton

JP

Fixed Battery Approach

Iowa-Benton Flexible Approach

The H-R Fixed Battery Approach

• Criterion keying approach

(27 tests trimmed to 13)*

*Heinz Werner (1949) based on clinical rather than actuarial judgment W.C Halstead (1947). Brain and intelligence: A quantitative study of

the frontal lobes. University of Chicago Press, Chicago, IL

The Rise and Fall of the H-R Approach

0

50

100

150

200

250

300

350

400

1930-1949 1950-59 1960-69 1970-79 1980-89 1990-99 2000-09 2010-2014

Nu

mb

er o

f P

ub

licat

ion

s

Year of Publication

• TCN=AACN 2005 SALARY SURVEY

The Case of JP and the Emergence of the Modern American Neuropsychological Tradition

Ward Halstead Arthur Benton

JP

Fix Battery Approach

Iowa-Benton Flexible Approach

Iowa-Benton Flexible Approach Arthur Benton

• calls for the administration of a battery of tests that are tailored to the presenting disorder

• developed various new point scale measures

( i.e., prosopagnosia and

visual spatial abilities ) Line Orientation Task Facial Recognition Test

The European Phenomenological Approach and Its Migration to the United States

Dr. Kurt Goldstein (1878-1965) A student of Carl Wernicke (Chief of Neurology at the Berlin Hospital of Moabit)

Heinz Werner (1890-1964) Professor, University of Hamburg

Kurt Goldstein and Testing

“The usual scoring method based on a scale of difficulty which has been standardized on a statistical basis offers no adequate instrument for determining the nature of the degree of impairment in a patient. Unless one takes into account the entire procedure, the specific reasons for the difficulty the patient encounters, one cannot simply read off from a score which task represents a greater difficulty and which a lesser. Any statistical evaluation has to be based upon a qualitative analysis of test results.”

Goldstein, K & Scheerer, M, 1941, page 19

Kurt Goldstein’s Tests

Goldstein -Scheerer Tests

“[Americans] wondered suspiciously about his (Goldstein’s) many-sided interests, which extended from medical research to psychology and philosophy. What was he really they asked: a physician, a psychologist or a philosopher?”

Marianne Simmel (Harvard), Kurt Goldstein, p. 15.

“He was very big in Vienna.”

Professor Heinz Werner

Clark University

Collaborators: Seymour Wapner and Bernard Kaplan

Heinz Werner’s Microgenesis

“Any human activity such as perceiving, thinking, acting, etc., is an unfolding process and whether it takes seconds or hours or days, occurs in developmental sequence” (Werner, 1956)

Norman Geschwind and the Rise of American Behavioral Neurology

The first clinical CT scanners were installed between 1974 and 1976

Chief of Neurology at the Boston VA Hospital

The Boston Process Approach

Edith Kaplan (1924-2009)

Geschwind and Edith Kaplan, Neurology 1962 A human cerebral disconnection syndrome

P.J.K., a 41- y/o white married police officer who underwent a left frontal craniotomy and later developed apraxia of the left side and agraphia of the left arm

Edith Kaplan’s Phenomenological Interpretation of Common Psychological Tests

Measures Edith Kaplan Authored or Co-authored

Authors Test Name

Publisher

Year of

Publication

Kaplan, Goodglass, &

Weintraub, (1983)

Boston Naming Test Lea &

Febiger

1983

Helm- Estabrooks ‎& Kaplan The Boston Stimulus Board Riverside 1989

Helm- Estabrooks ‎& Kaplan Card Board for Memory US patent 1991

Delis, Kramer, Kaplan, & Ober California Verbal Learning Test Pearson 1987

Powell et al. Microcog Pearson 1993

Leach el al. The Kaplan-Baycrest

Neurcognitve Assessment

Pearson 2000

Delis, Kaplan and Kramer The Delis- Kaplan Executive

Function System

Pearson 2001

The Limitations of the Boston Process Approach

Professor, I am sure you were aware when

you accepted the position that it was

publish or perish

Baxter vs . Temple (2005) Rejection of the Boston Process Approach

New Hampshire Supreme Court

• Shelby Baxter suffered from lead paint poisoning

• She was tested by Barbara Bruno-Golden, Ed.D.

• On the day of trial, the defendants moved to exclude the testimony of Dr. Bruno-Golden as being unreliable

• The court concluded that the BPA “While generally accepted in the appropriate scientific literature as a sound clinical approach to evaluating injuries for brain injury, it fails to show that it is generally accepted in the making of a forensic assessment.".

Current Trends in American Neuropsychology

Field of highest degree earned

Clinical psychology 62.1

Clinical neuropsychology 10.7

School psychology 5.0

Counseling psychology 11.2

Neuroscience 1.5

Other 9.5

Total 100.0

A survey of INS, NAN, and APA Division 40 Rabina, Barrb, Burtonc Archives of Clinical Neuropsychology Jan 2004

• 68% favored a flexible battery approach

• 20% favored a flexible approach (i.e., based upon the needs of an individual case)

• 11% favored a standardized battery (i.e., HRNB, LNNB)

• 51.2% employed technicians (Ruff et al., 2006)

Rise of Corporate Tests – The Pearson and PARS Corporations

The Wechsler Tests

1 WAIS-R/WAIS-III 63.1

2 WMS-R/WMS-III 42.7

3 Trail Making Test 17.6

4 CVLT/CVLT-II 17.3

5 WISC-III 15.9

6 HRNB 15.5

7 WCST 11.5

8 ROCF 10.4

9 MMPI/MMPI-2 5.9

US popularity of measures

Cautions and Concerns Regarding the New Wechsler IQ and Memory Scales (Loring and Bowers, 2010)

• Face validity is established by recruiting “advisors” (multiple authors) who later promote the new measures

• The availability of normative data is confused with validity

• Paucity of solid validity studies

• Normative data are proprietary and not open to scrutiny

• In each release previously “new” subtests are discarded due to their failure and replaced with another set of “improved” measures

Memory for Faces WMS-III Family Pictures WMS-III

10 sec

The Wechsler Memory Scale III

Modern Neuroimaging Studies

The Quantified Process Approach Deconstructing mental processes and creating new tests

A philosophical school of neuropsychological assessment that emphasizes the link between brain structures and behavioral functions by quantifying phenomenological aspects of behaviors and validating the data using criterion (brain lesion) and predictive (outcome/ecological) validity studies

Methodologies

• The “Satellite” Testing Paradigm

• The Composition Paradigm

• The Decomposition Paradigm

• Developing new cognitive measures that are consistent with current neurocognitive models and allow for process analysis

Poreh (2000, 2006)

The “Satellite” Testing Paradigm • Simple tasks are introduced to compliment existing tests so as

to clarify a given test performance . The Satellite Test methodology is based on the utilization of difference scores

The Reliability of Difference Scores Decreases as the Correlation Between the Tests Increases

-

The Composition Paradigm

• Data are compiled and new indices that reflect underlying constructs are developed

The Rey Complex Figure Test

ROCF Scoring Systems

Osterrieth (1944): Reproduction Type

• Elements with which the copy started • Drawing the rectangle separate from the details • Piece-by-piece versus holistic approach

Categorical scale ranging from I to 5

Visser ( 1973 ): Total Score

• Drawing each straight line continuously • Completeness of each line • Order in which a subset of lines are drawn

Numerical scale ranging from 0 to 30

Binder (1982): Configural Score

Drawing continuously or contiguously each of the following: horizontal bisector, vertical bisector. two main diagonals. vertices of the large pentagon

Numerical scale ranging from 0 to 5

Bennett-Levy (1984): Strategy Total Score

Drawing all straight lines continuously Drawing symmetric components contiguously Element with which copy is started

Numerical scale ranging from 0 to 34

Shorr et al. (1992): Perceptual Cluster Ratio

Drawing continuously or contiguously 20 lines and line junctures, most of which are i11 the framework: corrected for number of lines drawn

Numerical scale ranging from 0 to 100

Bylsma et al. ( 1995): Qualitative Score

Drawing I3 multi-line elements with contiguous lines; higher weights are given for the large rectangle. bisectors, and diagonals• Order in which the large rectangle, bisectors, and diagonals are drawn•

Numeric scale ranging from 0 to 24

ANGELA K. TROYER AND HEATHER A. WJSHART 1997

Development of Theory Driven Empirical Indices

6 stages of a piecewise copying Bennet Levin 1984

Chi (1997) A detailed analysis of the

task to determine the problem space so as to develop a computational scoring model

Define a solution path to identify which sequence of operators a particular subject utilizes

Arnold Gesell

ROCF Scoring Systems

ANGELA K. TROYER

Osterrieth (1944): Reproduction Type

• Elements with which the copy started • Drawing the rectangle separate from the details • Piece-by-piece versus holistic approach

Categorical scale ranging from I to 5

Visser ( 1973 ): Total Score

• Drawing each straight line continuously • Completeness of each line • Order in which a subset of lines are drawn

Numerical scale ranging from 0 to 30

Binder (1982): Configural Score

Drawing continuously or contiguously each of the following: horizontal bisector, vertical bisector. two main diagonals. vertices of the large pentagon

Numerical scale ranging from 0 to 5

Bennett-Levy (1984): Strategy Total Score

Drawing all straight lines continuously Drawing symmetric components contiguously Element with which copy is started

Numerical scale ranging from 0 to 34

Shorr et al. (1992): Perceptual Cluster Ratio

Drawing continuously or contiguously 20 lines and line junctures, most of which are i11 the framework: corrected for number of lines drawn

Numerical scale ranging from 0 to 100

Bylsma et al. ( 1995): Qualitative Score

Drawing I3 multi-line elements with contiguous lines; higher weights are given for the large rectangle. bisectors, and diagonals• Order in which the large rectangle, bisectors, and diagonals are drawn•

Numeric scale ranging from 0 to 24

The Five Point Test

FPT – A Nonverbal Fluency Test

Prof Marianne Regard

Ruff’s Revision of the Five Point Test (employing a Satellite Test Paradigm)

• Failure to analyze all of the permissible operators results in interpretation problems

• Reliability 0.58 to 9.69 (Ruff 1996)

1 min per section

• D-KEFS Design Fluency relies on motor planning and cognitive flexibility. The two additional trials do not contribute to performance (Suchy, Kraybill, Gidley, Larson, 2010)

• Unlike the original design fluency tests (Jones-Gotman and Milner, 1977), the D-KEF Design Fluency is not sensitive to left-frontal lobe damage

Baldo, Shimamura, Delis and Kaplan, 2001 1 min per section

D-KEFS Design Fluency

Clinical Examples

Total Repeat Err Unique # elements Addition Sub. Rotation

18-28 (N=39)

41.3 3.3 0 38.0 20.3 2.5 .67 12.36

30-60 (N=7)

36.3 5.6 0 42.0 14.9 1.7 .86 7.14

65-75 (n=20)

34.0 2.8 .10 31.0 16.6 1.55 .20 9.6

75-90 (N=19)

20.3 3.0 1.7 17.3 8.0 .72 .33 2.9

Verbal Fluency Tests

Fluency

Comprehension

Repetition POOR INTACT

Poor Good

Poor Good Good Poor

Anomia Conduction

Transcortical

Sensory Wernicke

Transcortical

Motor

Broca

Transcortical

Mixed

Global

Poor Good

Poor Good Poor Good

Benson 1979

Verbal Fluency Tests

• Discrepancies in phonemic and semantic fluency are an indication of anterior as opposed to posterior damage to speech areas. Butters et al., 1987; Monsch et al., 1992; Gourovitch, Goldberg,

& Weinberger, 1996; Corcoran & Upton, 1993 -

• The above conclusion has not always been supported. Joanette & Goulet, 1986

Benton (1968) - Phonemic fluency Rosen, (1980) - Semantic fluency

Applying the Clustering and Switching Methodology

• Clustering (Laine, 1988) - the generation of consecutive words within a particular subcategory.

• Switching (Gruenewald & Lockhead, 1990) - the ability to shift between subcategories.

art differ in syllable

arm bat differ in vowel

bit sand rhymes

stand sum homonyms

some

pigeon gull nightingale canary

Farm animals

insects

birds

Semantic Phonemic

hen rooster goose

fly cockroach black beetle

Avg. cluster 7/3=

2

2

3

Applying the Clustering and Switching Methodology

• Troyer et al. (1997) - young adults generate more words and switch more often. Older adults produce larger clusters on phonemic fluency tasks than younger adults.

• Troyer et al. (1998) - Phonemic fluency switching is impaired among frontal lobe damaged patients. Semantic-fluency clustering is impaired among temporal lobe damaged patients.

• Robert et al. (1998) – Schizophrenics • Rich (1999) - Longitudinal study of

patients with Huntington's disease.

pigeon gull nightingale canary

Farm animals

insects

birds

Semantic

hen rooster goose

fly cockroach black beetle

Avg. cluster 7/3=

The Decomposition Paradigm

• An approach which investigates the relationship between test items of a given measure according to underlying facets, resulting in the development of new subscores

FSSA map for RHI patients during the recall stage for right-, middle-, and left-spatially located elements

FSSA map for RHI patients during the copy stage for global and local elements

Examination of the Global and Local Features of the Rey Osterrieth Complex Figure Using Faceted Smallest Space Analysis

Decomposing The Trail Making Test

Decomposition of the Trail Making Test Amir Poreh, Ph.D., Ashley Miller, M.A., Philipp Dines, M.D., Ph.D., Jennifer Levin, Ph.D. (2012)

Only the last section of Trails B (11, K, 12, L, and 13) correlates with other measures of executive functioning

Latency per section

0,06

0,38 0,52

1,72

1 2 3 4

Patterns of TMT Performance among Dementia Patients (n=820)

• Type I - Failure on the 1-A-2-B-3 C

• Type II - Failure on 11-K-12-L-13 (Most common)

• Type III – Errors on the 1-A-2-B-3 C, improvement and then errors on H-9-I-10-J

Eye Tracking and the TMT

Decomposing the Rey Auditory Verbal Learning Test

• RVLT and CVLT sensitivity to left temporal lobe epilepsy

Loring et al. (2008) - 204 patients

Test Focus n M SD t-value P value Cohen’s d

AVLT Left Right

91 98

42.9 47.7

10.6 9.9

3.21 0.002 .47

CVLT Left Right

113 99

43.8 43.1

11.1 9.9

2.20 0.03 .30

The increased sensitivity of the AVLT to left temporal abnormality may reflect its increased sensitivity to deficits in relational learning

Decomposing the Rey Auditory Verbal Learning Test

Decomposing Rey Auditory Verbal Learning Test

Larry R. Squire and Arthur P. Shimamura 1986

• Rey Auditory Demo

The Quantified Process Approach – A Blueprint for Developing New Measures

Successful attempts in decomposing tests depends on whether a test is “pure”. Namely, it relates to a specific neuroanatomically based model

Assessment of executive functions: Review of instruments and identification of critical issues.

Chan et al. 2008.

Simulation of Real-life Decision Making

• The Iowa Gambling Task (Antoine Bechara et al. 1994)

Ventromedial prefrontal cortex

Tanabe et al. 2007 Bechara,Damasio, Tranel, & Damasio1997

Logical Memory – Are we measuring what we need to measure?

WMS 1945 I immediate recall of two stories WMS- R (Russell, 1974, 1988) 2 stories followed by delayed recall and recognition (yes no) WMS-III The 2nd story is read twice WMS-IV Two stories with the 1st read twice and a new geriatric version (1st story read twice)

The Poreh Geriatric Complex Figure

POREH’S GERIATRIC COMPLEX FIGURE

THE RBANS LIKE COMPLEX FIGURE

The Poreh Nonverbal Memory Test

• The Search for a “Pure” Visual Memory Test: Pursuit of Perfection. (Heilbronner, 1992)

• Factor analytical studies rebuff the validity of existing nonverbal memory measures – WMS Visual Reproduction

– Revised Visual Retention Test (BVRT: Benton, 1974)

– Memory for Designs (MFD: Graham & Kendall, 1960)

• Task Demand Characteristics – Test should include a delayed interval (Heilbronner 1992)

– Repetition is essential for assessing learning (Poreh, 2005)

• Nature of the Test Stimuli – Geometric designs lend themselves to being easily verbalized – (De

Renzi 1968; Cermak & Tarlow, 1978)

• Influence of Visuoconstructive Abilities – Tests that require visuoconstructive response (drawing) reduce the

proportion of variance accounted for by nonverbal memory skills

Criteria for Existing Measures

Morris Water Maze Spatial memory test in rats, developed in 1981 by Richard Morris.

Poreh Nonverbal Memory Test

The PNMT test is composed of a deck of cards with an array of squares. The final version of the test includes 4 cards with high-local cues and 5 cards with random arrays of 10 squares

Poreh Nonverbal Memory Test Alternate Version

Data (rodent) from the Morris Maze Test

Age Group PNMT RAVLT

Linear Quadratic Logarithmic Linear Quadratic Logarithmic

18-20 R=.921 F=16.8 p=0.03

R=.971 F=33.6 p=0.03

R=.968 F= 92.0 p= 0.002**

R=.901 F=27,3 p=0.01

R=.997 F=361.3 p=0.003**

R=.991 F=316.9 p=0.0001**

Rey Immediate

Recall-T Score

Rey Delay

Recall-T Score List A Delay

RAVLT

1-5

RAVLT

Trial 1

Poreh-Total-Delay -.288* -.454** -.288* -.236 -.120

Poreh 1 - 5 -.081 -.252* -.192 -.356** -.324

Poreh Trial 2-1 -.023 -.081 -.052 -.373** -.398**

Correlations for PNMT, ROCF and RAVLT

Visual Spatial Deficits in Abstinent Alcoholics

R² = 0,9379

R² = 0,9097

0,00

5,00

10,00

15,00

20,00

25,00

30,00

0 1 2 3 4 5 6

Score

Trial

Control

Experimental

Log. (Control)

Log. (Experimental )

Control Group N=32

ROCF 3min

ROCF Delayed

PNMT Complex Total -0.33** -0.38**

PNMT Simple Total -.260 -0.39**

Experimental Group (N=35)

PNMT Complex Total -0.49* -.540*

PNMT Simple Total -0.46* -0.43*

* 0.01 level (1-tailed) ** 0.05 (1 tailed)

PNMT Learning Curve for Complex Configurations

Signal-Detection Theory

• Following Polack & Norman (1964) and Green & Swets (1966) as well as Shum et al. (2010), scoring of the PNMT can also be achieved using a nonparametric measure of recognition performance (Proportion of correct acceptance or p(A).

p(A) = 0.5 (1 + HR – FP)

Where HR represents hit rate and FP false positive rate (FP)

Process Scores: Average Distance Between Target and First Click

Distance between target and 1st click

Process Scores – Average Distance Between Target and First Two Clicks

1

2 3

Conclusions

The future of neuropsychological assessment depends on the ability of researchers and clinicians to : (1) Develop and implement computer

programs that enable efficient access and management of large sets of data

(2) Alter tests to match advances in neuroscience

THE END