LSMSSkills Measurement

TrainingRachid Laajaj (Paris School of Economics -

INRA)

What are skills and which skills? Cognitive, non cognitive and technical skills Why does it matter to measure skills? How to asses the quality of measurement

instruments? What are the gaps and challenges? The LSMS-ISA attempt to fill some of these

gaps

Outline

COGNITIVE SKILLS (Hard skills) NON-COGNITIVE SKILLS (Soft skills) TECHNICAL SKILLS (task specific)

“Nature vs nurture” is not a correct distinction but cognitive skills are affected more at early ages and non-cognitive skills more at later ages.

Comples interaction in the generation of these skills

Definition of the skills by type

The terms IQ, general intelligence, general cognitive ability, general mental ability, or simply intelligence are often used interchangeably, reflecting the ability to solve abstract problems.

Since the different aspects of cognition are highly correlated, a general intelligence factor labeled “g” is a variable that summarize positive correlations among different cognitive tasks.

Early 20th century, psychologist Charles Spearman noticed positive correlation across similarly unrelated school subject and attributed it to general intelligence.

Cognitive Skills

Digit Span test: forward or backward. Can be auditive or visual.

Raven’s Progressive Matrices Math questions Reading tests Vocabulary tests

From more to less pure cognitive test

Measurement of Cognitive Skills in the field: Examples of tests

Non cognitive skills are personality traits that are weakly correlated with measures of intelligence, such as the IQ index (very broad)

Heckman highlights the importance of perseverance, motivation, time preference, risk aversion, self-esteem, self-control, preference for leasure

Five factor model is a broadly accepted taxonomy Renewal of interest for entrepreneurship,

locus of control, aspirations, etc.

Non-Cognitive Skills

Technical skills are the basic knowledge required to perform a task: very field specific by definition.

Why using proxies rather than the knowledge that really matters? Example with the use of seeds and fertilizer in Mozambique.

Technical skills

Good body of evidence in cognitive skills, mostly but not exclusively on developed countries.

Most evidence on the cognitive side: Hanushek and Kimko (2000) use math and science test scores, and find it to predict growth much better than years of education.

Numerous studies establish that measured cognitive ability is a strong predictor of schooling attainment and wages, conditional on schooling (Cawley, Heckman, and Vytlacil 2001).

Increasing number of studies in developing countries.

Does it Matter? What are the evidence? Cognitive skills

Heckman, Stixrud & Urzua find that a change in non-cognitive skills from lowest to highest level has an effect on behavior comparable to or greater than a corresponding change in cognitive skills.

The marshmallow experiment:

The Abecedarian projectChicago Child-Parent Center

Trainings > limited success

Does it Matter? What are the evidence?Non-Cognitive skills

More evidence on cognitive studies, less on non-cognitive ones, and very little in technical skills.

Very little in developing countries and rural areas, where less data is available

Virtually nothing on intra-household skills Very limited data that is comparable across

developing countries

What are the gaps?

Related to the measurement:- Measurement errors- Imperfect proxies- Applicability to field conditions

Related to evidence of its importance Omitted variables Reverse causality Challenges in the use of instruments (isolate

one improvement, change in the short run)

Technical difficulties

Consistency across time:High correlation if you replicated the measure

within a period short enough that it should not have change

Validity: Are you measuring what you intend to measure?

Should correlate well with other measures of the same skill. Should predict well related behaviors. Ex: back to the locus of control

What is a good measurement?

Our LSMS-ISA initiative: Test and re-test of questions for consistency Identify the best tradeoff between time and

precision of the measurement. Will investigate intra-household skills and their

complementarities

Combine it with a randomized technology adoption intervention: predictive role of the skills:

- The importance of “observing the unobservables”- Which one matters for technology adoption- Which one matters for productivity

Toward the improvement of skills measurement in developing countries

Interesting new work on how interventions shapes behaviors.

Examples:- Macours and Vakis (2008): Aspirations and

leadership- Laajaj (2012): Endogenous time horizon

More coming soon!

Feedback from intervention to behavior?

IQBack

Memory span: longest list of items that a person can repeat back in correct order immediately after presentation on 50% of trials. Called digit span when numbers are used. Measures the short term memory.

Digit Span Test

Back

60 Multiple choice questions in order of difficulty, test the reasoning ability.

Raven’s Progressive Matrices (1)

Raven’s Progressive Matrices (2)

Raven’s Progressive Matrices (3)

Raven’s Progressive Matrices (4)

Back

The Big Five Personality traits

NEO PI-R

Neuroticism Extraversion Openness to Experience Agreeableness Conscientious-

nessAnxiety Warmth Fantasy Trust Competence

Angry Hostility Gregariousness Aesthetics Straightforwardness Order

Depression Assertiveness Feelings Altruism Dutifulness

Self-Consciousness Activity Actions Compliance Achievement

Striving

Impulsiveness Excitement-Seeking Ideas Modesty Self-Discipline

Vulnerability Positive EmotionsValues Tender-Mindedness Deliberation

Big Five personality traits and their facets

Back

For each one of the following statements, please tell me which one you most agree with:

E191 - Each person is primarily responsible for his/her success in life |___|2 - One's success or failure in life is a matter of his/her destiny

E20

1 - To be successful, above all, one needs to work very hard

|___|2 - To be successful, above all, one needs to be lucky

Locus of Control

Back1

Back2

Technical Knowledge Questions for Seeds and Fertilizer Use

Back

Returns of cognitive abilitie in Developing countries (Hanusheck & Woesman 2008)

Back

The precision vs time tradeoff

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

0.2

0.4

0.6

0.8

1

1.2

Correlation with complete indicator as a function of the number of questions (illustration)

Correlation with complete indicator

Back