I/O chapter 4

Criteria:

Standards for Decision Making

Selection, Criteria,

Job Analysis, Job Evaluation

Objectives

• Clarify the use of criteria and selection process

• List the uses of job analysis information

• Describe the sources and ways of collecting job analysis

information

• Discuss the different job analysis methods

• Describe the evidence for reliability and validity of job

analysis methods

• Explain how job evaluation is used to set salary levels for

jobs

Selection

• What is selection? – Using scientific methodology to choose one alternative (job candidate) over another.

• Job Analysis • Measurement • Statistics

• Why is selection important? – Decreases the likelihood of hiring “bad” employees – Increases the likelihood that people will be treated fairly when hiring decisions are made

• Reduces discrimination • Reduces likelihood of discrimination lawsuits

• What do I/O psychologists need to know about selection? – How to select predictors of job performance (criteria problem) – How to accurately indentify and validate predictors for specific jobs (job analysis)

• Rely on cognitive and personality variables

– How to reliably and validly measure these predictors – How to use these predictors to make selection decisions

Criteria

• Standards used to help make evaluative judgments about objects, people, or events.

Example:

What is good Teaching? Student A: Preparedness, Relevance, Clarity

Student B: Enthusiasm, Inspiration, Ability

Note: There is a need to level-off or agree in one general criteria to define the context of ‘good teaching’

Criteria

• Criteria - standards used to judge the quality of (discriminate among) alternatives.

• For I/O psychologists, this means judging the quality of employees, programs, and units in the organization

Points to Ponder

• Values and taste also dictate people’s choice of criteria.

• Even people who use the same standards in making judgements do not always reach the same conclusion.

Conceptual vs Actual Criteria

• Conceptual Criterion

– The theoritical standard that researchers seek to understand through their research.

– An abstract idea that can never be actually be measured.

• Actual Criterion

– The operational or actual standard of measure or assess.

– Served as measure of the conceptual criteria.

Criteria of College Student

Conceptual Criteria Actual Criteria

Intellectual Growth Grade point average or QPI

Emotional Growth

Adviser rating of emotional maturity

Citizenship Number of volunteer organization joined in college

Conceptual Criterion

Actual Criterion

CRITERION

DEFICIENCY

CRITERION

RELEVANCE

CRITERION

CONTAMINATION

• Criterion Deficiency: The degree to which the actual criteria fail to overlap the criteria – that is, how deficient the actual criteria are in representing the conceptual ones.

• Criterion Relevance: The degree to which the actual criteria and the conceptual critria coincide.

• Criterion Contamination: The part of the actual criteria that is unrelated to the conceptual criteria.

Intellectual Growth

Grade point Average

Issue: Good College Student

What If:

Student A know nothing about

the subject while student B

has prior knowledge. By the

end of the learning period,

student A might gained more

intellectual knowledge but

student B might get a higher

grade since he knew the topic

at hand. Therefore, using GPA

as our criterion would falsely

conclude which student grew

more intellectually.

Emotional Growth

Adviser’s Rating

Citizenship

number of Volunteer Org.joined in College

Example: College Success

College Sucess

Extra-curricular

Involvement

Co-curricular Involvement

Emotional Stable

Above Average QPI

Numerous Relationship

(BF/GF)

Spiritually Developed

Have latest gadgets

Morally Up-right Physically

Healthy

Positive Self-

Esteem

Travel Abroad

Classification of Criteria I/O Psychologists try to

choose criteria that assess performance excellence.

Criteria are typically classified in one of two ways

Objective

Subjective

More easily quantifiable

Production

Number of touchdowns

Number of units produced

Sales

Tenure/Turnover

voluntariness

functionality

Absenteeism

Accidents

Theft

Judgements made about

employees performance

general factor

(effectiveness)

specific factors

quantity of work

quality of work

Note: More complex jobs

require more criteria for

effective evaluation

Illegal Criteria Title VII of the 1964 Civil Rights Act prohibits using

selection practices that have an unequal impact on members of a different: Race

Color

Sex

Religion

National Origin

Types of Illegal Discrimination Disparate Treatment (Opportunities)

Discrimination decisions based on one of five prohibited categories

Disparate Impact (Outcomes) Illegal discrimination is any practice (without a business justification) that has

unequal consequences for members of protected groups.

Roger Parloff, Fortune senior editor: Though disparate treatment and disparate impact cases are both aimed at

eradicating the same thing, there is potential tension between them. The goal of disparate treatment cases is to guarantee every worker equal

opportunity, but not equal outcomes. The focus of disparate impact cases is on equal outcomes.

If one pursues equal outcomes too single-mindedly, one can compromise the principle of equal opportunity by inducing the use of quotas.

Determining Disparate Impact The 4/5ths Rule

Disparate impact occurs if the selection ratio for any minority group is less than 4/5ths of the selection ratio of the majority group

100 male applicants

20 males selected

50 female applicants

20/100 = .20

.20 * 4/5ths(.80) = .16

At least 16% of people from minority group should be selected using a

given procedure.

50 * .16 = 8

At least 8 females should be selected

Summary Criteria

Reliable and valid predictors of job performance.

All criteria suffer from:

Deficiency

Contamination

Criteria typically classified as:

Objective

Subjective These labels can be misleading

There are several illegal criteria

There are two types of illegal discrimination

Disparate treatment

Disparate impact

Job Analysis

Our Textbook: Definition

• Job analysis – A formal procedure by w/c the content of a job is defined in

terms of tasks performed and human qualifications needed to perform the job.

– Harvey (1991) definition: the collection of data describing observable job behaviors performed by workers, including both what is accomplished as well as what technologies are employed to accomplish the end results, and verifiable characteristics of the job environment w/ w/c workers interact, including physical , mechanical, social, and informational elements.

– A procedure useful in identifying the criteria or performance dimensions of job.

What Is Job Analysis?

• Job analysis is the method for describing jobs and/ or the human attributes necessary to perform them – The procedure must be systematic

– A job is broken into smaller units

– The analysis results in some written product, either electronic or on paper

– Job analysis techniques can be used to collect information that job oriented or person oriented, depending on the purpose of the job analyst

• The Job-oriented Approach – Provides information about the nature of tasks does on the job

• Describes common features that cut across tasks that provide a picture of what people do on a job


– Tasks can be divided into a hierarchy in which higher-level descriptions are broken down into smaller pieces of the job

– Levine (1983) divides the major functions of job into 4 levels of specificity: • 1. Duty 2. Tasks 3. Activity 4. Element

– Duty is a major component of a job

» Accomplished by performing one or more associated tasks

– Task is a complete piece of work that accomplishes some particular objective

– Task can be divided into activities which are the individual parts that make up the task

– To accomplish this activity, a number of very specific actions, or elements are involved

– Contain a great deal of very specific information about what happens on a particular job

– The level of job actions produces a long and detailed report


• The Person-Oriented Approach

– Description of the attributes, characteristics or KSAOs necessary for a person to

perform a particular job successfully

– KSAOs are the knowledge, skills, abilities and other personal characteristics

necessary for a job

• First 3 characteristics mainly on job performance itself

• Other relate to job adjustment and satisfaction

• Knowledge is what a person needs to know to do a particular job

• Skill is what a person is able to do an the job

• Ability is a person’s aptitude or capability to do job tasks or learn to do job tasks

• Other personal characteristics relevant to the job that is not covered by the other three

– KSAO is an attribute or characteristic that a person needs in order to do a

particular task or tasks

Purposes of Job Analysis?

• Career Development

– Career ladder- a progression of position is established for individuals who acquire the necessary skills and maintain good job performance

– Not everyone climbs to the top of the ladder

• Limited opportunities for promotion and inability to achieve the necessary KSAOs

– Job analysis provide a picture of the KSAO requirements for jobs at each level of career ladder

• Legal Issues – Laws prohibiting discriminatory employment practices, especially in the hiring of

employees

– Job analysis provides a list of relevant KSAOs that can be used as the basis for hiring in place of irrelevant personal characteristics

– Legal concept in US employment is that of essential function, which is an action that must be done on a job, especially deciding whether to hire a person with a disability


– The nonessntial function ia an action that might be done occasionally, but is

not important for a person in that position to do

– Job analysis is used to identify essential functions and KSAOs and thus can

help ensure that decision affecting people are based on personal factors that

are job relevant

– When KSAOs are derived from a properly conducted job analysis, employer

actions based on those KSAOs are likely to be legal

• Performance Appraisal

– A well-designed performance appraisal system will be used on a job analysis

– Job-oriented analysis provides a list of the major components of a job, which

can be used as dimensions for performance evaluation

– The behavior-focused performance appraisal methods are based on a job

analysis


– Critical incidents that represent different levels of job performance, from

outstanding to poor

• The good incident would describe something a person did that worked well

• Selection

– Person-oriented job analysis should be the first step in the design of an employee-

selection system

– KSAOs for a job are identified, procedures can be chosen to determine how well job

applications fit the requirements for the job

– A person-oriented job analysis produces a list of KSAOs for a particular job

• Expected to have at the time of hiring

• Characteristics that will be developed on the job through experience and training

• Training

– KSAOs that applicants do not have when they apply for a position are areas for

training after they are hired


– Training program is based on a through analysis of the KSAO

requirements for a job

– Training efforts might be directed if the characteristics can be

acquired

• Research

– An additional use of job analysis information is in research

– Determine the role of job requirements or task characteristics in

organizational phenomena

– Ranging from employee motivation and performance to health

and safety

Sources of Job Analysis Info?

• Who provides the information?

– SME (Subject Matter Experts)

– Job analysis information comes from one of four sources:

a. Job analysts c. Supervisors

b. Job incumbents d. Trained observers

– Job analysts and trained observes actually do the job or spend time

observing employees doing job and translate those experiences into a

job analysis

– Incumbents and supervisors are considered subject-matter experts,

people with detailed knowledge about the content and requirements

of their own jobs or the jobs that they supervise


• How do people provide job analysis information?

– Perform Job

• Benefits:

– The job analyst can perform the tasks as an employee would or under simulated conditions

– The analyst gains insight into the nature of the tasks and how they interrelate

– Provides an apperception for the context in which people do their jobs

• Not often used:

– Experiencing the job by doing it can be costly and time-consuming

– Some jobs are dangerous, particularly for an inexperienced person

– Dose not clearly indicate that tasks can differ among employees with the same job title

– Observe

• Collect information about a job is to observe people doing it

• Observing employees can give insights into the context in which job tasks are performed

• It can also be expensive and time-consuming


– Interview

• Interviewing subject-matter experts who are familiar with the job

• Interviews are carried out by job analysts or trained interviewers

• Used to generate listed of all tasks and activities done by everyone who has

the same job title

• Other tasks might be performed by every employee, but only on race occasions

– Questionnaire

• Most efficient means of collecting job analysis information

• No other technique can provide as much information about jobs with as little

effort on the part of the job analyst

– Multiple methods

• Often used so that the limitations of one are offset by the strengths of

another

Methods Job Analysis?

• Job Components Inventory (JCI) – Allows for the simultaneous assessment of the job requirements and person’s

KSAOs

– The inventory includes KSAOs for both jobs and individuals

– 5 components of job features are represented: • Use of tool and equipment

• Perceptual and physical requirements

• Mathematics

• Communication

• Decision making and responsibility

• Functional Job Analysis – Provides both a description of a job and scores in several dimensions for the job and

potential workers

– Procedure can be used to make comparisons among jobs


• Occupational Information Network – Is a computer-based resource for job-related information on approximately

1,100 groups of jobs sharing common characteristics

– Occupation characteristics deal with other kinds of information are concerned with characteristic of job tasks

– Look up a particular job and get a description and detailed information about 6 domains

• Position Analysis Questionnaire – Contains 189 items dealing with the task requirements or elements of jobs

– The elements of the PAQ are general and allow comparisons of different jobs on a common set of dimensions or KSAOs


– Cover a wide variety of task requirements

• The inputting and processing of information

• The use of equipment and tools

• General body movement

• Interpersonal interaction

• Work context

– PAQ generates a standard list of KSAOs, jobs can be compared on their KSAO requirements

• Task inventions – Is a questionnaire that contains a list of specific tasks that might be done on the job that

is being analyzed

– Rating might be made on dimensions:

• Amount of time spent doing the task

• Critically of the task for doing a good job

• Difficulty of learning the task

• Important of the task


• Combination Job Analysis Method (C-JAM)

– Both interviews and questionnaires to collect information about

KSAOs and tasks

– Produces a detailed picture of the KSAOs for a job and the task

performed

• Choosing a Job Analysis Method

– Job analysis experts to rate the effectiveness of seven job

analysis methods for 11 purposes

– Choice of method requires consideration of several factors,

including cost and purpose

Our Textbook: Job Analysis Procedure

CLERICAL

Secretary

Person 1: Task A, B, C

Person 2: Task A, B, C

Receptionist

Person 3: Task D, E, F

Person 4: Task D, E, F

Data Entry

Person 5: Task G, H, I

Person 6: Task G, H, I

Job Family

Job

Position

Task

A. Types correspondence

B. Schedule meetings C. Takes dictation

D. Answers telephone

E. Greets Visitors F. Maintains register

G. Enters data H. Updates files I. Reconciles

statements

• Task Oriented Procedure – A procedure or set of operations in job analysis designed to

identify important or frequently performed tasks as a means of understanding the work performed.

• Functional Job Analysis – A method of job analysis that describes the content of jobs in

terms of PEOPLE, DATA, THINGS.

• Worker-oriented procedure – A procedure or set of operations in the job analysis designed to

identify important or frequently utilized human attributes as a means of understanding the work performed. The use of KSAOs.

Our Textbook: Job Analysis Procedure

• Interviews (SMEs)

• Direct Observation

• Questionnaires or inventories

• Taxonomy

• Position Analysis Questionnaire

• Occupational Information Network

Our Textbook:

Job Analysis Collection of Information

Reliability and Validity of Job Analysis

Information

• Reliability – Test-retest reliabilities ranged from 0.68 to 0.90

– Reasonably consistent in their job analysis rating when they repeated them over time

– Correlations among rating of different people ranged from 0.46 to 0.79

• Validity – The best evidence for the validity of job analysis comes from

studies that compared different methods or sources information

– Job analysis ratings might be less valid than I/O psychologists usually assume

– Need to improve job analysis procedures, the various methods are important tools used by I/O psychologists

Choosing Predictors of Job Performance

When selecting new employees, I/O psychologists use

criteria that will identify effective on-the-job

performance

Performance is a function of the following:

Knowledge

Skills

Abilities

Motivation

Situational Constraints

Performance = (KSA)*Motivation – Situational Constraints

Job Analysis

Describes:

the tasks that are performed

type of work

tools used

working conditions

human qualities (KSAOs or competencies) needed to

perform the work

Tells us what tasks people do and the knowledge,

skills and abilities they need to accomplish those

tasks.

Purposes of Job Analysis

Recruiting

Career development

What does it take to move up?

Legal defense

Essential functions: What tasks must be done?

Performance appraisal

Selection

What sorts of people should we hire?

Training

What knowledge and skills are needed?

Research

Job-Oriented Job Analysis

Job components (for a carpenter)

Duty: construct houses

Task: build kitchen cabinets

Activity: assemble cabinets

Element: drill holes

Person-Oriented Job Analysis

KSAO’s (for a carpenter)

Knowledge: Have information to do a task

Skill: Practiced act or behavior.

Ability: Stable capacity to do task.

Other personal characteristics: personality, interests, etc.

Examples Of KSAOs For Different Occupations

Job

Knowledge

Skill

Ability

Other Personal

Characteristics

Lawyer

Constitutional

rights

Writing clearly

Communica-

tion

Willingness to

work long

hours

Nurse

Surgical

procedures

Drawing blood

Remain calm

in a crisis

Lack of

squeamishness

in the sight of

blood

Plumber

Pipe design

Soldering

joints

Hand-eye

coordination

Willingness to

get dirty

Police

Officer Knowledge of

legal arrest

procedures

Writing clearly

Vigilence Willingness to

risk personal

safety

Hiring the Best

Job: College Professor

What are the major duties of a college professor?

What tasks are performed to complete each duty

Develop a set of KSAO’s necessary for these tasks.

should be useable for recruiting and evaluating

Challenges?

What other information would you want? How would you

get it?

Data Collection Approaches

Questionnaire

diaries

Interview

critical incidents

Observation

Analyst does work

Who do you collect data

from?

Subject Matter Experts

-incumbent

-supervisor

-co-worker

Occasions for Formal Job Analysis

Major Restructuring

after dramatic growth

downsizing

new positions

Large Selection Procedure

Dramatic changes in technology

Passage of Time

Job Evaluation

Job Evaluation

• Refers to a family of quantitative techniques that used to determine

the salary levels of jobs

• Job evaluation determines are relative salaries for different jobs by

mathematically combining job information

• Point method

– First, a panel determines the compensable factors for the job

– Second, a panel judges the degree to which each job has each compensable

factor

• Quantitative scale so that each job has each compensable factor

– Third, the points for the factors are summed for each job

– Fourth and final, the plot the actual salaries for each job against the point

totals for each job

– The point system is just one of many different job evaluation methods

Job Evaluation

• Comparable Worth

– Used to demonstrate pay discrimination against women

– Means that different but comparable jobs should be paid the

same

• Held predominantly by women contribution as much to the organization

as jobs held primarily by men, the salaries for the jobs should be the same

– Using the mathematical procedures to calculate how much

adjustment each of the underpaid jobs should receive

• Might undervalue lower-paid predominantly women-held jobs and

overvalue the higher-paid, predominantly men-held jobs

Future Issues and Challenges

• Job analysis is one of the most frequently used tools of practicing I/O psychologists – Concerned with developing new methods rather than with the validity of old methods

• Rater training is another area of possible research – Better understanding of how people make their ratings would suggest useful ways of

training raters

• Increased call for organizations to keep their employee actions, such as promotions and selection – Ensured that decisions about whom to hire or promote will be based on the KSAOs for

a job

• Electronic tools in conducting job analysis – More accurate picture of the time spent in various activities

• More focus on describing competencies rather than on what are often minimum KSAO requirements found in a typical job analysis

Our Textbook: Job Evaluation

• Job Evaluation

– A procedure for assessing the relative value of jobs in an organization for the purpose of establishing levels of compensation.

• External Equity

– A theoretical concept that is the basis for using wage and salary surveys in establishing compensation rate.

Our Textbook: Job Evaluation

• Internal Equity

– A theoretical concept that is the basis for using job evaluation in establishing compensation rate.

• Compensable Factor

– A dimension of work used to assess the relative value of a job for determining compensation rates.

– In practice: Effort, Skills, responsibility, working condition

– Others: Know-how, problem solving, accountability, additional compensable elements

Our Textbook: Job Performance Criteria

• Objective Performance Criteria

– A set of factors used to assess job performance that are (relatively) objective or factual in character.

• Subjective Performance Criteria

– A set of factors used to assess job performance that are product of someone’s (e.g. supervisor, peer, customer) subjective rating of these factors.

Our Textbook: 8 MAJOR Job Performance Criteria

1. Production

2. Sales

3. Tenure or Turnover

4. Absenteeism

5. Accidents

6. Theft

7. Counterproductive Workplace Behaviors

8. Customer-Service Behavior

Predictors:

Psychological

Assessments

Objectives

• To increase your understanding of psych

testing and its use in the workplace,

including:

– The benefits of using psych testing

– How psych tests can be most effectively used in

the selection and development process

– The types of tests available

– The limitations of psych tests

• What is testing?

• Why do we have so many tests?

• What are the pros and cons to

testing?

• How can we use testing to

improve . . .?

• What types of tests do we take?

Sept 2009 © Copyright 2009 Psylutions Pty Ltd.

Commercial in Confidence.

What is the cost of a bad hire?

• Scenario – You have interviewed your preferred candidate. He/she looks good on paper, interviews well and seems to fit the role requirements.

• You have been looking to fill the role for a while and are keen to make a placement.

BUT

• This role has high exposure in the organisation, managing a large customer focused project.

• You have been unsuccessful in finding the right candidate until now, the project is experiencing delays which is ultimately costing you money.

• The candidate looks good and you choose not to invest additional cost and time into psych testing.



Possible Outcomes

• Best case scenario – Candidate comes on board, fits in

well with the team, project is a roaring success and

you get promoted.

• Worst case scenario – Candidate comes on board, has

a number of behavioural issues that emerge in relation

to their management of others, management of clients,

management of the project which were not obvious in

the interview. Candidate leaves abruptly. Company

loses money. Project gets shelved, you get fired.

So, what is the cost of a bad hire?



What is the cost of a bad hire?

• Direct costs – advertising, engaging recruitment agency

• Indirect costs – time spent assessing resumes,

interviewing, training new employee

– Loss of IP

– payouts?

– Performance lag time.

What if you could minimise these costs overall by using

psych testing?



Client Testimonials

• Case 1:

– Client was taking on a new employee who looked fantastic on

paper and interviewed well, and decided not to put them through a

psych test. Candidate lasted 6 weeks.

• Case 2:

– Client was made aware of a candidate’s vulnerabilities for a

particular role through psych testing, but decided to proceed with

offering the candidate the role regardless. Candidate’s

vulnerabilities emerged on the job, and they were managed out of

the organisation within three months.

• Case 3:

– Client reported that the strengths and development opportunities

identified by the psych test were true of the individual in the role 6

months later.



How can you afford to not use psych

testing as part of your selection and

development process?



Benefits of using psych testing in the

recruitment process

• Adds rigour to the entire recruitment process

– Psych tests will return the same results if a candidate

takes the test now and in 6 months time. They are

reliable.

– Psych tests have been demonstrated to predict on the

job performance.

– Psych tests are a valid measure of behaviour, they

measure what they purport to measure - on the job

performance.



Benefits of using psych testing in the recruitment process

• Provides a methodology for predicting

performance against key competencies

– Competencies refer to a set of behaviours

required to successfully perform job tasks.

– Psych tests are designed to objectively measure

candidate’s behaviour in relation to the key

competencies of the role.




• Provides a measure of performance and potential

– Performance refers to past behaviours as measured by

interviews, references checks and resumes.

– Potential refers to a candidate’s capacity to perform

on the job and develop skills.

– Although they can be measured separately, we

recommend the combination of performance and

potential which allows you to take a more holistic

assessment approach to understanding the candidate’s

fit for the role.



Below the Line

• Assesses Knowledge, Skills,

Abilities and Other (KSAOs)

• People are hard to read:

Using psych testing is about

getting to the “below the

line” assets such as motives,

attitudes, drivers

• Psych testing can also be

used in support of “gut feel,”

to inform decisions.




• Assesses a wide range of job relevant skills and behaviours

– learning & cognitive style preferences

– emotional intelligence

– personal & leadership style

– team skills

– sales skills

– customer service skills

– motivation

– Microsoft Office skills

– IT programming

– values

– derailers




• Is resource effective –for volume

recruitment eg. Graduates

– Eg. Online abilities can save time and money in

screening a large amount of candidates in an

objective manner.



What psych tests are available?

• Abilities (verbal, numerical, abstract)

• Personality or work style preferences

• Sales behaviours/preferences

• Customer service

• Motivation

• Computer software skills

• Or any other assessments you can think of!

Psychological Testing

• A psychological test is a

standardized measure of a

sample of a person’s behavior

that is used to measure the

individual differences that exist

among people.

Types of Psychological

Testing

• There are two types of

Psychological tests.

–Mental Ability tests

–Personality tests

Why use tests?

• Psychological tests are used in research, however, most serve a practical purpose.

Mental Ability Tests

• Includes three subcategories.

–Intelligence tests

–Aptitude tests

–Achievement tests

Intelligence tests

• Measure general mental abilities. They are intended to measure intellectual potential.

Examples

• Emily is four years old. Her big

sister Amy is three times as old as

Emily. How old will Amy be when

she is twice as old as Emily?

• WOLF is to FLOW as 8526 is to:

2856 - 6258 - 5862 - 5682 - 6852

Examples

• Hanger is to closet as tree is to:

Branch - Bushes - Forest -

Ground - Nest

• What would be the next number in

this series? 15 ... 12 ... 13 ... 10 ...

11 ... 8 ... ?

Aptitude tests

• Assess talent for specific kinds of learning. (clerical speed, mechanical reasoning, etc.)

Examples

• Are You a Self-Starter?

• Climbing the ladder would bring a load of responsibility and pressure that I

wouldn't want to carry.

• If my boss or supervisor told me I were being promoted, the fact that they had so much confidence in my abilities

would:

Achievement tests

• Gauge a

person’s

mastery and

knowledge of

various subjects

Examples

• Who was the 43rd President of

the United States?

• What is 5x6 divided by 2?

• How many branches of

Government exist in the U.S.?

Personality Tests

• Measure aspects of personality, including motives, interests, values, and attitudes.

Examples

• Do you become upset when. . ?

• Do you feel like you lose control when. .?

• Are you happy when . . ?

In our TextBook

• Speed Test vs. Power Test

• Individual Test vs. Group Test

• Pen & Paper Test vs.

Performance Test

Test Design

• In order for a test to be accurate, it

must meet the three standards

below.

– Standardization

– Validity

– Reliability

Standardization

• Standardization refers to the uniform procedures used in administrating and scoring a test.

• Test norms: information used to rank scores in relation to other scores on the test.

• Can you think of examples

Validity

• Refers to the

ability of a test

to measure

what it was

designed to

measure.

• Examples

• What psychologist

promoted

introspection?

• Who developed the

four mechanisms for

dreaming?

• What school of

psychology does

Skinner belong to?

Validity Construct Validity

• Degree to w/c a test is accurate and faithful

measure of the construct it purports to

measure.

Criterion-related Validity

• The degree to w/c a test forecasts or is

statistically related to a criterion.

Validity Validity Coefficient

• A statistical index that reveals the degree of

association between two variables. Often used

in the context of prediction.

Content Validity

• The degree to w/c subject matter experts

agree that the items in a test are a

representative sample of the domain of

knowledge the test purports to measure.

Validity Face Validity

• The appearance that items in a test are

appropriate for the intended use of the test by

the individuals who take the test.

Reliability

• Reliability

refers to the

measurement

consistency of a

test (or other

techniques).

• Example

• You take a personality

test and are scored as

“assertive”. Three

weeks later you take

the same test and are

scored as “passive”. A

drastic change is

probably a result of an

unreliable test.

Testing Reliability

Test-retest

–Comparing subjects’ scores on two administrations of a test.

Correlation Coefficient

–A numerical index of the degree of relationship (-1, +1)

Testing Reliability

Equivalent-form

–Reveals the equivalence of test scores between two versions or forms of the test.

Internal-consistency

–Reveals the homogeneity of the items comprising a test.

Testing Reliability

Inter-rater

–Reveals the degree of agreement among the assessments of two or more raters.

Visual example

Visual example

Think!

• Why do we have so many tests?

• How can we use testing to improve

. . .?

• How does psychological testing

apply to school, careers, sports,

etc?

Ethical Standards

• Invasion of Privacy – a condition

associated w/ testing pertaining to asking of

questions on a test that are unrelated to the

test’s intent or are inherently intrusive to the

test taker.

• Confidentiality - a condition associated

w/ testing pertaining to w/c parties have

access to test results.

Test Content (Major types of construct)

• Intelligence Test

• Mechanical Aptitude Test

• Sensory/Motor Ability Test

• Personality Inventories

• Integrity Test

• Physical Abilities Test

• Multiple-Aptitude Test Batteries

• Computerized Adaptive Testing

Other methods

• Interviews (Unstructured vs. Structured)

• Situational Interviews (Experience based vs

situational questions)

• Work Samples

• Situational Exercise

• Biographical Information

• Letter of Recommendation

• Drug Testing

Controversy

The use of the following test…

• Polygraphy or Lie Detection

• Graphology

• Emotional Intelligence

Four major standard in

selecting a test

• Validity

• Fairness

• Applicability

• Cost

By the way…

• What is the difference between psych

testing and psych assessment or

evaluation?

• What are the general reasons of the use of

psychological tests?

Business

I/O chapter 4