Human Resources

Human Resources

Adverse Impact

Some Stuff You Should Know

Human Resources

What Exactly is Adverse Impact?

A substantially different rate of selection in hiring, promotion or other employment decision which works to the disadvantage of members of a race, sex, or ethnic group (Uniform Guidelines Q&A #10).

Human Resources

Where Does it Exist?

Written tests typically have the highest degree of adverse impact.

Highest level of impact tends to be against African Americans and Hispanics (Sackett, 2001).

Physical ability tests typically have adverse impact against women, especially when they measure upper body strength.

Human Resources

Why Does it Exist?(Guion, 1998)

1. Chance

2. Measurement problems inherent to the test (e.g., poor reliability)

3. The nature of test use (e.g., ranking vs. pass/fail)

4. Differences in distribution sizes (e.g., 100 males and only 10 females)

5. True population differences in distributions of the trait being measured

Human Resources

What Does it Describe?

Adverse impact simply describes differences between groups on a testing process.

It is not a legal term that implies guilt or a psychometric term that implies unfairness or test bias.

Many employment tests result in adverse impact.

Adverse impact is not normally due to forms of bias inherent to the test (Sackett, 2001).

Human Resources

Two Types of Adverse Impact Analyses

Selection Rate Comparison and Availability Comparison.

Selection Rate Comparison – evaluates the selection rates between two groups on a selection procedure (or other employment decision such as layoffs).

Involves two groups: a focal group (e.g., females or minorities) and a reference group (e.g., males or Caucasian).

Human Resources

Selection Rate Comparison

Four variables are entered into this type of adverse impact analysis:

1. Number of focal group members selected

2. Number who were not selected

3. Number of reference group members selected

4. Number who were not selected

Human Resources

Selection Rate Comparison Example

Took Test Passed Test %

Males 75 60 .80

Females 50 30 .60

A passing rate for females which is less than 64% (or 80% of the passing rate for males) is evidence of adverse impact.

Human Resources

Availability Comparison

Availability Comparison – evaluates one group’s representation in a position to their availability for that position (e.g., 13% of the incumbents in the Accountant classification are Hispanic; 15% of the qualified Accountant Applicants are Hispanic.

Useful for showing the extent to which one group may be underutilized (e.g., 2% in the above example).

Human Resources

Availability Comparison

Three variables are entered into this type of Adverse Impact analysis:

1. Total number of incumbents in a position

2. Number of focal group members in the position

3. Percentage of focal group members who are available for the position

Human Resources

Two Types of Adverse Impact Analyses

The Selection Rate Comparison is the only type of adverse impact analysis that can be used alone to demonstrate adverse impact.

The Availability Comparison only shows a prima facie reason to investigate further into an employer’s practices to see why a “gap” may exist.

Human Resources

Connecticut v. Teal (1982)

The employer must determine whether adverse impact exists at each step of a multiple-step selection process.

Evidence of adverse impact at any step of the selection process constitutes adverse impact.

Employers cannot hide adverse impact within a selection process by demonstrating none at the bottom line.

Human Resources

Adverse Impact Example

Males Females

Took Test

Passed Test

%Took Test

Passed Test

%

Written Test 200 130 .65 125 60 .48*

Performance Test 130 95 .73 60 32 .53*

Oral Board 95 76 .80 32 28 .88

Eligibility List 76 28

* Denotes adverse impact

Human Resources

80% or 4/5th Rule of Thumb

. . . A “rule of thumb” as a practical means for determining adverse impact for use in enforcement proceedings . . . It is not a legal definition of discrimination, rather it is a practical device to keep the attention of enforcement agencies on serious discrepancies in hire or promotion rates or other employment decisions (Overview, Section ii).

Human Resources

80% or 4/5th Rule of Thumb

. . . A selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact, while a greater than four-fifths rate will generally not be regarded by Federal enforcement agencies as evidence of adverse impact. Small differences in selection rate may nevertheless constitute adverse impact, where they are significant in both statistical and practical terms . . . (Section 4D).

Human Resources

80% Rule vs. Statistical Significance

Courts have scrutinized the 80% rule in Title VII litigation because the rule is greatly impacted by small numbers and does not consider the statistical significance of the passing rate disparity between the two groups.

Human Resources

80% Rule vs. Statistical Significance

For example:

“Rather than using the 80 percent rule as a touchstone, we look more generally to whether the statistical disparity is ‘substantial’ or ‘significant’ in a given case” (Bouman v. Block, 1991).

Human Resources

Statistical Significance

If the result of a statistical test is “statistically significant,” it is unlikely to have occurred by chance.

Obtaining a finding of statistical significance in a research study signifies a point at which the researcher is capable of stating that a legitimate trend, and not a chance relationship, actually exists (with a reasonable level of certainty).

Human Resources

Statistical Significance

Statistical significance tests result in a p-value (p for probability).

These p-values can range from 0 to +1.0.

A p-value of 0.01 means that the odds of the event occurring by chance are only 1% or 1 out of 100.

A p-value of 0.05 means that the odds of the event occurring by chance are only 5% or 5 out of 100.

Human Resources

Statistical Power

The use of statistical tests to determine whether statistical significance exists in a data-set is highly contingent on whether the analysis has sufficiently high levels of statistical power to find it.

Statistical Power – the ability to reveal a statistically significant finding if there is one to be found.

Human Resources

Statistical Power

A powerful adverse impact analysis is one that has a high likelihood to uncovering adverse impact if it really exists.

When applying the statistical power concept to adverse impact analysis, there are 3 factors that impact the statistical power of the analysis.

Human Resources

3 Factors That Impact Statistical Power

1. Effect size. For selection rate comparisons, this pertains to the size of the “gap” between the selection rates of the 2 groups (e.g., male passing rate of 80% vs. female passing rate of 60%).

2. Sample size. The number of focal and reference group members (plays a key role in the analysis).

3. Type of statistical test used. Some statistical tests are more powerful than others. Another issue is whether a one-tail or two-tail test of significance is used.

Human Resources

Sample Size

A large sample size is the most effective way to increase the power of an adverse impact analysis.

There is no absolute bottom threshold regarding the minimum sample size needed for statistical investigations.

In Bradley v. Pizzaco of Nebraska, Inc. (1991), the court said: “There is no minimum sample size prescribed either in federal law or statistical theory.”

Human Resources

Sample Size

OFCCP (1993) indicated that the minimum number for adverse impact analysis is 30 with at least 5 expected for selection (i.e., hired, promoted).

The Uniform Guidelines Q & A #20 (1979) states that a sample size of 20 is too small for adverse impact analysis.

What have the courts said?

Human Resources

Sample Size

A sample size of 6 was found to be too small in Gault v. Zellerbach (1998).

A sample size of 7 or 11 was insufficient to demonstrate a pretext in Martin v. United States Playing Card Co. (1998).

A sample size of 13 was too small in Tinker v. Sears, Roebuck, & Co. (1997).

A sample of 8 was insufficient in Anderson v. Premier Industrial Corp. (1995).

An 8-person sample was too small to support a discrimination case in Osborne v. Brandeis Machinery & Supply Corp. (1994).

Human Resources

Sample Size

Uniform Guidelines:

Where the user’s evidence concerning the impact of a selection procedure indicates adverse impact but is based upon numbers which are too small to be reliable, evidence concerning the impact of the procedure over a longer period of time and/or evidence concerning the impact which the selection procedure had when used in the same manner in similar circumstances elsewhere may be considered in determining adverse impact (Section 4D).

Human Resources

Statistical Tests

There are 2 categories of statistical significance tests that can be used – exact and estimated.

Exact tests are the most powerful statistical tests to use for adverse impact calculations.

Exact tests provide the precise probability value of the analysis.

The exact statistical significance test to use for selection rate comparisons is the two-tail Fisher Exact Probability Test for 2 X 2 contingency tables.

Human Resources

Fisher Exact Test

Males Females

Pass 60 35

Fail 15 20

The 2 X 2 contingency table will always have two groups and two categories.

The two-tail Fisher Exact Test is the statistical procedure most frequently used in litigation for establishing statistically significant levels of adverse impact.

Any probability value that is less than .05 is statistically significant and indicates a difference in passing rates between two groups that is not likely to be occurring by chance.

Human Resources

One-Tail vs. Two-Tail Test

A one-tail statistical test used for adverse impact analysis investigates the possibility of discrimination occurring in just one direction (e.g., against women when making a men vs. women comparison).

A two-tail test makes the assumption that discrimination could have occurred in either direction.

The courts have been almost totally consistent in their requirement of using a two-tail test for significance because it mirrors the philosophy of Title VII, which is to evaluate a disparity against any group, not just one group.

Human Resources

One-Tail vs. Two-Tail Test

Some court decisions endorsing the two-tail method:

Chang v. University of Rhode Island (1985)

Palmer v. Shultz (1987)

Mozee v. American Commercial Marine Service Co. (1991)

Csicseri v. Bowsher (1994)

Hoops v. Elk Run Coal Company Inc. (2000)

Moore v. Summers (2000)

Human Resources

Practical Significance

Rule of One: The Rule of One is often used to determine the

practical significance of any detected adverse impact.

The Rule of One states that if adverse impact is negated by having one more candidate pass the exam component (or be selected), the adverse impact has no practical significance.

Human Resources

So, What Does This Suggest?

For small sample sizes, use the 80% or 4/5th rule of thumb.

Apply the “Rule of One” to determine practical significance.

For larger sample sizes (≥ 30) you can use the two-tail Fisher Exact probability Test.