Recruitment & Selection

Work Sample Tests and Potential Adverse Impact

Adverse impact in employee selection is a crucial concern for human resource management. Adverse impact occurs when a protected class of applicants is hired or selected at a substantially lower rate compared to other groups of applicants, which can potentially lead to costly court charges and lawsuits. Thus, reducing actual and potential adverse impact against protected groups is a key priority for most HR departments. This goal has led companies to search for the most effective selection tools possible that maximize differentiating among applicants of different qualifications and abilities while minimizing adverse impact. Some selection tools, such as general cognitive ability tests, have reputations for producing adverse impact. Other tools, such as work sample tests, have generally been considered to be effective while producing little adverse impact. Work sample tests require applicants to perform brief exercises that are similar to functions they would be performing in their jobs. Examples of work sample tests include:

Role playing exercises (e.g., supervisor or customer service roles)
In-basket exercises (e.g., writing memos or scheduling employee work hours)
Technical skills exercises (e.g., troubleshoot a computer problem or solve a work-related math problem)

Thus, work sample tests are often recommended for use by many sources in place of other tools and instruments that are seen as more likely to result in adverse impact. However, research has discovered that the risk of adverse impact using work sample tests may actually be much greater than once thought.

Problems with Past Research

Several studies have claimed that work sample tests exhibit less adverse impact than other HR tools. However, this research contained some important limitations, including:

Using only job incumbent ratings and not job applicants, which can affect statistical results by attenuating the range of scores.
Some research grouped minorities together for analysis (for instance, combining African American and Hispanic data rather than analyzing it separately).
Some of the research combined work sample tests with other types of selection assessments, which prevented adverse impact information from being calculated just for work sample tests.

Current Research Findings

Recent evidence indicates that incumbent work sample tests show higher adverse impact than was expected. Applicant work sample scores have nearly double the commonly expected difference between whites and blacks. Work sample tests demonstrating the largest differences are in-basket exercises and technical and scheduling sample tests. Oral briefings and role-playing exercises, however, both exhibit low difference scores. The recent research also analyzed work sample exercises based on what constructs they were measuring. A construct here refers to underlying psychological attributes like personality, communication ability, cognitive ability, etc. Work sample tests that exhibit large group differences appear to tap into constructs involving cognitive ability and writing skills/knowledge, while work sample tests that exhibit lower group differences tap into constructs like leadership and interpersonal oral communication.

Implications for Practice

As research has indicated that adverse impact can be a bigger problem with work sample tests than previously thought, some recommendations for reducing the risk of adverse impact with work sample tests include:

Selection decision makers need to consider what constructs will be evaluated with a particular work sample test. Any given work sample test will likely involve several different constructs (e.g., in-basket exercise involving cognitive ability, personality, communication, and/or psychomotor skills).
Measured constructs must be closely aligned to key job functions and duties. For example, is general cognitive ability or certain social skills really necessary for satisfactorily completing the tasks of a particular job?
Record detailed information about applicants, incumbents, ratings, and specific types of work sample tests for adverse impact analyses.

The recent research does not indicate that work sample tests (or any other HR selection tool) will necessarily produce adverse impact. However, the results do indicate that HR professionals need to be more aware of the potential for adverse impact and to not take for granted the idea that work sample tests will be more acceptable.

Interpretation by:

Donnie Johnson

DeGarmo Group

This was a summary of the research and practice implications from: Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of Black-White differences in overall and exercise scores. Personnel Psychology, 61, 637-662.