Recruitment & Selection
Work Sample Tests and Potential Adverse Impact
- Role playing exercises (e.g., supervisor or customer service roles)
- In-basket exercises (e.g., writing memos or scheduling employee work hours)
- Technical skills exercises (e.g., troubleshoot a computer problem or solve a work-related math problem)
Problems with Past ResearchSeveral studies have claimed that work sample tests exhibit less adverse impact than other HR tools. However, this research contained some important limitations, including:
- Using only job incumbent ratings and not job applicants, which can affect statistical results by attenuating the range of scores.
- Some research grouped minorities together for analysis (for instance, combining African American and Hispanic data rather than analyzing it separately).
- Some of the research combined work sample tests with other types of selection assessments, which prevented adverse impact information from being calculated just for work sample tests.
Current Research FindingsRecent evidence indicates that incumbent work sample tests show higher adverse impact than was expected. Applicant work sample scores have nearly double the commonly expected difference between whites and blacks. Work sample tests demonstrating the largest differences are in-basket exercises and technical and scheduling sample tests. Oral briefings and role-playing exercises, however, both exhibit low difference scores. The recent research also analyzed work sample exercises based on what constructs they were measuring. A construct here refers to underlying psychological attributes like personality, communication ability, cognitive ability, etc. Work sample tests that exhibit large group differences appear to tap into constructs involving cognitive ability and writing skills/knowledge, while work sample tests that exhibit lower group differences tap into constructs like leadership and interpersonal oral communication.
Implications for PracticeAs research has indicated that adverse impact can be a bigger problem with work sample tests than previously thought, some recommendations for reducing the risk of adverse impact with work sample tests include:
- Selection decision makers need to consider what constructs will be evaluated with a particular work sample test. Any given work sample test will likely involve several different constructs (e.g., in-basket exercise involving cognitive ability, personality, communication, and/or psychomotor skills).
- Measured constructs must be closely aligned to key job functions and duties. For example, is general cognitive ability or certain social skills really necessary for satisfactorily completing the tasks of a particular job?
- Record detailed information about applicants, incumbents, ratings, and specific types of work sample tests for adverse impact analyses.
DeGarmo GroupThis was a summary of the research and practice implications from: Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of Black-White differences in overall and exercise scores. Personnel Psychology, 61, 637-662.