Leadership Tips

Want to use a pre-employment test? Here's what to look for

A test may ask all the right questions and assess all the skills you are looking for, but it needs to be able to perform well statistically. Any theories, regardless of the field of study, only become fact if they can stand up to rigorous analysis.

The American Psychological Association (APA) sets the standards for educational and psychological testing. Thus, all tests that are used for hiring purposes need to follow these rules. For a summary of the APA standards, click here.

The following is a general overview of what to look for in a test, based on the criteria set by the APA:

Reliability and Validity

First, ask your test provider to give you the statistical report for the test you are interested in. Although wading through a test's statistical data can be intimidating, you don't need to be a statistician in order to assess the quality of the test. Here is the key information to look for:

• The overall "Cronbach's Alpha" (a statistical measure of reliability or "internal consistency"). This should be at 0.70 or higher. Reliability measures attempt to determine if the test questions correlate or work well together.

• The validity of the test tells us whether it's measuring what it's supposed to measure. Here's how to determine whether a test is valid:

  • Which population has the test been used on? If the test is job-specific (e.g. for sales reps), people with sales experience should be part of the test population, and should score better than people without sales experience.
  • How big is the sample size? A sample must be at least 500 people, and the larger the test (i.e. the more skills and traits being assessed), the larger the sample should be.
  • If the test is intended for hiring purposes, does it comply with the Equal Employment Opportunity guidelines set by the EEOC (The U.S. Equal Employment Opportunity Commission)? To calculate compliance, statistical analyses are run to determine whether the percentage of a minority or protected group that attains a certain cut-off point (e.g. a score of 80 on the test) is not less than 80% of the majority group. As per EEO terminology, members of a "protected class" include women as well as men, different ethnic groups, individuals with a disability, and people over the age of 40.

Test Structure

Test creators or distributors should be able to provide a sample of the questions, a sample report, and/or a demo that allows you to take a test yourself. Here's what to look for in terms of how the test is built:

General Overview

In order to decide whether a test fits your needs, make sure to investigate the following:

• The length of the test. Realistically, a personality, aptitude, or IQ test should not be less than 10 minutes. At the same time, however, a 3-hour test is impractical.
• Whether the test is evaluating all the traits and skills that you need to assess in a candidate
• Whether benchmarks are available, and if you have the ability to create custom benchmarks based on your own pool of employees
• Whether they offer interview questions that are tailored to a candidate's test results

Test Questions

Make sure that the questions on the test are:

• Relevant to what the test is supposed to be assessing (e.g. a career-related test should not be asking about a person's personal or romantic life)
• Clear and easy to read, and appropriate in terms of reading level required
• A mix of direct (e.g. I am punctual) and indirect questions (scenarios, ability-based questions). Otherwise, test-takers may be pointed to the "right" answer.

It's also important for a test to include measures that assess "faking good," like Impression Management or Social Desirability scales. This is especially true in cases where the stakes are high. A candidate who really wants a job may try to answer the test questions in a way that will make him or her look good (rather than choosing responses that reflect his/her actual thoughts, feelings, and behaviors).

Test Report

Verify whether the test report offers sufficient information for you to make a clear decision about a candidate's potential. Ideally, test reports should include the following information:

• Clear, easy-to-understand graphs with the test-taker's score
• Definitions of each trait and skill assessed
• Clear interpretations of a person's score written in layman terms. This is important if you are not trained in interpreting psychological assessments. What does a high score indicate? What does a low score indicate? How does this impact a person's performance and potential for success?
• Clear indications of areas that need improvement
• Tips on how a person can improve

Other important information

Aside from asking the test provider for a statistical report and demo of the assessment, here are a few other important questions to ask:

• "What research and theories form the basis of this test?" While it's important to assess skills and traits that are supported by research and relevant to a position (e.g. conscientiousness has been linked to success in many job positions), keep in mind that in some cases, tests will assess skills or traits that are not supported or have never been used in research before - and are thus added for exploratory purposes. In cases like this, the test creator needs to be able to provide statistical support of the reliability and validity of the scale. In addition, if significant revisions have been made to a test (new questions and scales), fresh data must be collected and analyzed to ensure that the additions have improved the reliability and validity of the test.
• "Has the test been revised at some point? When was the last revision? What were the revisions?" Just like car models are upgraded regularly, a test must undergo revisions, especially if statistical analyses reveal that some questions are not working well. In addition, new research theories as well as social and economic changes, can affect the context of test questions, scoring norms, and benchmarks.