Uses of Pre-employment Tests in Selection Procedures

by Bill Angus

I nformation in making hiring decisions is generally regarded to come from three sources...

  1. History and Resume
  2. Standardized tests,
  3. Interviewer impressions.
Of these three sources, Interview has traditionally been the measure most relied on in making hiring decisions. However, research suggests that test scores are actualy the best predictors of job success.

Because three sources of information about an applicant's suitability are avilable, we need not rely only on one source of information. What we hope to find in examining information about an individual is convergence. If all sources of information seem to indicate roughly the same things about an individual, then the probability that these "things" are correct increases. By contrast, divergence (where one source seems to contradict another) presents a problem that must be dealt with by personnel managers. Further investigation will hopefully uncover possible explanations for any divergent information.

History

If we could get an accurate history, that would theoretically be the best indicator of performance. Psychology and learning theory tells us that past behavior is the best predictor of future behavior. However an "honest and complete" history can be difficult to obtain. A more serious problem, when checking references, is that the comments you receive are impossible to scale. An honest informant will say positive as well as negative things about an applicant. There is no easy way for you to put this information into a perspective which will allow you to make a reasonable judgment about the emplolyee or to rank prospective employees.

Interview

Interview results are problematic in the same way as work-history and references. That is, you have no sure-fire way to scale the applicant's responses so you can meaningfully rank your applicants. Interview techniques also present the danger of leading you to hire the person who can best "tell you what you want to hear". This might or might not relate very well to what you want the person to do once you hire him/her.

Although all interview techniques suffer from the above weaknesses, perhaps the best results will come from behavior-based interviewing (BBI). BBI questions are designed to eleict information about certain "key competencies." Questions might include the following (excerpt from The Ultimate Job Search Kit by D. Stimac):

Research/written communication competence.

Oral communication competence. Time-management / commitment to task Management / leadership ... the list can go on and on.

Objective Tests

Objective tests can be subdivided into two types:

Aptitude tests are used to identify the abilities of a prospective employee. These are sometimes referred to as "tests of maximum performance". Aptitude tetsts will generaly tell you how quickly a person may be expected to learn to do any tasks which you require. Viewed another way, these tests will identify who is unlikely to learn to do a job satisfactorilly, or within a satisfactory time-frame. There are general aptitude tests (sometimes called intelligence or IQ tests) and also tests for specific aptitudes.

General aptitude tests include the Wonderlic Personnel Test (WPT). This type of test can also be referred to as an Intelligence or IQ-test. There are also tests of specific aptitude, i.e. mechanical comprehension tests like the Wiesen Test of Mechanical Aptitude (WTMA).

Results of aptitude tests should be used to debunk the information obtained in history and interview. Note that candidates can "fake bad" on aptitude tests, but unless they have seen the test before, they cannot give a false-positive evaluation. A problem with using only aptitude tests is that these tests do nothing to measure important personality characteristics such as motivation, honesty, reliability, job involvement, personal interest, customer service orientation, team spirit, etc. Further, there is some criticism that aptitude tests do a less than perfect job of measuring the elusive variable known as "common sense" which can be an important facet of aptitude in the workplace. Our best estimates suggest that aptitude test results by themselves probably account for about 25% of the variance in job performance (Sternberg, American Psychologist vol 50 #11, Nov. 1995). While this seems remarkably low -- research shows that aptitude remains the single best objective predictor we have of success on the job. Because history and interview information cannot be scaled well, aptitude test scores end up being far better predictors of job performance.

Aptitude tests should generally allow an individual to demonstrate a measurable amount of success or failure at cutting-score points. When selecting aptitude tests, it is important to find a test-form which allows you to rank your applicants. If applicants get most of the test items correct, it may be that the test is too "easy" to measure the individual accurately. That is, your applicants may have more ability than the test can measure. Do not set cutting score points which require apllicants to get nearly all items correct. Rather, find a harder test and set your cutting score lower. A difference of 1 or 2 raw-score points on a test-form should never make the difference between a good solid "hire" and a "wash-out".

The above paragraph mentions number of items correct (raw score), which pertains to test selection. However, when using tests, you should never interpret raw-scores. Test results are usually interpreted in terms of percentile ranks (%'ile score) and/or standard-score. For those not trained in testing, I suggest sticking to percentile rank interpretation. A candidate scoring at the 90th %’ile can be expected to score better than 89% of the reference group which the standardization sample represents and worse than 9% of the reference group. It is not true that getting 90 percent of the test items correct means that the individual scores at the 90th %’ile. %’ile scores are derived from a table in the test manual which relates the number correct on a test to a ranking in the reference population on which the test was normed.

If you are unsure of how to use %’ile scores, get an organizational psychologist or a good personnel manager to help you. You will need to get help selecting one or more test-forms, and help setting cutting scores for each of the tests you choose.

A cutting-score is the particular number which you identfy as representing the score below which that an individual should not be hired. Some people use a model in which two cutting scores are set (1) a range of scores where caution is indicated, and (2) a score below which the candidate cannot be considered.

Some final notes about aptitude test scores are that we should never assume that the highest test-scores necessarily indicate the best prospective employee. Also, you should not average test-results obtained from different aptitude tests. Rather, it is better to consider the graph or "profile" of all scores produced by the applicant.

Personality tests, including and Honesty/Reliability/Integrity tests, Tests of sales orientation or customer service orientation, etc.

There are personality tests for which validity study has shown high-scoring employees to actually perform better on the job, after some period of employment. Unless this specific type of validity study is available for the specific test you plan to use, I would suggest that the test not be considered. Personality assessments can certainly be conducted with tests which have not shown themselves to be generally predictive of success in a particular job. However, assessment using these tests should generally be performed by people such as industrial/organizational psychologists. When such tests are used, it is the psychologist's professional opinion and overall assessment on which you must rely (it is not the individual test scores).

Most personality tests are designed to be used by psychologists. However, there are some tests avilable which can be interpreted by non-psychologists. These include the Performance and Tenure scales of the Employment Inventory (EI), listed in the catalog of tests. Specific personality tests which may be included with the EI include a "Sales" scale and "Customer Service Orientation" scale. In practice, it is important to make sure not to look for high sales scores when filling customer service positions, and vice versa. Some people may score high on both scales, but this is not always the case. In retail stores which do not rely on commision sales, it is most likely that the customer-service scale will be predictive of success on the job.

Integrity tests tests tend to be quite expensive when compared to aptitude tests. Simple aptitude tests usually cost something in the neighbourhood of $4 (at this writing). A well-reputed integrity test is likely to cost in the neighbourhood of $25 per administration. Test developers have justified the price of integrity tests based on development costs, lower volumes, security concerns (these tests are usually only computer scorable and the answer key is not usually released). Potential savings from using integrity tests arise because businesses who use these tests are less likely to hire employees who may steal, or be unreliable, or be more likely to take stress leave or otherwise be a higher risk for abusing the company benefits plan.

HR managers should note that the administration of one or more personality tests is not the same thing as a comprehensive pre-employment assessment provided by an I/O psychologist. Such assessments generally include interview, as well as ability and personality measures. These assessments typically cost in the neighborhood of $500 - $1,000 per candidate, and are based on an hourly rate charged by the psychologist.

To get in touch with an I/O psychologist, you might try your state or provincial College of Psychologists ( http:www.apa.org or http://www.cpa.ca ) Some business consultants and even a few psychologists are representing firms which are selling relatively high-priced test batteries ($300 plus per administration). This is not the same thing as a thorough pre-employment assessment by a psychologist. If you are simply receiving the computer-generated results of a canned test battery, then the price should be much lower than if you are receiving tailored testing, candidate interview, and a written expression of professional opinion by the psychologist.

The information provided in this article is not meant as a complete reference. If you need further help, please contact our office. If we cannot provide a satisfactory answer, ask us to refer you to one of the many competent Organizational Psychologists we deal with regularly, or use the Web Links page to locate the contact information for the Professional Psychologist’s Association in your area. This can usually be found by following the link to the American Psychological Association (APA) or the Canadian Psychological Association (CPA).

Copyright © 2006, M.D. Angus &Associates Ltd. This article may be freely distributed provided this copyright notice is included.