Journal of Applied Psychology 1992. Vol. 77. No. 4, 545-553 Copyright 1992 by the American Psychological Association, Inc. 0021-9010/92/S3.00 On the Choice of Scales for Task Analysis Juan I. Sanchez and Scott L. Fraser Florida International University One hundred one incumbents of 25 service jobs rated their respective tasks on relative time spent, difficulty of learning, criticality, and overall importance. Although scale convergence varied as a function of job title, task criticality and importance ratings were similar and presented low to moderate levels of convergence with both time-spent and difficulty-of-Iearning ratings. Different composites of task importance were compared. All composites and the overall judgments of im- portance were moderately correlated with each other and showed similar levels of interrater agree- ment. Several conclusions regarding the choice of scales and the use of composites in task analysis are drawn. The determination of essential work behaviors or tasks in a job is necessary to support a variety of personnel decisions. For instance, in its discussion of technical standards for validity studies, the Uniform-Guidelines on Employee Selection Proce- dures (1978, p. 38300) indicates that there should be a review of job information to determine mea- "\ sures of work behavior(s) or performance that are relevant to the job or group of jobs in question. These measures or criteria are relevant to the extent that they represent critical or important job duties, work behaviors or work outcomes as developed from the review of job information. The need to link selection measures to important work behav- iors has also been reinforced by the courts (Kleiman & Faley, 1985). As a result, public and private employers alike have re- sorted to task analysis as an aid in developing job-related em- ployment tests (Drauden, 1988). Task analysis has not been propelled only by increasing con- cerns with test validity. Some of the most recent applications of task analysis include (a) development of testing plans for licen- sure and certification examinations (Kane, Kingsbury, Colton, & Estes, 1989), (b) design of legally defensible performance ap- praisals reflecting specific dimensions of job performance (Bernardin & Beatty, 1984), (c) training needs analysis (Gold- stein, 1986, p. 40; Levine, 1983, p. 84), and (d) determination of the relative importance of job activities to estimate dollar-va- lued performance ratings in utility models (Cascio & Ramos, 1986). The enactment of a new piece of legislation, namely, the Americans With Disabilities Act (ADA; U.S. Department of Justice, 1991), may open new avenues for task analysis. The This study was supported by grants awarded to Juan I. Sanchez and Scott L. Fraser by the Marketing Science Institute (Grant No. 4-754) and by the Florida International University Foundation (Grant No. 265-50-09). We wish to acknowledge Edward L. Levine and three anonymous reviewers for their comments, and 25 job analysts for their dedication to this project. Correspondence concerning this article should be addressed to Juan I. Sanchez, Florida International University, Department of Psychol- ogy, University Park, Miami, Florida 33199. ADA calls for determination of the essential functions of a job to ascertain the employability of disabled applicants. Task anal- ysis seems to be a cost-effective tool for making such determina- tions. The present study was focused on an important procedural issue. In the final phase of task analysis, subject matter experts (SMEs) rate a list of tasks (i.e., a task inventory) on dimensions such as time spent, difficulty, and importance. A question that remains unanswered is which scales should be included in these task inventories. The aim of this study was to contribute to the need for infor- mation on task analysis scales. First, we attempted to provide information on scale interrater agreement across a wide sample of nonmilitary jobs. Second, we explored scale interrelation- ships and whether scale convergence differed as a function of job and rater characteristics. Last, we investigated the correla- tions among different composites of task importance proposed in the literature and their robustness to job and rater facets. Rating a multitude of tasks on several scales is a very tedious, time-consuming process. Therefore, the selection of scales to be employed in task inventories should be carefully and logi- cally planned. In the past, the choice of task analysis scales has been guided by (a) historical precedents set by the use of scales in the military (Christal, 1974), (b) suitability of the scales to the purpose of the job analysis (Goldstein, 1986, p. 41), or (c) reasoned notions about what dimensions constitute task im- portance (Kane et al., 1989; Levine, 1983). Although these ap- proaches may be reasonable, informed choices about task scales should be better driven by knowledge about the scales' psychometric characteristics and interrelationships (Schmitt, 1987). Previous studies concerning these areas are reviewed next. Test-Retest and Interrater Agreement on Task Inventory Ratings Wilson, Harvey, and Macey (1990) recently reviewed studies examining test-retest reliabilities of task inventories. They found the range of reliabilities to differ considerably (rs = .30 to .90) as a function of rating scale (task difficulty having the low- 545 This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.