CETE Observatory: Crane Operator Certification and Evaluation – OSHA Issues Final Rule

The Occupational Safety and Health Administration (OSHA) just issued a final rule, part of which took effect on December 9, 2018 (a further requirement takes effect on February 7, 2019), to clarify an employer’s duty to ensure safe operation of cranes through training, certification or licensing, and performance evaluation. Under this clarification (of a draft rule issued in 2010), heavy equipment operators of cranes across a range of settings in the construction sector will be required to earn and maintain certification (either specifying type and capacity of the crane or just the type of crane). This modification to the draft rule should reduce the regulatory burden while maintaining safety for the protection of operators, co-workers, equipment, and the public.

CETE works with certification bodies in the construction sector and foresees implications for its research-practice projects. Staff over the last few years completed projects with the Electrical Industries Certifications Association, Inc. (EICA) to assist in the accreditation of two certification programs (crane and digger-derrick) under ISO 17024:2012. A project in 2016-17 involved a DACUM analysis (conducted by CETE staff member John Moser) with an online verification survey and alignment of tasks to Knowledge-Skill statements (CETE staff members Brooke Parker and James Austin) that was reported to EICA as one part of their application to the accreditation body knows as the American National Standards Institute (ANSI). More recent projects completed by CETE staff member Bridget McHugh involved analysis of rater scoring from a multi-exercise performance assessment (scored on time and errors / deductions). The analyses estimated rater agreement to support score interpretations of part of the certification exam process (multiple performance exercises). Accreditation was granted to EICA in 2017, therefore both certification programs must submit reports annually as part of surveillance and monitoring (which is required to maintain accreditation standing).

Note that most certification bodies do not provide evidence on prediction of job performance (concurrent or predictive job outcomes) because their rationale for score interpretation is focused on content and response process (i.e., similar to Webb’s Depth of Knowledge concept). Specifically, certification bodies use a model of score interpretation in which individuals are evaluated against items created to sample the body of knowledge-skill-judgment. The focus is on whether an individual “at the point of testing” is judged to be ready to practice in the occupational area. One exception is the National Board for Respiratory Care (NBRC), which has supported research on job performance outcomes. For this reason, score reporting quality standards (National Commission on Certifying Agencies) and best practices call for providing only pass-fail status for passing candidates and additional information for failing candidates. This rule opens a possibility for evaluation of the prediction of job performance outcomes for certified individuals, although problems of range restriction and truncation will persist if certifications are required in hiring. CETE staff believes that such research would be important in specifying the value proposition of hiring certified individuals for employers.

Several broad implications for testing practice-research projects include 1) increasing accreditation support for certification bodies in various ways (for either accreditation body – National Commission on Certifying Agencies or ISO / ANSI in the United States), 2) studying relationships between employers and certification bodies to investigate how they collaborate (successfully or not), and 3) developing-validating performance evaluation systems for use by various stakeholders in the construction sector – crane operator employers (contractors) and training providers (for example, various sponsors of apprenticeships). Research by Olson, Fine, Myers, and Jennings (1981) on performance standards for heavy equipment operators (blade equipment), as well as research reported in construction management journals, may be useful in this regard as is research.

Contributors: James Austin, Angela Stansell