The statewide testing system, dating back to 1995, was built and operated by CETE for the Ohio Department of Education’s (ODE’s) Office of Career-Technical Education (CTE). The Ohio Career-Technical Competency Assessments (OCTCA) are free to local education agency users through funding provided by the ODE. Since 2009, we have moved the focus of these content-valid tests of achievement to the pathway rather than the specialization level. This shift is in keeping with the idea that a broader preparation will better serve student learners as they transition to postsecondary experiences ranging from labor market entry to further training and education. The latest reform is a shift from program- to course-based structure, which means that the tests are now end-of-course-aligned to the Career Field Technical Content Standards expressed in course outlines and, where possible, aligned to postsecondary learning outcomes to facilitate Career-Technical Credit Transfer (CT)2.

A complete summary of the project is beyond the scope of this update, but several highlights are noteworthy. First, over 7,800 multiple-choice items were created from July 1, 2013, to June 30, 2014. A four-day process with Ohio instructors (i.e., secondary and postsecondary levels) is used to write and review items. Written at two levels of challenge, which are based on Bloom’s taxonomy and Webb’s depth of knowledge, items may include scenarios and graphics. The item-by-item review process, which provides opportunities for input from the field and quality assurance, establishes content validity and recommendations for two cutoff scores (i.e., Proficiency and Advanced Proficiency). Second, field testing was completed on enhanced item banks for nearly 90 courses with the assistance of 6,265 secondary students. During field testing, the entire item bank for a course is made available so Ohio CTE instructors can schedule their students following instruction. The benefit of field testing is that it provides an opportunity to generate item statistics (e.g., difficulty, differentiation of high-low scorers) that can be used together with instructor judgments to create operational forms. Third, continuous user support for Ohio secondary districts is available. This support consists of frequently asked questions and a quick-start page, telephone support, user workshops held twice a year (streamed and archived), and a user survey. A set of shorter pathway-level practice tests will be developed over the next year to provide a testing encounter that students and instructors can both experience. Fourth and finally, pretests to help measure student growth will be rolled out over the next few years to coincide with the transition to end-of-course tests and provide evidence for the Ohio Teacher Evaluation System.

In summary, Ohio’s statewide testing system continues to improve while the use of the system continues to grow. Over 63,000 high school students were tested during the 2013–2014 assessment cycle as part of the Pathway Testing Project. The staff of CETE’s Assessment Services is proud to offer our expertise in the design, development, and deployment of content-valid tests of achievement. These tests provide data that can be turned into information for CTE programs across the state to help them operate efficiently and effectively.

Contributor: Jim Austin