Student Occupational Competency Testing is required by Pennsylvania Department of Education's Bureau of Career and Technical Education (BCTE) for all concentrations in approved career and technical education programs in Pennsylvania.
To assist schools in meeting this requirement, BCTE developed a number of resources, including A Guide to Student Occupational Competency Testing in Pennsylvania (PDF). The guide is provided to assist school administrators in preparing for the required testing and can also be used as a resource that addresses procedures for participating in federal and state mandated testing.
Pennsylvania utilizes two methods of measuring the occupational competency of career and technical education students. The two methods are assessments from the National Occupational Competency Testing Institute (NOCTI) and the National Institute of Metalworking Skills (NIMS). NOCTI's Job Ready tests are designed to measure an individual's knowledge of basic processes, including the identification and use of terminology and tools. Job Ready tests can be used for both secondary and postsecondary education as well as business and industry applications.
Students who successfully meet the state's requirements are awarded a Pennsylvania Skills Certificate (PDF). This certificate is an important addition to their portfolios when exiting the secondary career and technical education program.
Customized NOCTI Test Usage Policy
The Pennsylvania Department of Education (PDE), Bureau of Career and Technical Education (BCTE) contracted with NOCTI for the development of several customized tests. These tests are intended for use by Pennsylvania secondary schools with PDE-approved career and technical education (CTE) programs in conjunction with the Technical Assistance Program (pre-tests) and the Student Occupational Competency Testing Program (post-tests) in Pennsylvania. Post-tests may also be used by these schools that offer a corresponding adult program. Secondary schools with CTE programs that are not PDE-approved and postsecondary institutions are restricted from using these tests.
Ordering Information
Pennsylvania customized tests and test materials can only be ordered on Pennsylvania’s pre-test, study guide, and post-test order forms for students who meet the criteria of Pennsylvania’s testing policies. Pennsylvania customized tests are NOT available on NOCTI’s standard order form.
Requests to use Pennsylvania customized tests for students that do not meet the criteria of Pennsylvania’s testing policies must be submitted and reviewed by BCTE. Requests are to be sent to Lori Bell at lvb6@psu.edu.
For more information contact Lori Bell, Penn State Greater Allegheny: (412) 675-9065 or lvb6@psu.edu.
PA Cut Scores
BCTE develops Pennsylvania-specific performance levels and cut scores in conjunction with the Student Occupational Competency Testing program. The performance levels for career and technical education and of program testing are Basic, Competent, and Advanced. Those students who achieve the Advanced level will achieve a Pennsylvania Skills Certificate (PDF).
NOCTI Job Ready Student Assessments are revised approximately every two years. A new cut score must be developed each time an assessment is revised/updated. This is done using a statistical method called the Nedelsky Method (1954) which is specifically designed for assessments which use multiple choice items. A qualified panel of 12 or more subject matter experts (SME) is assembled using business and industry representatives as well as educators from across Pennsylvania. Committees are trained on the Nedelsky method in live training sessions and the actual assessment review is done online using the NOCTI Online Criterion Based Cut Scores System. Individual subject matter expert scores are calculated automatically by the NOCTI Online Cut Score System and are sent to Penn State Greater Allegheny and to BCTE. Cut scores are finalized by BCTE and released to schools and NOCTI. NOCTI uses these cut scores to generate score reports for each school.
Nedelsky Benchmarking Process Description
A panel of at least 12 qualified SMEs with clear instructions is used for this process. Each SME is instructed to evaluate each question by crossing out the responses that a minimally competent examinee should be able to eliminate as obviously incorrect. Each question is then given the reciprocal of the number of items/choices not crossed out. The sum of the reciprocals over all items in the assessment is denoted as the probable score of a minimally qualified examinee for a single SME. These values are averaged over all SMEs to arrive at the mid-point of the Competent Range. Then, two standard errors of measurement are subtracted from this average and two standard errors of measurement are added to this average to define the Competent Range. The low point of this range is the number of items that must be answered correctly on the assessment for the student to be deemed Competent. The Bureau has determined the Advanced level to be 0.1 above the high point of the Competent Range. The Basic level is 0.1 below the low point of the Competent Range. The cut scores determined through this methodology are used for each assessment until that assessment is revised. Then a new cut score is determined.