In July 2012, we signed an interagency agreement with the Bureau of Labor Statistics (BLS) to test occupational data collection methods that could lead to the development of a new Occupational Information System (OIS). The new OIS will replace the outdated Dictionary of Occupational Titles (DOT) in our disability determination process. In fiscal year 2013, BLS began testing the feasibility of using the National Compensation Survey (NCS) platform as a means to gather the occupational data we need for our OIS.
Background – The DOT and O*NET
Our disability program policy requires that we follow five steps of sequential evaluation to determine whether adult claimants qualify for disability benefits. Through step three, we make eligibility decisions based on information about the severity of claimants’ medical impairments. At step four, we compare claimants’ functional abilities with the demands of their past work, as they describe it, or as generally performed in the national economy, and determine if they can return to that work. If they cannot return to their past work, at step five we determine if there are other types of work in the national economy that the claimant can perform.
Currently, we base these medical-vocational decisions at steps four and five on the occupational information found in the Department of Labor’s (DOL) DOT and its companion volume, the Selected Characteristics of Occupations (SCO). Although DOL did not design the DOT for our use, we adapted our disability program to it by incorporating many of its concepts and definitions into our regulations and policy. However, DOL stopped updating the DOT in 1991 and replaced it with the Occupational Information Network (O*NET), which was designed for training and career exploration. O*NET’s definitions of some occupational measures do not conform to requirements in our regulations, and we are not able to use O*NET in its current format in our disability adjudication process.
More than half of the decisions we make at the initial level and over 80 percent at the hearing level are medical-vocational decisions that require current occupational information about work that exists in the national economy. Thus, we face the critical challenge to develop or adapt an OIS that can replace the DOT.
Advisory Panel Established
In August of 2008, we assembled a project team to develop the OIS, and in December 2008, we established the Occupational Information Development Advisory Panel (the Panel), in accordance with the Federal Advisory Committee Act. The Panel made recommendations to us regarding OIS development and held regular public meetings. These meetings served the important purpose of allowing external stakeholders, such as disability advocates and vocational experts, to share their advice and concerns regarding our OIS development. During the first three years of OIS development, our project staff performed research to identify our specific disability program needs for an OIS. The charter for the Panel expired on July 6, 2012.
Interagency Agreement with BLS
Although the Panel recommended that we develop a new OIS tailored specifically for our disability program needs, Congress expressed concern over the projected cost of the project and suggested that we work with DOL to develop the updated occupational information we need. In July 2012, we signed an interagency agreement with BLS to test the feasibility of using the NCS platform to collect updated occupational information.
Specifically, BLS will collect:
- An indicator of "time to proficiency," defined as the amount of time required by the typical worker to learn the techniques, acquire the information, and develop the facility needed for average job performance, comparable to the Specific Vocational Preparation (SVP) used in the DOT.
- Physical Demand (PD) characteristics/factors of occupations measured in such a way to support SSA disability program needs, comparable to measures in Appendix C Physical Demands of the SCO.
- Environmental conditions that replicate as closely as possible those listed in Appendix D Environmental Conditions of the SCO, or specific revisions or additions to these factors as agreed upon by SSA and BLS.
FY 2013 Accomplishments
In FY 2013, The OIS team conducted a three-phase field test to evaluate survey design options; begin developing collection protocols and aids; collect data to test and refine the protocols and aids; and assess initial feasibility of using the NCS as a platform to collect the information we need for the OIS.
- Phase 1 - Initial Proof of Concept Testing: The BLS created and tested an initial set of data collection protocols and collection aids to assess the feasibility of using NCS field economists to describe the survey to respondents and collect occupational information suitable for our needs. BLS conducted testing for this phase in the Washington, D.C. area from November 28, 2012 through December 6, 2012. BLS staff completed interviews with 27 establishments, collecting detailed information on our data elements for 104 occupations.
- Phase 2 - Collection Protocol Testing: The BLS continued to refine the collection protocols and aids and tested collection of the new data elements while performing a selection of occupations from each respondent. BLS conducted testing in the Indianapolis, IN and Portland OR/Vancouver WA metropolitan areas from January 28, 2013 through March 21, 2013. BLS field economists completed 227 interviews representing 240 establishments, collecting detailed job information for 1,094 occupations.
- Phase 3 - Broad-Scale Testing: The BLS expanded the data collection to six cities and tested its ability to select a sample of occupations within each establishment, collect the new data elements that we need, and collect other NCS data elements that are of research interest to us, such as wages and job-leveling information. A second objective of Phase 3 testing was to assess the feasibility of collecting the data we need in addition to all of the NCS data elements needed to produce the Employment Cost Index, Employer Costs for Employee Compensation, and various benefits products. At the conclusion of the testing period, BLS completed 667 interviews across all test cities, collecting information from 690 establishments and 3,259 occupations.
The results of the FY 2013 testing are promising and demonstrate that the collection of the data elements that we need using a probability selection of occupations in conjunction with selected NCS data elements is viable.
FY 2014 Testing Plans
Testing will continue throughout FY 2014 with the primary goal of improving data quality while minimizing the burden on respondents. In FY 2014, BLS will work to resolve any outstanding issues from FY 2013 testing, and test the collection of data elements that describe the mental and cognitive demands of work. In mid-2014, the BLS will begin a pre-production test that will involve collecting data from a nationwide sample of 2,500 establishments. BLS will design this pre-production test to mirror as closely as possible the survey production environment. The pre-production test is set to conclude in FY 2015.
Partnership with DOL Employment and Training Administration (ETA)
Our past research and work done by the Panel indicate that we cannot use O*NET in our disability program without significant modifications. The Panel offers a detailed discussion of our concerns with O*NET and Q&As with National Academy of Science staff on issues with O*NET.
However, we believe there are aspects of O*NET that we could effectively integrate into our OIS and avoid duplicating work already done by DOL. BLS will test classifying occupations using O*NET’s framework. We will work with the ETA to incorporate O*NET task descriptions and lay titles and identify other O*NET descriptors that might be useful for disability adjudication. The occupational data that BLS collects will supply the critical information we need regarding the exertional and nonexertional requirements of occupations, SVP, and environmental factors based on our current policy definitions. We hope our new OIS will incorporate the O*NET elements that we determine we can use with the new occupational characteristics collected by BLS.