Earnings and Disability Program Participation of Youth Transition Demonstration Participants after 24 Months

by Jeffrey Hemmeter
Social Security Bulletin, Vol. 74 No. 1, 2014

Using data from Social Security Administration (SSA) program records, this article evaluates employment and SSA disability program payment outcomes for youths participating in SSA's Youth Transition Demonstration (YTD) project. Participants were randomly assigned to treatment or control groups at each of six project sites. Treatment-group youths in the YTD received extra employment-related and other supports in order to improve educational and vocational outcomes during their transition to adulthood. The YTD had a positive impact on the proportion of youths with earnings in the first and second years after random assignment at three sites, with two of those sites having positive impacts in both years. Additionally, the treatment groups in four sites had higher Supplemental Security Income participation rates 24 months after random assignment.


Jeffrey Hemmeter is an economist with the Office of Program Development, Office of Research, Demonstration, and Employment Support, Office of Retirement and Disability Policy, Social Security Administration.

Acknowledgments: The author thanks Chris Silanskis, Joyanne Cobb, Susan Kalasunas, Thomas Fraker, Arif Mamun, Paul Davies, Deborah Cortwright, and Chris Tamborini for their helpful comments on this article and an earlier paper.

The findings and conclusions presented in the Bulletin are those of the authors and do not necessarily represent the views of the Social Security Administration.

Introduction

Selected Abbreviations
BHBF Broadened Horizons, Brighter Futures
CDR Continuing Disability Review
CPI-W Consumer Price Index for Urban Wage Earners and Clerical Workers
CTP Career Transition Program
CUNY City University of New York
DI Disability Insurance
MPR Mathematica Policy Research
SGA substantial gainful activity
SPI State Partnership Initiative
SSA Social Security Administration
SSI Supplemental Security Income
WINS Work Incentive Network of Supports
YTD Youth Transition Demonstration

Many youths with disabilities need assistance managing their transitions to adulthood. Services available through the education system typically end when youths complete secondary school and are seldom coordinated with adult-based services (Osgood, Foster, and Courtney 2010; GAO 2012). The difficulty of transition to adulthood is compounded for youths receiving means-tested Supplemental Security Income (SSI), who must cope with the added barriers of poverty. Relatedly, many youths who receive SSI drop out of school and encounter the criminal justice system (Loprest and Wittenburg 2007), factors that reduce the likelihood of employment in early adulthood (Hemmeter, Kauff, and Wittenburg 2009). As a result, former child SSI recipients tend to have low labor force participation well into adulthood (Davies, Rupp, and Wittenburg 2009) and remain on the SSI rolls for substantial periods as adults (Rupp and Scott 1995).

To help address these issues, the Social Security Administration (SSA) conducted the Youth Transition Demonstration (YTD) project to identify interventions that improve the educational and vocational outcomes for youths aged 14–25 who receive or potentially qualify for SSI payments or Social Security Disability Insurance (DI) benefits.1 YTD participants received a variety of employment, education, and other services. To encourage work while they participated in the project, youths were also entitled to waivers of certain SSI and DI rules that would otherwise limit the amount of earnings they could keep while remaining eligible for program payments.

Participants were randomly assigned to treatment and control groups at six YTD project sites. Control group members were subject to standard SSA program rules and received the employment services ordinarily available in their communities; treatment group members were eligible for the program rule waivers and enhanced employment supports. Participant surveys conducted 1 year after random assignment indicated that treatment groups received higher levels of employment services than control groups at all sites, but experienced no significant reductions in benefit receipt. Although employment increased for participants at three sites, no other statistically significant outcomes emerged (Fraker and others 2011a, 2011b, 2011c, 2012a, 2012b, 2012c). Early results, however, may be misleading. Project impacts may fade as participants no longer receive services. Alternatively, impacts may increase as youths mature and, over time, apply the skills obtained in the project. As part of the long-term research strategy for the YTD, in this article I expand on those early results, using SSA program data to describe the impact of the YTD on earnings and on SSI and DI program participation in the 2 years following random assignment.2 Future reports will assess these outcomes using both administrative and survey data.

At three of the six sites, the YTD had a positive impact on the proportion of youths with earnings 2 years after random assignment. At two of those sites, and at one site among the other three, the YTD also had positive impacts on earnings in the first year after random assignment. One of those sites guaranteed a summer job in the first year to all treatment group members interested in employment; however, that guarantee was not in place in the second year. Average earnings for the treatment group at that site were lower than those for the control group, consistent with the successful motivation of marginal workers (who would be expected to have lower average earnings) to attempt employment. Additionally, the treatment groups had higher SSI participation rates and payment amounts than control group members in four sites, consistent with the intended effect of the SSI program rule waivers.

YTD Project Overview

SSA started the YTD projects in 2003 to determine whether providing extra employment-related and other supports to youths receiving or potentially qualifying for SSI or DI benefits would improve educational and economic outcomes during their transition to adulthood, thereby eventually reducing dependence on SSA programs. The YTD began with seven sites: one each in California, Colorado, Iowa, Maryland, and Mississippi, and two in New York. The sites provided supports and services to help “youth with disabilities maximize their economic self-sufficiency as they transition from school to work.” Each site worked under a cooperative agreement with SSA to “collaborate among federal, state, and local agencies to develop and implement sustainable improvements in the delivery of transition services and supports” and to “test ways to remove other barriers to employment and economic self-sufficiency” (SSA 2003).

Although services and supports differed somewhat across sites, each site provided SSA program benefits counseling, career counseling, personalized planning, family counseling, opportunities for family involvement in client services and activities, and coordination of services (Martinez and others 2010). Participants in each site were also eligible for waivers of SSA program rules that would otherwise restrict their eligibility for payments if earnings exceeded certain thresholds. Also, the termination of SSI or DI eligibility due to a finding of medical improvement in a continuing disability review (CDR) or age-18 redetermination was deferred while youths participated in the YTD. Table 1 describes the standard SSA disability program work incentives and the associated YTD waivers of certain program rules.

Table 1. SSI and DI work incentives and the effects of associated YTD program rule waivers
Work incentive Description Policy change under YTD waiver
SSI
Student Earned Income Exclusion (SEIE) Enables recipients who are students to exclude a certain amount of earnings from countable income and thus avoid reductions in SSI payments. In 2009 and 2010, SSA excluded the first $1,640 of a student's earnings each month, to a maximum of $6,600 in a year. SEIE eligibility ends when a recipient attains age 22. Age limit is waived for YTD participants for as long as they attend school regularly.
General Earned Income Exclusion (GEIE) Enables most SSI recipients to exclude from countable income the first $65 of earnings plus one-half of additional earnings. YTD participants can exclude from countable income the first $65 of earnings plus three-quarters of additional earnings.
Plan to Achieve Self-Support (PASS) Enables SSI recipients to exclude from countable income and resources amounts paid for certain expenses, such as the cost of owning a car, pursuing an education, and purchasing assistive technology, to achieve a specific SSA-approved work goal. YTD participants can also use a PASS to explore career options or pursue additional education.
Individual Development Account (IDA) Provides a trust-like account for SSI recipients to save for a specific goal, such as purchasing a home, going to school, or starting a business. SSA matches earnings deposited in an IDA, often at $2 for every $1 deposited by the participant. The money accumulated in an IDA is excluded when determining SSI eligibility, and the earnings deposited during a month are excluded when determining the SSI payment amount. A YTD participant may also use an IDA to save for other approved goals.
SSI and DI
Continuing Disability Reviews and Age-18 Redeterminations (Section 301) Benefits based on disability may continue despite a negative Continuing Disability Review or age-18 medical redetermination if:
  • the beneficiary is participating in any of certain programs, and
  • SSA determines that continued participation will increase the likelihood that the individual will remain off the disability rolls permanently once benefits stop.
These “likelihood” determinations normally must be made on a case-by-case basis.
If SSA determines that medical disability has stopped and the participant is no longer eligible for assistance, he or she can continue to receive both cash benefits and health care services while participating in the YTD.
SOURCE: SSA (2013) and “YTD Modified SSI Program Rules (Waivers) Descriptions” (http://www.socialsecurity.gov/disabilityresearch/ytdmodifiedssi.html).

After a brief pilot phase, MDRC (a nonprofit contractor) reviewed the seven sites “to determine the feasibility of conducting a national random assignment evaluation of YTD and explore each project's appropriateness for and interest in such an evaluation” (Martinez and others 2010, 4). Based on that review, SSA terminated two sites (Iowa and Maryland) “because of difficulty they had reaching the goals stated in their cooperative agreements” (SSA 2008). Two other sites (California and Mississippi) continued as originally intended because of their overall strong service design; however, they were unable to implement the revisions recommended in MDRC's report, and are not discussed here.3 The other three projects—Colorado Youth Work Incentive Network of Supports (WINS); Transition WORKS in Erie County, New York; and the City University of New York (CUNY) YTD Project in Bronx County, New York—continued with a slightly revised YTD design. I refer to these three as the phase 1 sites.

The revised YTD included a stronger evaluation design, a technical evaluation, and greater emphasis on employment services (SSA 2008). Mathematica Policy Research (MPR) oversaw the implementation and evaluation of the revised YTD project; subcontractors provided services at each site. Additionally, TransCen, Inc. provided technical assistance to each of the projects, focusing on employment supports. After a national search, SSA selected three other sites—Career Transition Program (CTP) in Montgomery County, Maryland; Broadened Horizons, Brighter Futures (BHBF) in Miami-Dade County, Florida; and West Virginia Youth Works—to join the YTD project. I refer to these three as the phase 2 sites.

Recruitment and Enrollment

The original YTD was designed to serve all eligible youths. The project revision brought about the random assignment of participants into treatment and control groups. This change was difficult for some of the sites, but three of the original sites (the phase 1 sites) successfully modified their projects to accommodate the new recruitment and enrollment design. For example, the Erie County site changed from a classroom-based design to individualized services to allow random assignment into treatment and control groups.

MPR recruited potential participants from lists of SSI recipients and DI beneficiaries aged 14–25 in five of the sites' service areas. MPR randomized those lists and recruited the youths into the YTD project. After the youths voluntarily consented to be part of the project, MPR randomly assigned them to either the treatment group or the control group. If the sibling of a randomized youth was also eligible for the project, the sibling was assigned to the randomized youth's group but was not included in the research sample. MPR also conducted a baseline survey of the youths before randomization.

The new Maryland site followed a slightly different protocol. Its service population included youths considered to be at risk of needing, but not yet receiving, SSI or DI benefits. Site staff recruited youths directly into the project primarily from the local school district. After they consented to be part of the project, youths were processed as described for the other sites.

After randomization, staff at all sites contacted the treatment-group youths and enrolled them into project services. Each of the six sites enrolled between 79 and 89 percent of randomized treatment-group youths into services.

Project Services

Most of the types of services provided at YTD projects were those recommended by the National Collaborative on Workforce Disability for Youth (NCWD 2005), although some were drawn from “best practices” of other interventions for youths with disabilities. The YTD project's core interventions addressed the barriers youths face in their transition from school to work. Chart 1 depicts the barriers and the YTD intervention components, along with the transition environment and key project outcomes. The YTD specifically addressed these six barriers:

Chart 1.
YTD design objectives
Flowchart linked to information in narrative format.
SOURCE: Adapted from Rangarajan and others (2009).

Each of the YTD sites offered services to break down these real or perceived barriers to varying degrees. They offered individualized work-based experiences, including internships, job shadowing, job coaching, and competitive paid employment. They offered empowerment training to help participating youths learn to make their own choices (as opposed to having a parent or guardian choose for them). The sites also addressed the barriers by working with the families to break down misunderstandings about program rules; encouraging the families to participate in planning for the youths' self-sufficiency; working closely with local community services to link the educational and work supports for youths with disabilities, smoothing the transition to needed services; and providing case management to coordinate health and other social services. Based on results from the phase 1 sites showing lower-than-expected levels of employment, the phase 2 sites received enhanced technical assistance focused on finding ways to promote competitive employment experiences.

Youths in the treatment groups were eligible for waivers that allowed them to keep more of their earnings or save their earnings for a work or educational goal without affecting their SSI payment. To accompany those waivers, the sites provided DI and SSI benefits counseling. Virginia Commonwealth University trained the benefits counselors, as it had for an earlier program that provided counseling to DI and SSI beneficiaries under the Ticket to Work Act.4

The short-term objectives for the YTD project included encouraging participation in employment-promoting activities, increasing paid employment and income, improving attitudes and expectations, and enabling better educational outcomes. In the long term, the YTD project sought to increase participants' paid employment and total income, improve their self-determination, increase their general participation in productive activities (education, training, or employment), and reduce their contact with the justice system.

Site Descriptions

Although all the project sites conformed to a general YTD model, their implementations differed. This section provides a broad overview of each site's services and includes a reference to the site's detailed interim report.

Colorado Youth WINS

The Colorado site was run by the WINS Partners at the University of Colorado Health Sciences Center. Colorado Youth WINS served youths aged 14–25 who received SSI payments or DI benefits in Larimer, El Paso, Pueblo, and Boulder counties. A team of staff members provided services in each of the counties' One Stop Workforce Centers.5 The team included a disability program navigator,6 a benefits counselor, and one or more career counselors. Four hundred sixty-eight youths were randomized into the Colorado Youth WINS treatment group; 401 enrolled in services (86 percent). The control group comprised 387 youths. Random assignment occurred between August 2006 and March 2008, and services ended in the fall of 2009. Youths were eligible for services for at least 18 months.

To help fill gaps in youths' access to services from existing sources—such as the Division of Vocational Rehabilitation, local school systems, and other providers—Colorado Youth WINS focused primarily on case management, disability program navigation, and benefits counseling. Project staff also developed person-centered plans to help the youths identify educational, employment, and benefit goals and needs. Family members were included in most of the discussions. Career counselors provided vocational assessments and career exploration activities (such as job-site visits). By locating in the One Stop Workforce Centers, Colorado Youth WINS provided easy access to job development and placement services. For more information on the Colorado Youth WINS project, see Fraker and others (2011a).

New York: CUNY YTD Project

The CUNY site worked with youths aged 16–19 who received SSI payments or DI benefits. It was run by CUNY's John F. Kennedy Institute for Worker Education at the Hostos and Lehman campuses in Bronx County. Project staff included benefits counselors, parent advocates, career developers, and students at the partner campuses. An advisory committee of community groups, campus experts, and public agencies provided direction for the program and suggested potential service partners and ways to link the program with community services. Four hundred ninety-two youths were randomized into the CUNY treatment group; 387 enrolled in services (79 percent). The control group comprised 397 youths. Random assignment occurred between July 2006 and November 2008, and services ended in May 2010.

Treatment-group youths received direct services for 1 school year, after which summer employment and limited follow-up services were available. Youths attended Saturday sessions offering recreational and social activities and workshops on self-determination, benefits planning, and career development. Students from the partner campuses who enrolled in a course on working with youths with disabilities led the social and recreational activities; many of those students (and other staff) had disabilities themselves. While youths attended these activities, family members met to discuss benefits and issues related to the youths' self-determination. Youths also developed person-centered plans for identifying and achieving their goals. Parent advocates checked in with families to ensure they participated and met with the people who could address the youths' (or parents') needs. Services culminated with an offer of 7 weeks of summer employment through New York City's Summer Youth Employment Program or in an on-campus job.7 For more information on the CUNY project, see Fraker and others (2011b).

New York: Erie County Transition WORKS

The Erie County project served youths aged 16–25 who received either SSI payments or DI benefits in Erie County, New York (which includes the city of Buffalo). The Erie 1 Board of Cooperative Educational Services ran the project. They partnered with the Parent Network of Western New York, Neighborhood Legal Services, the Community Employment Office, and other agencies to provide services emphasizing self-empowerment. Youths were eligible for 18 months of services, with some employment supports available after that. Four hundred fifty-nine youths were randomized into the treatment group; 380 enrolled in services (83 percent). There were 384 youths in the control group. Random assignment occurred between January 2007 and March 2008, and services ended in the fall of 2009.

The Erie County service delivery schedule was very structured, with youths attending person-centered planning and self-determination workshops before receiving employment- or education-related services. The youths set short- and long-term goals for themselves (with the help of a counselor) and created a transition plan to meet those goals. Youths were also trained to organize important documents related to their benefits in a binder, to which they could easily refer as needed. Job developers helped conduct formal vocational assessments to learn about the youths' interests, arrange for informal work experiences (such as job shadowing, job-site tours, and mock interviews), and set up paid employment and internships when participants were ready. If youths were interested in continuing their education, transition coordinators helped them explore their options (for example, earning a high school equivalency degree or enrolling in college). For more information on the Erie County project, see Fraker and others (2011c).

Maryland: Montgomery County CTP

St. Luke's House, Inc., a local mental health provider, ran the CTP. This project served youths aged 16–22 who either were in the last 2 years of high school or had left high school in the past 12 months. Unlike the other projects, the CTP served a subset of the overall YTD target: youths with severe emotional disturbances who might be eligible for SSI payments or DI benefits. Roughly 12 percent of participants received either SSI or DI at the time of random assignment. Treatment-group participants were eligible for up to 18 months of services plus 2 years of follow-up services. MPR randomized 422 youths into the treatment group, 374 of whom enrolled in services (89 percent); 383 youths were in the control group. Random assignment occurred between April 2008 and December 2010, and services ended in March 2012.

After enrolling in the CTP, youths received individualized services focused on employment, beginning with an interview with a career transition specialist to determine the areas in which they needed support to accomplish their goals. The career transition specialist would meet with each youth once a week to develop service needs and interim goals. The youths then received a variety of preemployment vocational assessments (such as mock interviews, work trials, or job shadowing) to prepare them for competitive employment. CTP staff worked directly with employers and community employment resources to help place youths in jobs. Concurrent with those employment services, staff provided benefits counseling, case management, referrals to social and health services, and family supports. For more information on the CTP, see Fraker and others (2012a).

Florida: Miami-Dade County BHBF

Abilities, Inc. of Florida (now ServiceSource) ran the BHBF project. They worked with SSI or DI beneficiaries aged 16–22. Youths received up to 18 months of services, plus follow-up services as needed. BHBF enrolled 388 (84 percent) of the 460 youths randomized into the treatment group; 399 youths were in the control group. Random assignment occurred between April 2008 and September 2010, and services ended in March 2012.

As part of a strong benefits-counseling component, BHBF partnered with the Human Services Coalition and the National Disability Institute to provide financial literacy training along with many career preparation and placement services. The program included one-on-one and group services. For example, although youths developed individual plans for reaching their goals, BHBF also hosted job fairs to connect them with employers and provided work-based experiences, enabling youths to explore different jobs. For more information on BHBF, see Fraker and others (2012b).

West Virginia Youth Works

The Human Resources Development Foundation, Inc. ran the Youth Works project in 19 counties, where they provided up to 18 months of services to youths aged 15–25 receiving SSI or DI benefits. The project enrolled 388 (85 percent) of the 455 youths randomized in the treatment group; 397 youths were assigned to the control group. Random assignment occurred between March 2008 and September 2010, and services ended in March 2012.

Youth Works' services began with goal identification, in which youths received one-on-one benefits counseling and their interests were assessed. This culminated in a person-centered plan for future services and goals. Job placement activities followed to help improve youths' job search and “soft” skills. Youths were taught how to search for positions on their own, but were supported by customized employment specialists. Once working, the youths received job coaching and other employment supports. For more information on West Virginia Youth Works, see Fraker and others (2012c).

Data and Methods

The data for this study come from SSA program records matched to the randomization and enrollment dates provided by MPR. SSA's Master Earnings File provides data on all earnings reported on an individual's W-2 tax record (including non-Federal Insurance Contribution Act taxable earnings). The Supplemental Security Record and the Master Beneficiary Record respectively contain the program participation histories for SSI recipients and DI beneficiaries. Dates of death come from the Numerical Identification System (Numident) file, which contains SSA's Death Master List.

The outcomes of interest in this study are earnings and SSA program payments. Earnings include any income reported as wages or self-employment earnings on the Master Earnings File. Wage data are from W-2 records. If an individual worked more than one job in a given year, wage data from each W-2 are totaled to provide a single measure of annual earnings.

Total SSA program payments represent the sum of SSI payments due and DI benefits paid. SSI payments due are those the recipient should have received in a month if he or she reported all earnings and exclusions in a timely manner and if work incentives (including the YTD waivers) were applied correctly. Using this measure smooths the payment stream for each recipient.8 Smoothing the SSI payment data is especially important given the potential complications arising from YTD waiver implementation. Taking such a step is not as important for DI. There are relatively few DI beneficiaries in this sample, and because they are generally not subject to a monthly earnings test, their monthly benefit amounts are much less likely to vary. Additionally, only one of the five program rule waivers applied to DI beneficiaries, and it affected benefit receipt or nonreceipt, not the payment amount. Thus, actual DI benefits should not fluctuate to the extent that SSI payments do. Results based solely on payments made (for both SSI and DI) are available upon request from the author.

As noted, MPR used random assignment to place individuals into treatment and control groups. Therefore, comparisons of the outcomes between the treatment and control groups are unbiased and measure the average impact we would expect to see in a program in which some individuals choose not to receive services for which they are eligible; that is, the results measure the intent-to-treat impact.9 I do not use weights and the results do not control for any known differences in the characteristics of the two groups at the time of randomization.10 All dollar values are adjusted to 2012 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W).

The administrative records capture program participation on a monthly basis, so the SSI and DI results include all YTD participants who remain alive through the 24th month after random assignment. Although random assignment occurred irregularly over a 2-year period for each site, administrative earnings data are available only as calendar-year amounts. Therefore, I measure earnings in the first and second calendar years after random assignment. For example, if an individual was randomized in August 2009, the first and second years after random assignment are 2010 and 2011, respectively. For many participants, earnings in the second year represent those in their first year without program services. The few individuals who died over the course of this study are included in the analysis up to the month (for program participation) or year (for earnings) before death.

Results

This section presents the YTD project outcomes as of 24 months (for SSI and DI participation) and 2 years (for earnings) after participants were randomly assigned to treatment or control groups.

Characteristics of YTD Participants

Table 2 shows the characteristics of YTD participants. There were few statistically significant differences between the treatment and control groups, and those occurred only in the phase 1 sites. In CUNY, treatment-group youths were more likely than control-group youths to have mental disorders (other than intellectual disability) as their primary impairment, by 8.4 percentage points. In Erie County, treatment-group members were 7.4 percentage points more likely than control-group members to have nonintellectual disability mental disorders. Treatment-group youths in Colorado were 7.5 percentage points more likely to be male than control-group members; they were also 6.4 percentage points more likely to have an intellectual disability. There were no statistically significant differences in participant characteristics in the phase 2 sites.

Table 2. Selected characteristics of YTD participants, by site
Characteristic Treatment group Control group Difference p-value
Percent Standard error Percent Standard error Percentage points Standard error
  Phase 1 sites
  New York: CUNY
Male 68.1 2.1 66.5 2.4 1.6 3.2 0.62
Disability
Intellectual disability 25.4 2.0 29.7 2.3 -4.3 3.0 0.15
Other mental disorders 53.5 2.2 45.1 2.5 8.4*** 3.4 0.01
Nervous system and other sensory disorders 6.5 1.1 8.6 1.4 -2.1 1.8 0.24
Other disabilities 14.6 1.6 16.6 1.9 -2.0 2.5 0.42
Age at random assignment
15 or younger 15.9 1.6 19.4 2.0 -3.5 2.6 0.17
16 44.5 2.2 40.6 2.5 4.0 3.3 0.24
17 31.7 2.1 33.2 2.4 -1.5 3.2 0.63
18 or older 7.9 1.2 6.8 1.3 1.1 1.8 0.52
Number 492 397 . . . . . . . . .
  New York: Erie County
Male 62.1 2.3 61.7 2.5 0.4 3.4 0.91
Disability
Intellectual disability 36.4 2.2 38.5 2.5 -2.2 3.3 0.52
Other mental disorders 39.2 2.3 31.8 2.4 7.4** 3.3 0.02
Nervous system and other sensory disorders 8.1 1.3 10.4 1.6 -2.4 2.0 0.24
Other disabilities 16.3 1.7 19.3 2.0 -2.9 2.7 0.27
Age at random assignment
15 or younger 1.3 0.5 1.0 0.5 0.3 0.7 0.72
16 12.4 1.5 10.2 1.5 2.3 2.2 0.30
17 10.5 1.4 11.7 1.6 -1.3 2.2 0.56
18 or older 75.8 2.0 77.1 2.1 -1.3 2.9 0.67
Number 459 384 . . . . . . . . .
  Colorado
Male 60.3 2.3 52.7 2.5 7.5** 3.4 0.03
Disability
Intellectual disability 33.8 2.2 27.4 2.3 6.4** 3.1 0.04
Other mental disorders 30.3 2.1 34.9 2.4 -4.5 3.2 0.16
Nervous system and other sensory disorders 17.5 1.8 17.1 1.9 0.5 2.6 0.86
Other disabilities 18.4 1.8 20.7 2.1 -2.3 2.7 0.40
Age at random assignment
15 or younger 10.0 1.4 10.1 1.5 -0.0 2.1 0.99
16 8.5 1.3 7.8 1.4 0.8 1.9 0.67
17 5.3 1.0 5.9 1.2 -0.6 1.6 0.70
18 or older 76.1 2.0 76.2 2.2 -0.2 2.9 0.96
Number 468 387 . . . . . . . . .
  Phase 2 sites
  Maryland
Male 67.6 2.3 66.5 2.4 1.1 3.3 0.73
Disability
Intellectual disability 2.9 0.8 3.9 1.0 -1.1 1.3 0.40
Other mental disorders 19.5 1.9 19.9 2.0 -0.4 2.8 0.89
Nervous system and other sensory disorders 1.0 0.5 0.5 0.4 0.4 0.6 0.48
Other disabilities 76.7 2.1 75.7 2.2 1.0 3.0 0.74
Age at random assignment
15 or younger 0.5 0.3 0.8 0.5 -0.3 0.6 0.58
16 18.3 1.9 19.4 2.0 -1.0 2.8 0.71
17 27.4 2.2 26.2 2.2 1.2 3.1 0.70
18 or older 53.8 2.4 53.7 2.6 0.1 3.5 0.97
Number 420 382 . . . . . . . . .
  Florida
Male 58.0 2.3 61.4 2.4 -3.4 3.4 0.32
Disability
Intellectual disability 32.6 2.2 34.6 2.4 -2.0 3.2 0.54
Other mental disorders 45.7 2.3 43.6 2.5 2.0 3.4 0.55
Nervous system and other sensory disorders 9.1 1.3 7.5 1.3 1.6 1.9 0.40
Other disabilities 12.6 1.5 14.3 1.8 -1.7 2.3 0.47
Age at random assignment
15 or younger . . . . . . . . . . . . . . . . . . . . .
16 4.1 0.9 4.0 1.0 0.1 1.4 0.93
17 13.0 1.6 13.3 1.7 -0.2 2.3 0.92
18 or older 82.8 1.8 82.7 1.9 0.1 2.6 0.96
Number 460 399 . . . . . . . . .
  West Virginia
Male 58.5 2.3 57.2 2.5 1.3 3.4 0.71
Disability
Intellectual disability 31.2 2.2 30.0 2.3 1.2 3.2 0.70
Other mental disorders 42.0 2.3 44.8 2.5 -2.9 3.4 0.40
Nervous system and other sensory disorders 10.3 1.4 9.6 1.5 0.8 2.1 0.71
Other disabilities 16.5 1.7 15.6 1.8 0.9 2.5 0.73
Age at random assignment
15 or younger 0.9 0.4 0.8 0.4 0.1 0.6 0.84
16 5.5 1.1 6.5 1.2 -1.1 1.6 0.52
17 10.1 1.4 11.1 1.6 -1.0 2.1 0.64
18 or older 83.5 1.7 81.6 1.9 1.9 2.6 0.46
Number 455 397 . . . . . . . . .
SOURCE: Author's calculations using SSA program records.
NOTES: Totals do not necessarily equal the sum of rounded components.
. . . = not applicable.
* = statistically significant at the .10 level (two-tailed t-test).
** = statistically significant at the .05 level (two-tailed t-test).
*** = statistically significant at the .01 level (two-tailed t-test).

Although the characteristics within each site were generally similar, differences between the sites are worth highlighting. Over three-fourths of the Erie County and Colorado sites' youths were aged 18 or older at the time of enrollment, and over 80 percent of the Florida and West Virginia participants were 18 or older at enrollment. Conversely, only about 7 percent of the CUNY youths and just over one-half of the Maryland youths were 18 or older. Over 40 percent of CUNY, Florida, and West Virginia participants had mental disorders other than intellectual disability. However, because many of the Maryland youths were not receiving SSI or DI, their disabilities are not identified in the administrative records, for which reason they are grouped in the “other disabilities” category. The different characteristics of the populations served (and the different program models) indicate the risk of comparing outcomes between one site and another.

SSI and DI Program Participation

Because five of the six sites recruited participants from SSA program lists, it is not surprising that almost all youths in those five sites received either SSI payments or DI benefits in the month of random assignment (Table 3). At those sites, between 76.7 percent and 95.1 percent of participants were due an SSI payment, and at all six sites, SSI receipt was more prevalent than DI receipt. The shares of youths receiving DI benefits ranged from 2.6 percent to 25.6 percent, and many were auxiliary beneficiaries eligible for DI benefits based on somebody else's earnings record, typically that of a parent. However, some youths received DI primary (worker) benefits, particularly in the sites with older participants. In Colorado, for example, 10 percent of youths received DI worker benefits (not shown; breakdowns for DI worker and auxiliary benefits are available upon request).

Table 3. SSA disability program participation of YTD participants in the month of random assignment, by site
Site and program Treatment group Control group Difference p-value
Percent Standard error Percent Standard error Percentage points Standard error
New York: CUNY
SSI 95.1 1.0 89.7 1.5 5.4*** 1.8 0.00
DI 3.7 0.8 4.0 1.0 -0.4 1.3 0.77
Either 95.5 0.9 90.2 1.5 5.4*** 1.8 0.00
Both 3.3 0.8 3.5 0.9 -0.3 1.2 0.82
New York: Erie County
SSI 89.3 1.4 87.0 1.7 2.3 2.2 0.29
DI 19.4 1.8 23.2 2.2 -3.8 2.8 0.18
Either 96.7 0.8 94.8 1.1 1.9 1.4 0.16
Both 12.0 1.5 15.4 1.8 -3.4 2.4 0.15
Colorado
SSI 84.2 1.7 85.8 1.8 -1.6 2.4 0.52
DI 23.5 2.0 25.6 2.2 -2.1 3.0 0.48
Either 95.5 1.0 95.3 1.1 0.2 1.4 0.91
Both 12.2 1.5 16.0 1.9 -3.8 2.4 0.11
Maryland
SSI 8.8 1.4 14.1 1.8 -5.3** 2.3 0.02
DI 3.8 0.9 2.6 0.8 1.2 1.2 0.34
Either 11.9 1.6 15.7 1.9 -3.8 2.4 0.12
Both 0.7 0.4 1.0 0.5 -0.3 0.7 0.61
Florida
SSI 76.7 2.0 79.2 2.0 -2.5 2.8 0.39
DI 11.1 1.5 9.8 1.5 1.3 2.1 0.53
Either 81.1 1.8 82.7 1.9 -1.6 2.6 0.54
Both 6.7 1.2 6.3 1.2 0.5 1.7 0.78
West Virginia
SSI 84.8 1.7 82.9 1.9 2.0 2.5 0.44
DI 22.2 1.9 20.7 2.0 1.5 2.8 0.58
Either 92.7 1.2 90.9 1.4 1.8 1.9 0.33
Both 14.3 1.6 12.6 1.7 1.7 2.3 0.47
SOURCE: Author's calculations using SSA program records.
NOTES: Totals do not necessarily equal the sum of rounded components.
* = statistically significant at the .10 level (two-tailed t-test).
** = statistically significant at the .05 level (two-tailed t-test)
*** = statistically significant at the .01 level (two-tailed t-test).

The few youths not receiving any disability benefit in the month of random assignment either (a) were not receiving a check because of temporary overpayment issues or (b) had recently left the program after a CDR or age-18 redetermination. The projects enrolled those youths because of the strong possibility they would return to SSI or DI (although perhaps only after a successful appeal of the CDR or redetermination decision). In CUNY, nonrecipient youths were more prevalent in the control group, accounting for a 5.4 percentage point difference in SSI participation in the month of random assignment. Treatment-group youths were 5.3 percentage points less likely to receive SSI than the control group in Maryland. Recall that that site did not recruit from program records, instead primarily targeting youths deemed “at risk” of needing SSI or DI.

In the CUNY, Erie County, Florida, and West Virginia sites, the treatment group was significantly more likely to receive SSI payments 24 months after random assignment than the control group (Table 4). In CUNY, the difference in SSI participation after 24 months was 10.9 percentage points and is highly significant. This difference is likely due to the use of program rule waivers; for example, the CUNY treatment-group youths were 3 to 10 percentage points more likely to use the various SSA work incentives (and their associated waivers) than control-group youths (Fraker and others 2011b).

Table 4. SSA disability program participation of YTD participants 24 months after random assignment, by site
Site and program Treatment group Control group Difference p-value
Percent Standard error Percent Standard error Percentage points Standard error
New York: CUNY
SSI 84.9 1.6 74.0 2.2 10.9*** 2.7 0.00
DI 2.2 0.7 3.8 1.0 -1.6 1.2 0.17
Either 85.3 1.6 74.3 2.2 11.0*** 2.7 0.00
Both 1.8 0.6 3.6 0.9 -1.7 1.1 0.11
Died 0.2 0.2 1.0 0.5 -0.8 0.5 0.11
New York: Erie County
SSI 80.5 1.9 74.3 2.3 6.3** 2.9 0.03
DI 20.8 1.9 22.3 2.1 -1.5 2.9 0.60
Either 89.7 1.4 83.8 1.9 5.9* 2.4 0.01
Both 11.6 1.5 12.7 1.7 -1.1 2.3 0.62
Died 0.4 0.3 1.8 0.7 -1.4** 0.7 0.05
Colorado
SSI 81.0 1.8 78.1 2.1 2.9 2.8 0.30
DI 25.3 2.0 26.6 2.3 -1.3 3.0 0.67
Either 92.7 1.2 90.4 1.5 2.3 1.9 0.23
Both 13.6 1.6 14.3 1.8 -0.7 2.4 0.76
Died 1.1 0.5 0.8 0.4 0.3 0.7 0.66
Maryland
SSI 17.0 1.8 18.1 2.0 -1.0 2.7 0.70
DI 4.6 1.0 3.9 1.0 0.6 1.4 0.66
Either 20.4 2.0 20.4 2.1 -0.0 2.9 0.99
Both 1.2 0.5 1.6 0.6 -0.4 0.8 0.65
Died 0.7 0.4 0.0 0.0 0.7* 0.4 0.10
Florida
SSI 72.8 2.1 61.2 2.5 11.7*** 3.2 0.00
DI 14.1 1.6 11.4 1.6 2.7 2.3 0.24
Either 77.5 2.0 66.8 2.4 10.7*** 3.1 0.00
Both 9.5 1.4 5.8 1.2 3.7** 1.8 0.05
Died 1.5 0.6 1.3 0.6 0.3 0.8 0.74
West Virginia
SSI 79.4 1.9 70.7 2.3 8.7*** 3.0 0.00
DI 27.0 2.1 23.0 2.1 4.0 3.0 0.18
Either 89.4 1.4 80.1 2.0 9.3*** 2.5 0.00
Both 17.0 1.8 13.6 1.7 3.4 2.5 0.17
Died 0.7 0.4 0.3 0.3 0.4 0.5 0.39
SOURCE: Author's calculations using SSA program records.
NOTES: Totals do not necessarily equal the sum of rounded components.
* = statistically significant at the .10 level (two-tailed t-test).
** = statistically significant at the .05 level (two-tailed t-test)
*** = statistically significant at the .01 level (two-tailed t-test).

Looking only at the 24-month outcomes, however, masks program participation patterns. For example, although the SSI receipt rate fell by 10.2 percentage points 24 months after random assignment for CUNY treatment-group youths, the drop was relatively greater for the control group (15.7 percentage points). Chart 2 illustrates the changes in SSI and DI benefit receipt at random assignment and after 24 months (from Tables 3 and 4). In five of the six sites, the difference between the proportions of treatment- and control-group youths receiving SSI payments is greater at 24 months. Again, this can be explained by the use of program waivers that maintained youths' SSI eligibility (Maryland, the exception, did not draw its participants from the SSI program rolls). For three sites, the difference between the treatment and control groups in the proportion of youths receiving DI benefits decreased over time. In the CUNY, Florida, and West Virginia sites, however, the differences increased. As noted in Fraker (2013) and discussed later in this article, the Florida and West Virginia sites in particular reported large employment impacts, which may have enabled youths there to earn enough quarters of coverage to qualify for DI benefits.11,12

Chart 2.
Percentage point differences in SSI and DI participation rates between YTD treatment groups and control groups in the month of random assignment and after 24 months, by project site
Two bar charts linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Each value reflects the treatment-group percentage minus the control-group percentage at the given site and time.
* = statistically significant at the .10 level (two-tailed t-test).
** = statistically significant at the .05 level (two-tailed t-test).
*** = statistically significant at the .01 level (two-tailed t-test).

Earnings

In the first year after random assignment, only the CUNY, Florida, and West Virginia sites had statistically significant impacts on the prevalence of earners (Table 5).13 Over 48 percent of CUNY treatment-group youths had earnings, compared with only 23.9 percent of control-group youths. CUNY guaranteed a paid summer job for any treatment-group youth who wanted one, so that finding may reflect a program offering unique to the CUNY site more than the outcome of an intervention. In Florida, 29.8 percent of the treatment-group youths had earnings, compared with 23.6 percent of control-group youths. In West Virginia, 44.0 percent of treatment-group youths had earnings, compared with 28.0 percent of control-group youths. In Maryland, although the result is not statistically significant, 58.8 percent of treatment-group youths had earnings, compared with 55.0 percent of the control group. Recall that the Maryland project served fewer youths receiving SSI payments than the other sites; that site was also located in a service-rich area. Those factors likely contribute to the absence of a statistically significant result. All of these results are similar to those given in the 12-month interim reports.

Table 5. Earnings of YTD participants 1 year and 2 years after random assignment, by site
Earnings characteristic and site Treatment group Control group Difference p-value
Percent Dollars Standard error Percent Dollars Standard error Percentage points Dollars Standard error
  1 year after random assignment
Participants with any earnings (%)
New York: CUNY 48.4 . . . 2.3 23.9 . . . 2.1 24.4*** . . . 3.1 0.00
New York: Erie County 44.2 . . . 2.3 40.6 . . . 2.5 3.6 . . . 3.4 0.29
Colorado 39.7 . . . 2.3 36.7 . . . 2.4 3.1 . . . 3.3 0.36
Maryland 58.8 . . . 2.4 55.0 . . . 2.5 3.8 . . . 3.5 0.27
Florida 29.8 . . . 2.1 23.6 . . . 2.1 6.2** . . . 3.0 0.04
West Virginia 44.0 . . . 2.3 28.0 . . . 2.3 16.0*** . . . 3.2 0.00
Earnings among all participants ($)
New York: CUNY . . . 714 54 . . . 566 94 . . . 148 104 0.15
New York: Erie County . . . 1,788 214 . . . 1,684 210 . . . 104 303 0.73
Colorado . . . 1,738 189 . . . 1,536 190 . . . 202 271 0.46
Maryland . . . 2,808 274 . . . 2,465 214 . . . 343 352 0.33
Florida . . . 1,503 179 . . . 1,161 152 . . . 342 239 0.15
West Virginia . . . 1,776 186 . . . 1,392 187 . . . 384 265 0.15
Earnings among participants with any earnings ($)
New York: CUNY . . . 1,477 88 . . . 2,361 331 . . . -885*** 251 0.00
New York: Erie County . . . 4,044 435 . . . 4,124 447 . . . -80 632 0.90
Colorado . . . 4,364 406 . . . 4,187 438 . . . 177 602 0.77
Maryland . . . 4,764 423 . . . 4,485 329 . . . 279 550 0.61
Florida . . . 5,026 480 . . . 4,927 470 . . . 98 699 0.89
West Virginia . . . 4,040 365 . . . 4,978 538 . . . -937 633 0.14
Participants with earnings exceeding SGA (%)
New York: CUNY 0.0 . . . 0.0 0.5 . . . 0.4 -0.5 . . . 0.4 0.11
New York: Erie County 3.5 . . . 0.9 3.4 . . . 0.9 0.1 . . . 1.3 0.95
Colorado 3.6 . . . 0.9 3.1 . . . 0.9 0.5 . . . 1.2 0.66
Maryland 5.3 . . . 1.1 4.7 . . . 1.1 0.5 . . . 1.5 0.73
Florida 3.3 . . . 0.8 2.3 . . . 0.7 1.0 . . . 1.1 0.37
West Virginia 2.9 . . . 0.8 2.5 . . . 0.8 0.3 . . . 1.1 0.76
  2 years after random assignment
Participants with any earnings (%)
New York: CUNY 35.4 . . . 2.2 29.0 . . . 2.3 6.4** . . . 3.1 0.04
New York: Erie County 38.1 . . . 2.3 36.7 . . . 2.5 1.4 . . . 3.3 0.67
Colorado 44.2 . . . 2.3 34.9 . . . 2.4 9.3* . . . 3.3 0.01
Maryland 37.4 . . . 2.4 36.9 . . . 2.5 0.5 . . . 3.4 0.89
Florida 27.8 . . . 2.1 23.3 . . . 2.1 4.5 . . . 3.0 0.13
West Virginia 22.9 . . . 2.0 14.9 . . . 1.8 8.0*** . . . 2.7 0.00
Earnings among all participants ($)
New York: CUNY . . . 974 120 . . . 1,003 145 . . . -29 187 0.88
New York: Erie County . . . 2,177 254 . . . 1,817 224 . . . 361 346 0.30
Colorado . . . 2,047 225 . . . 1,643 208 . . . 404 312 0.20
Maryland . . . 3,729 366 . . . 3,761 394 . . . -32 537 0.95
Florida . . . 2,050 242 . . . 1,580 201 . . . 470 320 0.14
West Virginia . . . 1,858 241 . . . 1,789 384 . . . 69 441 0.88
Earnings among participants with any earnings ($)
New York: CUNY . . . 2,748 295 . . . 3,427 418 . . . -679 497 0.17
New York: Erie County . . . 5,686 573 . . . 4,883 507 . . . 803 784 0.31
Colorado . . . 4,599 447 . . . 4,698 500 . . . -100 685 0.88
Maryland . . . 5,985 510 . . . 6,081 553 . . . -96 751 0.90
Florida . . . 6,006 566 . . . 5,538 510 . . . 468 793 0.56
West Virginia . . . 4,805 505 . . . 6,943 1,278 . . . -2,138* 1,172 0.07
Participants with earnings exceeding SGA (%)
New York: CUNY 2.2 . . . 0.7 2.0 . . . 0.7 0.2 . . . 1.0 0.83
New York: Erie County 4.6 . . . 1.0 4.7 . . . 1.1 -0.2 . . . 1.5 0.92
Colorado 4.7 . . . 1.0 4.4 . . . 1.0 0.3 . . . 1.4 0.82
Maryland 9.9 . . . 1.9 7.5 . . . 1.7 2.5 . . . 2.6 0.34
Florida 5.9 . . . 1.2 4.0 . . . 1.1 1.9 . . . 1.6 0.25
West Virginia 2.6 . . . 1.0 4.8 . . . 1.4 -2.2 . . . 1.7 0.19
SOURCE: Author's calculations using SSA program records.
NOTES: Earnings are shown in 2012 dollars.
Totals do not necessarily equal the sum of rounded components.
SGA = substantial gainful activity; . . . = not applicable.
* = statistically significant at the .10 level (two-tailed t-test).
** = statistically significant at the .05 level (two-tailed t-test)
*** = statistically significant at the .01 level (two-tailed t-test).

Two years after random assignment, both CUNY and West Virginia treatment-group youths continued to be more likely to have earnings than their control-group peers; in addition, Colorado treatment-group youths increased their labor force participation in the second year and were also more likely to have earnings than their control-group peers. The sustained impact of CUNY is somewhat surprising because the first-year impacts were thought to be partly due to the guaranteed employment experience. In all sites except Colorado, fewer treatment-group youths had earnings 2 years after random assignment than did after 1 year.

There was no significant difference in mean earnings or in the percentage of participants with earnings that exceeded the level identified by SSA as representing substantial gainful activity (SGA) in either the first or second year after random assignment for any site.14 For differences in mean earnings, when looking only at participants with earnings, two results are statistically significant: CUNY after 1 year and West Virginia after 2 years. Surprisingly, both of those impacts are negative, indicating that the projects reduced the average earnings of the treatment groups. This outcome suggests that the projects may draw more “marginal” workers into the labor force; those workers would be expected to have low earnings, thus lowering the average for all those with earnings.15

To explore this hypothesis further, I looked at the earnings distributions of YTD youths in five earnings categories: $0; $1–$250; $251–$1,000; $1,001–$5,000; and over $5,000. Those results, available upon request, show that CUNY and West Virginia appear to have shifted workers out of the $0 group and primarily into the $1,001–$5,000 group in both years after random assignment. Similarly, Colorado (which had a significant impact on any earnings but an insignificant negative impact on average earnings of those with earnings in the second year) also appears to have shifted youths into the higher earnings categories in the second year. The other sites did not have statistically significant differences in the earnings distributions of the treatment and control groups.

SSA Program Payments

As Tables 3 and 4 show, fewer treatment-group youths left the SSI program than did control-group youths. Treatment-group youths may also have retained higher SSA program payments because they were eligible for special waivers that allowed them to keep more of their earnings and stay on a disability program longer than normal program rules allow. Specifically, the waivers exempted treatment-group youths from program payment reductions that were due to earnings—reductions to which control-group youths, under the normal program rules, remained subject.

Charts 3–8 show the average SSA program payments for the treatment and control groups for the 12 months before and the 24 months after random assignment, as well as for the month of random assignment itself (which is designated as month 0).16 The vertical bars indicate months in which the difference is statistically significant.

Chart 3.
CUNY participants: Average amount of SSA program payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test: months -6, -5, -4, 3, 4, 5, 7, 8, 10, 12, 13, and 14).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test: months -3, -2, -1, 0, 1, 2, 6, 9, 11, 15, 16, 17, 18, 19, 20, 21, 22,
  23, and 24).
Chart 4.
Erie County participants: Average amount of SSA payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test: months 15, 17, and 18).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test: months 20 and 21).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test: months 19, 22, 23, and 24).
Chart 5.
Colorado participants: Average amount of SSA program payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test: months 10 and 17).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test).
Chart 6.
Maryland participants: Average amount of SSA program payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test: months 3, 4, and 5).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test).
Chart 7.
Florida participants: Average amount of SSA program payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test: months -8, -6, -3, -2, 0, 1, 14, and 16).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test: months -7, 17, 18, 19, 20, 21, and 22).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test: months 23 and 24).
Chart 8.
West Virginia participants: Average amount of SSA program payments in 12 months preceding and 24 months following random assignment (in dollars)
Line chart linked to data in table format.
SOURCE: Author's calculations using SSA program records.
NOTES: Payments are adjusted to 2012 dollars using the CPI-W.
thin, light gray line = difference is statistically significant at the .10 level (two-tailed t-test: month 9).
wider, dark gray line = difference is statistically significant at the .05 level (two-tailed t-test: months 10, 11, 12, 13, 14, 15, 16, 18, 19, and 20).
widest, light gray line = difference is statistically significant at the .01 level (two-tailed t-test: months 17, 21, 22, 23, and 24).

The differences between the average SSA program payments for the treatment and control groups varied across sites. For CUNY, the difference was statistically significant for all months beginning in the sixth month before random assignment. By contrast, for Colorado, the difference was only statistically significant in two instances (months 10 and 17 after random assignment). Treatment-group payments tended to be significantly greater than those for the control group, especially in the second year after random assignment. For example, in Florida, the treatment group had statistically higher payments in all months after month 16; and in West Virginia, the treatment group had statistically higher payments in all months after month 9.

Only in Maryland, where most youths were not receiving any SSA program payments at random assignment, were the control group's payments consistently higher than the treatment group's payments over most of the period (although the difference is generally not statistically significant). Overall, the sites with significant increases in the percentage of youths with earnings in the first year (CUNY, Florida, and West Virginia; see Table 5) also had statistically significant higher SSA program payments at 12 months (differences of $39, $25, and $38, respectively; see Charts 3, 7, and 8). Those differences grew to $84, $61, and $61, respectively, at month 24.

Regression-Adjusted Results

Although youths were randomly assigned into the treatment and control groups, the two groups differed significantly at the baseline in many of the demographic characteristics reported in the 12-month interim reports (a few instances are identified in Table 2). Additionally, SSI payment or concurrent SSI/DI benefit receipt differed in the month of random assignment for CUNY and Maryland participants. Researchers often use multivariate regression analysis to improve the precision and efficiency of their estimates when there are known differences (Orr 1999).

The baseline surveys revealed differences between the treatment and control groups in several characteristics. Although the differences varied across sites, those characteristics included age; sex; race; language; living arrangement; self-reported health status; need for assistance in hearing, walking, or other functions; ability to perform certain independent activities; volunteer work, job training, and disability program payments received in the past year; paid work in the past month; expectations about future work; and parent's education and employment. Differences were generally small and within the number of differences one would expect to be due to chance. The Maryland site had the highest number of statistically different characteristics, with 6 (of 31 measured characteristics). However, those differences may substantially affect the results. The estimates presented in the 12-month interim reports were regression-adjusted, meaning these variables were included in a regression of the outcome on the treatment dummy. The coefficient on the treatment dummy is the regression-adjusted intent-to-treat impact.17

To test if known differences alter the results, I compared the raw-difference impacts with the impacts when controlling for sex, disability, age at random assignment, and the value of the outcome variable in the month of random assignment (results available upon request). In only two cases did an insignificant raw impact become significant with the regression adjustment (for Erie County, the receipt of DI worker benefits; and for Colorado, the receipt of SSI payments); and although the magnitude of those two impacts changed, in neither case did it change direction.18 In one case, a significant raw impact became insignificant with the regression adjustment (for West Virginia, the receipt of auxiliary DI benefits). None of these changes substantially alters the general thrust of the raw-difference results.

Discussion

This article examines how YTD projects affect earnings and SSA disability program participation for youths 24 months after random assignment into treatment or control groups. The results are consistent with the objectives of the YTD project as a whole. Given the young age and minimal work experience of most YTD participants, the small change in earnings is unsurprising. However, the results suggest that some sites succeeded in moving marginal workers into the labor force, increasing the prevalence of earners but reducing average earnings amounts.

Fraker (2013) showed that, at least in the first year after random assignment, employment impacts were positively correlated with average intensity of employment services. Interestingly, for most sites, the share of the treatment group that had earnings dropped in the second year after random assignment, coinciding with the end of service delivery for most youths. For example, the share of youths with earnings in the CUNY treatment group dropped 13.0 percentage points, and in West Virginia, that share dropped 21.1 percentage points. Although the control groups generally mimicked that second-year drop in employment, these two sites sustained significant impacts on employment. Additionally, the significant impacts on the prevalence of earners only in the second year after random assignment in the Colorado site—after program services ended for most youths—suggest potential delayed project impacts. Whether the earnings impacts will rebound in future years or fade out completely remains an open question.

The differences between treatment and control groups in SSI participation increased over 24 months in all sites except Maryland. The higher shares of treatment-group youths receiving SSI payments and the generally higher payment amounts are consistent with the intent of the YTD project for SSI participants. The program waivers allow the treatment-group youths to keep more of their income and remain in the program longer than the control-group youths. Combined with the earnings results, the waivers may indicate better employment outcomes for treatment-group youths. Although not statistically significant, the negative SSI participation impact in Maryland is also consistent with the YTD's intent of reducing the need for SSI among youths not already receiving it. Future research will determine if project services and waivers improve longer-term employment outcomes.

It is too early to determine the overall success of the YTD project, but the results provide evidence of increased earnings and employment in some sites. Although this is an important outcome, the results presented here do not consider other sources of income, as would be necessary to more fully assess progress toward self-sufficiency. The 12-month interim reports took, and the final report will take, advantage of survey information on work experiences, living arrangements, and nondisability program transfer payments to provide greater insight on participant self-sufficiency.

The sample sizes for each YTD project site were chosen to enable the detection of impacts in the range of 7–12 percent (Rangarajan and others 2009). Thus, although the projects may have been well implemented, smaller impacts cannot be detected by design. For example, the differences between the treatment and control groups in the shares with earnings in the second year after random assignment range from 0.5 to 9.3 percent; only the estimates close to the lower bound of minimum detectable impacts (or the upper bound of the observed range of outcomes) are significant.

Prior studies on transition-age SSI recipients have generally reported observational outcomes (for example, Loprest and Wittenburg 2007). The YTD is one of the first initiatives to establish a causal relationship between services and employment for that population.19 An earlier large-scale study, the State Partnership Initiative (SPI), included some projects that used an experimental design; interestingly, the few projects with those services were found to increase employment but not earnings, at least in the short term (Peikes and others 2005).20 This outcome may have resulted from individuals limiting their earnings to maintain benefit eligibility (Wittenburg, Mann, and Thompkins 2013). Although the SPI tested versions of some of the program waivers included in the YTD, such testing took place at only one of the SPI sites that used an experimental design. For the SPI population, the effects of waiver availability (separate from services received) on earnings and program participation, along with any longer-term impacts, are unclear.

For two reasons, the YTD may yield more positive results as time passes. First, many project participants are or recently were still in school and thus may not yet have been ready or able to have substantial earnings. Although the phase 2 project sites focused more on competitive employment, all YTD sites encouraged work experiences, including unpaid internships and temporary employment. Benefits from those experiences may not be realized until much later, and thus may not yet be captured in SSA's program records. Second, the YTD's sustained impacts on SSI participation after services ended may indicate increased use of the SSI rule waivers that encourage work. The surveys informing the interim and final reports will provide greater detail on the experiences of treatment- and control-group youths up to 3 years after random assignment.

More extensive and focused analyses could help policymakers better understand how well YTD-type services work among those who use them. Planned future reports will use the baseline surveys to estimate 3-year project impacts, which may improve the estimates. YTD services were meant to enable youths to permanently change the path they would otherwise have followed. As such, any project impacts may last well into the future, and further examination of this population may yield important findings for similar future interventions.

Notes

1 SSA previously funded similar projects such as the Transitional Employment Training Demonstration, Project NetWork, and the State Partnership Initiative, all of which served broader populations than that served by the YTD. The Department of Labor's Structured Training and Employment Transitional Services demonstration served a similar age range, but was limited to youths with low intelligence test scores who did not necessarily receive SSI or DI benefits. Those projects had varying impacts, but generally improved employment outcomes for participants. Wittenburg, Mann, and Thompkins (2013) summarize those and other employment programs for people with disabilities.

2 A paper presenting preliminary results was based on partial data for three of the sites (Hemmeter 2012); this article fully updates that paper.

3 See Bucks Camacho and Hemmeter (2013) for a review of the California and Mississippi projects.

4 See SSA's Work Incentives Planning and Assistance website for more information on that program and the training its participants received (http://www.socialsecurity.gov/work/WIPA.html).

5 One Stop Workforce Centers, sponsored by the Department of Labor's Employment & Training Administration, provide various job placement and related support services for both job seekers and employers.

6 The position is named for the Department of Labor initiative under which it was established (for more information, see http://www.doleta.gov/disability/new_dpn_grants.cfm).

7 The Summer Youth Employment Program is a subsidized employment program available on a lottery basis to all New York City youths.

8 SSA generally pays amounts due when underpayments are detected but does not necessarily recoup all overpayments.

9 For a review of this methodology, see Orr (1999) or Duflo, Glennerster, and Kremer (2006).

10 The 12-month interim reports cited in the site descriptions assess project impacts based primarily on survey data using regression adjustments to control for several statistically significant differences between the treatment and control groups. Although MPR correctly implemented randomization, some differences between the treatment and control groups remained. Unfortunately, many of the differing characteristics are not available in the administrative records upon which the current analysis is based. Regression-adjusted results using the available administrative data and the differences in characteristics between the treatment and control groups are discussed later.

11 Although DI eligibility typically requires 20 quarters of coverage (with 10 occurring in the past 5 years), that requirement is relaxed for younger individuals. Until age 22, youths need only 6 quarters of coverage to become eligible for DI.

12 See Gertler and others (2011) for a discussion of the “difference-in-difference” estimate exemplified by this difference between the differences at the baseline and at month 24. This estimate controls for time-invariant heterogeneity in the sample by differencing out any underlying trends. In calculating this estimate, I find that YTD participation increased SSI payment receipt by 4 to 14 percentage points over 24 months; the impact on DI benefit receipt was much smaller, and sometimes negative.

13 In the year of random assignment, the difference in the percentages of the treatment and control groups with earnings was significant in the Florida and West Virginia sites (5 and 7 percentage points, respectively; not shown). For the statistics included in Table 5, no other differences in employment-related outcomes in the year of random assignment were significant. Whether those differences reflect early program results cannot be determined, given the yearly measure of earnings.

14 SGA is a monthly measure of the upper limit of work activity that precludes an individual from being determined disabled for SSI and DI eligibility. In 2012, the SGA amount was $1,010 for nonblind beneficiaries and $1,690 for blind beneficiaries. For DI beneficiaries, earnings above the SGA level can result in a suspension of benefits after a trial work period. For SSI recipients, earnings above the SGA level will not necessarily result in suspension of payments; rather, payments are generally reduced by $1 for every $2 earned above $65 until the payment reaches $0. For the purposes of this analysis, I annualize SGA by multiplying by 12.

15 Consider the following simplified example: Given four people, two of whom work earning $5 and two of whom do not work, the average earnings of the workers equal $5 ([$5+$5]/2). If one nonworker starts to work, but earns only $2, then even though more people are working, average earnings drop to $4 ([$5+$5+$2]/3).

16 Results in this article differ from those in the 12-month interim reports for several reasons. First, although all the interim reports used SSI payments due, those data may have changed as SSA became aware of new earnings information. Second, this article's use of actual DI benefit payments rather than DI amounts due may lead to some minor differences. Third, the authors of the interim reports adjusted payments for inflation to 2008 dollars, and this article adjusts payments to 2012 dollars using the CPI-W.

17 Note that a regression without any other covariates would yield the same coefficient on the treatment dummy as the raw difference between the treatment and control groups.

18 Similarly, the interim reports include a comparison of the raw and regression-adjusted estimates and find few instances in which the direction and significance of the results differed.

19 See Luecking (2009) for a review of the evidence on the role of employment services and experiences for youths in the transition from school to work.

20 SPI projects typically served SSI recipients or DI beneficiaries aged 18–65, so the comparability of some results with those of the YTD may be limited. Furthermore, participation at some SPI sites was restricted to persons with mental disorders, those receiving vocational rehabilitation services, or other groups (Peikes and others 2005).

References

Bucks Camacho, Christa, and Jeffrey Hemmeter. 2013. “Linking Youth Transition Support Services: Results From Two Demonstration Projects.” Social Security Bulletin 73(1): 59–71.

Davies, Paul S., Kalman Rupp, and David Wittenburg. 2009. “A Life-Cycle Perspective on the Transition to Adulthood Among Children Receiving Supplemental Security Income Payments.” Journal of Vocational Rehabilitation 30(3): 133–151.

Duflo, Esther, Rachel Glennerster, and Michael Kremer. 2006. “Using Randomization in Development Economics Research: A Toolkit.” NBER Technical Working Paper No. 333. Cambridge, MA: National Bureau of Economic Research.

Fraker, Thomas. 2013. “The Youth Transition Demonstration: Lifting Employment Barriers for Youth with Disabilities.” Center for Studying Disability Policy Issue Brief No. 13-01. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Peter Baird, Alison Black, Arif Mamun, Michelle Manno, John Martinez, Anu Rangarajan, and Debbie Reed. 2011a. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Colorado Youth WINS. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Alison Black, Joseph Broadus, Arif Mamun, Michelle Manno, John Martinez, Reanin McRoberts, Anu Rangarajan, and Debbie Reed. 2011b. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on the City University of New York's Project. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Alison Black, Arif Mamun, Michelle Manno, John Martinez, Bonnie O'Day, Meghan O'Toole, Anu Rangarajan, and Debbie Reed. 2011c. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Transition WORKS. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Peter Baird, Arif Mamun, Michelle Manno, John Martinez, Debbie Reed, and Allison Thompkins. 2012a. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on the Career Transition Program. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Todd Honeycutt, Arif Mamun, Michelle Manno, John Martinez, Bonnie O'Day, Debbie Reed, and Allison Thompkins. 2012b. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on Broadened Horizons, Brighter Futures. Washington, DC: Mathematica Policy Research.

Fraker, Thomas, Arif Mamun, Michelle Manno, John Martinez, Debbie Reed, Allison Thompkins, and David Wittenburg. 2012c. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on West Virginia Youth Works. Washington, DC: Mathematica Policy Research.

GAO. See Government Accountability Office.

Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, Christel M. J. Vermeersch. 2011. Impact Evaluation in Practice. Washington, DC: World Bank.

Government Accountability Office. 2012. Students with Disabilities: Better Federal Coordination Could Lessen Challenges in the Transition from High School. GAO-12-594. Washington, DC: GAO. http://www.gao.gov/products/GAO-12-594.

Hemmeter, Jeffrey. 2012. “Earnings and Disability Program Participation of Youth Transition Demonstration Participants After 24 months.” Baltimore, MD: SSA. https://www.socialsecurity.gov/disabilityresearch/documents/YTD%20Final.pdf.

Hemmeter, Jeffrey, Jacqueline Kauff, and David Wittenburg. 2009. “Changing Circumstances: Experiences of Child SSI Recipients Before and After Their Age-18 Redetermination for Adult Benefits.” Journal of Vocational Rehabilitation 30(3): 201–221.

Loprest, Pamela J., and David C. Wittenburg. 2007. “Posttransition Experiences of Former Child SSI Recipients.” Social Service Review 81(4): 583–608.

Luecking, Richard G. 2009. The Way to Work: How to Facilitate Work Experiences for Youth in Transition. Baltimore, MD: Paul H. Brookes Publishing Co.

Martinez, John, Thomas Fraker, Michelle Manno, Peter Baird, Arif Mamun, Bonnie O'Day, Anu Rangarajan, and Davis Wittenburg. 2010. The Social Security Administration's Youth Transition Demonstration Projects: Implementation Lessons from the Original Projects. Washington, DC: Mathematica Policy Research.

[NCWD] National Collaborative on Workforce and Disability for Youth. 2005. Guideposts for Success. Washington, DC: Institute on Educational Leadership.

Orr, Larry L. 1999. Social Experiments: Evaluating Public Programs with Experimental Methods. Thousand Oaks, CA: Sage Publications.

Osgood, D. Wayne, E. Michael Foster, and Mark E. Courtney. 2010. “Vulnerable Populations and the Transition to Adulthood.” The Future of Children 20(1): 209–229.

Peikes, Deborah, Sean Orzol, Lorenzo Moreno, Nora Paxton. 2005. State Partnership Initiative: Selection of Comparison Groups for the Evaluation and Selected Impact Estimates. Princeton, NJ: Mathematica Policy Research.

Rangarajan, Anu, Thomas Fraker, Todd Honeycutt, Arif Mamun, John Martinez, Bonnie O'Day, and David Wittenburg. 2009. The Social Security Administration's Youth Transition Demonstration Projects: Evaluation Design Report. Princeton, NJ: Mathematica Policy Research.

Rupp, Kalman, and Charles G. Scott. 1995. “Length of Stay on the Supplemental Security Income Disability Program.” Social Security Bulletin 58(1): 29–47.

[SSA] Social Security Administration. 2003. “Supplemental Security Income (SSI), Youth Transition Process Demonstration (YTPD).” Federal Register 68(194): 57950–57953 (October 7). http://edocket.access.gpo.gov/2003/03-25194.htm.

———. 2008. “Supplemental Security Income, Youth Transition Demonstration.” Federal Register 73(50): 13601–13603 (March 13). http://edocket.access.gpo.gov/2008/E8-5036.htm.

———. 2013. 2013 Red Book: A Summary Guide to Employment Supports for Persons with Disabilities Under the Social Security Disability Insurance and Supplemental Security Income Programs. SSA Publication No. 64-030. Baltimore, MD: SSA. http://www.socialsecurity.gov/redbook.

Wittenburg, David, David R. Mann, and Allison Thompkins. 2013. “The Disability System and Programs to Promote Employment for People with Disabilities.” IZA Journal of Labor Policy 2(4): 1–25.