Earnings and Disability Program Participation of Youth Transition Demonstration Participants after 24 Months

Introduction

Many youths with disabilities need assistance managing their transitions to adulthood. Services available through the education system typically end when youths complete secondary school and are seldom coordinated with adult-based services (Osgood, Foster, and Courtney 2010; GAO 2012). The difficulty of transition to adulthood is compounded for youths receiving means-tested Supplemental Security Income (SSI), who must cope with the added barriers of poverty. Relatedly, many youths who receive SSI drop out of school and encounter the criminal justice system (Loprest and Wittenburg 2007), factors that reduce the likelihood of employment in early adulthood (Hemmeter, Kauff, and Wittenburg 2009). As a result, former child SSI recipients tend to have low labor force participation well into adulthood (Davies, Rupp, and Wittenburg 2009) and remain on the SSI rolls for substantial periods as adults (Rupp and Scott 1995).

To help address these issues, the Social Security Administration (SSA) conducted the Youth Transition Demonstration (YTD) project to identify interventions that improve the educational and vocational outcomes for youths aged 14–25 who receive or potentially qualify for SSI payments or Social Security Disability Insurance (DI) benefits.¹ YTD participants received a variety of employment, education, and other services. To encourage work while they participated in the project, youths were also entitled to waivers of certain SSI and DI rules that would otherwise limit the amount of earnings they could keep while remaining eligible for program payments.

Participants were randomly assigned to treatment and control groups at six YTD project sites. Control group members were subject to standard SSA program rules and received the employment services ordinarily available in their communities; treatment group members were eligible for the program rule waivers and enhanced employment supports. Participant surveys conducted 1 year after random assignment indicated that treatment groups received higher levels of employment services than control groups at all sites, but experienced no significant reductions in benefit receipt. Although employment increased for participants at three sites, no other statistically significant outcomes emerged (Fraker and others 2011a, 2011b, 2011c, 2012a, 2012b, 2012c). Early results, however, may be misleading. Project impacts may fade as participants no longer receive services. Alternatively, impacts may increase as youths mature and, over time, apply the skills obtained in the project. As part of the long-term research strategy for the YTD, in this article I expand on those early results, using SSA program data to describe the impact of the YTD on earnings and on SSI and DI program participation in the 2 years following random assignment.² Future reports will assess these outcomes using both administrative and survey data.

At three of the six sites, the YTD had a positive impact on the proportion of youths with earnings 2 years after random assignment. At two of those sites, and at one site among the other three, the YTD also had positive impacts on earnings in the first year after random assignment. One of those sites guaranteed a summer job in the first year to all treatment group members interested in employment; however, that guarantee was not in place in the second year. Average earnings for the treatment group at that site were lower than those for the control group, consistent with the successful motivation of marginal workers (who would be expected to have lower average earnings) to attempt employment. Additionally, the treatment groups had higher SSI participation rates and payment amounts than control group members in four sites, consistent with the intended effect of the SSI program rule waivers.

YTD Project Overview

SSA started the YTD projects in 2003 to determine whether providing extra employment-related and other supports to youths receiving or potentially qualifying for SSI or DI benefits would improve educational and economic outcomes during their transition to adulthood, thereby eventually reducing dependence on SSA programs. The YTD began with seven sites: one each in California, Colorado, Iowa, Maryland, and Mississippi, and two in New York. The sites provided supports and services to help “youth with disabilities maximize their economic self-sufficiency as they transition from school to work.” Each site worked under a cooperative agreement with SSA to “collaborate among federal, state, and local agencies to develop and implement sustainable improvements in the delivery of transition services and supports” and to “test ways to remove other barriers to employment and economic self-sufficiency” (SSA 2003).

Although services and supports differed somewhat across sites, each site provided SSA program benefits counseling, career counseling, personalized planning, family counseling, opportunities for family involvement in client services and activities, and coordination of services (Martinez and others 2010). Participants in each site were also eligible for waivers of SSA program rules that would otherwise restrict their eligibility for payments if earnings exceeded certain thresholds. Also, the termination of SSI or DI eligibility due to a finding of medical improvement in a continuing disability review (CDR) or age-18 redetermination was deferred while youths participated in the YTD. Table 1 describes the standard SSA disability program work incentives and the associated YTD waivers of certain program rules.

After a brief pilot phase, MDRC (a nonprofit contractor) reviewed the seven sites “to determine the feasibility of conducting a national random assignment evaluation of YTD and explore each project's appropriateness for and interest in such an evaluation” (Martinez and others 2010, 4). Based on that review, SSA terminated two sites (Iowa and Maryland) “because of difficulty they had reaching the goals stated in their cooperative agreements” (SSA 2008). Two other sites (California and Mississippi) continued as originally intended because of their overall strong service design; however, they were unable to implement the revisions recommended in MDRC's report, and are not discussed here.³ The other three projects—Colorado Youth Work Incentive Network of Supports (WINS); Transition WORKS in Erie County, New York; and the City University of New York (CUNY) YTD Project in Bronx County, New York—continued with a slightly revised YTD design. I refer to these three as the phase 1 sites.

The revised YTD included a stronger evaluation design, a technical evaluation, and greater emphasis on employment services (SSA 2008). Mathematica Policy Research (MPR) oversaw the implementation and evaluation of the revised YTD project; subcontractors provided services at each site. Additionally, TransCen, Inc. provided technical assistance to each of the projects, focusing on employment supports. After a national search, SSA selected three other sites—Career Transition Program (CTP) in Montgomery County, Maryland; Broadened Horizons, Brighter Futures (BHBF) in Miami-Dade County, Florida; and West Virginia Youth Works—to join the YTD project. I refer to these three as the phase 2 sites.

Recruitment and Enrollment

The original YTD was designed to serve all eligible youths. The project revision brought about the random assignment of participants into treatment and control groups. This change was difficult for some of the sites, but three of the original sites (the phase 1 sites) successfully modified their projects to accommodate the new recruitment and enrollment design. For example, the Erie County site changed from a classroom-based design to individualized services to allow random assignment into treatment and control groups.

MPR recruited potential participants from lists of SSI recipients and DI beneficiaries aged 14–25 in five of the sites' service areas. MPR randomized those lists and recruited the youths into the YTD project. After the youths voluntarily consented to be part of the project, MPR randomly assigned them to either the treatment group or the control group. If the sibling of a randomized youth was also eligible for the project, the sibling was assigned to the randomized youth's group but was not included in the research sample. MPR also conducted a baseline survey of the youths before randomization.

The new Maryland site followed a slightly different protocol. Its service population included youths considered to be at risk of needing, but not yet receiving, SSI or DI benefits. Site staff recruited youths directly into the project primarily from the local school district. After they consented to be part of the project, youths were processed as described for the other sites.

After randomization, staff at all sites contacted the treatment-group youths and enrolled them into project services. Each of the six sites enrolled between 79 and 89 percent of randomized treatment-group youths into services.

Project Services

Most of the types of services provided at YTD projects were those recommended by the National Collaborative on Workforce Disability for Youth (NCWD 2005), although some were drawn from “best practices” of other interventions for youths with disabilities. The YTD project's core interventions addressed the barriers youths face in their transition from school to work. Chart 1 depicts the barriers and the YTD intervention components, along with the transition environment and key project outcomes. The YTD specifically addressed these six barriers:

• Youths with disabilities often have low expectations for their economic future.
• Many youths with disabilities lack awareness of or access to employment services or work-based experiences.
• The handoff to general adult services is uncoordinated for many youths.
• Youths typically have inadequate access to social and health services.
• The reduction in SSI payments for recipients with certain thresholds of earned income can be viewed as a financial disincentive to work.
• Many youths and their families believe that working will result in the loss of their SSI payment or Medicaid.

Each of the YTD sites offered services to break down these real or perceived barriers to varying degrees. They offered individualized work-based experiences, including internships, job shadowing, job coaching, and competitive paid employment. They offered empowerment training to help participating youths learn to make their own choices (as opposed to having a parent or guardian choose for them). The sites also addressed the barriers by working with the families to break down misunderstandings about program rules; encouraging the families to participate in planning for the youths' self-sufficiency; working closely with local community services to link the educational and work supports for youths with disabilities, smoothing the transition to needed services; and providing case management to coordinate health and other social services. Based on results from the phase 1 sites showing lower-than-expected levels of employment, the phase 2 sites received enhanced technical assistance focused on finding ways to promote competitive employment experiences.

Youths in the treatment groups were eligible for waivers that allowed them to keep more of their earnings or save their earnings for a work or educational goal without affecting their SSI payment. To accompany those waivers, the sites provided DI and SSI benefits counseling. Virginia Commonwealth University trained the benefits counselors, as it had for an earlier program that provided counseling to DI and SSI beneficiaries under the Ticket to Work Act.⁴

The short-term objectives for the YTD project included encouraging participation in employment-promoting activities, increasing paid employment and income, improving attitudes and expectations, and enabling better educational outcomes. In the long term, the YTD project sought to increase participants' paid employment and total income, improve their self-determination, increase their general participation in productive activities (education, training, or employment), and reduce their contact with the justice system.

Site Descriptions

Although all the project sites conformed to a general YTD model, their implementations differed. This section provides a broad overview of each site's services and includes a reference to the site's detailed interim report.

Colorado Youth WINS

The Colorado site was run by the WINS Partners at the University of Colorado Health Sciences Center. Colorado Youth WINS served youths aged 14–25 who received SSI payments or DI benefits in Larimer, El Paso, Pueblo, and Boulder counties. A team of staff members provided services in each of the counties' One Stop Workforce Centers.⁵ The team included a disability program navigator,⁶ a benefits counselor, and one or more career counselors. Four hundred sixty-eight youths were randomized into the Colorado Youth WINS treatment group; 401 enrolled in services (86 percent). The control group comprised 387 youths. Random assignment occurred between August 2006 and March 2008, and services ended in the fall of 2009. Youths were eligible for services for at least 18 months.

To help fill gaps in youths' access to services from existing sources—such as the Division of Vocational Rehabilitation, local school systems, and other providers—Colorado Youth WINS focused primarily on case management, disability program navigation, and benefits counseling. Project staff also developed person-centered plans to help the youths identify educational, employment, and benefit goals and needs. Family members were included in most of the discussions. Career counselors provided vocational assessments and career exploration activities (such as job-site visits). By locating in the One Stop Workforce Centers, Colorado Youth WINS provided easy access to job development and placement services. For more information on the Colorado Youth WINS project, see Fraker and others (2011a).

New York: CUNY YTD Project

The CUNY site worked with youths aged 16–19 who received SSI payments or DI benefits. It was run by CUNY's John F. Kennedy Institute for Worker Education at the Hostos and Lehman campuses in Bronx County. Project staff included benefits counselors, parent advocates, career developers, and students at the partner campuses. An advisory committee of community groups, campus experts, and public agencies provided direction for the program and suggested potential service partners and ways to link the program with community services. Four hundred ninety-two youths were randomized into the CUNY treatment group; 387 enrolled in services (79 percent). The control group comprised 397 youths. Random assignment occurred between July 2006 and November 2008, and services ended in May 2010.

Treatment-group youths received direct services for 1 school year, after which summer employment and limited follow-up services were available. Youths attended Saturday sessions offering recreational and social activities and workshops on self-determination, benefits planning, and career development. Students from the partner campuses who enrolled in a course on working with youths with disabilities led the social and recreational activities; many of those students (and other staff) had disabilities themselves. While youths attended these activities, family members met to discuss benefits and issues related to the youths' self-determination. Youths also developed person-centered plans for identifying and achieving their goals. Parent advocates checked in with families to ensure they participated and met with the people who could address the youths' (or parents') needs. Services culminated with an offer of 7 weeks of summer employment through New York City's Summer Youth Employment Program or in an on-campus job.⁷ For more information on the CUNY project, see Fraker and others (2011b).

New York: Erie County Transition WORKS

The Erie County project served youths aged 16–25 who received either SSI payments or DI benefits in Erie County, New York (which includes the city of Buffalo). The Erie 1 Board of Cooperative Educational Services ran the project. They partnered with the Parent Network of Western New York, Neighborhood Legal Services, the Community Employment Office, and other agencies to provide services emphasizing self-empowerment. Youths were eligible for 18 months of services, with some employment supports available after that. Four hundred fifty-nine youths were randomized into the treatment group; 380 enrolled in services (83 percent). There were 384 youths in the control group. Random assignment occurred between January 2007 and March 2008, and services ended in the fall of 2009.

The Erie County service delivery schedule was very structured, with youths attending person-centered planning and self-determination workshops before receiving employment- or education-related services. The youths set short- and long-term goals for themselves (with the help of a counselor) and created a transition plan to meet those goals. Youths were also trained to organize important documents related to their benefits in a binder, to which they could easily refer as needed. Job developers helped conduct formal vocational assessments to learn about the youths' interests, arrange for informal work experiences (such as job shadowing, job-site tours, and mock interviews), and set up paid employment and internships when participants were ready. If youths were interested in continuing their education, transition coordinators helped them explore their options (for example, earning a high school equivalency degree or enrolling in college). For more information on the Erie County project, see Fraker and others (2011c).

Maryland: Montgomery County CTP

St. Luke's House, Inc., a local mental health provider, ran the CTP. This project served youths aged 16–22 who either were in the last 2 years of high school or had left high school in the past 12 months. Unlike the other projects, the CTP served a subset of the overall YTD target: youths with severe emotional disturbances who might be eligible for SSI payments or DI benefits. Roughly 12 percent of participants received either SSI or DI at the time of random assignment. Treatment-group participants were eligible for up to 18 months of services plus 2 years of follow-up services. MPR randomized 422 youths into the treatment group, 374 of whom enrolled in services (89 percent); 383 youths were in the control group. Random assignment occurred between April 2008 and December 2010, and services ended in March 2012.

After enrolling in the CTP, youths received individualized services focused on employment, beginning with an interview with a career transition specialist to determine the areas in which they needed support to accomplish their goals. The career transition specialist would meet with each youth once a week to develop service needs and interim goals. The youths then received a variety of preemployment vocational assessments (such as mock interviews, work trials, or job shadowing) to prepare them for competitive employment. CTP staff worked directly with employers and community employment resources to help place youths in jobs. Concurrent with those employment services, staff provided benefits counseling, case management, referrals to social and health services, and family supports. For more information on the CTP, see Fraker and others (2012a).

Florida: Miami-Dade County BHBF

Abilities, Inc. of Florida (now ServiceSource) ran the BHBF project. They worked with SSI or DI beneficiaries aged 16–22. Youths received up to 18 months of services, plus follow-up services as needed. BHBF enrolled 388 (84 percent) of the 460 youths randomized into the treatment group; 399 youths were in the control group. Random assignment occurred between April 2008 and September 2010, and services ended in March 2012.

As part of a strong benefits-counseling component, BHBF partnered with the Human Services Coalition and the National Disability Institute to provide financial literacy training along with many career preparation and placement services. The program included one-on-one and group services. For example, although youths developed individual plans for reaching their goals, BHBF also hosted job fairs to connect them with employers and provided work-based experiences, enabling youths to explore different jobs. For more information on BHBF, see Fraker and others (2012b).

West Virginia Youth Works

The Human Resources Development Foundation, Inc. ran the Youth Works project in 19 counties, where they provided up to 18 months of services to youths aged 15–25 receiving SSI or DI benefits. The project enrolled 388 (85 percent) of the 455 youths randomized in the treatment group; 397 youths were assigned to the control group. Random assignment occurred between March 2008 and September 2010, and services ended in March 2012.

Youth Works' services began with goal identification, in which youths received one-on-one benefits counseling and their interests were assessed. This culminated in a person-centered plan for future services and goals. Job placement activities followed to help improve youths' job search and “soft” skills. Youths were taught how to search for positions on their own, but were supported by customized employment specialists. Once working, the youths received job coaching and other employment supports. For more information on West Virginia Youth Works, see Fraker and others (2012c).

Data and Methods

The data for this study come from SSA program records matched to the randomization and enrollment dates provided by MPR. SSA's Master Earnings File provides data on all earnings reported on an individual's W-2 tax record (including non-Federal Insurance Contribution Act taxable earnings). The Supplemental Security Record and the Master Beneficiary Record respectively contain the program participation histories for SSI recipients and DI beneficiaries. Dates of death come from the Numerical Identification System (Numident) file, which contains SSA's Death Master List.

The outcomes of interest in this study are earnings and SSA program payments. Earnings include any income reported as wages or self-employment earnings on the Master Earnings File. Wage data are from W-2 records. If an individual worked more than one job in a given year, wage data from each W-2 are totaled to provide a single measure of annual earnings.

Total SSA program payments represent the sum of SSI payments due and DI benefits paid. SSI payments due are those the recipient should have received in a month if he or she reported all earnings and exclusions in a timely manner and if work incentives (including the YTD waivers) were applied correctly. Using this measure smooths the payment stream for each recipient.⁸ Smoothing the SSI payment data is especially important given the potential complications arising from YTD waiver implementation. Taking such a step is not as important for DI. There are relatively few DI beneficiaries in this sample, and because they are generally not subject to a monthly earnings test, their monthly benefit amounts are much less likely to vary. Additionally, only one of the five program rule waivers applied to DI beneficiaries, and it affected benefit receipt or nonreceipt, not the payment amount. Thus, actual DI benefits should not fluctuate to the extent that SSI payments do. Results based solely on payments made (for both SSI and DI) are available upon request from the author.

As noted, MPR used random assignment to place individuals into treatment and control groups. Therefore, comparisons of the outcomes between the treatment and control groups are unbiased and measure the average impact we would expect to see in a program in which some individuals choose not to receive services for which they are eligible; that is, the results measure the intent-to-treat impact.⁹ I do not use weights and the results do not control for any known differences in the characteristics of the two groups at the time of randomization.¹⁰ All dollar values are adjusted to 2012 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W).

The administrative records capture program participation on a monthly basis, so the SSI and DI results include all YTD participants who remain alive through the 24^th month after random assignment. Although random assignment occurred irregularly over a 2-year period for each site, administrative earnings data are available only as calendar-year amounts. Therefore, I measure earnings in the first and second calendar years after random assignment. For example, if an individual was randomized in August 2009, the first and second years after random assignment are 2010 and 2011, respectively. For many participants, earnings in the second year represent those in their first year without program services. The few individuals who died over the course of this study are included in the analysis up to the month (for program participation) or year (for earnings) before death.

Results

This section presents the YTD project outcomes as of 24 months (for SSI and DI participation) and 2 years (for earnings) after participants were randomly assigned to treatment or control groups.

Characteristics of YTD Participants

Table 2 shows the characteristics of YTD participants. There were few statistically significant differences between the treatment and control groups, and those occurred only in the phase 1 sites. In CUNY, treatment-group youths were more likely than control-group youths to have mental disorders (other than intellectual disability) as their primary impairment, by 8.4 percentage points. In Erie County, treatment-group members were 7.4 percentage points more likely than control-group members to have nonintellectual disability mental disorders. Treatment-group youths in Colorado were 7.5 percentage points more likely to be male than control-group members; they were also 6.4 percentage points more likely to have an intellectual disability. There were no statistically significant differences in participant characteristics in the phase 2 sites.

Although the characteristics within each site were generally similar, differences between the sites are worth highlighting. Over three-fourths of the Erie County and Colorado sites' youths were aged 18 or older at the time of enrollment, and over 80 percent of the Florida and West Virginia participants were 18 or older at enrollment. Conversely, only about 7 percent of the CUNY youths and just over one-half of the Maryland youths were 18 or older. Over 40 percent of CUNY, Florida, and West Virginia participants had mental disorders other than intellectual disability. However, because many of the Maryland youths were not receiving SSI or DI, their disabilities are not identified in the administrative records, for which reason they are grouped in the “other disabilities” category. The different characteristics of the populations served (and the different program models) indicate the risk of comparing outcomes between one site and another.

SSI and DI Program Participation

Because five of the six sites recruited participants from SSA program lists, it is not surprising that almost all youths in those five sites received either SSI payments or DI benefits in the month of random assignment (Table 3). At those sites, between 76.7 percent and 95.1 percent of participants were due an SSI payment, and at all six sites, SSI receipt was more prevalent than DI receipt. The shares of youths receiving DI benefits ranged from 2.6 percent to 25.6 percent, and many were auxiliary beneficiaries eligible for DI benefits based on somebody else's earnings record, typically that of a parent. However, some youths received DI primary (worker) benefits, particularly in the sites with older participants. In Colorado, for example, 10 percent of youths received DI worker benefits (not shown; breakdowns for DI worker and auxiliary benefits are available upon request).

The few youths not receiving any disability benefit in the month of random assignment either (a) were not receiving a check because of temporary overpayment issues or (b) had recently left the program after a CDR or age-18 redetermination. The projects enrolled those youths because of the strong possibility they would return to SSI or DI (although perhaps only after a successful appeal of the CDR or redetermination decision). In CUNY, nonrecipient youths were more prevalent in the control group, accounting for a 5.4 percentage point difference in SSI participation in the month of random assignment. Treatment-group youths were 5.3 percentage points less likely to receive SSI than the control group in Maryland. Recall that that site did not recruit from program records, instead primarily targeting youths deemed “at risk” of needing SSI or DI.

In the CUNY, Erie County, Florida, and West Virginia sites, the treatment group was significantly more likely to receive SSI payments 24 months after random assignment than the control group (Table 4). In CUNY, the difference in SSI participation after 24 months was 10.9 percentage points and is highly significant. This difference is likely due to the use of program rule waivers; for example, the CUNY treatment-group youths were 3 to 10 percentage points more likely to use the various SSA work incentives (and their associated waivers) than control-group youths (Fraker and others 2011b).

Looking only at the 24-month outcomes, however, masks program participation patterns. For example, although the SSI receipt rate fell by 10.2 percentage points 24 months after random assignment for CUNY treatment-group youths, the drop was relatively greater for the control group (15.7 percentage points). Chart 2 illustrates the changes in SSI and DI benefit receipt at random assignment and after 24 months (from Tables 3 and 4). In five of the six sites, the difference between the proportions of treatment- and control-group youths receiving SSI payments is greater at 24 months. Again, this can be explained by the use of program waivers that maintained youths' SSI eligibility (Maryland, the exception, did not draw its participants from the SSI program rolls). For three sites, the difference between the treatment and control groups in the proportion of youths receiving DI benefits decreased over time. In the CUNY, Florida, and West Virginia sites, however, the differences increased. As noted in Fraker (2013) and discussed later in this article, the Florida and West Virginia sites in particular reported large employment impacts, which may have enabled youths there to earn enough quarters of coverage to qualify for DI benefits.^11,12

Earnings

In the first year after random assignment, only the CUNY, Florida, and West Virginia sites had statistically significant impacts on the prevalence of earners (Table 5).¹³ Over 48 percent of CUNY treatment-group youths had earnings, compared with only 23.9 percent of control-group youths. CUNY guaranteed a paid summer job for any treatment-group youth who wanted one, so that finding may reflect a program offering unique to the CUNY site more than the outcome of an intervention. In Florida, 29.8 percent of the treatment-group youths had earnings, compared with 23.6 percent of control-group youths. In West Virginia, 44.0 percent of treatment-group youths had earnings, compared with 28.0 percent of control-group youths. In Maryland, although the result is not statistically significant, 58.8 percent of treatment-group youths had earnings, compared with 55.0 percent of the control group. Recall that the Maryland project served fewer youths receiving SSI payments than the other sites; that site was also located in a service-rich area. Those factors likely contribute to the absence of a statistically significant result. All of these results are similar to those given in the 12-month interim reports.

Two years after random assignment, both CUNY and West Virginia treatment-group youths continued to be more likely to have earnings than their control-group peers; in addition, Colorado treatment-group youths increased their labor force participation in the second year and were also more likely to have earnings than their control-group peers. The sustained impact of CUNY is somewhat surprising because the first-year impacts were thought to be partly due to the guaranteed employment experience. In all sites except Colorado, fewer treatment-group youths had earnings 2 years after random assignment than did after 1 year.

There was no significant difference in mean earnings or in the percentage of participants with earnings that exceeded the level identified by SSA as representing substantial gainful activity (SGA) in either the first or second year after random assignment for any site.¹⁴ For differences in mean earnings, when looking only at participants with earnings, two results are statistically significant: CUNY after 1 year and West Virginia after 2 years. Surprisingly, both of those impacts are negative, indicating that the projects reduced the average earnings of the treatment groups. This outcome suggests that the projects may draw more “marginal” workers into the labor force; those workers would be expected to have low earnings, thus lowering the average for all those with earnings.¹⁵

To explore this hypothesis further, I looked at the earnings distributions of YTD youths in five earnings categories: $0; $1–$250; $251–$1,000; $1,001–$5,000; and over $5,000. Those results, available upon request, show that CUNY and West Virginia appear to have shifted workers out of the $0 group and primarily into the $1,001–$5,000 group in both years after random assignment. Similarly, Colorado (which had a significant impact on any earnings but an insignificant negative impact on average earnings of those with earnings in the second year) also appears to have shifted youths into the higher earnings categories in the second year. The other sites did not have statistically significant differences in the earnings distributions of the treatment and control groups.

SSA Program Payments

As Tables 3 and 4 show, fewer treatment-group youths left the SSI program than did control-group youths. Treatment-group youths may also have retained higher SSA program payments because they were eligible for special waivers that allowed them to keep more of their earnings and stay on a disability program longer than normal program rules allow. Specifically, the waivers exempted treatment-group youths from program payment reductions that were due to earnings—reductions to which control-group youths, under the normal program rules, remained subject.

Charts 3–8 show the average SSA program payments for the treatment and control groups for the 12 months before and the 24 months after random assignment, as well as for the month of random assignment itself (which is designated as month 0).¹⁶ The vertical bars indicate months in which the difference is statistically significant.

The differences between the average SSA program payments for the treatment and control groups varied across sites. For CUNY, the difference was statistically significant for all months beginning in the sixth month before random assignment. By contrast, for Colorado, the difference was only statistically significant in two instances (months 10 and 17 after random assignment). Treatment-group payments tended to be significantly greater than those for the control group, especially in the second year after random assignment. For example, in Florida, the treatment group had statistically higher payments in all months after month 16; and in West Virginia, the treatment group had statistically higher payments in all months after month 9.

Only in Maryland, where most youths were not receiving any SSA program payments at random assignment, were the control group's payments consistently higher than the treatment group's payments over most of the period (although the difference is generally not statistically significant). Overall, the sites with significant increases in the percentage of youths with earnings in the first year (CUNY, Florida, and West Virginia; see Table 5) also had statistically significant higher SSA program payments at 12 months (differences of $39, $25, and $38, respectively; see Charts 3, 7, and 8). Those differences grew to $84, $61, and $61, respectively, at month 24.

Regression-Adjusted Results

Although youths were randomly assigned into the treatment and control groups, the two groups differed significantly at the baseline in many of the demographic characteristics reported in the 12-month interim reports (a few instances are identified in Table 2). Additionally, SSI payment or concurrent SSI/DI benefit receipt differed in the month of random assignment for CUNY and Maryland participants. Researchers often use multivariate regression analysis to improve the precision and efficiency of their estimates when there are known differences (Orr 1999).

The baseline surveys revealed differences between the treatment and control groups in several characteristics. Although the differences varied across sites, those characteristics included age; sex; race; language; living arrangement; self-reported health status; need for assistance in hearing, walking, or other functions; ability to perform certain independent activities; volunteer work, job training, and disability program payments received in the past year; paid work in the past month; expectations about future work; and parent's education and employment. Differences were generally small and within the number of differences one would expect to be due to chance. The Maryland site had the highest number of statistically different characteristics, with 6 (of 31 measured characteristics). However, those differences may substantially affect the results. The estimates presented in the 12-month interim reports were regression-adjusted, meaning these variables were included in a regression of the outcome on the treatment dummy. The coefficient on the treatment dummy is the regression-adjusted intent-to-treat impact.¹⁷

To test if known differences alter the results, I compared the raw-difference impacts with the impacts when controlling for sex, disability, age at random assignment, and the value of the outcome variable in the month of random assignment (results available upon request). In only two cases did an insignificant raw impact become significant with the regression adjustment (for Erie County, the receipt of DI worker benefits; and for Colorado, the receipt of SSI payments); and although the magnitude of those two impacts changed, in neither case did it change direction.¹⁸ In one case, a significant raw impact became insignificant with the regression adjustment (for West Virginia, the receipt of auxiliary DI benefits). None of these changes substantially alters the general thrust of the raw-difference results.

Discussion

This article examines how YTD projects affect earnings and SSA disability program participation for youths 24 months after random assignment into treatment or control groups. The results are consistent with the objectives of the YTD project as a whole. Given the young age and minimal work experience of most YTD participants, the small change in earnings is unsurprising. However, the results suggest that some sites succeeded in moving marginal workers into the labor force, increasing the prevalence of earners but reducing average earnings amounts.

Fraker (2013) showed that, at least in the first year after random assignment, employment impacts were positively correlated with average intensity of employment services. Interestingly, for most sites, the share of the treatment group that had earnings dropped in the second year after random assignment, coinciding with the end of service delivery for most youths. For example, the share of youths with earnings in the CUNY treatment group dropped 13.0 percentage points, and in West Virginia, that share dropped 21.1 percentage points. Although the control groups generally mimicked that second-year drop in employment, these two sites sustained significant impacts on employment. Additionally, the significant impacts on the prevalence of earners only in the second year after random assignment in the Colorado site—after program services ended for most youths—suggest potential delayed project impacts. Whether the earnings impacts will rebound in future years or fade out completely remains an open question.

The differences between treatment and control groups in SSI participation increased over 24 months in all sites except Maryland. The higher shares of treatment-group youths receiving SSI payments and the generally higher payment amounts are consistent with the intent of the YTD project for SSI participants. The program waivers allow the treatment-group youths to keep more of their income and remain in the program longer than the control-group youths. Combined with the earnings results, the waivers may indicate better employment outcomes for treatment-group youths. Although not statistically significant, the negative SSI participation impact in Maryland is also consistent with the YTD's intent of reducing the need for SSI among youths not already receiving it. Future research will determine if project services and waivers improve longer-term employment outcomes.

It is too early to determine the overall success of the YTD project, but the results provide evidence of increased earnings and employment in some sites. Although this is an important outcome, the results presented here do not consider other sources of income, as would be necessary to more fully assess progress toward self-sufficiency. The 12-month interim reports took, and the final report will take, advantage of survey information on work experiences, living arrangements, and nondisability program transfer payments to provide greater insight on participant self-sufficiency.

The sample sizes for each YTD project site were chosen to enable the detection of impacts in the range of 7–12 percent (Rangarajan and others 2009). Thus, although the projects may have been well implemented, smaller impacts cannot be detected by design. For example, the differences between the treatment and control groups in the shares with earnings in the second year after random assignment range from 0.5 to 9.3 percent; only the estimates close to the lower bound of minimum detectable impacts (or the upper bound of the observed range of outcomes) are significant.

Prior studies on transition-age SSI recipients have generally reported observational outcomes (for example, Loprest and Wittenburg 2007). The YTD is one of the first initiatives to establish a causal relationship between services and employment for that population.¹⁹ An earlier large-scale study, the State Partnership Initiative (SPI), included some projects that used an experimental design; interestingly, the few projects with those services were found to increase employment but not earnings, at least in the short term (Peikes and others 2005).²⁰ This outcome may have resulted from individuals limiting their earnings to maintain benefit eligibility (Wittenburg, Mann, and Thompkins 2013). Although the SPI tested versions of some of the program waivers included in the YTD, such testing took place at only one of the SPI sites that used an experimental design. For the SPI population, the effects of waiver availability (separate from services received) on earnings and program participation, along with any longer-term impacts, are unclear.

For two reasons, the YTD may yield more positive results as time passes. First, many project participants are or recently were still in school and thus may not yet have been ready or able to have substantial earnings. Although the phase 2 project sites focused more on competitive employment, all YTD sites encouraged work experiences, including unpaid internships and temporary employment. Benefits from those experiences may not be realized until much later, and thus may not yet be captured in SSA's program records. Second, the YTD's sustained impacts on SSI participation after services ended may indicate increased use of the SSI rule waivers that encourage work. The surveys informing the interim and final reports will provide greater detail on the experiences of treatment- and control-group youths up to 3 years after random assignment.

More extensive and focused analyses could help policymakers better understand how well YTD-type services work among those who use them. Planned future reports will use the baseline surveys to estimate 3-year project impacts, which may improve the estimates. YTD services were meant to enable youths to permanently change the path they would otherwise have followed. As such, any project impacts may last well into the future, and further examination of this population may yield important findings for similar future interventions.