Possible State Intervention Options to Serve Transition-Age Youths: Lessons from the West Virginia Youth Works Demonstration Project
Social Security Bulletin, Vol. 78 No. 3, 2018
The Social Security Administration (SSA) funded the West Virginia Youth Works intervention as part of the Youth Transition Demonstration (YTD) to improve the employment and independent-living outcomes of youths with disabilities. This project was one of six that constituted the full YTD evaluation. This article examines Youth Works implementation and outcomes to provide a potential case study for other states interested in expanding services to youths with disabilities. We find that Youth Works enrollees reported increases in the use of employment services, in employment, and in income in the year after random assignment into the treatment group, and the effects were large relative to those of previous SSA demonstrations. However, the size of the effects had diminished in the third year after random assignment, by which time project supports were no longer in place, indicating the potential importance of follow-up supports.
Joyanne Cobb is deputy director of the Office of Program Development, Office of Research, Demonstration, and Employment Support, Office of Retirement and Disability Policy, Social Security Administration. David Wittenburg is a senior researcher and Cara Stepanczuk is a researcher at Mathematica Policy Research.
Acknowledgments: The authors thank Corrine Lewis and Susan Kalasunas for their helpful comments on this article and an earlier paper. We are also grateful to Kerry Kern for editing the document.
The findings and conclusions presented in the Bulletin are those of the authors and do not necessarily represent the views of the Social Security Administration.
Introduction
CES | customized employment specialist |
CWIC | community work incentive coordinator |
DI | Disability Insurance |
HRDF | Human Resource Development Foundation |
SSA | Social Security Administration |
SSI | Supplemental Security Income |
VR | vocational rehabilitation |
WIOA | Workforce Innovation and Opportunity Act |
YTD | Youth Transition Demonstration |
Policymakers increasingly look for options to improve the prospects of youths with disabilities, who face several potential barriers to making a successful transition to adulthood and independence, especially if they receive cash benefits from the Social Security disability programs. For example, young beneficiaries do not fare as well as youths without disabilities in terms of labor market outcomes (Loprest and Wittenburg 2007). In recognition of these challenges, several state and federal agencies have initiated demonstration projects that aim to improve services and outcomes for transition-age youths; that is, those aged 14–25. Legislatively, the Workforce Innovation and Opportunity Act of 2014 (WIOA, Public Law 113-128) emphasizes improving services and outcomes for transition-age youths, including requiring state vocational rehabilitation (VR) agencies to adequately prioritize services to youths with disabilities.
In 2003, the Social Security Administration (SSA) initiated the Youth Transition Demonstration (YTD) to assess various options for supporting youths with disabilities who received or were at risk of receiving Social Security disability benefits. The YTD included an evaluation that compared results for randomly assigned treatment and control groups at six project sites in different geographic regions. The evaluation's project sites varied substantially in their participant composition and the availability of existing supports.
This article summarizes findings on one of the six projects, West Virginia Youth Works, from interim and final YTD evaluation reports (Fraker and others 2012, 2014). We focus on findings that are particularly relevant to ongoing state initiatives and demonstration projects that seek to improve services for youths with disabilities. The article also complements another article in this issue of the Bulletin (Fraker and others 2018), which reviews the evaluation reports and examines results for all six YTD projects. We focus more heavily on the implementation findings that state policymakers and administrators might find helpful in designing their own programs. We chose the Youth Works project because it covered the largest geographic area and had promising results in the year after implementation. Hence, policymakers and administrators who are developing programs for youths with disabilities, particularly in response to WIOA, might consider some aspects of the Youth Works project worth replicating.
The lead organization for Youth Works, the Human Resource Development Foundation (HRDF), provided intervention supports to treatment-group youths in accessible settings, including at the youths' homes or workplaces (or by phone) in 19 counties throughout West Virginia. Employment supports were strongly emphasized throughout the service period, which lasted 18 months.
The findings suggest that implementing larger state interventions to serve greater numbers of youths with disabilities is feasible. They indicate that services can substantially improve employment outcomes, but they also raise important questions about whether short-term services can generate results that last into adulthood without requiring further transition and employment supports.
Background
The Supplemental Security Income (SSI) program provides cash payments to eligible low-income children and adults with disabilities. To qualify for childhood payments, an individual younger than 18 must have a medically determinable impairment that causes severe functional limitations and will result in death or is expected to last for a minimum of 12 consecutive months. When a child SSI recipient reaches age 18, SSA conducts a redetermination of eligibility using the adult definition of disability; about one-third of recipients are found ineligible because they do not meet the adult criteria (Hemmeter and Gilby 2009). In January 2014, shortly after the YTD evaluation concluded, approximately 1.3 million disabled youths aged 17 or younger received SSI payments at a cost of $864 million a month, or about $10 billion for the year (SSA 2014).
Motivation for YTD Evaluation Projects
Current and former child SSI recipients face less promising adult outcomes than do their counterparts without disabilities. The poor education, employment, and program-participation outcomes of child SSI recipients before and after age 18 indicate some of the challenges these youths face in moving to adulthood. For example, approximately two-thirds of child recipients “stay on” SSI after age 18 based on the initial age-18 redetermination, an appeal of that decision, or a new application (Hemmeter and Bailey 2015; Hemmeter, Kauff, and Wittenburg 2009; Hemmeter and Gilby 2009). Nearly one-third of child SSI recipients drop out of high school before reaching age 18, and 43 percent report a problem in school that resulted in suspension or expulsion (Hemmeter, Kauff, and Wittenburg 2009). Compared with other young adults, former child SSI recipients are substantially less likely to be employed, in school, or in service programs that could lead to education or employment; have substantially higher arrest rates; and have higher dropout rates (Loprest and Wittenburg 2007; Hemmeter, Kauff, and Wittenburg 2009; Wittenburg 2011).
The YTD included six projects that used an experimental design to provide services to youths with disabilities to improve the experience of transition into adulthood.1 All six projects followed the guideposts for effective transition programs developed in 2005 by the National Collaborative on Workforce and Disability for Youth. The guideposts specifically required work-based experiences (such as job training and volunteer work), activities that promote self-sufficiency (such as self-advocacy training), family involvement, and system linkages (connecting youths to other service providers) (Luecking and Wittenburg 2009).
However, the target populations varied by project (Table 1), as did the types of service emphasized and the geographic scope of the service areas. One project—in Montgomery County, Maryland—targeted youths with mental impairments who were not current SSI recipients but were judged to be at risk of receiving SSI or Disability Insurance (DI) benefits in the future. Of the five projects that served youths on SSI, two (in Colorado and West Virginia) served multiple counties and the other three (two in New York and one in Florida) served more densely populated areas within a single county.2 One project (in Bronx County, New York) served only youths aged 15–19, one (in Florida) served youths as old as 22, and the remaining three projects served youths as old as 25. Three projects were implemented in 2006 and 2007 (phase 1), and the other three began in 2008 (phase 2).
State, location(s), and name | Lead organization | Target population | Evaluation enrollees | ||||
---|---|---|---|---|---|---|---|
Total | Control group assignees | Treatment group | |||||
Assignees | Participants | Participation rate (%) | |||||
All sites | . . . | . . . | 5,103 | 2,347 | 2,756 | 2,318 | 84.1 |
Phase 1 projects | |||||||
Colorado | |||||||
Boulder, El Paso, Larimer, and Pueblo Counties: Colorado Youth WINS |
Colorado WIN Partners of the University of Colorado Health Sciences Center | SSI and DI beneficiaries aged 14–25 | 855 | 387 | 468 | 401 | 85.7 |
New York | |||||||
Bronx County: CUNY Youth Transition Demonstration Project |
The City University of New York's John F. Kennedy, Jr. Institute for Worker Education | SSI recipients aged 15–19 and their families | 889 | 397 | 492 | 387 | 78.7 |
Erie County: Transition WORKS |
Erie 1 Board of Cooperative Educational Services | SSI and DI beneficiaries aged 16–25 | 843 | 384 | 459 | 380 | 82.8 |
Phase 2 projects | |||||||
Florida | |||||||
Miami-Dade County: Broadened Horizons, Brighter Futures (BHBF) |
ServiceSource (formerly Abilities, Inc.) | SSI and DI beneficiaries aged 16–22 | 859 | 399 | 460 | 388 | 84.3 |
Maryland | |||||||
Montgomery County: Career Transition Program (CTP) |
St. Luke’s House, Inc. | High school juniors or seniors with severe emotional disturbances | 805 | 383 | 422 | 374 | 88.6 |
West Virginia | |||||||
19 counties: West Virginia Youth Works |
Human Resource Development Foundation, Inc. | SSI and DI beneficiaries aged 15–25 | 852 | 397 | 455 | 388 | 85.3 |
SOURCES: Mathematica Policy Research and project management information systems. | |||||||
NOTE: . . . = not applicable. |
Participants in all projects could also use one or more of five YTD waivers of restrictions on standard SSI and DI work incentives (Table 2). The waivers were intended to enhance the incentive to find and retain work and/or participate in YTD activities.
Work incentive | Description | Rule change under YTD waiver |
---|---|---|
SSI | ||
Student Earned Income Exclusion (SEIE) | Enabled SSI recipients who were students to exclude a certain amount of earnings from countable income and thus avoid reductions in SSI payments. In 2009 and 2010 SSA excluded the first $1,640 of a student's earnings each month, to a maximum of $6,600 in a year. SEIE eligibility ended when a recipient attained age 22. | Age limit was waived for YTD participants for as long as they attended school regularly. |
General Earned Income Exclusion (GEIE) | Enabled most SSI recipients to exclude from countable income the first $65 of earnings plus one-half of additional earnings. | YTD participants could exclude from countable income the first $65 of earnings plus three-quarters of additional earnings. |
Plan to Achieve Self-Support (PASS) | Enabled SSI recipients to exclude from countable income and resources amounts paid for certain expenses, such as the cost of owning a car, pursuing an education, and purchasing assistive technology, to achieve a specific SSA-approved work goal. | YTD participants could also use a PASS to explore career options or pursue additional education. |
Individual Development Account (IDA) | Provided a trust-like account for SSI recipients to save for a specific goal, such as purchasing a home, going to school, or starting a business. SSA matched earnings deposited in an IDA, often at $2 for every $1 deposited by the participant. The money accumulated in an IDA was excluded when determining SSI eligibility, and the earnings deposited during a month were excluded when determining the SSI payment amount. | A YTD participant could also use an IDA to save for other approved goals. |
SSI and DI | ||
Continuing Disability Reviews and Age-18 Redeterminations (Section 301) | Benefits based on disability could continue despite a negative Continuing Disability Review or age-18 medical redetermination if:
|
If SSA determined that medical disability had stopped and the participant was no longer eligible for assistance, he or she could continue to receive both cash benefits and health care services while participating in YTD. |
SOURCES: SSA (2017) and “YTD Modified SSI Program Rules (Waivers) Descriptions” (https://www.ssa.gov/disabilityresearch/ytdmodifiedssi.html). |
The West Virginia Youth Works Project
In selecting projects for the demonstration, the evaluation contractors worked with SSA to identify sites that were likely to provide some geographic and demographic diversity along with creative interventions that could improve participants' employment and other primary outcomes. Youth Works was a compelling choice for inclusion in the YTD evaluation because it combined an extensive geographic scope with a relatively limited set of services that would otherwise be available. Through its existing network of offices, HRDF was able to reach youths in 19 counties, which were divided into two geographic regions (Chart 1). Selecting a site in West Virginia was also strategically important because of the prevalence of individuals with disabilities among the state's population; self-reported disability among adults and children in 2009 was 19 percent in West Virginia compared with 12 percent nationally (Fraker and others 2012). Consistent with this high prevalence of self-reported disability, more than 80,000 residents of West Virginia (approximately 9,000 of whom were younger than 18) received SSI in 2010 (SSA 2012, Tables 7.B1 and 7.B8).
West Virginia Youth Works service areas
Prior to the Youth Works rollout, existing supports for youths in West Virginia were relatively limited and poverty rates were high, suggesting that a strong intervention could generate substantial improvement (Wittenburg and others 2009). The YTD evaluation team reviewed the pre–Youth Works services offered in the state, including those available from public schools, the West Virginia Division of Rehabilitation Services, Workforce West Virginia, and the state Bureau for Behavioral Health and Health Facilities. Although the services were available to all youths with disabilities, administrators noted that, in practice, services often were fragmented and uncoordinated—especially across county lines—and many agencies had waiting lists. Absent a school counselor or case manager to function as a service broker, youths might too often be left without any service options. Access to services was particularly challenging in rural areas, where service and transportation options were few.
The Youth Works intervention offered an opportunity to address potential gaps in existing services for youths with disabilities. Given the relatively high poverty rate and low employment among youths with disabilities, a successful Youth Works project could substantially improve the effects of existing services.
Youth Works Study Design
The evaluation team, led by Mathematica Policy Research and its subcontractor MDRC, interviewed SSI and DI beneficiaries aged 15–25 to assess their interest in participating in the YTD evaluation.3,4 A young person enrolled in the evaluation by completing a baseline survey and sending Mathematica a signed consent form affirming his or her decision to take part (Fraker 2013). Emancipated youths could sign the consent form themselves; otherwise, a signature by a legal guardian was required. For the 852 youths who provided consent to participate, the evaluation team randomly assigned 455 to a treatment group and the other 397 to a control group. West Virginia Youth Works staff provided at least some type of direct service to 388 of the 455 youths in the treatment group. Youths in the control group had access only to the existing services available to all SSI recipients in the community, such as Individual Education Plans and VR services. The effects of the YTD interventions for all 455 treatment-group youths are measured against those for the 397 control-group youths.
As noted earlier, HRDF staff implemented the interventions and supports for treatment-group youths in accessible settings, including at the youths' homes or workplaces or by phone, in 19 counties. An additional subcontractor (TransCen) provided technical assistance to support service delivery in each of the HRDF locations. Employment supports were strongly emphasized during the typical 18-month service period. HRDF staff customized the employment and other supports to address the youths' specific strengths, skills, and career interests.
The evaluation team expected that the YTD interventions would have short-term effects on employment-promoting service use, employment, and income (Fraker and Rangarajan 2009). They did not expect the interventions to have short-term effects on benefit receipt, given the availability of the YTD waivers to negate benefit reductions that otherwise would accompany earnings gains.
We interviewed HRDF staff, youths, and the youths' families to obtain qualitative perspectives about service delivery. We also tracked quantitative service-delivery data entered by project staff using its Efforts to Outcomes management information system. These data were used to assess whether the intervention included the YTD's core components and the extent to which Youth Works staff members were able to deliver services related to those core components.
In addition to using findings based on qualitative data collected by the evaluation team and on service-delivery data from the management information system, the YTD evaluation reports analyzed project effects using a combination of survey results and SSA data for 1-year and 3-year follow-up intervals.5 Prior to the random assignment of potential Youth Works participants into treatment and control groups, the evaluation team's baseline survey collected demographic and other information (such as school attendance) that were not included in the administrative records. The evaluation team also conducted two follow-up surveys. The first survey collected information on service receipt, educational attainment, employment and earnings, attitudes and expectations, and other outcomes for evaluation enrollees in the first year after random assignment. The second collected information on many of the same variables, plus outcomes related to self-determination, postsecondary education services and training, and contacts with the justice system, in the third year after random assignment. The outcome variables were aligned with best practices based on the guideposts for success developed by the National Collaborative on Workforce and Disability for Youth. Finally, the evaluation team used administrative data to track long-term employment and earnings as well as monthly disability benefit amounts and the use of SSA work incentives and YTD waivers.
The project analysis sought to capture all the avenues by which Youth Works could affect the youths participating in the specialized YTD services (Fraker and Rangarajan 2009). Based on the project's design and the stated goals of YTD, we focused on Youth Works' effects on selected primary and supplementary outcomes, shown in Table 3. The outcomes are arranged into four domains: employment-promoting service use, employment and earnings, personal income, and ongoing engagement in productive activities such as employment, education, and training. Those domains (and certain other outcomes) are patterned after those included in the full YTD evaluation (Fraker and others 2014). That report selected each outcome measure according to its importance to the successful transition from SSI child recipient to self-sufficient adult, and to its predicted timing. Hence, in summarizing our findings, we put more emphasis on the primary outcomes because they are more directly related to the original goals of YTD implementation, whereas the supplementary outcomes provide more exploratory indicators of project effects. Year-1 findings are presented in three domains: employment-promoting service use, employment and earnings, and personal income. Year-3 findings revisit the employment and earnings and personal income domains, and summarize long-term engagement in productive activities. We also briefly summarize additional outcomes from two of the YTD final report's other primary domains (not shown in Table 3): contact with the justice system and self-determination.6
Measure | 1-year follow-up survey | 3-year follow-up survey | Administrative data |
---|---|---|---|
Employment-promoting service use | |||
Primary outcome | |||
Used any employment-promoting service | ✔ | ||
Employment and earnings | |||
Primary outcomes | |||
Employed for pay, any time in the past year | ✔ | ✔ | |
Total earnings in the past year | ✔ | ✔ | |
Supplementary outcomes | |||
Total hours worked in paid jobs in the past year | ✔ | ✔ | |
Employed for pay at the time of survey | ✔ | ||
Employed in calendar year | ✔ | ||
Total earnings in calendar year | ✔ | ||
Personal income | |||
Primary outcome | |||
Total income in the past year (earnings, DI benefits, SSI payments) | ✔ | ✔ | |
Supplementary outcomes | |||
DI or SSI benefit received in the past year | ✔ | ||
Total DI or SSI benefit amount in the past year | ✔ | ||
Proportion of total income from earnings | ✔ | ||
Current public or private health insurance coverage | ✔ | ||
Receipt of public assistance (Temporary Assistance for Needy Families, Supplemental Nutrition Assistance Program, housing assistance) in the past year | ✔ | ||
Productive activities (employment, education, and training) | |||
Primary outcome | |||
Took part in paid or unpaid employment, education, or training in the past year | ✔ | ||
Supplementary outcomes | |||
Took part in education or training program in the past year | ✔ | ||
Completed high school by time of survey | ✔ | ||
Ever enrolled in college or technical school | ✔ | ||
SOURCE: Authors' tabulation based on Fraker and others (2014). |
Project impacts are expressed as the differences in outcomes between treatment-group and control-group members. We used regression adjustment to increase the precision of the estimates. Our estimates are based on sample sizes that vary depending on the data source. The administrative-data sample includes all 852 YTD-eligible youths, whereas the survey sample includes the 733 members of the original YTD evaluation recruits who responded to the 1-year follow-up survey.
The YTD participants in West Virginia included a diverse mix of current and former child SSI recipients aged 15–25 (Table 4). At baseline, the majority (63 percent) were not in school. Most participants were aged 18 or older (81 percent), white (80 percent), and had annual family income of less than $25,000 (72 percent). Average annual earnings among recipients were relatively low ($801), given that most were out of school. Recipients' income and earnings characteristics at baseline indicated potential need for the types of employment supports Youth Works was designed to provide. There were no statistically significant differences between the treatment and control groups.7
Characteristic | All | Treatment | Control | Difference | p-value a |
---|---|---|---|---|---|
Baseline survey data b | |||||
Sample size | 733 | c 389 | 344 | . . . | . . . |
Race | . . . | . . . | . . . | . . . | 0.75 |
White | 80.2 | 81.0 | 79.3 | 1.8 | . . . |
Black | 8.9 | 8.7 | 9.1 | −0.5 | . . . |
American Indian, Alaska/Hawaii Native, or Pacific Islander | 3.5 | 2.8 | 4.3 | −1.5 | . . . |
Asian | 0.0 | 0.0 | 0.0 | 0.0 | . . . |
Other or unknown | 7.4 | 7.5 | 7.3 | 0.2 | . . . |
School attendance | . . . | . . . | . . . | . . . | 0.03 |
None | 63.2 | 65.2 | 60.9 | 4.4 | . . . |
Regular high school | 25.9 | 27.3 | 24.4 | 2.9 | . . . |
Special high school | 0.5 | 0.0 | 1.1 | −1.1 | . . . |
Other school | 10.4 | 7.5 | 13.7 | −6.2 | . . . |
Annual family income | . . . | . . . | . . . | . . . | 0.26 |
Less than $10,000 | 38.0 | 35.0 | 41.4 | −6.4 | . . . |
$10,000–$24,999 | 33.7 | 34.8 | 32.5 | 2.3 | . . . |
$25,000 or more | 28.2 | 30.2 | 26.1 | 4.1 | . . . |
Administrative data | |||||
Sample size | 852 | 455 | 397 | . . . | . . . |
Sex | . . . | . . . | . . . | . . . | 0.61 |
Male | 55.3 | 56.2 | 54.2 | 2.0 | . . . |
Female | 44.7 | 43.8 | 45.8 | −2.0 | . . . |
Age | . . . | . . . | . . . | . . . | 1.00 |
15–17 | 18.8 | 18.8 | 18.8 | 0.0 | . . . |
18–21 | 41.9 | 41.8 | 42.1 | −0.3 | . . . |
22–25 | 39.3 | 39.5 | 39.1 | 0.3 | . . . |
Average age (years) | 20.5 | 20.5 | 20.5 | 0.0 | 1.00 |
SSI recipient status | |||||
Yes | 93.6 | 93.9 | 93.3 | 0.6 | 0.75 |
No | 6.4 | 6.1 | 6.7 | 0.6 | . . . |
Duration of payment receipt (years) | 7.9 | 8.0 | 7.8 | 0.3 | 0.59 |
Primary disabling condition | . . . | . . . | . . . | . . . | 0.87 |
Mental illness | 23.9 | 22.2 | 25.8 | −3.6 | . . . |
Cognitive or developmental disability | 42.0 | 42.9 | 41.1 | 1.8 | . . . |
Learning disability/Attention Deficit Disorder | 13.7 | 14.6 | 12.7 | 1.9 | . . . |
Physical disability | 16.1 | 15.9 | 16.3 | −0.4 | . . . |
Speech, hearing, or visual impairment | 4.3 | 4.4 | 4.1 | 0.2 | . . . |
Earnings in year before random assignment ($) | 801 | 720 | 893 | −173 | 0.33 |
SOURCES: YTD baseline survey and SSA records. | |||||
NOTES: Data are weighted to account for survey nonresponse.
. . . = not applicable.
|
|||||
a. Calculated using either a two-tailed t-test or a chi-square test. | |||||
b. Statistics reflect the baseline survey responses of the Youth Works enrollees who ultimately responded to the 1-year follow-up survey rather than those of all 852 baseline survey respondents. | |||||
c. Comprises treatment-group survey respondents (irrespective of service receipt) rather than only the participants who received services. |
Service Delivery
Youth Works provided comprehensive services to promote employment and foster self-sufficiency for youths with disabilities, emphasizing work-based experiences in particular. Youth Works staff customized the services to meet the specific needs of individual participants and often met with them in their homes, schools, community centers, and workplaces. The project operated in 19 counties, each of which was assigned to one of two administrative regions, covering the northern and southern portions of the state. Enrollment occurred during two phases in each region. The model of service delivery and the duration of services were identical in both phases and both regions, but the provision of services was occasionally more extensive during the second phase.
HRDF, in partnership with the Center for Excellence in Disabilities (CED) at West Virginia University, implemented the benefits counseling intervention at Youth Works. Although HRDF provided most project services, CED provided the benefits counseling for the youths and their families. The front-line service-delivery staff consisted of customized employment specialists (CESs) and job developers with business development skills and experience in human services from HRDF; and of benefits counselors known as community work incentive coordinators (CWICs)8 from CED. The CESs recruited youths and enrolled them as participants in the project. They then met one-on-one with the participants, often in their homes, to conduct assessments, provide case management, and prepare them for employment. The job developers worked primarily with employers to identify job opportunities for participants. They also coordinated with the CESs and worked directly with participants to provide job placement services. Finally, the CWICs provided planning and counseling on benefits from SSA and other public assistance programs and assisted Youth Works participants in obtaining the YTD waivers.
Front-line Youth Works staff from HRDF and CED delivered project services to individual youths in four stages. In the first stage, HRDF staff enrolled treatment-group members into the project and provided an initial assessment and benefits counseling. The initial assessment included a person-centered plan for services, which was driven by an individual's strengths and preferences. During the second stage, HRDF staff started job development and placement services designed in part to prepare youths for job searching and employment. Project staff also conducted job development activities and provided ongoing supports for participants who had found employment. During the third stage, project staff provided postemployment benefits counseling, job coaching, and worksite visits. In the final stage, as HRDF closed out services at 18 months after enrollment, staff reviewed the participant's person-centered plan, and benefits counselors provided guidance on the YTD waivers.
Youth Works staff also provided case management and supports throughout a youth's engagement with the project. These supports were all employment-related (for example, interview-skills and résumé-writing training). In addition, staff provided transportation services, supports for youths with goals of further education, and referrals to social and health care services. Although referrals were sometimes provided to family members for various services, HRDF staff targeted the vast majority of services recorded in the Efforts to Outcomes system directly to the youths.
Initial challenges in reaching some youths were resolved later in the project. For instance, in the first year of the project, CESs often called youths multiple times to try to schedule enrollment meetings, to no avail. The CESs turned to using in-person visits to the youths' homes to spark interest in the project, schedule enrollment meetings, and complete the enrollment process.
The project's implementation revealed the importance of establishing clear benchmarks that emphasized employment and designated the roles of the staff—particularly CESs and job developers, who had some overlapping responsibilities. For example, during the initial months, project staff had difficulty in effectively promoting employment for youths in their caseload because they had no clear benchmarks to aim for. Recognizing this deficit, Youth Works management and TransCen developed specific benchmarks involving paid job placements, work-based experiences, and employer contacts for CESs and job developers. Staff members supported having explicit goals, which they viewed as helpful.
Front-line staff also faced challenges in delivering services to youths in rural locations. Transportation was especially problematic for many Youth Works participants. Project resources included a flexible pool of funds to facilitate participant access to needed supports. The funds were used primarily to transport youths to project activities and their places of employment. Additionally, Youth Works staff proactively referred youths to service providers whose own outreach capacities might be limited. They also helped youths to navigate known community services such as VR. HRDF staff also followed up with youths after referral.
All participants in Youth Works received project services from at least one of the four service categories shown in Table 5. The employment-related and case management services delivered by Youth Works staff were generally more extensive than those provided in other YTD projects. On average, Youth Works staff made 46 service contacts of any type per service recipient, lasting a total of 34 hours. Of particular note was that the vast majority of these service hours were for employment-related services (24 hours), which were emphasized in the Youth Works model. Fraker (2013) showed that Youth Works had the highest employment-related service hours per service recipient among the six YTD sites; the other sites ranged from 4 hours to 21 hours per recipient.
Indicator | Any service | Employment-related | Case management | Education-related | Benefits counseling |
---|---|---|---|---|---|
Share of participants receiving service (%) a | 100.0 | 96.4 | 99.0 | 72.2 | 98.7 |
Number of contacts per service recipient | |||||
Average | b 46.1 | 28.9 | 15.9 | 3.6 | 7.1 |
Median | b 37.0 | 18.0 | 14.0 | 2.0 | 6.0 |
Hours per service recipient | |||||
Average | 33.7 | 23.6 | 6.0 | 2.0 | 2.9 |
Median | 17.9 | 8.3 | 4.2 | 0.5 | 2.8 |
Service time per contact (minutes) | |||||
Average | 29.7 | 37.0 | 18.9 | 29.4 | 19.1 |
Median | 15.0 | 15.0 | 15.0 | 20.0 | 15.0 |
Contacts exceeding 30 minutes (%) | 19.3 | 24.5 | 10.4 | 18.9 | 13.3 |
SOURCE: Youth Works Efforts to Outcomes management information system. | |||||
NOTES: Excludes service contacts of less than two minutes and mail contacts that were not related to benefits counseling.
Statistics on service contacts and times are per participant using that service.
|
|||||
a. Percentages reflect shares of the full sample of 388 participants. | |||||
b. Number of contacts capped at one per day per youth. |
Consistent with the Youth Works program model, nearly all participants (96 percent) received employment-related services, and the number and cumulative duration of service contacts per service recipient were greater for that category than for any other. Most of these youths received career-exploration and job-search services, which included discussions of their career interests and job opportunities, assistance in preparing résumés, and guidance on conducting job searches. Providers used the person-centered planning model, which allows individuals with disabilities to participate directly in their transition planning and is associated with positive employment outcomes. As noted, project staff made 29 contacts per recipient to deliver employment services, with a cumulative duration of 24 hours. Additionally, almost all Youth Works participants (99 percent) received case management services, and their frequencies and cumulative durations were relatively high. The most common type of case management service, by a considerable margin, was a general check-in, in which a staff member contacted participants or their families to determine how they were doing and whether they needed assistance.
Nearly all youths received benefits counseling, although the service time was relatively limited compared with employment-related and case management services. Education-related services were not central to the Youth Works program model; correspondingly, a lower percentage of participants used them, and their frequency and duration among the participants who used them were lower than those for the employment-related and case management services.
Findings
We examine whether the services provided by Youth Works, combined with SSA's waivers for YTD, had longer-term effects on youth outcomes by including results from the third year after enrollment. We first assess whether design elements particular to the intervention increased receipt of employment-promoting services in the first year. We then examine the trajectory of outcomes 1 year and 3 years after project entry for the following measures: paid employment and earnings, personal income, and engagement in productive activities. The year-1 results reflect outcomes observed while treatment-group youths were still receiving services, whereas the year-3 results reflect outcomes observed after the youths had completed all Youth Works services (which, as noted above, lasted 18 months). Finally, we include a summary of other social and self-determination outcomes in year 3 from the full YTD report (Fraker and others 2014).
Employment-Promoting Services (Year 1 Only)
Youth Works had a positive effect on the use of employment-promoting services, according to the year-1 follow-up survey. Slightly less than two-thirds of the treatment-group youths reported having used an employment-promoting service from any source (not just Youth Works) in the year following their enrollment in the evaluation (Table 6).9 Youth Works brought about a 30 percentage point increase in the use of employment-promoting services relative to the control group. This primary outcome combines supplementary measures of positive Youth Works effects, such as those on résumé-writing and job-search support (31 percentage points) and on benefits/incentives counseling (24 percentage points).
Measure | Unadjusted mean (%) | Regression-adjusted results | |
---|---|---|---|
Treatment-group mean minus control-group mean | p-value a | ||
Primary outcome | |||
Used any employment-promoting service | 63.6 | 29.8 | 0.00 |
Supplementary outcomes: Youth received— | |||
Résumé-writing and job-search support | 43.1 | 31.0 | 0.00 |
SSA program benefit or work incentive counseling | 39.0 | 23.7 | 0.00 |
SOURCE: YTD follow-up survey. | |||
NOTES: Data are weighted to account for survey nonresponse.
Treatment-group sample size = 388.
|
|||
a. Calculated using a two-tailed t-test. |
Employment and Earnings (Years 1 and 3)
Youth Works also had a positive effect on employment and earnings in the year after enrollment in the evaluation, but the size of the effect diminished by the third year. The outcome of primary interest was whether the youths had paid employment during the year. Nearly 43 percent of treatment-group youths worked for pay at some time in the year after random assignment, which was 19 percentage points higher than the result for control-group youths (Table 7). That estimated difference is statistically significant. In the third year after random assignment, the difference between the proportions of treatment-group and control-group youths who had worked for pay had largely dissipated to 5.7 percentage points, a result which falls just short of being statistically significant at the 10 percent level.10 We also found that earnings in the year after random assignment were about 50 percent higher for treatment-group youths than for the control group; the former group earned an average of $1,559 that year, or $524 more than we estimated for the control group. In the third year after random assignment, the magnitude of the difference dissipated and was not statistically significant. The year-3 results are notable because the treatment group's mean earnings were higher in year 3 than in year 1 ($1,971 versus $1,559), indicating that the difference dissipated over time because of a larger increase in the mean earnings of control-group youths.
Measure and interval | Unadjusted mean | Regression-adjusted results | |
---|---|---|---|
Treatment-group mean minus control-group mean a | p-value b | ||
Primary outcomes | |||
Worked for pay in past year (%) | |||
Year 1 | 42.7 | 19.1 | 0.00 |
Year 3 | 35.7 | 5.7 | 0.11 |
Total earnings in past year (2008 $) | |||
Year 1 | 1,559 | 524 | 0.01 |
Year 3 | 1,971 | 241 | 0.40 |
Supplementary outcomes | |||
Total hours worked in paid jobs in past year | |||
Year 1 | 233.9 | 80.2 | 0.01 |
Year 3 | 269.6 | 29.2 | 0.44 |
Working for pay at time of survey (%) | |||
Year 3 | 23.0 | 3.6 | 0.23 |
Worked for pay in calendar year (%) | |||
Year 1 | 45.3 | 17.6 | 0.00 |
Year 2 | 39.4 | 10.7 | 0.00 |
Year 3 | 36.2 | 7.6 | 0.06 |
Total earnings in calendar year (2008 $) | |||
Year 1 | 1,665 | 430 | 0.04 |
Year 2 | 1,790 | 199 | 0.46 |
Year 3 | 1,952 | 172 | 0.67 |
SOURCES: YTD follow-up surveys; SSA records; and Hemmeter (2014). | |||
NOTES: Data are weighted to account for survey nonresponse.
For item-specific data sources, see Table 3.
Survey sample sizes: year 1 = 389 (treatment group), 344 (control group); year 3 = 365 (treatment group), 311 (control group).
|
|||
a. Differences are shown in percentage points, dollars, or hours, as applicable. | |||
b. Calculated using a two-tailed t-test. |
Hemmeter (2014) reported further evidence of the yearly effects of Youth Works and supported the general trajectory of diminishing results by year 3. That study found a statistically significant difference in earnings in the second year after randomization which was closer in magnitude to the year-1 effects shown in Table 7. Specifically, Hemmeter estimated a difference in the prevalence of youths with earnings of 16 percentage points (44.0 percent of treatment-group youths compared with 28.0 percent of control-group youths) in year 2. Taken together, these findings indicate that year-1 and year-2 project effects were large compared with those in year 3, which perhaps is not surprising given that HRDF services ended after 18 months. The positive effects declined relatively quickly once services were no longer available to treatment-group participants. Outcomes for supplementary employment and earnings measures followed the same trend as those for the primary measures: a positive and statistically significant difference between treatment and control groups in the first year after enrollment, which largely disappeared by the third year. For example, treatment-group youths were employed in paid jobs for more hours than were control-group youths in the first year after random assignment (and the difference was statistically significant), but the two groups did not have significantly different outcomes in the third year after random assignment.
Personal Income
Youth Works had a positive effect on personal income—defined in this context as combined income from earnings and SSA program benefits—in the first and third years after random assignment (Table 8). The income data are based on youth-reported earnings in the survey and disability benefit amounts from SSA records. The income of treatment-group youths exceeded that of control-group youths by $717 in the first year, representing a positive relative effect of about 10 percent; the difference of $1,010 in the third year represents a positive relative effect of 14 percent for the treatment group.
Measure and interval | Unadjusted mean | Regression-adjusted results | |
---|---|---|---|
Treatment-group mean minus control-group mean a | p-value b | ||
Primary outcome | |||
Combined income from earnings and SSA program benefits in past year (2008 $) | |||
Year 1 | 8,060 | 717 | 0.00 |
Year 3 | 8,405 | 1,010 | 0.00 |
Supplementary outcomes (year 3 only) | |||
Received any disability benefits in past year (%) | 88.6 | 8.7 | 0.00 |
Amount of disability benefits in past year (2008 $) | 6,278 | 748 | 0.00 |
Earnings as a percentage of income | 16.5 | −0.8 | 0.74 |
Health insurance c coverage (%) | 90.5 | 2.9 | 0.22 |
Received any public assistance d in past year (%) | 50.2 | −2.8 | 0.44 |
SOURCES: YTD follow-up surveys and SSA records. | |||
NOTES: Data are weighted to account for survey nonresponse.
For item-specific data sources, see Table 3.
Survey sample sizes: year 1 = 389 (treatment group), 344 (control group); year 3 = 365 (treatment group), 311 (control group).
|
|||
a. Differences are shown in dollars or percentage points, as applicable. | |||
b. Calculated using a two-tailed t-test. | |||
c. Public or private. | |||
d. Includes Temporary Assistance for Needy Families, Supplemental Nutrition Assistance Program, and housing assistance. |
The supplementary outcomes, shown for year 3 only, provide some context for the factors that drove the effects on personal income. First, the share of treatment-group youths that received any disability benefits during the third postenrollment year exceeded the share of control-group youths by 9 percentage points, a difference that is statistically significant at the 1 percent level. The treatment group also received an average of $748 more than the control group in disability benefits in the third year after enrollment. This difference is also statistically significant at the 1 percent level. Positive effects on the prevalence and amount of benefits received are not surprising. We anticipated that the SSA rule waivers for YTD participants would result in increased benefits even in the third year after enrollment because they allowed youths to keep more of their benefits while earning work income. Of particular relevance is the Section 301 waiver, which delayed the effectuation of a negative age-18 SSI eligibility redetermination for 4 years after Youth Works enrollment. The greater benefit amounts and work earnings received by treatment-group youths (although not statistically significant) account for the project's effect on total income. Finally, Youth Works did not shift the main source of income from benefits toward earnings, and it had no effect on the prevalence of either public assistance receipt or health insurance coverage.
Productive Activities
Youth Works had a positive effect on engagement in productive activities which, as noted earlier, is a composite measure of a youth's participation in education, training, and paid or unpaid employment in the third year after enrollment in the evaluation. Table 9 shows that 54 percent of treatment-group youths participated in at least one productive activity, a difference of 8 percentage points over control-group youths, which is statistically significant at the 5 percent level. Among the supplementary outcomes, we observed a small but statistically significant difference in college or technical school enrollment, with the treatment group having a lower enrollment rate than the control group. Although we are cautious not to overinterpret this effect, one possible explanation is that the heavy emphasis on employment-promoting activities led some treatment-group youths to substitute employment for additional schooling.
Measure | Unadjusted mean (%) | Regression-adjusted results | |
---|---|---|---|
Treatment-group mean minus control-group mean | p-value a | ||
Primary outcome | |||
Had paid or unpaid work or participated in education or training in past year | 53.5 | 7.6 | 0.04 |
Supplementary outcomes | |||
Participated in education or training in past year | 27.0 | 5.1 | 0.09 |
Completed high school b | 69.3 | 3.3 | 0.34 |
Ever enrolled in college or technical school | 10.4 | −3.5 | 0.09 |
SOURCE: YTD follow-up survey. | |||
NOTES: Data are weighted to account for survey nonresponse.
Sample sizes: 365 (treatment group), 311 (control group).
|
|||
a. Calculated using a two-tailed t-test. | |||
b. Earned diploma or General Educational Development (GED) certificate or higher by time of survey. |
Other Outcomes
Fraker and others (2014) also examined two other outcome domains (not shown in our tables) that provide additional context for the Youth Works findings: contacts with the justice system and measures of self-determination. The authors did not find any statistically significant project effects in those domains. Specifically, they found a small difference in the prevalence of being arrested or charged with a crime during the 3-year follow-up period (4 percent for the treatment group versus 5 percent for the control group). The authors also showed that there was no effect on any of the self-determination measures, perhaps reflecting the limited number of services focused on this domain.
Discussion
The Youth Works project demonstrates that it is possible to scale service delivery over many counties in a single state and serve a large sample of youths with disabilities by using service providers that can deliver individualized supports. HRDF successfully set up an infrastructure and provided services, referrals, and follow-up to 388 youths with disabilities in 19 different counties. This is notable given that HRDF was traditionally an employment service provider with relatively limited experience providing services specifically for persons with disabilities. For Youth Works, HRDF staff adopted the person-centered planning model to provide ample employment opportunities to the youths and delivered substantial service hours. The staff exhibited expertise in business development and human services.
The findings from the Youth Works project are potentially relevant to the current implementation of the WIOA, which emphasizes serving transition-age youths. Under WIOA, state VR agencies are setting aside at least 15 percent of their funding to provide transition services to youths with disabilities. Although the characteristics of youths who enter VR may differ from those included in YTD, state VR agencies may nonetheless find the lessons here informative in identifying potential service providers with which to collaborate, particularly those with a customized employment-services background and experience serving at-risk youths. Additionally, SSA's Work Incentive Planning and Assistance program could enhance outreach by providing CWIC services specifically to youths in transition.
An important implementation lesson from the Youth Works experience was to use statistical benchmarks to reinforce project goals. Youth Works' focus on employment was emphasized in the technical assistance delivered to the project. In qualitative interviews, both staff members and youths expressed strong support for explicit goals that clarify the project's purpose. The benchmarks also supported front-line staff, as the technical assistance provider (TransCen) was able to link outcomes such as employment to the number of service hours provided.
The findings also indicate the potential implications of providing extensive short-term supports, in that substantial year-1 outcomes had diminished by year 3. Specifically, Youth Works had positive effects on the prevalence of paid employment during years 1 and 3, but effects on earnings did not persist into year 3. Youth Works also increased the youths' total income (through the YTD waivers' effects on benefits) and participation in productive activities. The promising 1-year results reflect the extensive, yet short-term, Youth Works service model's emphasis on employment—as well as the comparatively poor outcomes of the control group. However, the dissipation in effects by year 3 indicates that the intervention was less successful in influencing long-term outcomes, which may reflect the fact that Youth Works participants' eligibility for services was capped at 18 months.
In summary, the findings from Youth Works illustrate the potential advantages of developing and implementing a statewide employment-focused intervention to improve short-term outcomes for child and young-adult SSI recipients. Other states could presumably test a similar service-delivery model if they could develop a strong network of providers with staff who are able to implement extensive customized employment services with clear benchmarks. Despite some promising findings, the short duration of services (18 months) might have contributed to a general decline in effects from year 1 to year 3.
Notes
1 The YTD also included some project sites that were not part of the final evaluation because they did not use the experimental design. For details, see Martinez and others (2008) and Bucks Camacho and Hemmeter (2013).
2 Four of these projects (in Colorado; Erie County, New York; Florida; and West Virginia) also served DI beneficiaries.
3 Although the nationwide YTD evaluation targeted youths aged 14–25, individual sites were permitted to narrow that range; accordingly, West Virginia targeted the 15–25 age group.
4 Our discussion focuses on SSI recipients because they accounted for more than 93 percent of the Youth Works enrollees.
5 The YTD final report details the data sources for the evaluations (Fraker and others 2014).
6 The primary outcome in the self-determination domain is measured with an index that combines indicators of autonomy, internal locus of control, and external locus of control. For more details, see Fraker and others (2014).
7 However, Fraker and others (2012) found some small differences when they examined a broader set of variables. For example, treatment-group youths were more likely than control-group youths to report that their fathers had completed high school.
8 CWICs are trained and certified through the Work Incentive Planning and Assistance program, which was established in 2006 as a modification of the Benefits Planning, Assistance, and Outreach program of the Ticket to Work and Work Incentives Improvement Act of 1999.
9 The use of employment-promoting services shown in Table 6 differs from the use of employment-related services shown in Table 5. Table 6 reports any employment services that survey respondents recalled receiving in the past year. By contrast, Table 5 reports Efforts to Outcomes data on the types of services delivered to treatment-group service recipients only.
10 However, when we included unpaid work, we found that Youth Works had a statistically significant positive effect of 6.1 percentage points on the share of youths who were employed (not shown).
References
Bucks Camacho, Christa, and Jeffrey Hemmeter. 2013. “Linking Youth Transition Support Services: Results from Two Demonstration Projects.” Social Security Bulletin 73(1): 59–71.
Fraker, Thomas. 2013. “The Youth Transition Demonstration: Lifting Employment Barriers for Youth with Disabilities.” Center for Studying Disability Policy Issue Brief No. 13-01. Washington, DC: Mathematica Policy Research.
Fraker, Thomas M., Joyanne Cobb, Jeffrey Hemmeter, Richard G. Luecking, and Arif Mamun. 2018. “Three-Year Effects of the Youth Transition Demonstration Projects.” Social Security Bulletin 78(3): 19–41.
Fraker, Thomas, Arif Mamun, Todd Honeycutt, Allison Thompkins, and Erin Jacobs Valentine. 2014. Final Report on the Youth Transition Demonstration Evaluation. Washington, DC: Mathematica Policy Research.
Fraker, Thomas M., Arif A. Mamun, Michelle S. Manno, John Martinez, Deborah Reed, Allison Thompkins, and David C. Wittenburg. 2012. The Social Security Administration's Youth Transition Demonstration Projects: Interim Report on West Virginia Youth Works. Washington, DC: Mathematica Policy Research.
Fraker, Thomas, and Anu Rangarajan. 2009. “The Social Security Administration's Youth Transition Demonstration Projects.” Journal of Vocational Rehabilitation 30(3): 223–240.
Hemmeter, Jeffrey. 2014. “Earnings and Disability Program Participation of Youth Transition Demonstration Participants after 24 Months.” Social Security Bulletin 74(1): 1–25.
Hemmeter, Jeffrey, and Michelle Stegman Bailey. 2015. “Childhood Continuing Disability Reviews and Age-18 Redeterminations for Supplemental Security Income Recipients: Outcomes and Subsequent Program Participation.” Research and Statistics Note No. 2015-03. Washington, DC: SSA.
Hemmeter, Jeffrey, and Elaine Gilby. 2009. “The Age-18 Redetermination and Postredetermination Participation in SSI.” Social Security Bulletin 69(4): 1–25.
Hemmeter, Jeffrey, Jacqueline Kauff, and David Wittenburg. 2009. “Changing Circumstances: Experiences of Child SSI Recipients Before and After Their Age-18 Redetermination for Adult Benefits.” Journal of Vocational Rehabilitation 30(3): 201–221.
Loprest, Pamela J., and David C. Wittenburg. 2007. “Post-Transition Experiences of Former Child SSI Recipients.” Social Service Review 81(4): 583–608.
Luecking, Richard G., and David C. Wittenburg. 2009. “Providing Supports to Youth with Disabilities Transitioning to Adulthood: Case Descriptions from the Youth Transition Demonstration.” Journal of Vocational Rehabilitation 30(3): 241–251.
Martinez, John, Michelle S. Manno, Peter Baird, Thomas Fraker, Todd Honeycutt, Arif Mamun, Bonnie O'Day, and Anu Rangarajan. 2008. The Social Security Administration's Youth Transition Demonstration Projects: Profiles of the Random Assignment Projects. Washington, DC: Mathematica Policy Research.
[SSA] Social Security Administration. 2012. Annual Statistical Supplement to the Social Security Bulletin, 2011. Publication No. 13-11700. Washington, DC: SSA.
———. 2014. “Monthly Statistical Snapshot, January 2014.” https://www.ssa.gov/policy/docs/quickfacts/stat_snapshot/2014-01.html.
———. 2017. 2017 Red Book: A Summary Guide to Employment Supports for Persons with Disabilities under the Social Security Disability Insurance and Supplemental Security Income Programs. SSA Publication No. 64-030. Baltimore, MD: SSA. https://www.ssa.gov/redbook.
Wittenburg, David. 2011. “Testimony for Hearing on Supplemental Security Income Benefits for Children.” Presented at the Subcommittee on Human Resources, Committee on Ways and Means, U.S. House of Representatives. Washington, DC: Mathematica Policy Research.
Wittenburg, David, Arif Mamun, Thomas Fraker, Karen CyBulski, Sara Muller-Ravett, Ihno Lee, and Laura Guy. 2009. The West Virginia Youth Works Youth Transition Demonstration Project: Early Assessment Report. Washington, DC: Mathematica Policy Research.