The Use of Longitudinal Data on Social Security Program Knowledge

Introduction

Many federal programs and services promote the health, safety, and economic security of the American public. Individuals must be knowledgeable about the programs and services offered to make the most use of them. Since the 1990s, the Social Security Administration (SSA) has regularly evaluated public knowledge of its retirement and disability programs. Most recently, the agency has funded a longitudinal study of program knowledge using the Understanding America Study (UAS). This longitudinal research may enable SSA to expand its understanding of the public's program knowledge in a number of ways. These include:

• The dynamics of program knowledge among individuals over the life course and between population subgroups (for example, by educational attainment).
• The tools people use to learn about Social Security programs.
• How interventions might increase public knowledge about Social Security, and how to measure their short- and long-term effectiveness.

In this article, we present results from the first two waves of the UAS survey on Social Security program knowledge. This research sheds light on how the level of program knowledge varies across the life cycle. We begin by documenting historical SSA efforts to gauge the public's program knowledge. We then provide an overview of the UAS, highlighting the opportunities offered by a longitudinal study of program knowledge. After presenting initial UAS results, we discuss SSA's possible next steps in using the longitudinal study to measure public knowledge and tailor effective communication efforts.

Literature Review

Along with pensions and private savings, Social Security forms the metaphorical three-legged stool of retirement security (DeWitt 1996). For many, Social Security is the primary source of retirement income (SSA 2016). Understanding whether one is eligible for Social Security benefits, when to claim those benefits, and how much income to expect from them affects work and savings decisions before retirement (Gustman and Steinmeier 1999; Rohwedder and van Soest 2006)—and those decisions in turn affect the level of income in retirement. Social Security program knowledge thus plays an important role in retirement security. For decades, SSA has worked to better inform the public about its retirement and disability programs.

Past Efforts

In 1995, SSA undertook the largest effort in its then-60-year history to inform the public about its retirement program by introducing the Social Security Statement. The annual Statement provides projected estimates of the monthly benefits that a worker will receive based on his or her earnings history and age at claiming, along with a summary explanation of the benefits.¹ SSA then commissioned the Gallup Organization to conduct a series of cross-sectional surveys between 1998 and 2001 to gauge public knowledge of the Social Security programs. These surveys widely expanded the agency's understanding of the public's program knowledge. For instance, Smith and Couch (2014) analyzed the Gallup surveys and found that many younger workers understood the basics of the retirement and disability programs but did not understand certain aspects such as how benefits are calculated.

The Gallup surveys also shed light on the effectiveness of the Social Security Statement in increasing program knowledge. Although the surveys were cross-sectional, different iterations took place before and after the Statement was introduced, providing researchers with a natural experiment to test changes in populationwide knowledge.² Cook, Jacobs, and Kim (2010) found evidence that the Statement increased program knowledge. Smith and Couch (2014) found that the Statement particularly improved knowledge among the population with low levels of education. Other researchers tested the effectiveness of the Statement using the Health and Retirement Study (HRS), which collects longitudinal data but provides limited measurement of program knowledge.³ As in the Gallup-based studies, Mastrobuoni (2011) found that the Statement increased program knowledge. Conversely, Armour and Lovenheim (2016) found that some Statement recipients misunderstood the presentation of projected benefits. Biggs (2010) reported more ambiguous findings and suggested further research is needed to understand both the public's knowledge about Social Security and the effectiveness of the Statement in shaping that knowledge.

More recently, SSA further explored program knowledge using Internet panel studies. Funded by an SSA Retirement Research Consortium grant, Liebman and Luttmer (2015) conducted a randomized controlled trial (RCT) using a large Internet panel called Knowledge Networks (now known as the GfK KnowledgePanel) to see how an informational intervention might affect retirement behavior. They found that sending a brochure to persons aged 55–70 with information about the retirement earnings test (which applies to those who work after claiming benefits) increased employment by 4.2 percentage points. SSA also developed a program-knowledge survey as part of the American Life Panel (Greenwald and others 2010), which yielded substantial information about program knowledge.⁴

The UAS

In its latest effort, SSA is funding a program-knowledge survey as a component of the UAS, an Internet-based panel managed by the University of Southern California. The UAS panel is a representative sample of approximately 8,000 U.S. households.⁵ Researchers use an address-based sample to recruit panel members. Tablet computers and Internet access are provided to participants who need them. Panel members may choose to participate in a number of surveys covering a wide range of topics, for which they receive nominal compensation. Researchers administer the Social Security program-knowledge survey on a rolling basis every 2 years. The protocol is to administer the survey to all new panel members or to any panel member who has not taken that survey for 2 years. Researchers use the Census Bureau's Current Population Survey Annual Social and Economic Supplement as the benchmark for weighting. The reference population for the UAS pool of respondents is the U.S. population aged 18 or older excluding military personnel and institutionalized individuals.⁶

The UAS offers a number of advantages for researching public knowledge of federal programs such as Social Security. For example:

• It provides a preconstructed nationally representative panel, which saves time and money in recruitment and retention.
• Because the UAS is Internet-based, researchers can receive survey results quickly, typically within a matter of months.
• Unlike other longitudinal studies, the UAS allows investigators to add survey questions or entire surveys to the Internet panel relatively easily.⁷ Investigators may therefore use it to conduct RCTs.
• The UAS includes a wide array of surveys on numerous topics (for example, financial literacy, personality, and topics covered by Health and Retirement Study modules), which are publicly available. Investigators may match these data to their own data sets to strengthen their studies.

In this study, we use the first two waves of the Social Security program-knowledge survey to extend previous research on how the public understands the programs. Wave 1 is designated by UAS as survey 16 (UAS 16) and wave 2 is designated as UAS 94. We address the following questions:

1. How knowledgeable is the population about basic aspects of Social Security?
2. Does populationwide knowledge change over time?
3. How does an individual's knowledge vary across the life course?
4. Within age groups, how does knowledge vary by individual characteristics such as education and financial literacy?

Methods

More than 5,000 UAS panel members completed the UAS 16 Social Security program-knowledge survey, providing an overall response rate of 85.4 percent of the total UAS panel. We restrict our sample to individuals aged 25–65 who completed both UAS 16 and UAS 94. At the time of analysis, we had access to complete second-wave data for one UAS sampling batch of 1,279 panel members.⁸ Of the 929 participants who completed the first wave in 2015, 724 also completed the second wave in 2017 (a 77.9 percent follow-up response rate).⁹ If the characteristics of panel members who completed both waves differ in meaningful ways from those of members who completed only the first wave, the measures of program knowledge may be biased. Table 1 compares the demographic characteristics of panel members who completed only the first survey wave with those who completed both waves. The demographic variables used in this analysis derive from UAS 1, which focused on cognitive abilities, financial literacy, and psychology. Only the age distribution of the groups differs significantly, in that younger panel members (aged 25–35) are less likely to have completed both waves.¹⁰

The program-knowledge survey covers respondents' understanding of Social Security program basics and of benefit-claiming age (and its effect on benefit amounts) in particular. In this study, we focus on knowledge of program basics in nine different subject areas. Box 1 shows the Social Security program aspects covered in the survey, the wording of the associated questions, the response options, and the correct responses.

We measure program knowledge among three age groups that correspond with the age ranges for which SSA provides different versions of the Social Security Statement: 25–35 (young workers), 36–54 (midcareer workers), and 55–65 (workers near retirement age). Within these age groups, we also investigate variation across two broad educational attainment categories (high school diploma or less and some college or more)¹¹ and two levels (high and low) of financial literacy as determined by a 14-item UAS assessment derived from questions developed by Lusardi and Mitchell (2017). Each of these variables derives from UAS 1. The financial literacy assessment tests respondents' knowledge of annuities, individual retirement accounts, and life insurance policies, among other topics. A score at or above the sample median indicates high financial literacy. We use descriptive statistics to present our findings.

Results

Table 2 shows relatively high levels of knowledge for many basic aspects of Social Security. For instance, more than 80 percent of respondents know of the availability of Social Security disability benefits, the adjustment of benefit amounts by claiming age, the option to wait after retirement to claim benefits, the funding of Social Security through payroll taxes, and the availability of benefits for the minor children of beneficiaries. Americans are less knowledgeable of some other aspects of Social Security, however. Relatively few individuals understand that Social Security benefits adjust with inflation and that spousal benefits may be available, including to a widow(er) with no children. Only one in five wave 1 respondents, when given a choice of answers, identified the way Social Security benefits are calculated.

For most Social Security program aspects, knowledge did not change significantly between survey waves. There were some notable exceptions, however. Knowledge of how benefits are calculated increased by 13.4 percentage points, knowledge of the age adjustment increased by 7.5 percentage points, and knowledge of the presence of disability benefits increased by 5.4 percentage points.

On average, respondents correctly answered 71.4 percent of all questions in wave 1 and 75.8 percent of all questions in wave 2. Chart 1 shows that individuals aged 25–35 had the lowest levels of knowledge in both waves 1 and 2, but exhibited a significant increase in knowledge between survey waves. The middle age group (36–54) also experienced a significant—although smaller—increase. Knowledge also increased, but not significantly so, among the group approaching retirement (ages 55–65).

Within age groups, knowledge levels varied by education and financial literacy. Chart 1 shows that individuals with higher educational attainment had higher levels of Social Security knowledge and Table 3 shows that the differences by educational attainment are significant for all age groups and survey waves except for individuals approaching retirement (ages 55–65) in wave 2. Increases in knowledge between survey waves, however, were similar regardless of education. Among the youngest group, knowledge increased substantially, both for individuals who attended college and for those who did not. Yet, the knowledge gap by educational attainment remains by wave 2. Only for the group approaching retirement (ages 55–65) did individuals with no college close the knowledge gap with their college-educated peers by more than 1 percentage point (although still not significantly).

Knowledge patterns by financial literacy largely mirrored those by education. With the exception of younger individuals (ages 25–35) in wave 1, differences by financial literacy were significant in all age groups. Increases in knowledge did not vary substantially by an individual's level of financial literacy. In no age group did those with less financial literacy close the knowledge gap between waves. For the youngest group (ages 25–35), the difference in knowledge was not significant in wave 1, but became significant by wave 2.

One possible explanation for the relatively strong growth in program knowledge among young adults is that they are first encountering basic aspects of Social Security (such as the availability of disability benefits or the funding of the program through a payroll tax) that are more widely known by older individuals. Individuals may tend to learn less widely understood program aspects (such as spousal benefits) later. Evidence for this theory is limited, however, as Table 4 suggests. The youngest age group (25–35) shows the smallest increase in knowledge of the least understood concept (the benefit calculation). On nearly every other program aspect, however, the younger individuals exhibit an increase in knowledge similar to or larger than that of the older age groups. In all cases, differences between age groups for individual survey items were statistically significant.

Discussion, Limitations, and Future Research

For more than two decades, SSA has worked to gauge public knowledge of its retirement and disability programs. Investing in the UAS' longitudinal survey of program knowledge is the most recent of these efforts.

Our study is the first attempt to use longitudinal program-knowledge data to build on existing research and provide improved insights on what people know and how that knowledge changes over time. We measure Social Security program knowledge at different time points and examine differences by age, educational attainment, and level of financial literacy. We find that, for example, knowledge increases most among young individuals (ages 25–35). This finding echoes research by Smith and Couch (2014), who emphasize the importance of targeting informational outreach efforts to younger workers.

Although knowledge about Social Security increases among young individuals of varying levels of educational attainment and financial literacy, their knowledge still lags significantly relative to that of older individuals. Our findings suggest that young individuals with no postsecondary education or low levels of financial literacy are potential targets for informational interventions. For such individuals, who are more likely to rely predominantly on Social Security benefits for their future retirement income, these interventions could prove particularly important.

Our study faces a number of limitations. One challenge is that testing itself may affect the statistical validity of the findings. That is, knowledge may increase simply because panel members complete the survey multiple times and not because of agency outreach or by comparatively organic means such as learning from employers, peers, or family members. However, because panel members do not receive the correct answers upon completing the survey and only take the survey every 2 years, the potential learning effect is minimal.

Another limitation is that this study does not identify the means by which respondents learned program aspects. Future research could identify which factors and processes drive knowledge gains.

The availability and use of longitudinal program-knowledge data from the UAS will continue to expand. Chard, Rogofsky, and Yoong (2017) used UAS data to develop a sophisticated measure of Social Security program knowledge and retirement preparedness, which researchers may use in future studies to measure changes in knowledge and preparedness over time. In addition, SSA researchers are conducting RCTs to evaluate the effectiveness of alternative communications to improve program knowledge—especially on topics for which knowledge has consistently been low, such as the retirement earnings test and the effects of choosing a retirement claiming age. The use of RCTs in combination with longitudinal survey data on program knowledge can guide agency efforts to better inform the public about its programs.