The Impact of Response Error on Participation Rates and Contributions to Defined Contribution Pension Plans

Introduction

It is a well-known fact that employer-provided pension plans have shifted from traditional defined benefit (DB) plans, where the employer bears most of the risks of providing retirement benefits, toward defined contribution (DC) plans, where the employee bears all the risks (Munnell and Sunden 2004).¹ DB pensions provide retirement benefits based on a formula typically involving the final salary, age, and years of service. In contrast, DC pensions are tax-deferred savings accounts where employer and employee contributions into the account are invested, and retirement income depends on the account balance at retirement. The shift from DB to DC pensions has been identified with different data sources such as the Bureau of Labor Statistics' National Compensation Survey (Costo 2006); Form 5500 employer submissions to the Department of Labor (Kruse 1995; Turner and Beller 1989, 1992; Gustman and Steinmeier 1992; Employee Benefit Research Institute 1993; Rajnes 2002; Buessing and Soto 2006); and household surveys (Gustman, Steinmeier, and Tabatabai 2009; Dushi and Iams 2008; Purcell 2005, 2009; Copeland 2005, 2009; Verma 2006).

Many studies have used household survey data, in particular the Census Bureau's Survey of Income and Program Participation (SIPP), to assess participation in and contributions to DC plans for the entire labor force. Purcell (2005, 2009) and Copeland (2005, 2009), for example, use SIPP data to examine both DC plan participation and contributions. An advantage of SIPP data is the availability of pension plan coverage by type of plan for the entire labor force, which allows one to study its relationship with several socioeconomic and job characteristics. However, as is the case with many household survey data, there is the issue of reporting error. If SIPP-reported information about DC pension plans is incorrect, then trends in participation and contributions may be measured inaccurately and thus projections about future coverage and account balances in such plans may also be incorrect. Furthermore, parameter estimates of the determinants of participation and contributions to DC plans may as well be biased or inconsistent.

One approach used to assess the validity of respondents' reports regarding their pension type is to merge survey reports with employers' pension information. Previous research (Mitchell 1988; Gustman and Steinmeier 1989, 2004) has shown that a respondent's reports of plan type and plan characteristics often differ from those obtained from the employer's pension summary plan description (SPD).² Those analyses assume correct matching of employer plans to survey respondents and accuracy of the employer plans in representing the respondent's retirement plan. Rohwedder (2003) argues that inconsistencies may arise from errors with employer-reported data and the process of matching employer data to a particular respondent. Alternatively, one can rely on pension reports of those reaching retirement because the respondent report on pensions would be more salient when people are about to retire or have recently retired (Chan and Stevens 2004; Hurd and Rohwedder 2007).

In support of their hypothesis, Gustman, Steinmeier, and Tabatabai (2009) examine whether the differences between employer and respondent reports are due to lack of knowledge from respondents, due to survey questions and design, or due to the matching of survey and employer data. Their findings from Watson-Wyatt payroll data, which contain both employee and employer information, suggest that the problems associated with matching of SPDs to respondents are not the main reason for the mismatch in reported pension type. In addition, data from the 2004 pension module to the Health and Retirement Study (HRS) indicate that respondents, when asked about the name of their plan, did well in identifying 401(k) and DC plans, but less so with DB plans, suggesting that some of the reporting error is due to the failure of the question wording to clearly identify the plan type. Thus, the authors conclude that the respondents misreport pension plan types mainly because they do not understand their pension well, and employer-provided data are more accurate than respondent-reported data.

In the case of DC plans, the assumption that plan characteristics obtained from employers are more accurate than those reported by respondents is particularly problematic with respect to DC account balances. It is common that DC pension account holders receive an annual statement of the account balance, which suggests that respondents' reports would be more accurate than inferences from an SPD (Scholz 2004; Cunningham, Englehardt, and Kumar 2007).

Another approach in identifying DC pension participation, but not DB pensions, other than from survey self-reports, is to use information from Internal Revenue Service (IRS) W-2 tax records. Turner, Muller, and Verma (2003) use information reported from private-sector workers in SIPP (1993 and 1996 panels) combined with information from W-2 records on tax-deferred contributions to examine participation in DC plans. The authors find a 31 percent discrepancy rate between respondents' report of participation in DC plans and W-2 records, and they suggest that such a difference could be due to lack of knowledge and inaccurate reporting in SIPP by respondents.³ The authors, however, do not address whether imputations by the Census Bureau contribute to their findings. Dushi and Honig (2008), using data from the HRS matched with IRS W-2 tax records, examine reporting error of participation and contribution amounts among respondents aged 51–61 in 1992 and 2004 who at the time of interview were employed in the private sector. The authors find that respondents in 2004 (the younger cohort) were more likely to report correctly whether they were included in DC plans, but they were no more accurate in reporting whether they contributed to their plans. Furthermore, their findings indicate that respondents in both cohorts significantly overestimate their annual contributions. Unfortunately, given that HRS data are available only for the population aged 51 or older, the authors' results may not apply to younger workers and those in the public sector.

In this article, using information from SIPP reports linked to W-2 tax records, we examine the response error with respect to reported DC plan coverage and tax-deferred contribution amounts among full-time workers.⁴ This study contributes to the literature, by investigating in particular the extent of the error that is due to imputations of nonresponse questions by the Census Bureau, a common practice in SIPP. This is particularly important for both users of SIPP pension data and policymakers interested in income security of future retirees. If for example the distribution of DC pension coverage among the imputed observations is not similar to that of nonimputed (self-reported) observations, then the imputation may alter the overall distribution of DC pension coverage. If that is the case, then estimates of pension coverage and consequently estimates of income security of future retirees will be erroneous. Another contribution of this study is the inclusion of public-sector workers in the analysis, who were often omitted in previous research. We stratify our analysis by private- and public-sector workers because it is plausible that public-sector employees, who are more likely to have both a DB and DC plan through their employer, may exhibit a different degree of reporting error regarding their DC plans than private-sector employees.

We find that both the offer rate and participation rate of full-time private- and public-sector workers aged 21–64 are substantially higher when using W-2 tax records than from survey reports. Moreover, findings indicate that reporting errors regarding DC plan participation are more prevalent in imputed records (imputations by the Census Bureau) than in actual responses. A false-positive (type 1) error is a typical error among respondents with imputed information, whereas the false-zero (type 2) error is more likely among respondents with self-reported information. We also find that the probability of a type-1 (false positive) error increases with W-2 annual earnings, whereas the probability of a type-2 (false zero) error decreases with annual earnings. With regard to tax-deferred contribution amounts, we find that while at the median, respondents' reported contributions to DC plans were only slightly lower than those in their W-2 tax records, substantial misreporting is present at the 10^th and 90^th percentiles of the distribution of the difference between SIPP reports and W-2 records. Finally, at the median the absolute difference between individuals' self-reported and W-2 record contribution amounts comprises 29 percent and 35 percent of W-2 contributions in 1998 and 2006, respectively; but it is substantially greater at the upper part of the distribution. Also, the absolute error relative to W-2 contributions is significantly larger among respondents with imputed information than among those with reported (nonimputed) information.

The following section describes the information available in SIPP reports and W-2 records and alternative definitions of DC plan participation used in this article. Our findings and conclusions are then discussed.

Data

This study uses data from two panels of the Survey of Income and Program Participation—the 1996 and 2004 panels. We use data from two different panels because, given the changes in pension environment over the past decade, the extent of reporting error may differ in the two samples. Consequently, trends in DC pension participation will be subject to measurement error. SIPP collects information about pension coverage and contribution amounts of current workers in the seventh interview (in the topical module questions to wave 7), conducted from April through July 1998 (for the 1996 panel) and from February through May 2006 (for the 2004 panel). This analysis focuses on pension participation separately for full-time, private- and public-sector workers aged 21–64.

Respondents in SIPP are asked whether the employer offers a plan based on a formula that takes into account earnings and years on the job,⁵ an individual account plan where contributions are made to an account by the employee and his or her employer,⁶ or a cash balance plan with only the employer contributing to the account. Next, SIPP asks whether the employee is included in the plan. It is responses to these two questions that are typically used in the literature to measure offers and participation in DC plans.⁷

Then SIPP asks respondents if they currently make any tax-deferred contributions to the plan. An employee's tax-deferred contribution is a distinguishing feature of 401(k)-type plans. Respondents who said that either their employer did not offer a plan, their contributions were not tax deferred, or they did not make contributions to a retirement or pension plan are then asked a "follow-up" question about the availability of tax-deferred plans:

"I would like to make sure about a particular type of retirement plan that allows workers to make tax-deferred contributions. For example, you might choose to have your employer put part of your salary into a retirement savings account and you do not have to pay taxes on this money until you take it out or retire. These plans are called different names, including 401(k) plans, pre-tax plans, salary reduction plans, and 403(b) plans. Does your job offer a plan like this to anyone in your company?"

If the respondent indicates that this type of plan is offered, then SIPP asks if the respondent participates in the plan. Then, conditional on participation, respondents are asked whether contributions were made to the plan either by themselves or their employer and the respective amounts of contributions.

We use SIPP respondent's linked IRS W-2 tax records to assess the accuracy of survey-reported participation and tax-deferred contributions. Based on agreements between the Social Security Administration (SSA) and the Census Bureau, Social Security administrative records are linked to SIPP panels and are available to analysts for research on approved projects at restricted data sites. SIPP respondent reports are matched with the respondent's W-2 tax records including information on tax-deferred contributions to retirement plans. About 83 percent of adult respondents in the 1996 panel and 79 percent in the 2004 panel have their survey reports matched to their actual W-2 records. Analysis by Czajka, Mabli, and Cody (2008) find little selectivity bias from nonmatched data in SIPP.

Our analysis of linked tax records draws from SSA's Detailed Earnings Record (DER) file.⁸ Starting in 1990, the W-2 records available in the DER contain a variable that indicates the amount of tax-deferred contributions made to retirement plans and to health savings accounts (HSA) for each job a worker held in a given year. The 2006 W-2 record separately identifies contributions made to HSAs from contributions made to retirement accounts (such as 401(k), 403(b), 408, 457, and 501 accounts), but the 1998 W-2 record does not separately identify these two different types of deferred compensation. However, this discrepancy is not likely to affect our analysis because the HSA legislation took effect in 1997 (Committee on Ways and Means 2004, 23–24), which means that HSA participation was quite modest in 1998, and the bulk of W-2 deferred compensation reflects contributions made to retirement accounts.

We assess the response error in survey-reported information about DC plan participation rates and contributions by comparing SIPP respondents' tax-deferred contributions as recorded in their W-2 tax records with those that are reported by survey respondents. We examine several measures of pension participation. One definition of participation in a DC plan, typically used in the literature, is a respondent's self-report of being included in a retirement plan where contributions are made to an account by the employee and his or her employer. An alternative definition of participation in a DC plan that we use is a respondent's self-report of making tax-deferred contributions to the account (referred to as active participation).⁹ We also measure active participation by the presence of a positive tax-deferred contribution amount in the W-2 record either in the survey year or in the previous year.¹⁰

A standard practice in SIPP is that when respondents do not answer a question, the Census Bureau statistically imputes a response and flags the imputation. The Census Bureau usually imputes nonresponse questions with a hot-deck procedure.¹¹ The National Research Council briefly reviews this hot-deck procedure and concludes that it is inadequate because it is not "carefully tailored to the variable imputed" (Citro and Scholz 2009). The tax-deferred contribution question we use in classifying self-reported active participation in DC plans has been imputed by the Census Bureau for about 13–14 percent of SIPP respondents. Therefore, given that SIPP identifies observations for which the tax-deferred contribution variable is imputed, in this analysis we stratify respondents who self-reported tax-deferred contributions to a retirement plan from those for whom the values were imputed by the Census Bureau.

We examine two types of response errors in survey-reported pension information. The first type of error in self-reports is a false positive (or type-1 error) in which the W-2 record contains zero tax-deferred contributions, whereas the SIPP respondent reports a positive tax-deferred contribution to a DC account. Another type of error is a false zero (or type-2 error) in which the W-2 record contains a positive deferred contribution, whereas the SIPP respondent reports zero tax-deferred contributions to a DC plan. In addition to reporting the proportion of each type of error by sector of employment separately for imputed and nonimputed observations, we also estimate the probability of each type of error as a function of the imputation variable and several control variables. Finally, we compare the amount of tax-deferred contributions made in the survey year as reported by respondents in SIPP with the contribution amounts in the W-2 tax record for the same year. We assume that tax-deferred contributions to retirement plans in the W-2 records are accurate; however, because of possible errors in W-2 records, our findings should be considered suggestive.¹² We also estimate the impact of imputation on the magnitude of the error measured either as the difference between SIPP and W-2 record contributions or as the absolute difference of contribution amounts (in SIPP and W-2) relative to W-2 record contribution amounts. The multivariate analysis controls for sex, race and ethnicity, education, marital status, sector of employment, and W-2 annual earnings and tax-deferred contributions.

For all percentage estimates provided in the results section, we calculate standard errors using SUDAAN to account for the complex sampling procedure in the SIPP panels. We also perform significance tests of the differences between the estimates in the two panels, and because of large sample sizes, percentage differences that are greater than 1 percentage point are usually statistically significant at the 5 percent level. We do not perform parametric statistical significance tests for differences between the two different measures within a given year (such as between SIPP reports versus SIPP supplemented with W-2 records) because the estimates are for the same sample of respondents and the two measures are different only for a subset of the sample. In the latter case, interpretation of differences in estimates must rely on whether or not the percentage differences seem substantially important.

DC Pension Plan Offer and Participation

Offer and participation rates in DC plans, as reported by respondents in SIPP and as calculated from the W-2 record, are shown separately for private- and public-sector workers in 1998 and 2006 (Table 1).¹³ Fifty percent of full-time workers in 2006 reported that their employer offered an individual account pension plan (Table 1, row 1).¹⁴ Private-sector workers are about as likely as public-sector workers to report being offered a DC plan from their employer. The offer rate would be even lower if respondents in SIPP were not asked the follow up question. We find that about a 10th of respondents who initially reported not being offered a plan, then reported being offered a tax-deferred plan in the follow-up question. This suggests that these respondents know what type of plan they are offered, but are confused by the wording in the survey question. When we supplement the respondent's report with information in the W-2 record that indicates having a positive tax-deferred contribution amount (Table 1, row 2), we find that overall 67 percent of employees in 2006 were offered a DC plan from their employer—a 17 percentage-point increase in the offer rate compared with the self-reported rate.¹⁵ In addition, the offer rate of public-sector workers is about 6 percentage-points higher than that of private-sector workers. These findings suggest that some respondents do not understand the typical survey questions, whereas others do not know their plan type.

As noted earlier, SIPP respondents who report being offered a pension plan from their employer are asked whether they are included in the plan (often referred to in the literature as participation) and whether the plan is an individual account plan.¹⁶ Using this definition (Table 1, row 3), we find that about 35 percent of full-time workers reported being included (that is, participating) in a DC plan in 2006. In the same year, public-sector workers were more likely to report participating in a DC plan compared with private-sector workers (40 percent versus 34 percent).

Often employees included in a DC plan may select not to contribute to the account in a given year. Therefore, an alternative definition of participation in a DC plan, referred to as active participation, is whether or not the employee is making a tax-deferred contribution to the DC account in a given year (Honig and Dushi 2003; Turner, Muller, and Verma 2003). Using this second definition of DC participation, measured by whether or not the respondent reports making tax-deferred contributions to an individual account, we find that overall 39 percent of full-time workers actively participated in a DC plan in 2006 (Table 1, row 4). About 6 percentage-points more private-sector workers report making tax-deferred contributions to a DC plan than report being included in a DC plan (40 percent versus 34 percent).¹⁷ In contrast, a similar percentage of public-sector workers report making tax-deferred contributions to a DC plan and also report being included in a DC plan (37 percent versus 40 percent).¹⁸

Using this second definition of DC participation, but measured more precisely by whether or not the W-2 tax record indicates a positive tax-deferred contribution either in the interview year or in the previous year, we find that in 2006 about 46 percent and 49 percent of private- and public-sector workers, respectively, had contributed to a DC plan as indicated by their W-2 record. These participation rates are 12 and 9 percentage-points higher, respectively, for private- and public-sector workers, compared with the self-reported inclusion in a DC plan and 6 and 12 percentage-points higher than the self-reports of tax-deferred contributions. In sum, the traditional survey definition of participation in an individual account plan substantially underestimates DC plan participation rates compared with W-2 records of positive tax-deferred amounts, which we consider as the benchmark.¹⁹

Among private-sector workers, there were no significant differences in SIPP-reported offer or participation rates in DC plans between 1998 and 2006 (Table 1). Among public-sector workers, offer and participation rates, despite the definition used, were significantly higher in 2006 than in 1998, although the difference regarding active participation is smaller when using W-2 records. Contrary to our expectations, the active participation rate based on the information in W-2 records is similar between 1998 and 2006, even though automatic enrollment of workers into DC plans became more common after 1998.²⁰ Finally, it is worth noting here that, although DC plans have become more common particularly in the past decade, the extent of response error is quite similar in both the 1996 and 2004 SIPP panels.

Types of Errors: Active Participation

In order to assess the types of errors of respondents' reports, we now compare the self-reported active participation in a DC plan (whether respondent reported making a tax-deferred contribution at the time of interview) with information from the W-2 tax record (as defined in Table 1). About a quarter of private-sector workers (9 percent + 15 percent) and over one-third of public-sector workers (12 percent + 24 percent) misreported whether they made a tax-deferred contribution to a plan in 2006 (Table 2). In the same year, 9 percent and 12 percent of private- and public-sector workers, respectively, had a false-positive (type 1) error, that is, self-reporting making a contribution to the plan when in fact there was zero tax-deferred contribution in the W-2 tax record. An additional 15 percent and 24 percent of private- and public-sector workers, respectively, had a false-zero (type 2) error, that is, self-reporting zero tax-deferred contribution when in fact there was a positive amount deferred in the W-2 tax record. The proportions of respondents with these types of errors were almost the same in 1998.

Table 2 indicates that SIPP misreporting of making tax-deferred contributions in DC plans is larger among respondents for whom the amount of contribution has been imputed compared with those with self-reported (nonimputed) information. This suggests that an error is much more likely to occur when missing data are imputed rather than in respondent reports. Imputations in 2006 were incorrect as either a type-1 or a type-2 error for about 42 percent and 46 percent of private- and of public-sector workers, respectively (Table 2, row 2). A type-1 error was the typical error among those with imputed information—33 percent and 28 percent in the private and public sector, respectively. In contrast, respondents who self-reported the amount of contribution, and thus have no imputed information, have a much lower rate of type-1 errors—6 percent and 9 percent in the private and public sector, respectively (Table 2, row 3). Respondents with nonimputed information in 2006 were more likely to report zero contribution when in fact the W-2 record indicates a positive contribution, and therefore type-2 error is more common (16 percent and 25 percent in the private and public sector, respectively). Furthermore, type-2 errors were about 7 percentage-points more common among respondents with nonimputed information compared with respondents with imputed information. Similar patterns regarding participation error are evident in 1998. Taken together, these findings suggest that researchers should be cautious when using SIPP data to estimate DC pension plan participation, and the validity of their results would improve by using W-2 records. Similarly, the Census Bureau would improve its imputations of respondents' reports by using information in the W-2 records.

Multivariate Analysis of the Probability of Reporting Error of Active Participation

We now turn to multivariate analysis of the effect of imputation on the probability of reporting error. Table 3 reports probit estimates, separately for 1998 and 2006, of the relationship between each type of reporting error and the imputation variable of interest, while controlling for several socioeconomic characteristics.²¹ The dependent variable in columns 1 and 3, respectively for each year, equals one if respondents report making a contribution when there is no contribution in the W-2 record (type-1 error) and zero otherwise. The dependent variable in columns 2 and 4 is equal to one if respondents report making no contribution when the W-2 record indicates that a positive contribution was made (type-2 error) and zero otherwise. The independent variable of interest is defined as equal to one if the response regarding tax-deferred contributions is imputed, and zero otherwise.

For both 1998 and 2006, respondents with imputed tax-deferred contribution amounts are significantly more likely, by 50 percent, to have a type-1 error than those without imputed information (Table 3, columns 1 and 3). In addition, the probability of type-1 reporting error significantly increases with the amount of the W-2 annual earnings.

Estimates regarding the probability of having a type-2 error are shown in Table 3 (columns 2 and 4). For both 1998 and 2006, results indicate that, in contrast to a type-1 error, respondents with imputed information are significantly less likely (by 6 percent and 8 percent, respectively) to have a type-2 error than those without imputed information, a finding consistent with results in Table 2. Similarly in both years, the probability of a type-2 error significantly increases with the amount of W-2 tax-deferred contributions and decreases with the amount of W-2 annual earnings, but the magnitudes of these effects are negligible.

Amount of Tax-Deferred Contributions

The amount of tax-deferred contributions reported in SIPP will now be compared with the amount in the W-2 tax record for the same year, among respondents with positive contributions in both SIPP reports and W-2 records. The following three questions will also be addressed.

1. Is the distribution of tax-deferred contribution amounts in SIPP comparable with the distribution in the W-2 record?
2. Is an individual's contribution amount higher or lower than the amount in the W-2 record?
3. At the individual level, what is the extent of the relative difference of contribution amounts in SIPP and W-2 records?

The SIPP-reported contribution amount is for the reference period at the time of the survey, and it is assumed that this applies throughout the survey year. For one set of respondents, there is information on the amount contributed and the frequency of contributions made during the survey year from which an annual contribution amount is calculated. For another set of respondents, the percentage of salary contributed to the plan is obtained,²² and the contribution amount using this reported percentage is calculated by applying it to annual earnings in the highest paid job in the W-2 tax record for the survey year.²³

The first question is whether the distributions of tax-deferred contributions in the SIPP data and the W-2 data are comparable. If they are, either source can be used to estimate the amount of money contributed to DC plans among workers covered by DC plans. Table 4 reports, separately for each year, contribution amounts from SIPP (columns 1 and 5) and from the W-2 record (columns 2 and 6) at selected percentiles in each of the two distributions. Panel A reports results for the overall sample, whereas panels B and C report results separately for private- and public-sector workers. In 1998, contribution amounts in SIPP were 7 percent higher than contribution amounts in the W-2 record at the 10^th percentile and within 1 percent of each other at the 25^th percentile and above. In contrast, in 2006, contribution amounts in SIPP were 8 percent higher than W-2 contribution amounts at the 10^th percentile, the same at the 25^th percentile, and 5–11 percent lower at the median and above (panel A).²⁴ These findings suggest that the SIPP amounts of tax-deferred contributions are a close estimate of the true (W-2 record) contribution amounts for respondents in the 1998 sample, but they are an underestimate for the 2006 respondents.

The second question is whether the contribution amount for an individual is higher or lower in the SIPP data than in the W-2 data. This would provide an indication of whether SIPP data overestimate or underestimate the true retirement savings by individuals participating in DC plans. Table 4 (columns 3 and 7) reports the distribution of the difference between SIPP and W-2 record contribution amounts calculated for each individual. A negative (positive) value for the difference indicates that the SIPP contribution amount is smaller (larger) than the value in the W-2 record. In 2006, at the 10^th percentile the underreporting of contributions was substantial (a difference of -$4,320) decreasing to -$50 at the median. In the upper half of the distribution, SIPP contribution amounts are higher than those in the W-2 record, by $760 and $3,000 at the 75^th and 90^th percentile, respectively. Thus, the difference between SIPP and W-2 record contributions is substantial at the tails of the distribution of differences. Furthermore, the magnitude of the error (underreporting) in the 10^th and 25^th percentiles is larger than corresponding values (overreporting) in the 75^th and 90^th percentile. Although overall similar patterns are evident in 1998, the differences between SIPP and W-2 record amounts are much lower at the 10^th and 90^th percentile in 1998 than in 2006. These findings suggest that SIPP data may not provide a good base for estimating the extent that individuals save for retirement.

The third question addressed is the extent of the relative difference between the contribution amounts in SIPP and W-2 records. Another measure of reporting accuracy is the absolute difference between SIPP and W-2 record amounts as a percentage of the W-2 contribution amount measured at the individual level. Table 4 (columns 4 and 8) reports this measure of discrepancy at selected percentiles. In 2006, the ratio of the absolute difference to the W-2 amount was about 2 percent and 9 percent at the 10^th and 25^th percentile, respectively. The ratio increases to 35 percent at the median, 75 percent at the 75^th percentile, and 196 percent at the 90^th percentile, which suggests that there are substantial errors in reported tax-deferred contributions. The same pattern is evident in 1998, with the ratios only slightly lower compared with those in 2006.

The same pattern as observed earlier in Table 4 (panel A) is evident as well among private- and public-sector workers (panels B and C). The main difference between private- and public-sector workers is that the difference between SIPP and W-2 record amounts measured at the individual level (columns 3 and 7) is generally higher among workers in the public sector than those in the private sector, particularly in 2006, suggesting that public-sector workers are less accurate than private-sector workers. The same is true for the absolute difference as a percentage of the W-2 record (columns 4 and 8); the error is generally higher among workers in the public sector than their counterparts in the private sector and substantially higher in the upper tail of the distribution.

Multivariate Analysis of Reporting Error of Contribution Amounts

We estimate the relationship between reporting error of contribution amounts and several socioeconomic characteristics, using ordinary least squares separately for each year (Table 5).²⁵ Two measures of respondents reporting error are used as the dependent variable: the difference between SIPP and W-2 amounts (columns 1 and 3) and the absolute difference (between SIPP and W-2 amounts) as a percentage of the amount in the W-2 record (columns 2 and 4).²⁶

Regression estimates in Table 5 (columns 1 and 3) indicate that the difference in contributions (between SIPP and W-2) is not significantly different between respondents with imputed and nonimputed information, whereas the reporting error is significantly related to the two variables in the W-2 record: the amount of tax-deferred contributions and the annual earnings. The magnitude of the difference between self-reported and W-2 contributions decreases significantly as the amount of the W-2 record contribution increases. In other words, reporting error decreases as the true (W-2) value of contribution amount increases. Conversely, reporting error is significantly larger for high-income earners.

Estimates in columns 2 and 4 indicate that the reporting error measured as the absolute difference between SIPP and W-2 record contributions relative to the W-2 record contribution is significantly higher among respondents with imputed contributions than for those with nonimputed (that is, self-reported) contributions. Similar to results in columns 1 and 3, the absolute error relative to the true value of W-2 record contributions decreases significantly as the amount of W-2 contributions increases.

Conclusion

Pension income traditionally has been one of the pillars of retirement income. During the past three decades, as the type of pension plans available to employees has been shifting from traditional defined benefit plans toward defined contribution plans, the risk associated with these plans has also been shifting from employers to employees. Given the implications of this shift, trends in levels of pension participation by type of pension plan and the accuracy of such information—particularly regarding DC plans, which are becoming an increasing part of retirement income—are important to understanding economic well-being of future retirees. If survey data are reported with substantial error, then this understanding is compromised.

Using data from the Census Bureau's Survey of Income and Program Participation, a major survey data source containing information on DC pension plan offering and participation, linked to Social Security administrative data, we examine the extent of reporting error regarding participation in and contributions to DC plans.

Our findings indicate that the offer rate of DC plans is about 17 percentage-points higher when the survey data are supplemented by data in the W-2 tax records. Furthermore, evidence indicates that the question used in SIPP about pension plan type confuses a nontrivial proportion of respondents. The participation rate is underestimated by 4 percentage points when using SIPP reports of inclusion in a DC plan compared with SIPP reports of tax-deferred contributions (active participation) to such plans. Furthermore, using SIPP-reported tax-deferred contributions underestimates by 7 percentage points the active participation rate compared with that indicated by the W-2 records. Thus, reliance on survey reports of tax-deferred contributions only partially closes the gap between survey-defined and W-2 record–defined participation rates.

Our analysis of possible sources of reporting errors of participation in DC plans finds that the imputation process conducted by the Census Bureau creates substantial errors. About 42–46 percent of imputations on tax-deferred contributions to DC plans are in error compared with W-2 contribution amounts. Moreover, controlling for several socioeconomic characteristics, the probability of having a false-positive error is about 50 percent higher among respondents with imputed rather than nonimputed information, whereas the probability of a false-zero error is only about 8 percent higher.

Finally, by comparing SIPP tax-deferred contribution amounts with those in the W-2 record we find that, while the median difference between the two is minor, substantial error is present at the upper and lower quartiles of the distribution. Furthermore, we find that, at the median, the absolute difference (between SIPP and W-2 amounts) is about 29–35 percent of the W-2 contribution amounts, increasing substantially at the upper part of the distribution. These relative differences were higher in 2006 than in 1998 and are also higher among workers in the public sector than those in the private sector. In addition, regression results reveal that this relative error is significantly higher among respondents with imputed contribution amounts than among those who self-reported them.

These findings suggest that the Census Bureau's procedures would benefit from using W-2 record information on tax-deferred compensations to retirement plans for both imputations and editing of respondents' reports. Analysts should use caution when using SIPP data on pension coverage and should consider using SIPP data linked with W-2 records of tax-deferred contributions. Furthermore, the findings in this study, although derived from the SIPP data, have implications for other surveys that collect pension information (such as the Survey of Consumer Finances, the Panel Study of Income Dynamics, and the Health and Retirement Study) because it is plausible that these types of measurement errors may potentially be present in other surveys as well. Questions about pension type, participation, and contributions are complex with concepts that the layperson may not use. Consequently, respondents may be inclined to nonresponse and therefore missing data are generated. Although SIPP gets somewhat around this problem by using the follow-up question, the implication for other surveys would be to modify the wording to the question about pension type. In addition, imputations to replace missing data can also generate measurement errors. Thus, any analysis should consider carefully the form of imputations and the possible use of better information about DC plans from W-2 tax records.