Homeless People Whose Self-Reported SSI/DI Status Is Inconsistent with Social Security Administration Records

Summary

Clinicians routinely ask people with disabling psychiatric illnesses whether they receive Supplemental Security Income (SSI) or Social Security Disability Insurance (DI) benefits. We looked at self-reported receipt of SSI or DI by 7,220 homeless people with mental illness and compared those self-reports with information in Social Security Administration (SSA) databases. Overall agreement between the two sources was only fair (kappa = 0.60), and 41.3 percent (934/2,257) of clients reporting receipt of SSI or DI were not in SSA's databases. In multivariate analyses, people reporting receipt of SSI or DI that is unconfirmed by SSA administrative records had disproportionately more severe psychotic and medical illnesses than confirmed nonrecipients. Among recipients identified by SSA, those who did not report receiving SSI or DI were more likely to claim, apparently incorrectly, that they instead received Social Security retirement benefits. Clinicians should verify basic demographic information provided by clients, especially those who are psychotic or medically ill, because that information is often inaccurate.

Introduction

People disabled by psychiatric illness depend on Supplemental Security Income (SSI) and Social Security Disability Insurance (DI) benefits to meet their basic needs. Disability payments provide critical financial support in preventing homelessness among the indigent (Sosin and Grossman 1991) and contribute to improved outcomes when homeless mentally ill people receive treatment (Rosenheck, Frisman, and Gallup 1995). Clinicians routinely ask indigent new clients if they receive SSI or DI, and this information is incorporated into treatment planning.

Given the importance of disability payments to people disabled by psychiatric illnesses, it is ironic that no prior studies have been done on the validity of self-reported SSI/DI status among the mentally ill. Some studies have described the low reliability (Jenkins and others 2005) and accuracy (Pedace and Bates 2001; Card, Hildreth, and Shore-Sheppard 2004; Jackle and others 2004) of self-reported income among poor people, but there are no studies to inform clinicians by describing specific psychiatric and medical characteristics of people whose self-reported SSI/DI status is inaccurate. The underreporting of symptoms and the inconsistency of information provided are considerable when people with substance abuse (Stephens 1972; Rounsaville and others 1981) or psychiatric disorders (Strauss, Carpenter, and Nasrallah 1978) are asked to describe their psychiatric history and symptoms. However, there is little data concerning whether homeless people with mental illness inaccurately report basic demographic information and, specifically, whether they accurately report receipt of SSI and DI.

There are several potential explanations for why clients might report SSI/DI receipt inaccurately. The misreporting of SSI/DI benefits may reflect neuropsychological deficits. Inaccurate self-reports might track related constructs like the degree of knowledge about one's medical care, which is lower in people with cognitive deficits and reading difficulties (Baker and others 1995; Kalichman and others 2000; Baker and others 2002). Another possibility is that inaccurate self-reported income is influenced by subtle social pressures to underestimate income. Evidence for the underreporting of income by poor people is that families reporting low income in the Labor Department's Consumer Expenditure Survey reported much higher expenditures, and low income and high expenses are difficult to reconcile (Jencks 1997).

The first goal of this study, conducted in 2004, was to document the degree of agreement between a client's self-report that he or she received SSI or DI benefits and SSA administrative records of whether the person was receiving benefits. We then characterized those clients whose self-reported SSI/DI status was not consistent with SSA administrative records using comprehensive clinical data, self-reported SSI/DI status, and SSA administrative data from participants in a large study of individuals who were homeless and mentally ill. This study first determined what demographic and clinical factors were associated with self-reports of SSI/DI receipt and not being in the SSA database; it then identified what factors were associated with reporting not receiving benefits but having SSA records that indicate otherwise.

Methods

Participants and Sampling

Participants were enrolled in the ACCESS (Access to Community Care and Effective Services and Supports) demonstration study, a study of service delivery strategies for homeless people with mental illness (Randolph and others 2002). In ACCESS, agencies in 18 cities offered Assertive Community Treatment (Stein and Test 1980) to 100 participants per year for 4 years. Participants were eligible if they were homeless, had a severe mental illness, and were not engaged in psychiatric treatment at the time of enrollment. Eligible participants were identified and offered case management services. After providing informed consent, a comprehensive set of assessments was completed.

Data Collection

Research assistants using structured interviews collected data. Basic demographic data included age, sex, children in residence, race and ethnicity, years of education, longest full-time job, and veteran status. Homelessness was characterized by age at the first episode of homelessness, number of times homeless, lifetime number of years homeless, and years living in the current city of residence. Legal status questions included questions about having ever been convicted or incarcerated. History of arrests (McClellan and others 1980) and victimization (Lehman 1988) within the last 60 days were also documented. Self-reported data concerning the presence or absence of 17 medical disorders and whether the client was taking prescribed medication were also recorded. Other self-reported symptoms quantified social support (Vaux and Athanassopulou 1987; Lam and Rosenheck 1999), service utilization (Rosenheck and others 2002), a history of conduct disorder (Helzer 1981), and stability of family of origin (Kadushin, Boulanger, and Martin 1981). Participants reported the number of days in the last 60 that they had been housed and the number of days in the last 30 that they had been employed. Overall quality of life was also assessed by the question "Overall, how do you feel about your life right now?" on a scale ranging from 1 (terrible) to 7 (delighted) (Lehman 1988).

Psychiatric diagnoses were those of the admitting clinicians on the case management teams. Psychiatric measures were derived from the Addiction Severity Index (ASI) psychiatric composite problem index, a depression scale derived from the Diagnostic Interview Schedule (Robins, Helzer, and Croughan 1981), and a psychotic symptoms scale derived from the Psychiatric Epidemiology Research Interview (Dohrenwend 1982). Depression was quantified as the number of symptoms of depression out of 5 endorsed by the client, and interviewer ratings of psychosis were derived from 13 items ranked on a 0–4 Likert scale.

Substance abuse was assessed by questions drawn from the Addiction Severity Index (McClellan and others 1980), and a referring clinician rated the patient's substance use on 5-point clinical rating scales anchored by 1 (abstinence) and 5 (severe dependence) (Mueser and others 1995).

Service utilization was measured by questions concerning receipt of six types of services: assistance from a public housing agency, mental health services, general health care, substance abuse services, public income support, and vocational rehabilitation. The number of services received was calculated. Finally, the research assistant rated the reliability of the participant's data on a 5-point scale.

Income Data

Participants were asked to record how much income they had received during the past month from a list of possible sources. Participants were also asked to record earnings for the current month, even if the money had not yet been received. The sources listed included earned income, Social Security retirement benefits, Supplemental Security Income, Social Security Disability Insurance, social welfare benefits from state or county governments such as general welfare and Aid to Families with Dependent Children (AFDC), and nine other potential sources of income. Participants were asked if there was anyone who "handles your money for you (like a payee or guardian)" and, if so, whether the client's checks were mailed directly to this person.

SSA's Office of Research, Evaluation, and Statistics provided client-level data on beneficiary status by cross-matching Social Security numbers of ACCESS participants with those in SSA's Master Beneficiary Record and Payment History Update System, which record payments from the DI program, and the Supplemental Security Record, which records payments from the SSI program. SSA provided data only when its files contained a corresponding Social Security number verified by date of birth. SSA's algorithm for determining whether there is a cross-match—the Enumeration Verification System—did not require the supplied dates of birth to exactly match those in SSA's databases. A Social Security number match was verified when the years of birth agreed or when the months agreed and the years differed by one year.

Data Analysis

The purpose of the study was to determine whether participants could distinguish SSI from DI from other sources of income. We were not concerned with whether participants could distinguish SSI from DI, so receipt of SSI or DI was considered a single measure (SSI/DI). Kappa was calculated to characterize the overall agreement between self-reported and SSA verification of receipt of SSI/DI. The kappa statistic describes the agreement between two dichotomous variables with a range of zero (no agreement) to 1 (perfect agreement). Then, two similar analyses were conducted. The first analysis determined demographic and clinical factors that differentiated people who reported receiving SSI/DI but were not in the SSA database from those who did not report receiving SSI/DI and were also not in the SSA database. Chi-square and t-test comparisons between the two groups were conducted on a broad range of measures. Measures that differentiated the two groups at p<.05 were entered into a logistic regression, and backward elimination was used to identify the most salient correlates at p<.01. A similar approach was employed to compare two other groups: those reporting that they did not receive SSI/DI but in fact were in the SSA databases as receiving benefits and those who reported receiving SSI/DI and were confirmed by SSA records.

Results

Sampling and Overall Agreement Between Self-Report and SSA Databases

Altogether, 16 percent of participants ([934 + 193]/7,220) reported SSI/DI status that was not verified by the SSA database (Table 1). The majority of the discordant reports were from participants who reported having received SSI/DI but were not in the SSA database (13 percent of the total sample) and 3 percent who reported not having received SSI/DI but in fact were in the SSA database. Kappa was 0.60, indicating moderate agreement between self-reports and SSA records (Cicchetti and Sparrow 1981).

Sample Characteristics by Self-Reported and SSA-Verified SSI/DI Status

The sample characteristics shown in Table 2 indicate, as expected, relatively long durations of homelessness and high rates of psychiatric comorbidity and substance abuse. All the measures in Table 2, within the groups of those who had and had not received SSI or DI according to SSA, significantly differentiated the participant group whose self-report was concordant with SSA from participants whose self-report was discordant with SSA's administrative records.

Comparison Among Clients not Receiving SSI/DI According to SSA: Participants Self-reporting Receipt of SSI/DI versus Those not Self-reporting Receipt. In multivariate analyses, the measures that significantly (p<.01) distinguished the 934 individuals reporting receipt of SSI/DI (without SSA verification) from the 4,770 not reporting receipt (in concordance with SSA records) are listed in Table 3. The 934 participants with unverified reports of receiving SSI/DI were more impaired in several realms. They had disproportionately less education and employment and were disproportionately more likely to have been diagnosed with schizophrenia, human immunodeficiency virus (HIV), and seizure disorders.

Not all functional indices were worse among those with unverified claims. Within this population of homeless people, those who had unverified claims were likely to have used alcohol and cocaine for fewer years and to have been incarcerated for fewer days in the preceding 60 than were those who did not claim receipt of SSI/DI. Self-reported depressive symptoms and a diagnosis of major depression were associated with a lower likelihood of making an unconfirmed claim of receiving SSI/DI.

Benefit status differed between the two groups. Participants with unverified claims of receiving SSI/DI were more likely to report having a payee than were those who did not claim benefit receipt. Those with unverified claims also had received fewer benefits overall.

Comparison Among Clients Receiving SSI/DI According to SSA: Participants not Self-reporting Receipt of SSI/DI versus Those Self-reporting Receipt. Participants who did not report receiving SSI/DI in contradiction to SSA's records that they actually had received benefits were more likely to have reported receipt of Social Security retirement benefits and other social welfare benefits (Table 4). In a post hoc analysis, we considered the possibility that clients who thought they received Social Security retirement benefits were disproportionately aged 62 or older, and they were. Altogether, 17.4 percent (34/195) of participants who inaccurately reported nonreceipt of SSI/DI were aged 62 or older, but only 3 percent (39/1,322) of those with concordant reports of receiving SSI/DI were aged 62 or older (chi-square 77.8, p<.0001).

Discussion

Fully 41 percent (934/2,257) of clients who reported receiving SSI/DI benefits did not receive them according to SSA. Clients whose report of receiving SSI/DI was unconfirmed were more likely to have conditions associated with neurocognitive impairment: they were disproportionately psychotic, HIV-positive, diagnosed with a seizure disorder, and occupationally impaired. Clients who misreported basic demographic information may also not have understood the benefits they receive, the question asked, or how to translate their knowledge into a correct response. The clients whose report of receiving SSI/DI was not confirmed used cocaine and alcohol for disproportionately fewer years, but this finding is not inconsistent with a cognitive explanation for anomalous self-reports—some studies indicate that within populations of people with mental illness, those who use drugs may actually be higher functioning (Ries and others 2000).

Cognitive problems also may have been a factor when participants who had received SSI/DI according to SSA did not report receiving those benefits. These clients appear to have been confused by different types of "social" benefits and apparently indicated receipt of Social Security retirement benefits and social welfare benefits instead of the actual SSI/DI they were receiving.

The overreporting of SSI/DI receipt relative to administrative databases in this homeless, mentally ill population is in contrast to the underreporting of income among poor people generally (Hotz and Scholz 2002). For example, validation of data from the Survey of Income and Program Participation suggested that self-report responses underestimated SSI receipt by as much as 23 percent (Marquis and Moore 1990). The responses of homeless people with mental illness may be affected by neurocognitive difficulties that are less salient in poor people who are not defined by homelessness and mental illness.

One clinical implication of the problematic self-reports is that when a client reports receiving SSI or DI, the assertion should be verified. The client can be asked the amount of the check or how the check came to be awarded. Clients should also be questioned to make sure the check referred to is an SSI or DI check and not another kind of payment. Information about benefit receipt can be obtained when another person receives the benefit check or by examining the clients' Medicare card. Primary Medicare beneficiaries who are too young to qualify for retirement benefits presumably receive DI.

The low agreement between self-report and SSA databases among the homeless, mentally ill population has other far-reaching implications. Data concerning sources of income are collected in the U.S. Census and several surveys specifically targeting poor people (Hotz and Scholz 2002). Accurate data about use of public support payments is crucial to assessing the impact of policies such as welfare reform (Primus and others 1999) and changes in eligibility for SSI and DI (Watkins, Wells, and McLellan 1999). In health services research, self-reported Social Security numbers and dates of birth are frequently used to cross-match data from people with known clinical characteristics with another database of interest (Friedman and others 1996; Bach and others 2002). A systematic bias is unwittingly introduced to data when a failure to cross-match is not random.

Some clients who reported receiving SSI/DI but did not appear in SSA databases probably did not cross-match with SSA databases because they provided inaccurate Social Security numbers (SSNs) or inaccurate dates of birth. In the 1996 Survey of Income and Program Participation, a full 16 percent of the SSNs provided by survey participants appeared to be inaccurate because they did not match SSNs in the Summary Earnings Record (Huynh, Rupp, and Sears 2002). One reason to suspect that inaccurate SSNs were provided is that the 1,323 participants whose reported receipt of SSI/DI was validated by SSA administrative records were similar to the 934 whose self-reported receipt was not validated (Table 2). For instance, both groups included high proportions of clients who reported that someone else received their check and managed it for them (29.3 percent and 27.6 percent, respectively). The clients who are discordant with SSA records could have some sort of non-SSA fiduciary arrangement, but the 27.6 percent reporting that someone else receives their check is consistent with other estimates that approximately a third of adults under the age of 65 who receive SSA payments based on a psychiatric disability have been assigned a payee to manage their funds (Social Security Administration 2001a, Table 7; and 2001b, Table 32).

Social Security numbers have high sensitivity and specificity in validating death against the National Death Index (Williams, Demitrack, and Fries 1992), and SSA databases are highly regarded (Waldron 2001). Yet underreporting of deaths to SSA does occur and is not random—underreporting of death information provided to SSA by third parties (such as state vital record systems) is less likely when the deceased was a woman, black, younger, unmarried, or from the South (Curb and others 1985; Wentworth, Neaton, and Rasmussen 1983; Boyle and Decoufle 1990).

Benefits for the Supplemental Security Income and Disability Insurance programs provide a vital safety net for clients disabled by psychiatric disorders. It is important that each individual's benefit status be accurately determined for that client's clinical care and that studies dependent on demographic information provided by impaired clients be independently verified so that use of the Social Security safety net is accurately described.