Factors Affecting Initial Disability Allowance Rates for the Disability Insurance and Supplemental Security Income Programs: The Role of the Demographic and Diagnostic Composition of Applicants and Local Labor Market Conditions

Introduction

The probability of an initial disability allowance among Disability Insurance (DI) and Supplemental Security Income (SSI) applicants is affected by numerous factors at individual, state, and national levels. That includes demographic and diagnostic characteristics of applicants (age, sex, and type of impairment), national policies, local factors such as the state unemployment rate, and the implementation of disability determination policies by state Disability Determination Services (DDSs). In this article, I focus on the role of key factors that are outside of the direct control of DDS management in affecting initial allowances. I also discuss implications for the understanding of variations in state initial allowance rates.¹

Three of the most important factors believed to affect the probability of an initial allowance are (1) demographic characteristics of applicants, (2) the diagnostic mix of applicants, and (3) local labor market conditions. Existing descriptive statistical tabulations of initial determination results summarize the association of those variables and the initial allowance rate, but are not designed to isolate the independent effect of those factors. More is known about the causal effect of labor market shocks on the allowance rate. Consistent with previous studies (discussed in the next section), I use fixed-effects multiple regression analysis. The fixed-effects model controls for state- and/or year-specific effects. This nonexperimental methodology allows the estimation of the relationship between the dependent variable (initial allowance rate) and the independent variables of interest net of long-term differences among states and changes in national policies affecting each state. Because of the use of individual-level data on applicants, I can look at the relationship between my indicator of local labor market conditions—the unemployment rate—and the initial allowance rate, considering a level of detail on applicant characteristics (for example, age and diagnostic category) that has not been feasible in previous work that was based exclusively on state-level data. Because in this study (as in all related analyses) the nature of the evidence is nonexperimental, caution is warranted in causal interpretation.

There are a number of reasons for expecting a relationship between labor market conditions, disability applications, and the initial allowance rate. Adverse labor market conditions are expected to bring in marginally qualifying or marginally interested applicants or affect the timing of application following disability onset. Adverse labor market conditions may result in job loss or decreased access to full-time, part-time, or irregular job or work opportunities.² Those changes typically hit people with disabilities the hardest. Thus, people in the process of disablement may apply for disability benefits earlier than they otherwise would. Others may believe that they are only marginally qualified, but the opportunity cost of applying for disability benefits decreases sufficiently for application to appear worthwhile. A related, but distinct factor is that reduced employment and earnings increase the chances for an adult applicant to pass the substantial gainful activity (SGA) screen. These factors are relevant in the DI and SSI adult program context. The loss of income and reduced work opportunities also increase the chances that the individual would meet the SSI income screen. That factor affects adult SSI and concurrent applicants. Concurrent applicants meet the eligibility criteria of both DI and SSI, and thus are potentially eligible to receive both types of benefits. Job loss, poor labor market conditions, and the increased chance of meeting the SSI means test because of parental income loss may also affect adults with disabled children by increasing the relative attractiveness of applying for disability benefits on behalf of their child. An increase in the proportion of marginally qualified applicants is outside of the control of the DDS and should be expected to result in a reduced average probability of allowance. A negative relationship between changes in the unemployment rate and the probability of an initial allowance suggests that the Social Security Administration (SSA) is successful in screening out some marginally qualified applicants. The present study has been designed to address the overall relationship between the unemployment rate and the initial allowance rate. In this study, I do not test specific mechanisms underlying that relationship, although that is a worthwhile topic for more research in the future.

This study focuses on the initial disability determination outcome (initial allowance or initial denial).³ Thus, when I speak of "applicants" in the context of this analysis, I am referring to people subject to the initial determination decision.⁴ I refer to Title II of the Social Security Act as "DI" and Title XVI as "SSI." People applying for or receiving both DI and SSI benefits are referred as "concurrents" throughout this article. Adults who apply for benefits are classified as "DI-only," "SSI-only," or "concurrent" applicants.

The rest of the article is structured as follows: I review related literature in the next section and then address the data and methodology. I then present the empirical results and finally conclude.

Previous Literature

A paucity of research addresses factors affecting disability allowance rates. However, a larger body of previous research looks at factors affecting applications and awards. The broader literature is relevant for my topic for several reasons. First, key hypotheses about the effects of labor market conditions on initial allowance rates are predicated on the presumption of a positive relationship between adverse labor market conditions and disability applications. Second, separate estimates of the effects of labor market conditions on applications and on initial allowances is informative with respect to possible effects of labor market conditions on initial allowance rates. For example, if an estimated positive relationship exists between changes in the state unemployment rate and changes in the volume of disability applications, and a similar but smaller effect with respect to the volume of initial allowances, such a finding would suggest a negative relationship between high unemployment and the initial allowance rate. Finally, the methodological issues are similar with respect to estimating the effect of various factors on applications, allowances, and allowance rates.

Rupp and Stapleton (1995) summarized studies on the relationship between the unemployment rate, applications, awards, and caseload growth between 1974 and 1995, using primarily aggregate time-series methods and, in a few instances, cross-sectional approaches based on individual-level data. In addition, they presented new results based on state-level, fixed-effects modeling. Their fixed-effects analysis also provided much more detail and programmatic specifics than did previous studies. Specifically, while previous studies were limited to DI applications and awards, the authors also estimated models of initial determinations and the initial allowance rate separately for DI-only, SSI-only, and concurrent applicants. The fixed-effects results generally have shown a positive relationship between the state unemployment rate and both DI and SSI applications and awards. The estimates for initial allowance rates have shown zero effect in the year of change, but significant negative effects with lags of 1 year and 2 years for all three program groups. A lagged effect arises if changes in the unemployment rate resulted in increased applications in subsequent years. The authors estimated that a 1 percentage point increase in the unemployment rate was associated with no change in the initial allowance rate for DI-only applicants in the given year of the unemployment rate change, but it resulted in a 1 percentage point decrease in the initial allowance rate during the following year as well as during the second year after the unemployment rate change. Stapleton, Dietrich, and Lo (1995) and Stapleton and others (1998) provided detail on the data and methodology that was used to estimate the results reported previously by Rupp and Stapleton (1995).

Using individual-level data on a sample of men from the 1978 Survey of Disability and Work and a structural model of applications, awards, and state-contingent lifetime income flows, Kreider (1999) estimated, as expected, that the unemployment rate has had a positive effect on DI applications. Black, Daniel, and Sanders (2002) used county-level data; a natural experiment framework; and first differences with state, county, and year fixed effects to estimate the effect of labor market shocks to the local economy on program participation. The authors found a negative relationship between labor market participation and disability program participation both for DI and SSI.⁵ Their results suggested that the relationship between economic conditions and disability program participation is much stronger for permanent than for transitory economic shocks. Transitory shocks reflect short-term (for example, year-to-year) changes—such as typical business cycle effects—while permanent shocks are associated with lasting change, such as the disappearance of a major source of jobs in the local economy or the coal boom and bust studied by the authors. Burkhauser, Butler, and Weathers (2001/2002) used Health and Retirement Study data and a hazard model framework to analyze the effect of various factors on the timing of DI applications. They used the unemployment rate to capture the effects of labor market conditions on a worker's decision to apply for DI. Their results showed that a higher state unemployment rate substantially increases the risk of DI application following the onset of disability for both men and women and reduces the time from disability onset to application.

Strand (2002) used state-level data to assess variation in allowance rates for the 1997–1999 period. He used ordinary least squares (OLS) regression, which pooled the 3 years of data, with the allowance rate (and in some models, the filing rate) as the dependent variable, a number of state-level predictors, and year dummies. The models explained a substantial portion of state-to-state variation, and when adjusting for the independent variables, the variation in allowance rates among states was cut in half. The author also found that states with the highest and lowest allowance rates for DI and SSI tended to retain that status over time and that there was a negative association between filing rates and allowance rates. The unemployment rate coefficients were negative, as expected, and statistically significant.

Autor and Duggan (2003) used state-level difference equations to look at the effect of labor market conditions on DI applications. They found that state employment contractions were associated with increased DI applications. Their results were generally robust to fixed effects and other alternative specifications. Duggan and Imberman (2008) estimated annual time-series regressions of DI applications, awards, and recipients at the national level using more recent data (1984–2003) than previous studies. They produced results comparable with previous estimates reported by Rupp and Stapleton (1995); Black, Daniel, and Sanders (2002); and Autor and Duggan (2003). Duggan and Imberman's results showed that adverse labor market conditions have had significant effects on DI entry in the expected direction.

Soss and Keiser (2006) used a pooled cross-section analysis of both DI and SSI applications in a "seemingly unrelated regressions" framework. Their state-level models explained well over half of the variation in DI application rates and even more of the variation in SSI application rates. They found that increases in the state unemployment rate were associated with increases in both DI and SSI applications. Guo and Burton (2008, 2012) focused on state-level models of DI application rates with and without fixed effects and found that both specifications resulted in a substantial positive relationship between the unemployment rate and DI application rates. Using state-level data for the 1993–2009 period, Coe and others (2011) investigated the extent to which state-level variation in DI application rates reflected differences in health, demographic and employment characteristics of the population, policies, and politics. The authors presented estimates with and without state fixed effects. The models not including state fixed effects explained 70–80 percent of the state-level variation in application rates depending on specification and program group, whereas the models with fixed effects explained 85–93 percent of the variation. All three fixed-effects models estimated a positive unemployment rate coefficient, but only two of them (overall DI and DI/SSI concurrent) were statistically significant.

Thus, recent studies generally found a positive relationship between labor market shocks and disability applications. They used a variety of methodologies and their point estimates varied and were not always directly comparable, but the qualitative conclusion of a positive association was consistent across studies. Only two studies (Rupp and Stapleton (1995) and Strand (2002)) provided direct estimates of the relationship between the unemployment rate and disability allowance rates.⁶ Both considered a number of state-level variables and provided evidence consistent with the hypothesis of a negative link.

Data and Methodology

In this study, I build on previous work that used fixed-effects modeling to assess factors affecting disability applications, allowances, and the allowance rate. My initial disability determination model is as follows:

Y_ist = α + βX_ist + γUE_st+ δ_s + ζ_t + ε_ist,

where

Y_ist = initial decision for individual i, in state s in year t (ist) − Y_ist = 1 if initial decision is an allowance, Y_ist = 0 if initial decision is a denial;

X_ist = vector of characteristics for individual (ist) subject to the initial determination;

UE_st = the state unemployment rate for state (s) at year (t);

α = estimated intercept;

β = vector of estimated coefficients for individual characteristics;

γ = estimated coefficient for the unemployment rate;

δ_s= estimated fixed effect for state (s);

ζ_t = estimated fixed effect for year (t); and

ε_ist = a random error term.

In the individual-level models, I use a logit specification with the result of the initial disability determination (1 = allowance, 0 = denial) as the dependent variable. All models include state and time fixed effects. Additional independent variables include the natural logarithm of the state unemployment rate and individual indicators for sex (female = 1); age group (dummies for the following age groups for adults: 18–24, 25–29, 30–34, 35–39, 40–44, 45–49, 50–54, 55–59, 60–64, with 50–54 chosen as the reference group and dummies for the following age groups for children: 0–5, 6–12, 13–17, with 0–5 chosen as the reference group); primary diagnosis (15 dummies, with musculoskeletal chosen as the reference group⁷); and the presence of a secondary diagnosis indicator⁸ (yes = 1). While this study improves on previous work partly by including those individual-level variables that provide substantial controls for factors affecting allowance rates, there are unobserved aspects of case severity that are not explicitly accounted for in the econometric model. Program group is a stratifying variable (DI-only adult, SSI-only adult, concurrent adult, SSI child). All of the states and the District of Columbia are included as "states"; Puerto Rico and the US territories are excluded from the sample frame.

I also conduct additional analyses using state-level data to address issues related to the possibility of aggregation bias that may have affected previous studies, to assess the predictive properties of my models, and to address issues related to variation in allowance rates across space and time. Most of my state-level models use OLS as did related work by Strand (2002). In addition, I explore issues related to possible lagged effects using generalized least squares (GLS) to correct for autocorrelation.⁹

There are several innovative features of the present study. This is the first study that controls for the demographic and diagnostic mix of individual applicants—two sets of variables known to be strongly related to the probability of an initial allowance. Their inclusion allows me to analyze the role of the unemployment rate controlling for and conditional on the characteristics of applicants. The data cover a longer and more recent period than did previous studies. Finally, previous studies did not separately estimate models for SSI childhood allowance rates, while this study does.

The study universe consists of 1,736,554 records, a 5 percent random sample of initial determinations for the 1993–2008 period. The sample frame includes DI-only adults, SSI-only adults, concurrent adults, and SSI childhood initial determinations. The sample frame excludes DI-only and concurrent childhood initial determinations (consisting of 1,698 observations, or about 0.1 percent of the source 5 percent sample). The data source is the so-called "831 file"¹⁰ from SSA's National Disability Determination Services System File, which contains data on DDS initial determinations. As previously noted, technical denials made by SSA prior to sending application records to the DDS are not included in the sample frame. The state unemployment rate for the given year, obtained from Current Population Survey data, was added to each initial determination record. Table 1 describes the characteristics of the applicant sample by SSA program group.

Results

This section is organized around two topics: (1) estimated effects of demographic characteristics, diagnostic factors, and the unemployment rate on the initial allowance rate; and (2) differences in allowance rates among the states.

Estimated Effects of Demographic and Diagnostic Factors and the Unemployment Rate on the Initial Allowance Rate

Table 2 shows the estimated effect of demographic characteristics, diagnostic factors, and the state unemployment rate on the probability of an initial allowance from fixed-effects logit regressions. The models in this table include state and year fixed effects, as well as the following time-varying factors: demographic characteristics, primary diagnosis, the presence of a secondary diagnosis, and the state unemployment rate. The results show that both sex and age are associated with the probability of an initial allowance. Women have a lower probability of an allowance for all program groups. The positive relationship between age and the probability of an initial allowance is strong and remarkably similar across the three adult program groups.¹¹ Children aged 0–5 are more likely to be allowed than those aged 6–17.

There is substantial variation in the estimated marginal effect of primary diagnosis (Chart 1). Looking at DI-only adults, I observe that (net of the effect of other independent variables) having a musculoskeletal impairment (the reference category) is associated with a predicted probability of initial allowance that is the lowest of all primary impairment categories (except "unknown"). In contrast, having an intellectual disability, a disease of the genitourinary system, or neoplasms as a primary impairment substantially increases the probability of an initial allowance. The overall patterns are quite similar for SSI-only adults and concurrent adults. Remarkably, with a few exceptions, the patterns of estimated primary diagnosis effects are similar for SSI children. Having a secondary diagnosis has a relatively small estimated negative effect on the three adult program groups, while the coefficient for SSI childhood initial allowances is positive and significant.

Controlling for demographic and diagnostic characteristics of applicants, Table 2 also shows that an increase in the unemployment rate is associated with a decrease in the probability of an allowance for all four program groups. Table 3 explores that relationship further by comparing estimates of the unemployment rate coefficient from three different model specifications, by SSA program group. We start with a model that includes state and year fixed effects and the unemployment rate and then add demographic and diagnosis predictors in a sequential fashion to investigate the sensitivity of the unemployment rate coefficient. Model 1 includes state and year fixed effects and the unemployment rate as independent variables. Model 2 adds demographic predictors, while model 3 adds primary diagnosis variables and an indicator for the presence or absence of a secondary impairment. The top panel of Table 3 provides the results for the DI-only group. All three estimated coefficients are negative and statistically significant. The results are remarkably robust to model specification. The patterns are similar for SSI-only and concurrent adults. Notably, the estimated relationships for SSI children are quite similar to the results for adults, albeit the point estimates tend to be somewhat lower. The statistically significant negative relationship for children is remarkable because, unlike for adults, the reference person with a disability (the child) is not directly affected by the unemployment rate, and the person who may be affected by it (the adult parent) is not necessarily disabled. Thus, some of the explanations related to the effect of changes in the local unemployment rate on the employment status of the disabled applicant are not relevant in the same way as they may be for adults with disabilities. Plausible explanations involve the effect of job loss of adults on the financial eligibility of the child and the effect of adverse labor market conditions on the opportunity costs of disability application on behalf of the child. Chart 2 presents the corresponding estimates of marginal effects of the unemployment rate evaluated at the mean for each of the four SSA program groups.

Given the wide variation in the marginal association between allowance outcomes and diagnostic categories (as shown in Table 2) for all four program groups, a more detailed analysis of the relationship between diagnosis and labor market shocks is warranted. Specifically, an important question is whether the unemployment rate effect is conditional on the diagnostic category of the applicant. Table 4 explores that possibility by presenting the estimated unemployment rate coefficients from separate logit models for each primary diagnosis group by SSA program group (68 regressions). Chart 3 shows only the estimated effects that are statistically significant—different from zero at least at the 0.95 level of confidence. All of the statistically significant coefficients are negative as expected.¹² There is some variation among the diagnostic categories, suggesting that the unemployment rate has different effects on the allowance rate for different types of impairments. In particular, the estimated negative effect of the unemployment rate is consistently large across adult program groups for mental and musculoskeletal disorders. Importantly, all of the statistically significant estimates for SSI children are also negative (as shown in Table 4), albeit only three coefficients are significant (intellectual disabilities, mental impairments, and endocrine disorders). Thus, the evidence is strong for a tendency of a negative association between allowance rates and unemployment rate effects conditional on diagnosis for both adults and children.

Generally, increases in the unemployment rate are associated with reductions in the probability of an initial allowance outcome—a finding that is fairly robust. This is consistent with the hypothesis that deteriorating labor market conditions have a positive effect on applications¹³ through drawing in marginally qualified applicants and that SSA is successful in screening out many of those marginally qualified applicants in the initial determination phase.

Because previous related studies used state-level data, it is worthwhile to compare results from this study with those of studies employing similar models; I use a data set aggregated to the state level. The state-level results are summarized in Table 5, where I use the same underlying data set that was used for the individual-level analysis to facilitate comparability with that analysis. Table 5 shows that the state-level estimates of the effect of the unemployment rate on allowance rates are robust to weighting and model specification. The dependent variable in all of the models is the initial allowance rate. Models 1–5 use independent variables that are directly comparable to the individual-level predictors. All of the estimated effects are statistically significant and negative; they vary in a relatively narrow range and are similar to the estimated marginal effects from the individual-level models.

An additional issue I address is the three-way relationship between the unemployment rate, volume of applications, and initial allowance rate. From previous research, we know that there is strong evidence of a positive relationship between the unemployment rate and the number of initial applications; as the unemployment rate increases, more people apply for disability benefits. One question that comes to mind is whether the negative association between the unemployment rate and initial allowance rate is simply a reflection of that increased volume or whether the association continues to be negative and statistically significant after controlling for the volume of applications. A related question is whether increases in the volume of applications are associated with the initial allowance rate for reasons unrelated to increases in the unemployment rate. In order to answer these questions, both the unemployment rate and the volume of applications need to be included as independent variables in the regression model. Model 6 includes both variables, using the natural logarithm of the number of initial determinations in the state as the indicator of the volume of applications (see Table 5). Two sets of estimated coefficients (sets 1 and 2) are provided for model 6 to answer the two questions above. The first set shows the unemployment rate coefficient for the four program groups and the second set displays the initial determinations coefficients. The estimated unemployment rate effects are highly robust to the inclusion or exclusion of the initial determinations independent variable, suggesting that the relationship between the unemployment rate and initial allowance rates is not explained by the increased volume of applications associated with higher unemployment. Having said that, the volume of applications also has an independent effect. All four estimated coefficients are negative, suggestive of an independent negative relationship between changes in the number of initial determinations and changes in the initial allowance rate, although only two of these coefficients are statistically significant.

Finally, I conduct tests addressing the temporal dimension of the relationship between the unemployment rate and initial allowance rates. Recall that Rupp and Stapleton (1995) reported statistically significant negative lagged effects from state-level models, but no significant relationship between the unemployment rate and initial allowance rates for the same year. In contrast, both my individual- and state-level analyses show statistically significant negative relationships for the same year. Consequently, I conduct further analysis of my state-level data to shed light on the source of those differences. Similar to Rupp and Stapleton (1995), I use models that correct for autocorrelation and include lagged values of the unemployment rate predictor.

In Table 6, correcting for autocorrelation using generalized least squares (GLS) without including lagged values of the unemployment rate predictor does not affect the sign and statistical significance of the adult current-year unemployment rate coefficients, but produces coefficients that are smaller in absolute value. For childhood models using the current-year unemployment rate predictor, the GLS estimate is not statistically significantly different from zero in two of the models, but is statistically significant and positive in the model that controls for both demographic and diagnostic factors.

Next, I look at the effect of including lagged values of the unemployment rate predictor. In most cases, the inclusion of lagged values did not change the sign of the current-year (period t) coefficient, but tended to eliminate statistical significance. Most strikingly, when both period t-1 and period t-2 lagged predictors are added, all of the t-2 coefficients are consistently negative and in most cases statistically significantly different from zero. I measure the cumulative effect by the sum of the three coefficients and estimate it as -1.8 percent for DI-only adults, -1.3 percent for SSI-only adults, -0.7 percent for concurrent adults, and -0.8 percent for SSI children (author's calculation based on model 1 estimates in Table 6).

Based on this analysis, the current study and the Rupp and Stapleton (1995) estimates are consistent with respect to finding evidence to support the hypothesis of lagged effects of the unemployment rate on initial allowance rates, but inconsistent in that the 1995 study failed to show current-year effects. The inconsistency with respect to current-year effects may of course reflect the fact that the two studies are based on data reflecting very different time periods and economic conditions. In my view, however, there are reasons to have confidence in the current study's findings of current-year effects and to conclude that the failure of the 1995 study to find current-year effects may be attributable to data limitations of that study. With respect to this analysis, the current-year estimates are consistent and statistically significant using a large number of individual- and state-level models pertaining to a large number of subgroups. In addition, this study includes detailed controls for demographic variables and the diagnostic mix of applicants, while the previous study lacked this level of detail. Finally, the data for the 1995 study had some unique weaknesses, as discussed by Stapleton, Dietrich, and Lo (1995). The most important in this context is the fact that there was a temporal misalignment in that study between the data set pertaining to the unemployment rate and the administrative data on initial determinations that should bias current-year estimates toward zero. Specifically, while the unemployment rate data were compiled based on a calendar-year basis, the initial determination data reflected a fiscal-year concept that started 3 months earlier.

Several conclusions and issues for future research arise from the analysis presented in Table 6. The state-level estimates therein are generally consistent with the hypothesis of a negative relationship between disability allowance rates and the local unemployment rate. The evidence is strongest for DI and SSI adults and somewhat weaker for concurrent adults and SSI children. The analysis also suggests that both current-year and lagged unemployment rate effects are present, that is, the unemployment rate affects initial allowance rates during the same calendar year, but some of the effects on initial allowance rates materialize years later. The evidence of lagged effects on allowance rates is consistent with the finding of Burkhauser, Butler, and Weathers (2001/2002) that the risk of applying for DI benefits is highest during the year immediately after the onset of a work-limiting condition, but the median duration between onset and disability application is much longer. Because the issue of possible lagged effects of the unemployment rate and initial disability allowance rates has substantial policy relevance and, to my knowledge, is addressed only in this article and in Rupp and Stapleton (1995), more research on the temporal structure of the unemployment rate effect is warranted.

State-to-State Differences in Observed Allowance Rates

In this section, I address two additional aspects of state-to-state differences in allowance rates. The first explores prediction and the second deals with the understanding of variations of allowance rates across space and time. Neither aspect involves causal inference. The term "explanation"—commonly used in this context (for example, explanation of R²)—does not imply causation.

The issue of the predictive power of the regression models has to do with the ability of the independent variables included in the models to predict the allowance rate for individual states. In assessing the quality of the state-level predictions, following Strand (2002), I use the counterfactual of the average absolute difference between the state and national mean as the main measurement tool; in the absence of any additional information, the national mean is the best predictor of the state allowance rate. My model, however, considers a number of additional variables that allow me to calculate a predicted allowance rate for each state that itself varies around the national average. The expectation is that the average absolute difference between observed and predicted state allowance rates is smaller than the counterfactual that considers only the national average in the prediction. The magnitude of reduction is my key measure of the predictive ability of the model.

First, I look at the relationship between observed and model-predicted allowance rates by state and program group for selected years. Table 7 shows the average observed and model-predicted allowance rate for the states, the standard deviation of observed and predicted state allowance rates, the average absolute difference between the state and national mean, and the average absolute difference between the observed and predicted state average. All of the estimates are based on unweighted averages of the state-level statistics.

When I compare the average observed and predicted initial allowance rates for individual years, all of the differences are within 2 percentage points. Most are much smaller: 16 of the 20 estimates show a difference of 1 percentage point or less. The standard deviation of predicted state allowance rates is noticeably lower. An arguably better measure of the explanatory strength of the model is given by comparing the average absolute difference between the observed state and national mean (a model with no predictors) with the average absolute difference between the observed and model-predicted state averages. As in Strand (2002), I prefer the average absolute difference measure to the standard deviation measure because outliers unduly influence the standard deviation. The model results in substantial reduction. For example, for DI-only adults in the year 2000, state allowance rates on average varied 7.8 percentage points in either direction from the national average. The model accounts for 4.0 percentage points, explaining roughly half of the average absolute difference. In other words, the individual-level predictions aggregated to the state level reduce the average absolute difference in allowance rates by roughly half, as opposed to the counterfactual, which compares raw state allowance rates with the national average allowance rate. The magnitude of the results is comparable to Strand's results for DI and SSI allowance rates in 1999. The magnitude of the reduction in average absolute value of the difference between observed and predicted allowance rates is expressed both in percentage point and percent terms in the two columns with the heading "Reduction of average absolute difference attributable to model" (Table 7). The percent reduction varies by 10–59 percent, but most are substantial—9 of the 20 estimates are close to or above 50 percent.

Table 8 explores the sensitivity of predictive accuracy to the use of individual-level compared with state-level models¹⁴ and to decisions about the use of weights. It compares the results of five alternative methods to measure overall accuracy. In this analysis, weights are used in two distinct steps. The first is the estimation of the regression model. The individual-level models are "self-weighting," while the state-level models can be estimated unweighted or weighted. In this context, the term self-weighting means that (1) the individual-level models weight each initial determination equally, and (2) the result is that each state is represented proportionally to the number of initial determinations in that state.¹⁵ The second step is the derivation of summary statistics at the national level from the state-level estimates. The averaging is either unweighted (giving each state equal weight, regardless of size), or weighted (giving each state a weight proportional to the number of initial determinations in that state). I illustrate the sensitivity of results to the level of aggregation of the sample used for the regression model and weighting at two steps in the analysis based on calculations showing DI outcomes for the year 2000. Method 1 represents the kind of statistics the previous results in Table 5 were based on (individual-level logit model, self-weighting sample used for model estimation, and unweighted average of state-level results to derive the national average absolute difference). The findings from Table 8 are straightforward. First, the estimated average absolute difference between observed and model-predicted values is not sensitive to the use of individual-level versus state-level data in the modeling; there is only a 0.2 percentage point difference between the individual-level method 1 and the state-level method 3 results. The same is true in the comparison of method 2 and method 5. Second, the results are not sensitive to giving equal weight to each state or using weights proportional to the number of initial allowances in the state-level models; the result from method 4 is only 0.2 percentage points higher than the result from method 5. Third, the only factor that affects the magnitude of the average absolute difference in a meaningful way is the use of weighting in summarizing state-level averages to the national level. When state results are weighted by size (the number of initial determinations), there is a clear reduction in the average absolute difference compared with results that give equal weight to the states in summarizing the state-level results to the national level. The unweighted average of the state-level estimates from the individual-level model (method 1) is 4.0 percentage points, while the corresponding weighted average of the states from method 2 is 3.3 percentage points—a difference of 0.7 percentage points (a substantial percent reduction). The magnitude of difference between method 3 (unweighted state averages) and method 4 (weighted state averages) displays a similar pattern (0.5 percentage points). The reduction arising from weighting the state averages is partly explained by the fact that random error of state-level estimates is negatively related to size. That suggests that some of the unexplained variation in the unweighted state-level models is attributable to random variation; the weighted estimates give more weight to the larger states, while the unweighted estimates are unduly influenced by very small states, especially because the state-level estimates are based on an underlying 5 percent sample of individual-level observations rather than a larger sample.¹⁶

Next, I consider how my fixed-effects models contribute to the understanding of the overall variation in allowance rates across states. The R² is commonly used to assess the variation "explained" by the independent variables included in a regression model. This explanation is purely descriptive and does not reflect causal understanding. In the context of fixed-effects modeling, we can go a little further and decompose the total variation into the following four components:

1. variation attributable to unmeasured factors that reflect long-term differences among the states,
2. additional variation attributable to unmeasured factors that reflect year-specific national differences,
3. additional variation attributable to the effect of specific independent variables that vary by state and over time, and
4. unexplained variation that may result from unmeasured time-varying factors not explicitly considered in the model or from purely random fluctuations.

Table 9 shows the decomposition of total variation. State fixed effects reflect long-term differences among states and account for 41–52 percent of explained variation. Note that state fixed effects do not provide any specific information about the reasons for these long-term differences, which may be due to factors that are exogenous or endogenous to program management. Next, I find that year effects range widely by program group from 9 percent for concurrent adults to 29 percent for the SSI-child category. Year effects pick up the effects of changes over time that affect all states equally. Perhaps it is not surprising that the variation explained by year effects is by far the highest for the SSI-child program group, given the enormous volatility of national policies and their implementation from the Supreme Court's "Zebley" decision to the 1996 welfare reform. Note that state and time fixed effects combined explain 55–79 percent of the overall variation.¹⁷

I find that adding the three sets of time-varying independent variables to state and time fixed effects in a sequential fashion increases the R² by 6–18 percent depending on program group. The relative contribution of the three principle sources of time-varying factors is somewhat sensitive to the sequence of adding the three groups of independent variables to the equation.

Given the substantial contribution of state fixed effects to the overall variation in allowance rates, Chart 4 shows the average allowance rate by state over the 1993–2008 observation period. The chart demonstrates that long-term observed average differences among the states—in effect the estimated state fixed effects—are substantial. Further analysis of the 1993–2008 average state allowance rates suggests that the long-term average of allowance rates by state is correlated among the program groups. The unweighted r² between the DI and SSI adult series is 0.66. All of the program groups show a positive correlation with each other as well. The strongest correlation is between the SSI adult and concurrent adult averages by state (r² = 0.94). The unweighted r² between the DI-only adult and SSI child series is 0.67. The positive correlation coefficients suggest that relatively permanent differences among the states affect the four program groups similarly. Chart 5 displays scattergrams of the six possible two-way relationships between the state averages for the four program groups during the 1993–2008 period. Of particular interest is the apparent relationship involving the SSI child program group and the three adult program groups. While the visual impression from the chart suggests a relatively strong association between the SSI-only adult and SSI child series as expected, there is clearly some positive relationship even between the DI-only adult and SSI child series as well—a finding that reinforces the inference based on the estimated r² between the two series discussed earlier. The correlations themselves, of course, do not tell whether the association is due to historical differences in external factors outside of the control of DDS management or to state-to-state differences that are internal to DDSs, but nevertheless the consistent positive association in all six two-way comparisons is remarkable. Disentangling these influences is a worthwhile subject for future research.

Conclusions

Overall, I find that the probability of an initial allowance is clearly affected by demographic characteristics, diagnostic mix, and the local unemployment rate in the expected direction. All of these factors are outside the control of DDS decision makers; therefore, accounting for their role should help in comparing allowance rates across DDS units on a footing more equal than the comparison of raw initial allowance rates.

The empirical results show that—

1. Older adult disability applicants are more likely than younger adult applicants to experience a favorable outcome of the initial disability determination, regardless of SSA program group (DI, SSI, or both). The reverse is true for childhood SSI applicants. Adult applicants with a musculoskeletal primary diagnosis are less likely to experience a favorable initial disability determination outcome than applicants with other diagnoses, regardless of program group. In contrast, adult applicants with intellectual disabilities, neoplasms, and genitourinary diagnoses are relatively more likely to receive a favorable initial determination result. The relationship between primary diagnosis and the likelihood of an initial allowance is generally similar for adult and childhood applicants.
2. An increase in the state unemployment rate tends to be associated with a decrease in the initial allowance rate. This relationship holds for all three adult program groups and childhood SSI applications. The negative relationship between unemployment and initial allowances is particularly strong for "mental" diagnoses for all three adult program groups and for SSI children. The negative relationship is statistically significant, and it is comparable in magnitude for musculoskeletal diagnoses for all three adult program groups. Interestingly, only the DI-only adult coefficient is statistically significantly different from zero for neoplasms, and even that point estimate is relatively low in absolute value. Note that (a) neoplasms are invariably "doctor diagnosed" health conditions and (b) award decisions may be more likely to be made at the "meet the listings" step in the disability determination process, compared with determinations regarding mental and musculoskeletal impairments. Both of these indicate the relatively major role of objective factors for neoplasms, and therefore it is not surprising that we estimate award probabilities for the neoplasms diagnostic group that are relatively less likely to be responsive to business cycle conditions.
3. Estimates based on individual-level data are generally comparable to estimates from similarly specified models based on state-level models. An important advantage of access to data on individual characteristics of applicants is that it provides maximum flexibility for program group analyses and model specification.
4. Using the fixed-effects models substantially contribute to predictability of the variation of state allowance rates. A substantial portion of state-to-state variation in allowance rates is explained by relatively permanent differences among the states and year-to-year changes affecting all states. The state fixed effects reflect differences among the states that are long term, that is, they affect average state differences for the whole 1993–2008 study period. The year fixed effects reflect factors, such as legislative changes, that affect all states during a given year. The models allow for the measurement of variation attributable to these two sources, but do not provide specific reasons for their explanatory power. Time-varying factors accounted for by the models include demographic and diagnostic characteristics of applicants and the state unemployment rate, which provide additional contribution to explained variation. The three groups of variables included in the regression models (state fixed effects, time fixed effects, and time-varying factors) together explain 73–84 percent of total variation. Sixteen to 27 percent of the total variation in state allowance rates (depending on SSA program group) is not explained by fixed effects and time-varying factors explicitly considered in the models.

Various future research directions might be productive. An obvious extension to this study would be to replicate it using data that are more recent. Accounting for the role of state and year fixed effects and time-varying exogenous factors should be helpful in refining projections of initial allowance rates. A logical direction of follow up could extend this analysis to research on factors affecting ultimate allowance rates and lifetime benefit streams. Of particular interest with respect to lifetime benefit streams is the question of whether the exit rates of marginally qualifying applicants who were awarded benefits during periods of high unemployment are higher than the exit rates of others who were first awarded benefits during more favorable labor market conditions. Another potentially fruitful area of future research might focus on the temporal dynamics (lag structure) of the relationship between unemployment shocks and disability allowance rates. Yet another potentially promising area might involve studies designed to analyze various mechanisms that may be responsible for the relationship between the unemployment rate and initial determination results. Some may be external to the disability determination system; others may involve programmatic responses to external shocks, such as sudden increases in applications arising from worsening labor market conditions or other factors.

While this study does not focus on possible responses by DDS administrators or caseworkers, and the recent work of Maestas, Mullen, and Strand (2011) did not analyze the effects of local labor market conditions on DDS decision making, a file adding longitudinal data on caseworker characteristics to this analysis file might facilitate future work on the relationship between the business cycle and DDS decision making.