Results of the Office of Policy's 2001 User Satisfaction Survey

by
Social Security Bulletin, Vol. 64, No. 4, 2001/2002

Results of a 2001 Gallup poll indicate that the majority of users of the Social Security Administration's (SSA's) research, statistical, and policy products are satisfied with them and with the agency's performance in identifying and working on new and emerging areas of research and policy. Satisfaction varies with professional interests, length of time working with Social Security and Supplemental Security Income issues, work affiliation, and frequency of use of SSA's products. The report of the survey is available at www.socialsecurity.gov/policy.


The author is with the Division of Economic Research, Office of Research, Evaluation, and Statistics, Office of Policy, Social Security Administration.

Acknowledgments: The author wishes to acknowledge the contributions of the members of the User Satisfaction Survey Team: Susan Grad, Patrice Cole, Weltha Logan, Patricia Martin, Sonja Nesbit, and John Woods. Special thanks to John Woods for the many helpful suggestions and comments regarding this article.

The findings and conclusions presented in the Bulletin are those of the authors and do not necessarily represent the views of the Social Security Administration.

Summary

In the summer of 2001, individuals who had an active professional interest in research or policy issues related to Social Security or to Supplemental Security Income (SSI) were surveyed by the Gallup Organization, under contract to the Social Security Administration (SSA). The survey had two goals. One was to determine the extent to which SSA's research, statistical, and policy analysis work was focusing on topics and issues of widespread concern to users. The other was to gauge user satisfaction with the products of that work. Most of SSA's research and analysis is carried out by the Office of Policy.

Responses were obtained from 1,043 persons out of a sample of 1,800. Respondents were most likely to have been interested in Social Security or SSI issues for at least 10 years, to have used information from SSA multiple times during the preceding 2 years, and to have worked for the government, a college or university, or a nonprofit service organization.

Results suggest that most users were satisfied with the quality of SSA's research, statistical, and policy analysis information products.

Respondents were asked to characterize their professional interests in two ways--as related primarily to Social Security, SSI, or both programs, and as related primarily to retirement issues, disability issues, or issues in both areas. Satisfaction varied, sometimes substantially, with professional interests. On all measures, Social Security specialists were more satisfied than SSI specialists, and retirement specialists were more satisfied than disability specialists. Note that of respondents interested primarily in the SSI program, 93 percent were also more interested in disability issues.

The survey asked users to recommend ways of improving SSA's research, statistical, and policy analysis, and to recommend issues for SSA's research and policy agenda in the near future. Only 39 percent of users offered recommendations for improvement, and of those recommendations, no single issue predominated. In general, the most common recommendations were for increased data analysis in particular areas and for improved dissemination of information.

A large proportion of respondents (77 percent) suggested issues that SSA should include in its research and policy agenda. Although the recommendations were quite diverse, two types appeared to dominate--disability issues and programs, and Social Security solvency and reform proposals. Some of the suggestions for disability research and policy work seem to suggest that the lower satisfaction ratings of specialists in this area reflect generalized dissatisfaction with the structure and administration of disability programs themselves rather than dissatisfaction with the quality of SSA's research and policy products.

The Office of Policy is developing specific proposals for improving products and services, based on the survey's findings. By improving the quality and content of its statistical and analytical information, SSA will continue its efforts to meet the needs of a diverse research and policy community.

Introduction

An initial analysis of the data from the 2001 User Satisfaction Survey was provided by the Gallup Organization in December of that year and is posted on the Office of Policy's Web site at www.socialsecurity.gov/policy. The analysis consists primarily of percentage distributions of responses to each item in the questionnaire, with only limited attention given to relationships between variables. In general, the survey found relatively high levels of satisfaction, although satisfaction varied somewhat for different measures.

This article explores those and other survey findings in greater depth--in particular, how selected characteristics and professional interests of respondents may be related to their degree of satisfaction and their recommendations for improvements in the Social Security Administration's (SSA's) research and analysis efforts.

Survey Design

The target population for the survey--users and potential users of SSA's research, statistical, or policy information--cannot be easily identified or listed for sampling. The population would include individual subscribers to relevant SSA publications, but it would also include nonsubscribers who are active in Social Security-related policy and research and who may gain access to SSA's information in other ways--for example, in libraries, on the Internet, by contacting SSA staff, or through SSA briefings and presentations. To select a sample, the Office of Policy and Gallup developed four unduplicated lists: individual subscribers to a cross-section of publications, nonsubscribers compiled from lists of professional associations and attendees at various research and policy-related conferences, individuals on an agency stakeholders list who had expressed interest in receiving statistical publications, and a small group of decision makers--individuals in high-level positions at SSA and related federal agencies.1

A sample of 1,800 persons was selected from the four lists, and a questionnaire was mailed to them in July 2001. Respondents were given the option of completing a Web-based questionnaire, and telephone interviews were attempted for those who had not responded by mail or Internet within several weeks. Responses were obtained from 1,043 individuals (for a response rate of 58 percent). Almost all of the respondents (88 percent) were identified as active--that is, as having had a professional interest in research, statistics, or policy issues related to Social Security or to Supplemental Security Income (SSI) in the past 24 months. And almost all of the active respondents (92 percent) were identified as users of SSA's information--that is, as having received any research, statistical, or policy information from SSA publications, Web sites, or staff in the past 24 months. The final group of 840 users included 45 decisionmakers, 462 subscribers, 185 nonsubscribers, 145 stakeholders, and 3 respondents whose status was unknown.2

The questionnaire was designed to be relatively brief, to minimize the burden on respondents and encourage a high response rate. Testing indicated that most respondents were able to complete it in about 10 minutes. Keeping the questionnaire short limited the number and complexity of questions that could be included. The core content of the questionnaire was divided into two parts. The first included a series of questions about topics and issues of concern to the 913 respondents who were classified as active in the area of Social Security or SSI policy and research. The second part focused on the subset of 840 users of SSA's research, statistical, or policy information in the 2 years preceding the survey. This group was asked about the types of products they had used and about their satisfaction with various aspects of those products. A final question asked about their satisfaction with the overall quality of the information. In addition to the core content, the questionnaire included a few questions about the respondents themselves and an open-ended question eliciting their recommendations for improving SSA's research, statistical, or policy products and services.3

Characteristics of Respondents

Although the survey included only a few questions about respondents' characteristics, some interesting patterns emerged. Over 70 percent of respondents were employed in one of four categories: the executive branch of the federal government, colleges or universities, nonprofit service organizations, or state or local governments (see Chart 1). About one-half of the federal workers (11 percent of respondents) were employees of SSA. Roughly three-fourths of respondents had been interested in Social Security or SSI issues for at least 10 years, and almost half reported such interests for more than 20 years (see Chart 2).

Chart 1.
Respondents' work affiliation
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 1. Respondents' work affiliation (in percent)
Affiliation Percent
Federal government 23.3
University or college 20.9
Nonprofit service organization 13.6
State or local government 13.3
Private research or policy organization 7.6
No affiliation 5.2
Business or industry 4.3
Advocacy or lobbying group 4.0
Professional or trade association 3.7
Congressional agency 2.9
Other 1.4
NOTE: Total does not equal the sum of rounded components.
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.
Chart 2.
Length of time interested in Social Security or Supplemental Security Income issues
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 2. Length of time interested in Social Security or Supplemental Security Income issues (in percent)
Years Percent
Less than 5 8.6
5 to 9 16.0
10 to 20 30.0
More than 20 45.5
NOTE: Total does not equal the sum of rounded components.
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.

Those who had used information from SSA typically reported multiple uses during the preceding 2 years. Thirty-seven percent said they had sought or received research, statistical, or policy information three to nine times; almost half had done so even more frequently (see Chart 3). Users received the information in a variety of ways, the most common being published or hard copy materials, SSA's web sites, and individual assistance from SSA staff (see Chart 4). The majority used information in the form of both statistical tables and analytical reports; less than one-fifth used one or the other exclusively.

Chart 3.
Frequency of use of information from the Social Security Administration in the past 24 months
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 3. Frequency of use of information from the Social Security Administration in the past 24 months
Frequency Percent
Less than 3 times 16.0
3 to 9 times 37.2
10 to 19 times 21.1
20 to 50 times 15.9
More than 50 times 9.9
NOTE: Total does not equal the sum of rounded components.
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.
Chart 4.
Means of receiving information from the Social Security Administration
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 4. Means of receiving information from the Social Security Administration
Means Percent
Published or hard copy material 89.3
Web sites 66.7
Telephone or fax from staff 43.8
Briefings, presentation by staff 36.9
"E-News" newsletter 25.3
Other 6.5
 
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.

Professional and Programmatic Interests

Because users and potential users of SSA's research and policy work could be expected to have a range of professional interests, respondents were asked to characterize their interests in two ways--first, as related primarily to Social Security, SSI, or both programs, and second, as related primarily to retirement issues, disability issues, or issues in both areas. On the basis of their answers, the respondents were divided into program specialists (the 29 percent more interested in Social Security and the 17 percent more interested in SSI) and program generalists (the 55 percent interested in both programs). Respondents were divided similarly along areas of interest: specialists were the 25 percent more interested in retirement issues and the 36 percent more interested in disability issues; generalists were the 39 percent interested in both areas.

Although the majority of respondents were interested in both programs and in both areas, dividing them into specialists and generalists provided some useful insights. The overwhelming majority of respondents interested primarily in the SSI program were concerned particularly with disability issues (see Table 1). In contrast, 60 percent of respondents interested primarily in Social Security were concerned particularly with retirement issues. Of respondents interested in both Social Security and SSI, over half were interested in both retirement and disability issues.

Table 1. Distribution of respondents' professional interests, by program and area (in percent)
Area Specialist Generalist
(Social
Security
and SSI)
(N=494)
Social
Security
(N=262)
SSI
(N=150)
Specialist
Retirement issues (N=227) 60 1 13
Disability issues (N=326) 8 93 35
Generalist
Retirement and disability issues (N=353) 32 6 52
Total (N=906) 100 100 100
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.

Work Affiliation

Specialist and generalist groups varied somewhat by work affiliation. As indicated in Chart 5, the largest group of generalists worked in the executive branch of the federal government. The largest group of Social Security or retirement specialists were employed by universities, and the largest group of SSI specialists were employed by state or local governments. Disability specialists appeared to be more evenly divided among the federal government, state and local governments, nonprofit service organizations, and universities.

Chart 5.
Work affiliation, by professional interest
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 5. Work affiliation, by professional interest (in percent)
Work affiliation Total Social
Security
SSI Social
Security
and SSI
Retirement Disability Retirement
and
disability
Federal government, Executive branch 23 18 17 28 13 21 32
University or college 21 29 22 16 35 16 16
Nonprofit service organization 14 8 17 15 8 22 9
State or local government 13 2 31 14 4 23 11
Private research or policy organization 8 13 4 6 14 3 8
 
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: SSI = Supplemental Security Income.

Interest in Particular Topics and Specific Populations

The survey also measured interest in specific topics. Overall, respondents showed the greatest interest in current programs or proposals for change, the economic well-being of the aged and disabled, work-related issues, and the economic impact of the Social Security or SSI program.

Interest in some of the topics differed substantially by program and area, however. Consistent with earlier findings, the topics of greatest interest to SSI specialists tended to be similar to those of greatest interest to disability specialists, and the topics that most interested Social Security specialists tended also to interest retirement specialists. The four topics with the greatest spread in interest are shown in Table 2. Note, for example, that whereas Social Security financing ranks only fifth in interest overall (at 63 percent), it is a topic of great concern to Social Security and retirement specialists (at 79 percent and 80 percent, respectively).4 Specialists in Social Security or retirement issues showed much greater interest in private sources of income security and in other countries' income security systems than did SSI or disability specialists. Generalists, whether interested in both Social Security and SSI or in both retirement and disability, showed more interest in other government income security programs in the United States than the total sample did; SSI specialists showed less interest (23 percentage points less).

Table 2. Interest in selected topics and populations, by program and area (in percent)
Interest All
respondents
Program Area
Social
Security
SSI Social
Security
and SSI
Retirement Disability Retirement
and
disability
Interest in topics
Social Security financing a 63 79 20 67 80 38 75
Other government income security programs in U.S. a 52 48 29 61 53 43 60
Private sources of income security a 47 67 9 48 70 19 58
Income security systems in other countries a 41 56 14 42 57 24 48
Current programs or proposals for change 92 92 87 94 94 91 94
Economic impact of Social Security/SSI 71 72 60 73 73 64 76
Work-related issues 78 74 70 83 73 82 78
Economic well-being of the aged or disabled 82 80 77 85 82 82 83
Other SSA-related topics b 4 2 4 5 1 5 6
Interest in populations
Disabled adults 64 39 63 76 19 81 75
Women 56 67 26 60 71 35 68
Low-wage workers 55 55 33 63 60 41 67
Racial or ethnic minorities 55 49 43 61 52 46 66
Disabled children 54 19 87 60 11 80 54
Older workers 48 67 12 50 69 18 65
Early retirees 39 63 4 39 62 10 55
The oldest old (aged 85 or older) 36 45 12 40 51 13 52
Immigrants, noncitizens 36 24 30 43 24 28 52
Widowed, divorced, or never married persons 34 42 5 39 47 12 49
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: SSI = Supplemental Security Income.
a. Topics with greatest spread in interest.
b. Includes Medicare and Medicaid; health-related issues; other specific program provisions; other disability-related issues; issues and programs related to children; benefits and equity or group issues; demographic, economic, or social issues; and other interests.

The questionnaire then measured respondents' interest in selected populations. Overall, respondents showed the most interest in disabled adults, women, low-wage workers, racial or ethnic minorities, and disabled children, as indicated in Table 2. Again, there were some wide differences between specialists and generalists. Social Security or retirement specialists exhibited considerable interest in women, older workers, and early retirees, whereas SSI or disability specialists exhibited far less. As would be expected, retirement and disability generalists exhibited a greater interest in most topics than the total sample did.

Satisfaction with SSA's Research and Policy Agenda and Suggested Issues for the Future

For the satisfaction questions in the survey, five responses were provided: very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, somewhat dissatisfied, and very dissatisfied. Overall, 62 percent of respondents were very or somewhat satisfied with SSA's performance in identifying and working on new and emerging research and policy issues over the preceding 2 years. Satisfaction varied by specialty area, with 76 percent of retirement specialists, 67 percent of Social Security specialists, and 52 percent of disability or SSI specialists indicating satisfaction (see Table 3). Of the five largest employment categories (Chart 1), only respondents affiliated with a private research or policy organization exhibited greater satisfaction (74 percent) than the total sample; none of the other four groups exhibited notably less. In general, those who used information from SSA at least 20 times in the preceding 2 years were more satisfied than those who used information less than 20 times. Satisfaction with SSA's performance on new and emerging issues did not vary much by length of interest in issues relevant to Social Security or SSI.

Table 3. Satisfaction with the Social Security Administration's work on new research and policy issues, by professional interest, work affiliation, and frequency of use (in percent)
Characteristics Very
satisfied
Somewhat
satisfied
Neither
satisfied nor
dissatisfied
Somewhat
dissatisfied
Very
dissatisfied
All respondents 18 44 24 11 4
Professional interest
Social Security 26 41 23 7 2
Supplemental Security Income 12 40 31 11 7
Social Security and SSI 15 47 22 13 3
Retirement 26 50 17 6 2
Disability 9 43 27 16 5
Retirement and disability 21 42 25 9 3
Work affiliation
Federal government 18 41 24 12 6
University or college 16 49 20 13 2
Nonprofit service organization 14 44 24 13 5
State or local government 14 43 30 12 1
Private research or policy organization 27 47 17 3 5
Other a 20 44 26 8 2
Frequency of use
Less than 3 times 15 45 26 10 4
3 to 9 times 17 43 27 11 3
10 to 19 times 18 49 23 8 3
20 to 50 times 23 49 13 12 3
More than 50 times 22 40 17 14 8
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
a. Includes Congressional agency, business or industry, professional or trade association, advocacy or lobbying group, the press or other media, no work affiliation, international organization, and other.

After exploring respondents' interests and their satisfaction with SSA's recent research and policy agenda, the survey asked them to list the most important research or policy issues they thought SSA should be working on in the next year or two. Twenty-three percent did not respond to this open-ended question; those who did cited diverse and wide-ranging issues (see Table 4). About one-fourth of respondents identified program solvency and Social Security reform issues as the most important research or policy areas for the near term. Other priorities were disability, income security, and work-related issues, with the latter two often linked to concerns for the disabled.

Table 4. Recommended research and policy issues for the near term
Recommendation Percent
Program financing, solvency; Social Security reform, privatization 27
None or no response 23
Disability, disability program a 17
Income security, economic well-being b 13
Employment, work incentives c 12
Other issues regarding SSA's programs and policies d 10
Maintaining or strengthening programs e 6
The aged, retirement f 6
Demographics g 5
Health, health care h 4
Other i 12
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTES: Multiple responses allowed; total is greater than 100 percent.
SSA = Social Security Administration.
a. Disability definition, determination; disabled children; other disability issues.
b. Includes non-SSA sources of income security; concerns about economically vulnerable groups; distributional issues.
c. Employment, training, work incentives, disability and work, earnings issues.
d. Issues regarding specific programs, policies, provisions, operations, outcomes.
e. Maintaining, strengthening programs; educating the public about the value of programs.
f. The decision to retire, retirement age, retirement earnings test.
g. Life expectancy, marital status, immigration, baby boomers.
h. Medical insurance, long-term care, mental health.
i. Data file issues, economic issues, services for the aged and disabled.

Response rates and suggestions varied somewhat, largely according to satisfaction with SSA's performance in addressing new and emerging issues (data not shown), although concerns about disability and Social Security solvency or reform continued to dominate. Among the 114 respondents who were somewhat or very dissatisfied with SSA's performance, only 10 did not answer. The most frequent suggestion from the dissatisfied (54 instances) focused on some aspect of the disability program, including both SSI and Disability Insurance. This was followed by suggestions related to Social Security trust fund solvency and the effects of proposed Social Security reforms such as individual accounts and privatization (25 instances). Specific suggestions regarding the disability program focused most often on the disability determination process, including the definition of disability and the adjudication process, followed by employment of the disabled and incentives or disincentives to work, disabled children, the Ticket to Work program, and financing of disability programs.

Among the 189 respondents who were neither satisfied nor dissatisfied with SSA's performance, 76 percent offered suggestions. Most frequently, they referred either to some aspect of disability programs (52 instances) or to Social Security trust fund solvency and the effects of proposed reforms such as individual accounts and privatization (50 instances). Specific suggestions related to disability focused most often on the determination process and on work and disability.

Over 80 percent of the 141 respondents who were very satisfied with SSA's performance offered suggestions. The most frequent suggestions mentioned Social Security trust fund solvency or the effects of proposed Social Security reforms such as individual accounts and privatization (79 instances). Smaller numbers mentioned disability-related issues (16 instances).

Satisfaction with Particular Aspects of SSA's Products

Users of SSA's research, statistical, or policy information were asked to rate their satisfaction with the agency's statistical tables and analytical articles or reports. Respondents evaluated seven specific aspects of these products: accuracy, clarity, comprehensiveness, objectivity, timeliness, usefulness, and accessibility (see Chart 6). Users of both statistical tables and analytical products were most satisfied with the accuracy of the information and least satisfied with the effort it took to find what they were looking for (accessibility). On all seven aspects, slightly more users were very satisfied with statistical tables than with analytical articles. Total satisfaction (very and somewhat satisfied) was roughly equal between users of statistical tables and users of analytical articles on all seven aspects.

Chart 6.
Satisfaction with seven aspects of statistical tables and analytical articles or reports received from the Social Security Administration in the past 24 months
Statistical tables
Stacked bar chart with tabular version below.
Analytical articles or reports
Stacked bar chart with tabular version below.
Show as table
Table equivalent for Chart 6. Satisfaction with seven aspects of statistical tables and analytical articles or reports received from the Social Security Administration in the past 24 months (in percent)
Aspects Very
satisfied
Somewhat
satisfied
Statistical tables
Accuracy 65.5 23.1
Usefulness 36.8 48.5
Objectivity 61.3 23.7
Clarity 44.5 33.9
Comprehensiveness 37.7 39.8
Timeliness 31.6 42.3
Accessibility 26.0 43.1
Analytical articles or reports
Accuracy 53.0 33.9
Usefulness 38.6 44.5
Objectivity 32.0 49.1
Clarity 49.8 31.1
Comprehensiveness 30.9 46.8
Timeliness 28.9 44.8
Accessibility 24.9 46.5
 
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.

Satisfaction varied somewhat by professional interest. The largest differences overall concerned objectivity and accessibility. Specifically, 75 percent of SSI specialists and 78 percent of disability specialists were very or somewhat satisfied with the objectivity of statistical tables, compared with 88 percent of Social Security specialists and 92 percent of retirement specialists. In addition, 60 percent to 63 percent of disability and SSI specialists were very or somewhat satisfied with the ease of finding information (statistical tables), compared with 80 percent of Social Security or retirement specialists.

Some differences were even more pronounced when the measure was restricted to those who were very satisfied. In rating the accuracy, objectivity, and clarity of information, Social Security and retirement specialists were more likely to be very satisfied with statistical and analytical materials than were SSI and disability specialists:

Differences on the remaining aspects (comprehensiveness, timeliness, usefulness, and accessibility) were not as large, although in all cases more Social Security or retirement specialists were very satisfied than SSI or disability specialists. In order to keep the survey questionnaire brief, the Office of Policy did not ask any follow-up questions; therefore, the survey provides few clues as to what respondents found unsatisfactory about particular aspects of quality. However, in an open-ended question that appeared later in the survey, several respondents asked for more nontechnical or nonstatistical discussions of issues and for the use of more bold print and bullets to improve the clarity of information from SSA.

Satisfaction with Specific Publications

Respondents were asked to indicate their overall satisfaction with 10 selected Office of Policy publications they might have used in the preceding 2 years. In contrast to the other satisfaction questions in the survey, Gallup did not have a screening question to determine this question's applicability to each individual; instead, a "did not use" category was added at the end of the satisfaction response categories for each of the publications. This design feature, intended to keep the questionnaire shorter and less complex, may have inadvertently introduced some measurement error.

The possibility of measurement error is suggested by the data in Table 5. On the one hand, the "did not use" option appears to have worked as intended for a large number of respondents: that is, the more specialized publications (such as the statistical compilation OASDI Beneficiaries by State and County) were used least often, and the publications with broader interest (such as the Social Security Bulletin) were used most often. However, some respondents who had not used a particular publication may have simply skipped that rating, and still others who were uncertain about their usage may have taken what they felt was a neutral position. In fact, the rate of "missing" responses was much higher on this question than on the survey's other satisfaction questions, and the proportion choosing the middle response on the satisfaction scale ("neither satisfied nor dissatisfied") was also higher. Perhaps more important, the incidence of middle responses for particular publications was roughly correlated with "did not use" responses--that is, the less frequently used a publication was, the more likely it was to have missing responses and ambiguous ratings on the satisfaction scale.

Table 5. Satisfaction with specific Office of Policy publications (in percent)
Measure Social
Security
Bulletin
Annual
Statistical
Supplement
Fast Facts
and Figures
About
Social
Security
Social
Security
Programs
in the U.S.
Social
Security
Programs
Throughout
the World
SSI
Annual
Statistical
Report
SSI
Recipients
by State
and County
OASDI
Beneficiaries
by State
and County
Income
of the
Population
55 or Older
Income
of the
Aged
Chartbook
Rate of response
Did not use publications 15 22 25 38 49 36 43 50 42 45
Rating missing 7 8 9 10 11 11 11 13 11 11
Gave satisfaction rating 78 70 66 52 40 53 45 38 48 44
All responses
Very or somewhat satisfied 83 83 81 72 64 75 70 66 71 74
Neither satisfied nor dissatisfied 15 15 16 25 30 22 25 29 25 22
Very or somewhat dissatisfied 2 2 3 3 6 3 5 5 4 4
Responses excluding "neither satisfied nor dissatisfied"
Very or somewhat satisfied 98 97 97 96 92 96 93 92 95 95
Very or somewhat dissatisfied 2 3 3 4 8 4 7 8 5 5
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: SSI = Supplemental Security Income; OASDI = Old-Age, Survivors, and Disability Insurance.

In light of these concerns, caution should be used in interpreting the satisfaction ratings for publications. An initial analysis suggests fairly wide differences among them, ranging from a high of 83 percent who were very or somewhat satisfied with the Social Security Bulletin and the Annual Statistical Supplement to a low of 64 percent who were very or somewhat satisfied with Social Security Programs Throughout the World (Table 5). However, if calculations are restricted to respondents who took a clear position (excluding the "neither satisfied nor dissatisfied" category), the differences between publications narrow considerably, ranging from 98 percent who were very or somewhat satisfied with the Social Security Bulletin to 92 percent for two of the more specialized publications (see lower panel, Table 5). With either approach, the proportion expressing clear dissatisfaction with the 10 publications was relatively small, ranging from 2 percent to 8 percent who were somewhat or very dissatisfied.

An effort was made to explore the reasons for variations in satisfaction by examining respondents' ratings of specific aspects of analytical and statistical products--the possibility, for example, that a respondent's lower satisfaction with a particular publication might be related to his or her lower satisfaction with timeliness or some other attribute. However, almost all respondents reported use of multiple publications, making it impossible to establish a link between a given publication and evaluations or recommendations made elsewhere in the survey.

Satisfaction with the Overall Quality of Information from SSA

Altogether, 86 percent of users indicated that they were very or somewhat satisfied with the overall quality of SSA's research, statistical, or policy information, 10 percent were neither satisfied nor dissatisfied, and 4 percent were very or somewhat dissatisfied. Ratings varied somewhat by professional interest, length of time interested in Social Security or SSI issues, frequency of use of information from SSA, work affiliation, and satisfaction with SSA's work on new and emerging issues. Overall quality ratings did not vary much among those who used only statistical tables, only analytical articles or reports, or both.

Differences by specialty were not great on the measure of total satisfaction (very satisfied and somewhat satisfied), but they were fairly pronounced on the very satisfied measure (Chart 7), with more retirement and Social Security specialists than disability or SSI specialists reporting great satisfaction. The two generalist categories were intermediate in their ratings of very satisfied. Dissatisfaction was quite low for all groups of respondents.

Chart 7.
Satisfaction with overall quality, by program and area
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 7. Satisfaction with overall quality, by program and area (in percent)
Program and area Very
satisfied
Somewhat
satisfied
Neither
satisfied
nor
dissatisfied
Somewhat or
very
dissatisfied
All respondents 37 49 10 4
Program
Social Security 49 40 10 2
SSI 27 56 13 5
Social Security and SSI 34 51 10 5
Area
Retirement 53 40 7 1
Disability 23 59 13 6
Retirement and disability 41 46 10 3
NOTE: Totals do not necessarily equal the sum of rounded components.
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: SSI = Supplemental Security Income.

Total satisfaction was high and tended to rise with years of interest in Social Security or SSI issues (see Table 6). In fact, users with more than 20 years' interest accounted for the largest percentage of people who were very satisfied with the overall quality of SSA's information. Again, dissatisfaction was quite low for all groups.

Table 6. Overall satisfaction with quality of information from the Social Security Administration (in percent)
Measure Very
satisfied
Somewhat
satisfied
Neither
satisfied nor
dissatisfied
Somewhat
dissatisfied
Very
dissatisfied
All respondents 37 49 10 4 a
By length of time interested in Social Security or SSI issues
Less than 5 years 34 46 15 3 2
5 to 9 years 33 50 13 5 0
10 to 20 years 36 52 9 3 0
More than 20 years 41 46 9 4 a
By frequency of use
Less than 3 times 39 41 16 3 0
3 to 9 times 35 51 10 4 a
10 to 19 times 35 54 9 2 0
20 to 50 times 41 52 7 1 0
More than 50 times 43 39 7 10 1
By work affiliation
Federal government 35 46 14 4 1
State and local government 32 54 7 7 0
University or college 39 49 8 4 0
Private research organization 48 42 8 2 0
Nonprofit service organization 22 59 15 4 0
By satisfaction with SSA's work on new issues
Very satisfied with work 90 10 0 0 0
Somewhat satisfied with work 35 63 3 0 0
Neither satisfied nor dissatisfied with work 18 56 23 3 0
Very or somewhat dissatisfied with work 8 46 23 21 2
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: SSI = Supplemental Security Income; SSA = Social Security Administration.
a. Less than 0.5 percent.

Although satisfaction varied somewhat with frequency of use of SSA's products, it was difficult to discern a consistent pattern (Table 6). Respondents who had used information more than 50 times in the past 24 months accounted for both the greatest proportion who were very satisfied and the greatest proportion who were somewhat or very dissatisfied. As a result, total satisfaction among the most frequent users is closer to satisfaction among the least frequent users than it is to the total satisfaction of intermediate users.

Over 80 percent of users in the five largest employment categories--private research or policy organizations, university and college employees, state and local employees, nonprofit service organizations, and the executive branch of the federal government--were very or somewhat satisfied with the quality of SSA's information (Table 6). Employees of SSA expressed roughly the same levels of satisfaction as all users. Private research organizations had the highest percentage of users who were very satisfied, and nonprofit service organizations had the lowest.

Finally, there appears to be a correlation between satisfaction with SSA's recent work on new and emerging issues and satisfaction with the overall quality of SSA's information (Table 6). The correspondence is highest among those who expressed greatest satisfaction with SSA's work on new issues and those who were most satisfied with overall quality. A much smaller proportion was dissatisfied on both measures. More interesting, however, is the finding that a majority of those dissatisfied with SSA's work on new issues were satisfied with the overall quality of SSA's information.

Recommended Improvements

The survey's final question was open-ended, seeking recommendations for improving SSA's research, statistical, or policy products and services. Overall, 61 percent of users did not respond or indicated that they had no recommendations (see Table 7). The suggestions that were offered were wide-ranging and often quite specific. Grouped into general categories, the greatest number of users suggested expanding the content of information, improving the dissemination of information, or improving the quality of information. Some differences can be seen between the recommendations of the somewhat or very dissatisfied and other users, but the sample size of the dissatisfied is too small to reach any conclusions.

Table 7. Recommendations for improving research, statistical, or policy information, by satisfaction with quality (in percent)
Recommendation All users
(N=838)
Very
satisfied
(N=305)
Somewhat
satisfied
(N=401)
Neither
satisfied nor
dissatisfied (N=83)
Very or
somewhat
dissatisfied (N=32)
None or no response 61 69 59 59 25
Expand content of information a 16 14 17 16 22
Improve dissemination of information b 14 13 14 14 28
Improve quality of information c 10 7 11 14 16
Increase resources d 3 3 3 2 13
Other 8 7 5 10 24
SOURCE: Author's tabulations using data from the 2001 User Satisfaction Survey conducted by the Gallup Organization.
NOTE: Multiple responses allowed; total is greater than 100 percent.
a. Provide additional data or analysis in particular areas, including some priority issues for near term (Table 4), more data by geographic area or Congressional district, more international data.
b. Improve Web site; publicize what's available and how to get it; make data files more accessible, easier to use; other dissemination issues.
c. Make clearer, more concise, easier to use; more timely, frequent; more objective; other quality issues.
d. More or better qualified Social Security Administration staff; more support for external research.

A few users repeated the concerns they had expressed earlier in the survey about increasing access to SSA's administrative data files. Such data files are maintained for internal, administrative purposes, and the law restricts access to them by outside researchers and analysts in the interests of maintaining citizens' confidentiality. Assuming appropriate safeguards could be implemented, users particularly wanted access to public-use data files in four areas mentioned in the questionnaire: SSA data linked to survey data (73 percent), characteristics of Social Security beneficiaries (68 percent), characteristics of SSI recipients (62 percent), and the earnings of covered workers (50 percent). Differences in interest by specialty were not surprising; for example, SSI specialists were not particularly interested in the characteristics of Social Security beneficiaries, and Social Security or retirement specialists were not as interested in the characteristics of SSI recipients.

Potential Users of Information from SSA

The survey defined potential users of research, statistical, or policy information as respondents who, in the preceding 2 years, had had a professional interest in research, statistics, or policy issues related to Social Security or SSI but who had not received any research, statistical, or policy information from SSA publications, SSA Web sites, or SSA staff. In total, 72 respondents (8 percent of the sample) were identified as potential users. Although the sample size was small, the following characteristics of potential users emerged:

The survey also asked these respondents why they had not used SSA's information in the past 24 months. Of those who gave a reason, the most common was that the information was "not related to my needs," as indicated in Chart 8. A similar proportion did not know the information was available or had not received it. About one-half of all potential users gave no particular reason (or did not respond to the question) for not using SSA's information.

Chart 8.
Potential users' reasons for not using information from the Social Security Administration
Bar chart with tabular version below.
Show as table
Table equivalent for Chart 8. Potential users' reasons for not using information from the Social Security Administration
Reasons Percent
No response 41.1
Not related to my needs 17.8
No particular reason 13.7
Lack of knowledge as to its availability 8.2
Have not been receiving this information 6.8
Other 15.1
 
SOURCE: Adapted from the Gallup Organization report on the 2001 User Satisfaction Survey.
NOTE: Multiple responses allowed.

Notes

1. Excluded from these lists were employees of SSA's Office of Policy, persons living outside the United States, and persons who had participated in the pretest of the questionnaire. Members of Congress and their staffs were also excluded from the sample. In the early stages of planning the survey, it was learned that interviews and focus group sessions with key Congressional staff were being conducted by SSA's Office of Communication during 2000-2001 in an effort to evaluate SSA's services to Congress. To avoid duplication of effort and undue respondent burden, Congressional participation was not requested for this survey.

2. A more comprehensive presentation on the population, sample, and data collection is provided in Gallup's final report, at www.socialsecurity.gov/policy. The Gallup report found relatively small differences among the four sample groups on measures of satisfaction and recommendations for improvements; thus, it was deemed valid to treat their responses as representative of a single, combined sample. At the same time, because of the limitations on specifying the population and sampling for this survey, statistical inferences are not made for the theoretical population of all users and potential users.

3. A copy of the questionnaire is attached as Appendix B to Gallup's final report, at www.socialsecurity.gov/policy.

4. SSI is financed out of general revenues; Social Security has a trust fund financed by a dedicated payroll tax.