Information Quality Guidelines

Background

The Office of Management and Budget (OMB) has issued government-wide information quality guidelines under section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001. The OMB section 515 guidelines have been prepared to ensure and maximize the quality, utility, objectivity, and integrity of information disseminated by federal agencies. The guidelines direct each federal agency to issue its own section 515 guidelines. In response to the OMB directive, the guidelines presented here (highlighted in bold) describe procedures the Social Security Administration (SSA) will employ to ensure the quality of its information products, including their utility, objectivity, integrity, transparency, and reproducibility. The accompanying text describes steps that SSA is currently taking that conform to these guidelines.

Introduction

SSA provides information to the public about the current experience of its programs, the projected scope and impact of those programs in the future, and the effect of proposed changes to the Social Security system. Information products describe the impact of SSA programs on our economy, society, and beneficiary populations and provide detailed demographic and economic information on beneficiaries and covered workers. Those products are used by government planners and policymakers as well as by actuaries, economists and other social scientists, the media, and the public to analyze Social Security programs and their impact on the nation.

Utility

Utility involves the usefulness and availability of the information to its intended audience. To produce useful, policy-relevant information, agencies must stay informed of information needs and develop new data, models, and information products where appropriate.

SSA will keep informed of information needs through active and ongoing contact with the user community and will provide vehicles for user input into our information programs.
SSA keeps abreast of information needs with respect to the analysis of SSA programs through a variety of means, including carrying out internal analyses of information requirements, convening and attending conferences, conducting user surveys, working with advisory committees, and sponsoring outreach activities. Contact information is available, where appropriate, on a variety of information products to allow for questions, comments, and suggestions from users.

SSA's analytical and statistical publications and other information products will be reviewed to ensure that they remain relevant and timely and that they address current information needs.
On the basis of internal product reviews and consultation with users, and in response to changing needs and emphases, the content of ongoing information products is changed, new products are introduced, and some products are discontinued. SSA prepares special reports and topical studies that address emerging information needs stemming from proposed changes in the law and related policy debates. SSA identifies requirements for simulation models to support the preparation of analytical reports and policy studies and modifies its current models or develops new models accordingly. When major gaps in data related to SSA populations are identified, SSA may conduct special-purpose surveys to address those needs.

SSA's information dissemination process will make information products widely available and broadly accessible.
Most reports and other data products are available both as printed and electronic documents. They are announced on the SSA Web site, and most electronic versions can be accessed and downloaded directly. All documents posted on our Web site since June 21, 2001, are in compliance with section 508 and are therefore available to an audience that includes persons who have a visual impairment and read online using assistive technology.

Objectivity

Objectivity involves a focus on ensuring that information and its presentation are accurate, reliable, and unbiased and that information products are presented in an accurate, clear, complete, unbiased, and well-documented manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, by having information products prepared by qualified people using proven methods, and by carefully reviewing the content of all information products.

• Use of reliable data sources
  • Information products disseminated by SSA will be based on reliable, accurate data that have been validated.
    Much of the information disseminated by SSA is based on Social Security administrative data files. Those files contain information used to manage SSA programs, including data to determine benefit entitlement, and to compute and pay benefits. SSA conducts ongoing quality assurance reviews of claims and postentitlement information in its data systems to ensure their accuracy. SSA employs an outside contractor to assess the agency's quality control methodology and processes and to validate its review procedures. SSA administrative data are also covered under SSA's financial management systems and conform to their high standards of financial accountability. Those systems are mandated by the Office of Management and Budget and are designed to provide complete, reliable, consistent, timely, and useful management information to enable agencies to carry out their fiduciary responsibilities. Information products disseminated by SSA also employ reliable data from external sources; in particular, administrative data and survey data from other federal agencies as well as survey data from established survey organizations.
  • Surveys sponsored by SSA will be conducted using methodologies that are consistent with generally accepted professional standards for all aspects of survey design and implementation.
    SSA employs and documents accepted professional standards and practices for all major survey activities, including sample frame development, sample design, questionnaire design and testing, data collection, analysis of sampling and coverage errors, nonresponse analysis, imputation of missing data, weighting, and variance estimation. SSA surveys follow guidelines and policies set forth in the Paperwork Reduction Act (PRA) and other regulations related to the conduct of government surveys. PRA submissions for SSA statistical surveys shall state that the survey will be conducted using methodologies that are consistent with generally accepted professional standards for all aspects of survey design and implementation and that survey data quality will conform to SSA section 515 information quality standards.
  • All data employed in the preparation of information products will be compiled using statistically sound procedures implemented by qualified professional staff.
    When analysis requires using samples from administrative data files, SSA employs statistically acceptable methods to design and select the samples. Data samples are designed and compiled by staff knowledgeable about the content, structure, and limitations of the administrative data files employed. In addition, those staff members maintain working relations with agency personnel who create, update, and maintain those files to ensure that their understanding of files is current and complete. When information products require administrative files linked to external data sources, SSA employs sound procedures for extracting and linking data from external sources based on a thorough understanding of the relevant components of the data sources.
• Preparation of statistical estimates
  • All estimation and sampling procedures will be prepared using statistically sound procedures designed by qualified professional staff.
    Samples from administrative files are evaluated to ensure that the samples are representative and subject specialists compare estimates with comparable information from prior years and from other sources to ensure reasonableness and reliability. Computer programs for sampling from administrative files and using data from external data files are reviewed and tested by technical staff. Data files incorporating external data sources are reviewed to ensure that extraction and linkage processes have been implemented correctly.
  • Data sources, sampling errors, and disclosure limitation methods will be documented in publications, either for the publication as a whole or for individual tables.
    Documentation in SSA publications contains information on data sources including definitions and specifications of variables. Report documentation also includes, where appropriate, information on sampling errors and a description of rules or techniques for avoiding disclosure of confidential information.
• Preparation of simulation models
  SSA uses several simulation models to estimate the effects of demographic and economic trends and legislative and policy options on Social Security programs and beneficiary populations now and in the future. Models are based on SSA's and its contractor's best judgments of current and future behavioral relationships and methods of projecting key program outcomes.
  • For all simulation models, implementation procedures will be tested, and the models' performance will be thoroughly evaluated.
    All simulation models are extensively tested and reviewed within SSA to verify that the computer programs that were developed to implement models conform to the stated objectives. Where appropriate, historical simulations are developed to evaluate the success of a model in producing reasonable projections. Where appropriate, based on a model's complexity and scope of application, SSA convenes technical committees to review a model's performance and evaluate whether it meets its objectives. Models are periodically updated to reflect input from internal and external reviews and research findings on behavioral relationships.
  • Documentation will be available for all simulation models.
    Contracts for the development of simulation models provide for detailed documentation that describes the goals and objectives of the model, the data sources being used, and the methodologies and assumptions employed. Documentation for simulation models developed by contractors are available from SSA upon request and will be made available on the Internet. Documentation for simulation models developed within SSA will be available upon request.
• Preparation of analytical reports and policy studies
  • Analytic reports will be prepared by subject specialists who use sound statistical and analytic methods and are knowledgeable about the data sources and models being used.
    Reports are prepared by staff using a variety of analytical techniques ranging from simple tabulations and descriptive summary statistics to multivariate statistical methods and econometric models. Staff preparing analytic reports and policy studies are expert in their use of relevant administrative data files, external data sources, and projections from simulation models.
  • Analytical techniques will be reviewed to ensure that they are appropriate for the data and the analysis to which they are applied and they will be documented.
    All analytical reports and policy studies are reviewed by qualified staff to ensure that the analysis is valid, complete, unbiased, objective, and relevant. Reports and studies that are considered to be more technically complex and are likely to have a greater impact are also reviewed by external technical committees to provide additional perspective and expertise. Analytic techniques are clearly described in reports and data sources are identified in SSA publications. When analyses are based on projections from simulation models, the assumptions used to produce the projections are identified, as well as the rationale for the assumptions used and the impact of using alternative assumptions.
• Editorial review for accuracy and clarity of information in publications
  • All information products will be edited and proofread before release to ensure clarity and coherence of the final report.
    Text is edited to ensure that the report is easy to read and grammatically correct, thoughts and arguments flow logically, and information is worded concisely and lucidly. Tables and charts are edited to ensure that they clearly and accurately illustrate and support points made in the text and include concise but descriptive titles. Tables and charts indicate the unit of measure and the universe being examined, and all internal labels (column headings, row stubs, and panel headings) accurately describe the information they contain. All changes made to a manuscript during the editing process are checked by a proofreader and reviewed and approved by the author.
• Policy for correcting errors
  • A comprehensive errata policy informs users of both printed and Web-based publications when an error has been found and corrected.
    An errata notice is included in all print publications clearly directing users to the Web version of a publication to check for additions or corrections to the previously published data. If an errata has been issued, an errata notice is placed on the first page of the Web version and the corrected version of the data is posted on the Web.

Integrity

Integrity, as used in the OMB quality guidelines, refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. SSA administers social insurance and social welfare programs that touch the lives of almost every American. The agency is highly protective of the confidentiality of information it holds through its policies and practices.

To ensure the integrity of its administrative information, SSA will employ rigorous controls that have been identified as representing sound security practices.
SSA has in place programs and policies for securing its resources as required by the Government Information Security Reform Act (P.L. 106-398, title X, subtitle G). Those security procedures address all major components of information security and apply to all SSA operating components. In addition, SSA is subject to statutory and regulatory requirements to protect the sensitive information it gathers and maintains on individuals. Those requirements are contained principally in the following documents:

• Privacy Act of 1974
• Computer Security Act of 1987
• Office of Management and Budget (OMB) Circulars A-123, A-127, and A-130
• Government Information Security Reform Act
• Federal Managers' Financial Integrity Act (FMFIA) of 1982
• Section 6103 of the Internal Revenue Code
• Section 1106 of the Social Security Act
• SSA's Regulation 1, codified at 20 C.F.R. Part 401
• IRS Tax Information Security Guidelines for Federal, State, and Local Agencies

Transparency and Reproducibility

Information products that are subject to these SSA section 515 information quality guidelines include statistical and actuarial information. The guidelines focus on reports, studies, and summaries prepared for public dissemination to inform the public about the impact of Social Security programs and to use in formulating broad program policy.

For the purpose of these guidelines, transparency refers to a clear description of the methods, data sources, assumptions, outcomes, and related information that will allow a data user to understand how an information product was designed and produced. Reproducibility of information refers to the ability, in principle, for a qualified individual to use the documentation of methods, assumptions, and data sources to achieve comparable findings. In practice, opportunities for direct reproducibility are often limited by restrictions on access to confidential information.

SSA will make the information it disseminates and the methods used to produce this information as transparent as possible so that they can, in principle, be reproducible by qualified individuals. SSA guidelines call for clear documentation of data and methods used in producing estimates and projections. Their implementation will ensure the transparency and reproducibility of our disseminated information. Some statistical publications are based on publicly available data, and the computer programs that produce the statistics may be made available on request; accordingly, information in those publications is fully reproducible by the public. In addition, SSA is planning to release public-use extracts of some of its data files that will increase options for actual reproducibility of estimates and projections while still protecting confidentiality. Statistical surveys sponsored by SSA will be conducted using documented methodologies that are consistent with generally accepted professional standards for all aspects of survey design and implementation.

Many estimates and projections included in SSA's information products are not directly reproducible by the public because the underlying data sets used to produce them are confidential. Others may not be directly reproducible because of the complexity and detail of the methods and data. In those cases, we will employ rigorous checking for robustness, including greater emphasis on periodic review by outside panels of technical experts and on documentation of methods, assumptions, data sources and related information.

Much of SSA's disseminated actuarial, statistical, and analytical information is potentially influential because it has an impact on important public policies or important private-sector decisions relating to the Social Security program. Information products that are deemed to have a greater impact on public policies are subject to more extensive internal review and, where appropriate, review by external technical panels prior to release.