The U.S. Census Bureau should be paying more attention to the needs and opinions of the people and organizations that use its data, according to a recent National Academy of Sciences report.
The report, which focused on the American Community Survey, recommended that the bureau sharpen its sometimes confusing advice on evaluating data quality, consider trimming the number of tables it produces, and make it easier for users – both inexperienced and more advanced – to access statistics they need.
The American Community Survey is “an invaluable resource,” but the bureau should do more to make sure the survey meets researchers’ needs, including establishing a formal advisory group of users, according to the panel of data scientists who wrote the report at the Census Bureau’s request. The National Academy of Sciences is a private nonprofit organization, established by Congress, that provides independent advice on science and technology.
The Census Bureau is still studying the National Academy’s recommendations and has not yet decided which ones to pursue, agency spokeswoman Shelly Hedrick said last week in an email.
The review was meant to help the bureau, which is in the midst of its own examination of the American Community Survey. The bureau’s largest household survey – which is based on data from more than 2 million households – provides national and local estimates on education, immigration, housing and other topics that are widely used by government agencies, nonprofit organizations, academic researchers and businesses. The survey also plays a role in allocating $400 billion a year in government funding and in enforcing civil rights laws, among other things.
But people who use this census data say the bureau’s website leaves much to be desired, the report said. For example, inexperienced users “have difficulty navigating the many options,” and more sophisticated users “feel constrained by the limited flexibility and features” that are offered.
Meanwhile, the academy panel said, the job of producing more than 11 billion estimates a year from the survey is stretching the bureau’s capacity. The agency has never evaluated which data products are most useful. The report suggested that the bureau do such an evaluation and consider trimming the number of data tables it publishes.
The Census Bureau took one step in that direction this month by proposing to stop publishing estimates that combine three years of survey data; these data provide a larger sample size that lets researchers analyze data in more detail. The agency will continue to publish five-year estimates for that purpose.
Dropping some data products would free bureau resources for other projects to help data users, the report said. One such project it recommended was development of a more user-friendly data query system so advanced-level researchers could run customized tables and perform high-level statistical analysis.
The report also told the bureau to offer clearer advice on the limits and quality of its data. For example, although the bureau publishes margins of error for its survey estimates, its guidance on how to use these estimates is sometimes “confusing and uninformative,” the academy panel said. The report also questioned the bureau’s practice of not publishing some data it deems too imprecise, such as estimates about a small population group with a large margin of error. It would be better to publish the data and offer users more help in evaluating its quality, the report said. (The bureau also withholds data to protect respondent confidentiality; the report did not challenge that practice.)
The academy’s report was commissioned chiefly to study ways to improve the survey’s patchy data about small geographic areas and small population groups as well as to recommend other improvements. In addition to urging more user-friendliness, the report made two dozen recommendations for improvements in sample design and survey-question selection, as well as in data collection, processing and analysis.
The report comes as the bureau is reviewing public comments about its proposal last year to drop several questions from the survey about marriage and other topics. The bureau is to announce its recommendations by April to the Office of Management and Budget, which makes the final decision. Changes would take place in 2016.
