Due to the increasing need for up-to-date data in a fast-paced world, the U.S. Census Bureau created the American Community Survey (ACS) in the early 2000s to replace the “long-form” Census survey. Like the long-form once did, the ACS gathers information about housing, education, income, occupations, and much more from a sample of the U.S. population. However, the ACS is a rolling survey, conducted on a continual, monthly basis. This enables data to be published every year, rather than every 10 years.
It is important to note that the last census to use the long-form survey, the 2000 Census, had a sample size of 1 in 6 households, while the 2010 and 2020 censuses, which relied on the ACS, used samples that were only about 1 in 11 households.
It is no surprise, therefore, that the Census Bureau’s ACS guidebooks recommend “extreme caution” when using any estimate where the margin of error is 10% or more of the estimate. Unfortunately, in small geographies like villages, towns, census tracts, and census blocks this is frequently the case. It is concerning, therefore, that some educators and professionals don’t include the margin of error in their presentations, plans, and reports. Even when they do, such numbers may be unheeded by most audiences, who are not familiar with the significance of error estimates.
ACS data looks just like the long-form data we used to rely on, but “significantly smaller sample sizes yield less precise measures of common variables than Census 2000 sample data,” reports an ESRI white paper. To test this in the field, Kevin recently compared several 2017 5-year ACS datasets for three small villages in Sheboygan County (Adell, Cascade, and Waldo) with “real world” data from the Sheboygan County Geographic Information System. In each case, the ACS data varied substantially from the County GIS data. Below is one example.
Since all of the margin of errors in this table are 10% or more of the ACS estimates, by the Census Bureau’s own guideline the ACS housing unit data for all three villages falls into the “extreme caution” category. In the case of Cascade, for instance, the ACS is saying the total housing units could be as high as 342 or as low as 266. The actual estimate of 304 would indicate that the village grew by a robust 12% since 2000 when it had 272 units. In reality the village only grew by a modest 5%.
Local officials, as well as businesses and land developers, benefit from learning about their communities, and accurate data helps them plan for the future. Although “short form” census data for items like population, age, and gender, which are still based on responses from all U.S. households, are still considered to be reasonably accurate, it appears that other datasets previously collected via the long form but now derived from ACS sampling are often unreliable for villages, towns, census tracts, and census blocks.
