Photo by timothy muza on Unsplash
This post explains the differences between the American Community Survey (ACS) and the Decennial Census and how those differences will impact your work.
Background: Long Form and Short Form
Remember back to the Census 2000 when some people were complaining about the Census form taking forever to fill out? Well, those people received the Census 2000 Long Form which was sent to about 18 million households and collected detailed demographic, economic and housing data. The rest of the US households received the simple Census 2000 Short Form, which was used to count the population and collect basic demographic data. So for the Census 2000, 2 forms were used: the Long Form and the Short Form.
Fast-forward to the Census 2010. The Census 2010 did not use the Long Form, only the Short Form. Enter the ACS. The American Community Survey (ACS) is an annual demographic survey of the United States. It provides the detailed demographic, economic and housing data that was once supplied by the Decennal Census Long Form.
| 2000 | 2010 | |
|---|---|---|
| Count + Basic Data | Short Form | Census 2010 |
| Detailed Data | Long Form | ACS 5 year estimates |
However, the ACS is only sent to 3 million households each year, a statistically small sample especially when compared to the Census 2000 Long Form which was sent to 18 million households in 1 year. So to produce the detailed demographic data, the ACS combines data from several years to produce multi-year estimates. For example, the ACS combines data collected from 2006 + 2007 + 2008 to get 3-year ACS estimates for geographies with at least 20,000 persons. ACS 5 year estimates, which will be out in December 2010, will provide data for geographies with fewer than 20,000 persons.
Similar Data; Different Methodologies & Format
The ACS and the Decennial Census provide the same types of data data–that is, demographic, economic and housing data for the US. However, the methodology of these 2 tools differs greatly.
| ACS | Decennial Census | |
|---|---|---|
| What isIt? |
Continuous demographic survey |
Count of the population |
| Data |
Average characteristics OVER TIME |
Counts characteristics for a SPECIFIC DATE |
| How Often |
Every 1 year (more timely) |
Every 10 years (less timely) |
| House-holds | 11 million households over 5 years |
18 million households in 1 year |
| Sampling Error |
Higher Error (1.75 percent higher than Census 2000*) |
Lower Error |
| Accuracy | Less Accurate | More Accurate |
*[Revised: this information (and much of the blog post) is based on the Census Bureau’s Brian McKenzie’s awesome presentation here]
To sum up the table above, the ACS more timely but less accurate than the Decennial Census, because the data is collected over a series of years (instead of just 1 year), and fewer households are surveyed.
Because the ACS is less accurate, ACS data includes margins or error and is released at 90% confidence limits. Here’s an example of ACS data.
| Austin city | ||||
| Estimate | Error | Percent | Error | |
|---|---|---|---|---|
| Hispanic/Latino | 261,672 | +/-3,404 | 35.0 | +/-0.4 |
This means that we can say with 90% confidence that the number of Hispanic persons in Austin is between 258,268 and 265,076 (or 261,672 plus or minus 3,404). We can also say with 90% confidence that the percentage of Hispanic persons in Austin is between 34.6% and 35.4%.
How Does All of this Impact You?
- ACS demographic data is less reliable than Decennial Census data. It’s going to be more important than ever use ACS data as a starting point only and to support the data fieldwork, interviews, and data from other sources.
- Doing basic calculations with ACS data is going to require an elementary understanding of statistics. For example, calculating percentage change between Census 2000 data and ACS estimates will require basic statistics, because the numbers are not comparable (they are apples to oranges).
If you’re interested, let me know in the comments if a blog post on how to accurately calculate comparisons using Decennial Census data and ACS estimates would be helpful. Or you can check out the Census’ guidance on the topic. Any questions about the differences between ACS and Decennial Census data? Are there still points of confusion?
It is not exactly true that the ACS is less accurate than the 2000 decennial census long form. The ACS has a higher response rate according to Robert Scardamalia (The American Community Survey: general commentary on the findings from external evaluations, Population Research and Policy Review V25 p 293) and the responses received were more complete requiring less imputation. We think of the long form as accurate because the Census Bureau didn’t tell us how inaccurate it was by publishing margins of error. On the other hand, the data for the ACS is collected across 12, 36 or 60 months and so it is hard to talk about it in relation to a specific point in time.
A bigger issue I think is figuring out which estimate to use. When the ACS is in full swing, there will be nine estimates that include a given year for places with a population of over 65,000 people (1 1-year release, 3 3-year releases, 5 5-year releases). What rules should we use in selecting an estimate?
In addition, there are issues around income that haven’t been fully resolved. The long form asked how much a person made in the previous year (e.g., in 1999) while the ACS asked how much a person made in the last 12 months. It has been well documented that people answer these questions differently. For the 3-year and 5-year releases income responses are adjusted to the final year of the compiled responses. How much are these two issues affecting estimates of people in poverty?
Finally, residency rules are different in the ACS and the long form. In the long form residency was pinned to your “usual residence” — the place you spend most of your time. In the ACS it is where you happen to receive the survey be that a summer home, a dorm or your regular old house in New Hampshire. It is not clear how this will impact estimates for places that have a lot of seasonal fluctuation in population. The GAO (American Community Survey: Key Unresolved Issues, Government Accountability Office pub. #GAO-05-82) suggests that the ACS will more accurately represent the population of these places.
Thanks so much for your thoughtful comment. I should have documented my data sources better; I’ve revised the blog post to include a reference.
The fact about the ACS being 1.75% less accurate than the Decennial Census came from Census Bureau’s Brian McKenzie’s awesome presentation here: http://fhwa.na3.acrobat.com/p47445864/ That presentation is about a month+ old, so if there’s more recent data about how the ACS compares to the Census (or if I’ve misunderstood the information), I’d love to know about it. And I agree with you 100% about publishing the Margins of Error will actually increase the perception of the data’s inaccuracy–while it actually paints a more realistic picture!
I’m very interested to see how the issues that you mention above will play out over the next year or so.
Kristen
Thanks for the link to the presentation! Yes, accuracy is difficult to talk about because there are so many different aspects of surveys that relate to how well the estimate reflects the true value. The ACS sample size is smaller so the margin of error is larger but the response rate is better so there is less room for non-response error. In addition, the ACS sample is drawn from the Master Address File that is continuously updated (as opposed to the decennial census where the MAF had to be recreated every 10 years and had more errors) so there is probably less sampling error.