Building Broadband Maps with Census and FCC Data

Photo by Mika Baumeister on Unsplash

The outbreak of Covid-19 has shifted more of the world online. Kids are going to school from home and need internet access for their classes. Staff are working remotely and meeting online. Coincidentally, interest in data about broadband availability and home computers has increased as government agencies, businesses and non-profits are identifying areas that lack fast internet.

There are 2 government datasets that I use to help our clients identify areas lacking broadband internet:

  • US Census American Community Survey
  • FCC’s Fixed Broadband Deployment Data

Many state and local governments use these datasets to evaluate broadband access in communities and institute policies and programs to increase access for areas with less connectivity. Businesses and non-profits can also use these statistics to analyze internet access in the communities that they serve. Below is a quick introduction to both of the datasets as well as how to map them.


About the US Census Computer, Internet & Broadband data

Dataset: US Census Bureau’s American Community Survey
Most Current Year: 2018 (with 2019 data to be released in December 2020)
First Year Data Are Available:
  • 2017 5 Year – For Small Geographies like zips/ZCTAs, Census tracts & block groups
  • 2013 1 Year – For Large Geographies like states, MSAs, counties & cities

You may be aware of the US Census that counts US residents every 10 years. The same agency collects internet and broadband statistics annually. The most current data can be found in the U.S. Census Bureau’s release of the 2018 American Community Survey (ACS) . The 2019 data set will be available in December 2020.

The computer and internet use questions were mandated by the 2008 Broadband Improvement Act and added to the ACS in 2013. The questions are not asked for the group quarters population and do not include data about people living in housing such as dorms, prisons, nursing homes, etc.

If you need data about the ownership and usage of all types of computers, including desktops, laptops, smartphones, tablets, etc., the 2018 ACS is the right dataset. The data also includes whether any member of the household has access to the internet. “Access” refers to someone in the household using or connecting to the internet, regardless of the service fee they pay.

The ACS provides information about the type of internet service used by the U.S population:

  • Cellular Data – a plan for a smartphone or other mobile device;
  • Broadband (high speed) Internet service-fiber optic or DSL;
  • Satellite;
  • Dial-up or some other service.

Let’s take a glimpse into the most popular computer and internet access tables that I pull from the 2018 American Community Survey.

The above tables can be even more valuable when you add additional dimensions like computer or internet subscription by:

  • Household Income,
  • Age,
  • Labor Force Status, and
  • Race/Ethnicity.

For example, you can get income by internet access or computing device by race/ethnicity.


About the FCC Data Fixed Broadband Deployment Data

Dataset: FCC’s Fixed Broadband Deployment Data
Most Current Year: 2018 (with 2019 data to be released in December 2020)
Update: Data appears to be updated twice a year
Geographies: Census blocks

Census Blocks are typically bounded by streets, roads or creeks. In cities, a census block may correspond to a city block, but in rural areas where there are fewer roads, blocks may be limited by other features. The population of a census block varies greatly. As of the 2010 census, there were 4,871,270 blocks with a reported population of zero,while a block that is entirely occupied by an apartment complex might have several hundred inhabitants.

The Federal Communications Commission (FCC) monitors regional and global communications in all 50 states, the District of Columbia, and U.S. territories. The commission, managed by Congress, stands as the United States’ primary authority for communications law, regulation, and technological innovation. The FCC use the Fixed Broadband Deployment statistics to measure the nationwide development of broadband access, as well as the successful deployment of the next generation of broadband technology.

Here’s a sample of the raw FCC data.  

The projects that I’ve done in the past with this data involved identifying areas without high speed internet or broadband. To do this, we aggregate the data by block group to list all of the providers and identify the largest Max Advertized Download Speed and Max Advertised Upstream Speed. So that would look like the example below where I’ve concatenated all businesses offering internet service for a single Census block and identified the max speeds for all providers.

Aside: a client mentioned that the FCC data can be misleading because the FCC allows a provider to report the highest download and upstream speeds if just 1 household in the block has internet access at those speeds. I haven’t verified if this statement is true or false.


Broadband Map

You can map data from the U.S. Census Bureau and the FCC datasets to identify areas with slow internet connection. Here’s what what a quick broadband map looks like:

Here are some helpful hints for using the map.

  1. Turn layers on and off by checking the check-boxes in the upper left.
  2. Turn on choropleth display (or colors) by clicking on the teardrop.
  3. Click on the bar graph to make handles appear to filter.

And here’s a quick video showing you how to use the checkboxes, teardrop and bar graph in the map to identify areas that lack broadband access.

Whew! Still reading? I’m impressed. So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data and build this map myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. Prices start at $299 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

10 of the Best US Crime Data Resources

Photo by Bill Oxford on Unsplash

“Happy families are all alike; every unhappy family is unhappy in its own way.” ― Leo Tolstoy, Anna Karenina

Not only are crime data about an unhappy topic, I also feel like each source of crime data that we pull for clients has a major drawback or “is unhappy in its own way.” While crime data is a bit outside of my area of expertise (which is demographics & business data), I do occasionally pull crime statistics for clients. Below is an overview of who uses crime data and why as well as 10 resources along with each source’s unhappy limitation.

Most Popular Uses of Crime Data

  • Businesses building models. Our business clients who are interested in crime data tend to be building models, and crime data is just one of many different types of data that they want to use in their models. For example, a prospect was interested in “associating medical trends with risk data to potentially identify problem areas sooner and determine where what type of resources are needed most based on risk and their associated medical needs. For instance, we may want to create a burn victims unit in areas with higher fire risk vs those with low fire risk, or staff more mental health professionals in higher crime.”
  • Non-Profits. We also pull crime statistics for non-profit clients who need this information for their community needs assessments or when writing grants.
  • Real Estate Developers. Crime data are important to developers who need to show that they are building in a low crime neighborhood so they can get their affordable housing developments approved by government agencies. Crime statistics can be central to approval and is often subject to dispute if the data shows crime is too high.
DataSourcePriceSmall Geographies Available?
Crime CountsFBI UCR/NIBRSFreeLaw Enforcement Agencies
 NCVSFreeNo. Entire US
 OJJDPFreeNo. States & US
Crime RiskApplied Geographic Solutions$$$Blocks groups, Census tracts, Zip Codes & more
 Neighborhood Scout$ SubscriptionNeighborhoods
Crime MapsCityProtectFreeMap of Incidents
 SpotCrimeFreeMap of Incidents
 LexisNexis Community Crime MapFreeMap of Incidents
Custom Data PullCubit$We pull the best geography for your project.
Table of 10 Crime Data Resources

CRIME COUNTS

FBI Uniform Crime Reporting Program/National Incident-Based Reporting System

Link: https://www.fbi.gov/services/cjis/ucr

Price: Free

If you need crime counts by type of crime for cities or counties and you aren’t worried about geographic differences, then the FBI crime datasets are the right fit. Data are provided by law enforcement agencies rather than standard geographies like a zip code. We most often pull FBI crime data for the geographies of County Agencies and City Agencies. There are also data for Metropolitan Statistical Areas or MSAs, but these geographies tend to be too large for the types of projects that we pull data for. There are also data provided for Universities and Colleges as well as State, Tribal and Other Agencies, but honestly, most of our clients just ignore these agencies and just get city and county data.

There are 2 drawbacks to this dataset:

  1. Crimes are reported by agency rather than rolled up to a geography. To quote Neighborhood Scout “…crimes are reported by individual law enforcement agencies, rather than by city or town, and many cities – even small ones – have more than one agency responsible for law enforcement (municipal, university, county, transit, etc.). Even FBI data are reported by agency not by city or town, providing an incomplete assessment of city-wide crime counts. It is an agency-centric rather than locality-centric reporting method. If you use FBI data, you only get city-wide general counts, and only from one agency in the city, so it is generally incomplete for the city overall, as well as not specific to a neighborhood or address.”
  2. There are no FIPS or unique geography ids that make it easy to join this dataset to other datasets like Census data. You have to do name joins which are error prone and painful. 

An Example of Data Collected by the FBI for the Uniform Crime Reporting Program

The UCR also provides additional details about the persons arrested, such as age, race and ethnicity, the weapons used and the value of items stolen.

The FBI uses a tool called the Crime Data Explorer in an effort to make crime data more user friendly. The Crime Data Explorer currently includes violent crime statistics (murder and nonnegligent manslaughter, rape, robbery, and aggravated assault) and property crime (burglary, larceny-theft, motor vehicle theft and arson)

Crime Data Explorer

 State crime rates can be compared to national crime rates.

This is an example of the Arrest Data found in the Crime Data Explorer tool.

National Crime Victimization Survey (NCVS)

Link: https://www.bjs.gov/index.cfm?ty=dcdetail&iid=245

Price: Free

The NCVS is an annual data collection conducted by the U.S. Census Bureau. The purpose of the NCVS is to fill in the gaps between crimes reported to law enforcement and those that are not. The collection reports national statistics only and doesn’t provide data for smaller geographies. Data includes nonfatal personal crimes (rape or sexual assault, robbery, aggravated and simple assault and personal larceny) and houseold property crimes (burglary/trespassing, motor-vehicle theft, and other theft).

Here is an example of the difference in crimes reported to the police and those that aren’t. The table below provides the crime rates for different types of crimes.

Office of Juvenile Justice and Delinquency Prevention (OJJDP)

Link: https://ojjdp.ojp.gov/research-and-statistics

Price: Free

Juvenile offenses, as well as crimes where juveniles are victims are included in the OJJDP’s crime report for each US state. The Statistical Briefing Book is a tool that provides access to online information about juvenile crime and victimization and about youth involved in the juvenile justice system.

Below is an example of the options provided to search for juvenile crime statistics, as well as an example of the data returned.

CRIME RISK

Applied Geographic Solutions (AGS)

Link: You can buy this data through us since we are a vendor.

Price: Base Price: $3,750 for all US zips; $1,250 for all US counties *

If you are doing model building and need crime data for small geographies like zip codes, the Applied Geographic Solution dataset with crime indexes can be a helpful resource. Using advanced analysis of a rolling seven-year database of FBI and local agency statistics, AGS provides relative crime risk, not just crime occurrence. Crime risk is an index of the probability of crime in a geographical region compared to the entire US. Zip code is the most popular geography that’s requested, but county and city data as well as smaller geographies are available as well. FIPS codes are also provided with the data, which makes it easy to connect geographies between datasets – super important for doing modeling work when you need data from multiple sources.

The AGS crime data comes from a private data vendor rather than a public agency. Unlike the FBI data above, there will be restrictions on how you use the data. The prices above are rough quotes, and I’ll have to adjust the pricing based on number of users and how you are using the data (e.g. internal modeling versus publishing online). Contact me if you are interested in continuing the conversation here.

Below is an example of the type of information that AGS provides:

Neighborhood Scout

Link: https://www.neighborhoodscout.com

Price: $39.99 – $119.99/month for a subscription report reports

Neighborhood Scout is geared toward real estate investors who want to identify opportunities with low crime risk. Other free web sites collect data from local law enforcement agencies, not by locality, which tends to leave holes in the data. Not all agencies elect to report data and some localities have more than one agency. Neighborhood Scout fills in the holes by using like neighborhood crime data leading to seamless 100% US coverage.

By entering an address, an investor can get a full report on the crime risk of a neighborhood. The report includes crime risk ratings for several crime types, a resident’s risk of becoming a victim and 5-year trends and forecasts.

Below is an example of the type of information that Neighborhood Scout provides:

CRIME MAPS

Links:

CityProtect

SpotCrime

LexisNexis Community Crime Map

Price: Free

City Protect, Spot Crime and LexisNexis Community Crime Map all provide interactive maps of crime data from law enforcement agencies. You can search for an address and get a map of where or what neighborhoods have the most crimes and what types of crimes were committed in the past. If you are getting a radius report, you might want to also check one of these map solutions for recent crimes in your area of interest. By doing a simple search on zip code, city, or state, you have the ability to see incident details, statistics and reports. Like the FBI UCR, the data available is dependent on reports provided by law enforcement agencies. Some areas may not have any crime information available due to lack of participation by the local agency.

Here’s what these resources look like.

City Protect

Spot Crime In addition to an interactive map, SpotCrime also has a Trend tab with a summary of crime trends.

LexisNexis Community Crime Map lets you filter date and type of incidence. It also has option for to buffer addresses.

CUSTOM DATA PULL

Cubit

Price: Starting at $299 and depends on geographies and years needed.

Geographies: All US geographies from large geographies like states and counties to small geographies like zips, census tracts and blocks.

Whew! Still reading? I’m impressed. So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. Prices start at $299 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

Census Datasets by Geography

The Census Bureau beat me to putting together this useful graphic of Census datasets by geography. If you wanted to know what Census datasets are available for zip codes, you can use the graphic below to quickly see that the American Community Survey, County Business Patterns & Longitudinal Employer-Household Dynamics Origin-Destination Employment Statistics are the only 3 datasets that you need to dig into further.

Census Datasets by Geography from Kristen Carney

Source: Andrew W. Hait’s Census Business Data at a Glance bber.unm.edu, 2018.

Cubit’s list of favorite datasets also helps you find datasets by geography and includes datasets other than just Census data. Got questions or want to tell me about your favorite dataset? Send me an email.

Population Projections by County and by City for the US

Photo by Mario Purisic on Unsplash

While the US Census Bureau doesn’t publish population projection data, each state individually produces projections that state agencies use to make policy decisions and that businesses can use to make business decisions. We include the population forecast data in Radius Reports and when requested in Custom Data projects. Below are 50 of our favorite population projection datasets for counties for each state in the US.

StateProjection YearsAgency that Produces Population ProjectionsLast Updated
Alabama2020-2040University of Alabama Center for Business and Economic Research2018
Alaska2019-2045Alaska Department of Labor and Workforce Development2018
Arizona2018-2055Arizona Commerce Authority2018
Arkansas2014-2065UALR Arkansas Economic Development Institute2015
California2019-2060California Department of Finance2019
Colorado2018-2050Colorado Department of Local Affairs2018
Connecticut2020-2040Connecticut State Data Center2018
Delaware2020-2050Delaware Population Consortium2018
District of Columbia2020-2045Metropolitan Washington Council of Governments2018
Florida2020-2045Florida Office of Economic and Demographic Research2019
Georgia2018-2063Georgia Governor’s Office of Planning and Budget2019
Hawaii2020-2045Hawaii State Department of Business, Economic Development & Tourism2018
Idaho2016-2026Idaho Department of Labor2018
Illinois2020-2030Illinois Department of Commerce and Economic Opportunity2017
Indiana2020-2050Indiana Business Research Center2018
Iowa
Kansas2014-2044Wichita State University, Center for Economic Development and Business Research2016
Kentucky2020-2040Kentucky State Data Center2016
Louisiana2010-2030Louisiana Division of Administration2008
Maine2021-2036Maine Department of Administrative and Financial Services2018
Maryland2015-2045Maryland Department of Planning2017
Massachusetts2015-2040University of Massachusetts Donahue Institute2018
Michigan2020-2045Michigan Department of Technology, Management and Budget2019
Minnesota2015-2050Minnesota State Demographic Center2017
Mississippi2020-2050The State Data Center of Mississippi2019
Missouri2010-2030Missouri Office of Administration2008
Montana2018-2060Montana Department of Commerce2019
Nebraska2015-2050Nebraska Center for Public Affairs Research2015
Nevada2019-2038Nevada Department of Taxation2019
New Hampshire2020-2040RLS Demographics, Inc.2016
New Jersey2019-2034New Jersey Department of Labor and Workforce Development2016
New Mexico2020-2040University of New Mexico, GeoSpatial and Population Studies2017
New York2018-2040Cornell University2018
North Carolina2019-2039North Carolina Office of State Budget and Management2019
North Dakota2020-2040North Dakota Department of Commerce, Census Office2016
Ohio2020-2050Ohio Development Services Agency2018
Oklahoma2012-2075Oklahoma Department of Commerce2012
Oregon2019-2069PSU, College of Urban & Public Affairs2019
Pennsylvania2015-2040Pennsylvania State Data Center2014
Rhode Island2020-2040Rhode Island Statewide Planning Program2018
South Carolina2010-2035South Carolina Department of Revenue and Fiscal Affairs2012
South Dakota2010-2035South Dakota State University2012
Tennessee2019-2070University of Tennessee Boyd Center for Business and Economic Research2019
Texas2018-2050Texas Demographic Center2018
Utah2015-2065Kem C. Gardner Policy Institute2017
Vermont2020-2030Vermont Agency of Commerce and Community Development2013
Virginia2020-2040Weldon Cooper Center for Public Service at the University of Virginia2019
Washington2018-2040Washington Office of Financial Management2017
West Virginia2020-2030West Virginia University Bureau of Business and Economic Research2017
Wisconsin2015-2040Wisconsin Demographic Services Center2013
Wyoming2019-2040Wyoming Department of Administration & Information2019
50 of our Favorite Population Forecasts by County Datasets

County Projections are handy and all, but what I really want are Population Projections by Cities.

I hear you! The smaller the geography, the better…as long as the data are accurate, right?

The first stop in your hunt for population forecasts for cities is to check the agency website in the table above. Some state agencies, like Arizona, provide population projections for cities along with counties. Be aware they might use an unexpected name for cities like sub-counties or places. So you’ll want to do some digging through data files here rather than just reading the file names. After digging and if you still don’t find the data you need, continue your search with the two options below.

If you need population projections for just 1 city…

You can often find population projection data for 1 city in the city’s comprehensive plan, and here’s how.

  1. Search for the city’s comprehensive plan in Google (e.g. City of Austin comprehensive plan).
    1. TIP! Sometimes, you have to navigate to the city’s website & then search for the plan on their site. It’s usually linked from the planning department’s webpage.
  2. When you find the plan, search in the text of the plan for the words “projection” or “population”.
    1. TIP! If the text of the plan isn’t searchable, look in the table of contents. Usually, there’s a section in the table of contacts that’s titled “environment” or “population.” Sometimes, this section is in an Appendix, but usually, it’s at the very beginning of the document. Read through only this section of the plan to see if you can find population projection data.
  3. Also consider how old the plan is. If the plan is older than 2010, consider looking for this data on the MPO’s website (next step) first and then coming back to the comprehensive plan if you can’t find it on the MPO’s website.  

If you need population projections for multiple cities in a region…

The metropolitan planning organization (MPO) often publishes population projections for multiple cities in a region to estimate traffic demand. MPOs are federally mandated and funded transportation policy-making organizations in the United States that are made up of representatives from local government and governmental transportation authorities (source). To find population forecasts in MPO planning documents, first you have to figure out if your city is in an MPO and if so, which MPO.

  1. If you don’t already know, do a Google search for what county the city is in (e.g. “What county is Austin, Texas in?”).
  2. Then google search for what MPO the county is in.
    1. TIP! You may have to google for a “map of MPOs for the state of _____” and use the map to figure this out.
  3. Go to the MPO’s website and find the long range transportation plan. It’s usually called something like MPO Year Plan (e.g. CAMPO 2040 Plan) or 2040 transportation plan.
  4. Open the plan, and search for the words “projection” or “population”.
  5. Sometimes, the population projections in the plan are for cities. Other times, the population projections are for Traffic Analysis Zones (or TAZs). If we can’t find population projections for cities, you can use the Traffic Analysis Zone map. See the sample below for what a TAZ population projection map could look like.

Don’t Have Time to Pull This Data Yourself?

Whew! Still reading? I’m impressed.

So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data for my area of interest,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. You can get population projections, as well as hundreds of other data points, in a custom data pull. Prices start at $299 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

US Census’ Building Permits Survey

Photo by Avel Chuklanov on Unsplash

People who don’t work with demographic data day in and day out often don’t know that the US Census Bureau collects many other datasets in addition to the Decennial Census that we all have to fill out. The Census’ Building Permit Survey is a lesser known dataset but still a valuable resource. The dataset contains the number of new housing units authorized by building permits, the type of housing (e.g. single family) and unit value.

Data Details: US Census Bureau’s Building Permits Survey

  • What: Building Permits
  • Who: Small-ish geographies include States, Metropolitan Areas (CBSAs), Counties and individual permit-issuing places (large cities)
  • When Updated: Monthly & Annually. Release Schedule.
  • Historic data: Yes but it’s complicated. See below for details.
  • Join on: FIPS id

What the Building Permit data look like

Possible problems with Building Permit data

The monthly files provide data for all counties that are requested to report monthly (about 900+). So there’s not new data each month for all counties. And there are no data at all for small population cities – which make sense because most of them don’t have that many new buildings going up.

On years that end with 4, the agency updates which places are included in the universe. Below is the reply that I received when asking about guidance for historical comparisons given the universe updates.

“It’s something to be aware of, but doesn’t negate longer run comparisons. Places coming in and out of the universe tend to be smaller and have a minor effect on the data, especially when looking at larger geographies. Most of the change from 2004 to 2014 universe was single family housing in the south, so it’s just something to keep in mind if you see a big jump from 2013 to 2014, just know that universe change could be part of that. For instance smaller places in TX or other larger Southern states that may have not issued permits in 2004, could have begun at some point between 2004 and 2014, so the 2014 data includes them. Odds are good that they weren’t having a significant amount of building and not issuing permits, so permit issuance probably coincided with the increase in building, and therefore less may be omitted than it seems. Just be aware of it if you notice any oddities around universe update years – those ending in 4 – but don’t avoid looking at trends because of it.”

Fun Visualizations

The following visualizations are from the 2018 dataset.

Historic Data Availability

While there are historic data available, here’s a table that shows what years data are available for which geographies.

How Building Permit Data are Collected

  • “Statistics on construction authorized by building permits are based upon reports submitted by local building permit officials in response to a mail survey. They are obtained using Form C-404,”Report of Building or Zoning Permits Issued and Local Public Construction.” When a report is not received, missing residential data are either (1) obtained from the Survey of Use of Permits (SUP) or (2) imputed. All other missing data are imputed.”
  • “The monthly statistics are based on data from a sample of 8,500 permit-issuing places, selected from a universe of approximately 19,000 places.”
  • “The annual statistics are not based upon a sample, but are obtained by directly cumulating the data for the 19,000 permit-issuing places.”

More Details

Other Datasets You Might be Interested in

Don’t Have Time to Pull This Data Yourself?

Whew! Still reading? I’m impressed.

So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data for my area of interest,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. You can get building permit data, as well as hundreds of other data points, in a custom data pull. Prices start at $299 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.