Cubit's Blog

Photo by Bill Oxford on Unsplash

“Happy families are all alike; every unhappy family is unhappy in its own way.” ― Leo Tolstoy, Anna Karenina

Not only are crime data about an unhappy topic, I also feel like each source of crime data that we pull for clients has a major drawback or “is unhappy in its own way.” While crime data is a bit outside of my area of expertise (which is demographics & business data), I do occasionally pull crime statistics for clients. Below is an overview of who uses crime data and why as well as 10 resources along with each source’s unhappy limitation.

Most Popular Uses of Crime Data

• Businesses building models. Our business clients who are interested in crime data tend to be building models, and crime data is just one of many different types of data that they want to use in their models. For example, a prospect was interested in “associating medical trends with risk data to potentially identify problem areas sooner and determine where what type of resources are needed most based on risk and their associated medical needs. For instance, we may want to create a burn victims unit in areas with higher fire risk vs those with low fire risk, or staff more mental health professionals in higher crime.”
• Non-Profits. We also pull crime statistics for non-profit clients who need this information for their community needs assessments or when writing grants.
• Real Estate Developers. Crime data are important to developers who need to show that they are building in a low crime neighborhood so they can get their affordable housing developments approved by government agencies. Crime statistics can be central to approval and is often subject to dispute if the data shows crime is too high.

Data	Source	Price	Small Geographies Available?
Crime Counts	FBI UCR/NIBRS	Free	Law Enforcement Agencies
	NCVS	Free	No. Entire US
	OJJDP	Free	No. States & US
Crime Risk	Applied Geographic Solutions	$$$	Blocks groups, Census tracts, Zip Codes & more
	Neighborhood Scout	$ Subscription	Neighborhoods
Crime Maps	CityProtect	Free	Map of Incidents
	SpotCrime	Free	Map of Incidents
	LexisNexis Community Crime Map	Free	Map of Incidents
Custom Data Pull	Cubit	$	We pull the best geography for your project.

CRIME COUNTS

FBI Uniform Crime Reporting Program/National Incident-Based Reporting System

Link: https://www.fbi.gov/services/cjis/ucr

Price: Free

If you need crime counts by type of crime for cities or counties and you aren’t worried about geographic differences, then the FBI crime datasets are the right fit. Data are provided by law enforcement agencies rather than standard geographies like a zip code. We most often pull FBI crime data for the geographies of County Agencies and City Agencies. There are also data for Metropolitan Statistical Areas or MSAs, but these geographies tend to be too large for the types of projects that we pull data for. There are also data provided for Universities and Colleges as well as State, Tribal and Other Agencies, but honestly, most of our clients just ignore these agencies and just get city and county data.

There are 2 drawbacks to this dataset:

1. Crimes are reported by agency rather than rolled up to a geography. To quote Neighborhood Scout “…crimes are reported by individual law enforcement agencies, rather than by city or town, and many cities – even small ones – have more than one agency responsible for law enforcement (municipal, university, county, transit, etc.). Even FBI data are reported by agency not by city or town, providing an incomplete assessment of city-wide crime counts. It is an agency-centric rather than locality-centric reporting method. If you use FBI data, you only get city-wide general counts, and only from one agency in the city, so it is generally incomplete for the city overall, as well as not specific to a neighborhood or address.”
2. There are no FIPS or unique geography ids that make it easy to join this dataset to other datasets like Census data. You have to do name joins which are error prone and painful. 

An Example of Data Collected by the FBI for the Uniform Crime Reporting Program

The UCR also provides additional details about the persons arrested, such as age, race and ethnicity, the weapons used and the value of items stolen.

The FBI uses a tool called the Crime Data Explorer in an effort to make crime data more user friendly. The Crime Data Explorer currently includes violent crime statistics (murder and nonnegligent manslaughter, rape, robbery, and aggravated assault) and property crime (burglary, larceny-theft, motor vehicle theft and arson)

Crime Data Explorer

 State crime rates can be compared to national crime rates.

This is an example of the Arrest Data found in the Crime Data Explorer tool.

National Crime Victimization Survey (NCVS)

Link: https://www.bjs.gov/index.cfm?ty=dcdetail&iid=245

Price: Free

The NCVS is an annual data collection conducted by the U.S. Census Bureau. The purpose of the NCVS is to fill in the gaps between crimes reported to law enforcement and those that are not. The collection reports national statistics only and doesn’t provide data for smaller geographies. Data includes nonfatal personal crimes (rape or sexual assault, robbery, aggravated and simple assault and personal larceny) and houseold property crimes (burglary/trespassing, motor-vehicle theft, and other theft).

Here is an example of the difference in crimes reported to the police and those that aren’t. The table below provides the crime rates for different types of crimes.

Office of Juvenile Justice and Delinquency Prevention (OJJDP)

Link: https://ojjdp.ojp.gov/research-and-statistics

Price: Free

Juvenile offenses, as well as crimes where juveniles are victims are included in the OJJDP’s crime report for each US state. The Statistical Briefing Book is a tool that provides access to online information about juvenile crime and victimization and about youth involved in the juvenile justice system.

Below is an example of the options provided to search for juvenile crime statistics, as well as an example of the data returned.

CRIME RISK

Applied Geographic Solutions (AGS)

Link: You can buy this data through us since we are a vendor.

Price: Base Price: $3,750 for all US zips; $1,250 for all US counties *

If you are doing model building and need crime data for small geographies like zip codes, the Applied Geographic Solution dataset with crime indexes can be a helpful resource. Using advanced analysis of a rolling seven-year database of FBI and local agency statistics, AGS provides relative crime risk, not just crime occurrence. Crime risk is an index of the probability of crime in a geographical region compared to the entire US. Zip code is the most popular geography that’s requested, but county and city data as well as smaller geographies are available as well. FIPS codes are also provided with the data, which makes it easy to connect geographies between datasets – super important for doing modeling work when you need data from multiple sources.

The AGS crime data comes from a private data vendor rather than a public agency. Unlike the FBI data above, there will be restrictions on how you use the data. The prices above are rough quotes, and I’ll have to adjust the pricing based on number of users and how you are using the data (e.g. internal modeling versus publishing online). Contact me if you are interested in continuing the conversation here.

Below is an example of the type of information that AGS provides:

Neighborhood Scout

Link: https://www.neighborhoodscout.com

Price: $39.99 – $119.99/month for a subscription report reports

Neighborhood Scout is geared toward real estate investors who want to identify opportunities with low crime risk. Other free web sites collect data from local law enforcement agencies, not by locality, which tends to leave holes in the data. Not all agencies elect to report data and some localities have more than one agency. Neighborhood Scout fills in the holes by using like neighborhood crime data leading to seamless 100% US coverage.

By entering an address, an investor can get a full report on the crime risk of a neighborhood. The report includes crime risk ratings for several crime types, a resident’s risk of becoming a victim and 5-year trends and forecasts.

Below is an example of the type of information that Neighborhood Scout provides:

CRIME MAPS

Links:

CityProtect

SpotCrime

LexisNexis Community Crime Map

Price: Free

City Protect, Spot Crime and LexisNexis Community Crime Map all provide interactive maps of crime data from law enforcement agencies. You can search for an address and get a map of where or what neighborhoods have the most crimes and what types of crimes were committed in the past. If you are getting a radius report, you might want to also check one of these map solutions for recent crimes in your area of interest. By doing a simple search on zip code, city, or state, you have the ability to see incident details, statistics and reports. Like the FBI UCR, the data available is dependent on reports provided by law enforcement agencies. Some areas may not have any crime information available due to lack of participation by the local agency.

Here’s what these resources look like.

City Protect

Spot Crime In addition to an interactive map, SpotCrime also has a Trend tab with a summary of crime trends.

LexisNexis Community Crime Map lets you filter date and type of incidence. It also has option for to buffer addresses.

CUSTOM DATA PULL

Cubit

Price: Starting at $599 and depends on geographies and years needed.

Geographies: All US geographies from large geographies like states and counties to small geographies like zips, census tracts and blocks.

Whew! Still reading? I’m impressed. So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. Prices start at $599 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

The Census Bureau beat me to putting together this useful graphic of Census datasets by geography. If you wanted to know what Census datasets are available for zip codes, you can use the graphic below to quickly see that the American Community Survey, County Business Patterns & Longitudinal Employer-Household Dynamics Origin-Destination Employment Statistics are the only 3 datasets that you need to dig into further.

Census Datasets by Geography from Kristen Carney

Source: Andrew W. Hait’s Census Business Data at a Glance bber.unm.edu, 2018.

Cubit’s list of favorite datasets also helps you find datasets by geography and includes datasets other than just Census data. Got questions or want to tell me about your favorite dataset? Send me an email.

Photo by Mario Purisic on Unsplash

Updated April 2023. While the US Census Bureau doesn’t publish population projection data, each state individually produces projections that state agencies use to make policy decisions and that businesses can use to make business decisions. We include the population forecast data in Radius Reports and when requested in Custom Data projects. Below are 50 of our favorite population projection datasets for counties for each state in the US.

State	Projection Years	Agency that Produces Population Projections	Last Updated
Alabama	2020-2040	University of Alabama Center for Business and Economic Research	2018
Alaska	2025-2050	Alaska Department of Labor and Workforce Development	2022
Arizona	2022-2060	Arizona Commerce Authority	2022
Arkansas	2014-2065	UALR Arkansas Economic Development Institute	2015
California	2020-2060	California Department of Finance	2020
Colorado	2022-2050	Colorado Department of Local Affairs	2022
Connecticut	2020-2040	Connecticut State Data Center	2018
Delaware	2025-2050	Delaware Population Consortium	2022
District of Columbia	2025-2045	Metropolitan Washington Council of Governments	2022
Florida	2022-2050	Florida Office of Economic and Demographic Research	2022
Georgia	2020-2060	Georgia Governor’s Office of Planning and Budget	2021
Hawaii	2020-2045	Hawaii State Department of Business, Economic Development & Tourism	2018
Idaho	2020-2029	Idaho Department of Labor	2020
Illinois	2020-2030	Illinois Department of Public Health	2021
Indiana	2020-2050	Indiana Business Research Center	2018
Iowa
Kansas	2020-2045	Wichita State University, Center for Economic Development and Business Research	2021
Kentucky	2025-2050	Kentucky State Data Center	2022
Louisiana	2010-2030	Louisiana Division of Administration	2008
Maine	2023-2038	Maine Department of Administrative and Financial Services	2021
Maryland	2025-2050	Maryland Department of Planning	2022
Massachusetts	2020-2040	University of Massachusetts Donahue Institute	2018
Michigan	2020-2045	Michigan Department of Technology, Management and Budget	2019
Minnesota	2020-2060	Minnesota State Demographic Center	2020
Mississippi	2020-2050	The State Data Center of Mississippi	2019
Missouri	2010-2030	Missouri Office of Administration	2008
Montana	2020-2080	Montana Department of Commerce	2020
Nebraska	2030-2050	Nebraska Center for Public Affairs Research	2022
Nevada	2022-2041	Nevada Department of Taxation	2022
New Hampshire	2025-2050	New Hampshire Office of Strategic Initiatives	2022
New Jersey	2019-2034	New Jersey Department of Labor and Workforce Development	2016
New Mexico	2020-2040	University of New Mexico, GeoSpatial and Population Studies	2020
New York	2018-2040	Cornell University	2018
North Carolina	2025-2050	North Carolina Office of State Budget and Management	2022
North Dakota	2020-2040	North Dakota Department of Commerce, Census Office	2016
Ohio	2020-2050	Ohio Development Services Agency	2018
Oklahoma	2012-2075	Oklahoma Department of Commerce	2012
Oregon	2022-2072	PSU Population Research Center	2022
Pennsylvania	2015-2040	Pennsylvania State Data Center	2014
Rhode Island	2020-2040	Rhode Island Statewide Planning Program	2018
South Carolina	2020-2035	South Carolina Revenue and Fiscal Affairs Office	2021
South Dakota	2010-2035	South Dakota State University	2012
Tennessee	2021-2070	University of Tennessee Boyd Center for Business and Economic Research	2022
Texas	2022-2060	Texas Demographic Center	2022
Utah	2020-2060	Kem C. Gardner Policy Institute	2022
Vermont	2020-2030	Vermont Agency of Commerce and Community Development	2013
Virginia	2030-2050	Weldon Cooper Center for Public Service at the University of Virginia	2022
Washington	2025-2050	Washington Office of Financial Management	2022
West Virginia	2020-2030	West Virginia University Bureau of Business and Economic Research	2017
Wisconsin	2015-2040	Wisconsin Demographic Services Center	2013
Wyoming	2019-2040	Wyoming Department of Administration & Information	2019

If you need projections for counties in different states, you’ll need to purchase projections that use the the same methodology so that they’ll be apples to apples comparable. We’ve teamed up with Cooke Demographics to provide such county population projections that include age and sex data for years 2022-2047. Learn more about our County Population Projections here: Link

Other companies that we have worked with in the past that provide county population projection data are AGS (Applied Geographic Solutions), Woods & Poole Economics, Nielsen and ESRI.

County Projections are handy and all, but what I really want are Population Projections by Cities.

I hear you! The smaller the geography, the better…as long as the data are accurate, right?

The first stop in your hunt for population forecasts for cities is to check the agency website in the table above. Some state agencies, like Arizona, provide population projections for cities along with counties. Be aware they might use an unexpected name for cities like sub-counties or places. So you’ll want to do some digging through data files here rather than just reading the file names. After digging and if you still don’t find the data you need, continue your search with the two options below.

If you need population projections for just 1 city…

You can often find population projection data for 1 city in the city’s comprehensive plan, and here’s how.

1. Search for the city’s comprehensive plan in Google (e.g. City of Austin comprehensive plan).
   1. TIP! Sometimes, you have to navigate to the city’s website & then search for the plan on their site. It’s usually linked from the planning department’s webpage.
2. When you find the plan, search in the text of the plan for the words “projection” or “population”.
   1. TIP! If the text of the plan isn’t searchable, look in the table of contents. Usually, there’s a section in the table of contacts that’s titled “environment” or “population.” Sometimes, this section is in an Appendix, but usually, it’s at the very beginning of the document. Read through only this section of the plan to see if you can find population projection data.
3. Also consider how old the plan is. If the plan is older than 2010, consider looking for this data on the MPO’s website (next step) first and then coming back to the comprehensive plan if you can’t find it on the MPO’s website.  

If you need population projections for multiple cities in a region…

The metropolitan planning organization (MPO) often publishes population projections for multiple cities in a region to estimate traffic demand. MPOs are federally mandated and funded transportation policy-making organizations in the United States that are made up of representatives from local government and governmental transportation authorities (source). To find population forecasts in MPO planning documents, first you have to figure out if your city is in an MPO and if so, which MPO.

1. If you don’t already know, do a Google search for what county the city is in (e.g. “What county is Austin, Texas in?”).
2. Then google search for what MPO the county is in.
   1. TIP! You may have to google for a “map of MPOs for the state of _____” and use the map to figure this out.
3. Go to the MPO’s website and find the long range transportation plan. It’s usually called something like MPO Year Plan (e.g. CAMPO 2040 Plan) or 2040 transportation plan.
4. Open the plan, and search for the words “projection” or “population”.
5. Sometimes, the population projections in the plan are for cities. Other times, the population projections are for Traffic Analysis Zones (or TAZs). If we can’t find population projections for cities, you can use the Traffic Analysis Zone map. See the sample below for what a TAZ population projection map could look like.

Don’t Have Time to Pull This Data Yourself?

Whew! Still reading? I’m impressed.

So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data for my area of interest,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. You can get population projections, as well as hundreds of other data points, in a custom data pull. Prices start at $599 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

Photo by Avel Chuklanov on Unsplash

People who don’t work with demographic data day in and day out often don’t know that the US Census Bureau collects many other datasets in addition to the Decennial Census that we all have to fill out. The Census’ Building Permit Survey is a lesser known dataset but still a valuable resource. The dataset contains the number of new housing units authorized by building permits, the type of housing (e.g. single family) and unit value.

Data Details: US Census Bureau’s Building Permits Survey

• What: Building Permits
• Who: Small-ish geographies include States, Metropolitan Areas (CBSAs), Counties and individual permit-issuing places (large cities)
• When Updated: Monthly & Annually. Release Schedule.
• Historic data: Yes but it’s complicated. See below for details.
• Join on: FIPS id

What the Building Permit data look like

Possible problems with Building Permit data

The monthly files provide data for all counties that are requested to report monthly (about 900+). So there’s not new data each month for all counties. And there are no data at all for small population cities – which make sense because most of them don’t have that many new buildings going up.

On years that end with 4, the agency updates which places are included in the universe. Below is the reply that I received when asking about guidance for historical comparisons given the universe updates.

“It’s something to be aware of, but doesn’t negate longer run comparisons. Places coming in and out of the universe tend to be smaller and have a minor effect on the data, especially when looking at larger geographies. Most of the change from 2004 to 2014 universe was single family housing in the south, so it’s just something to keep in mind if you see a big jump from 2013 to 2014, just know that universe change could be part of that. For instance smaller places in TX or other larger Southern states that may have not issued permits in 2004, could have begun at some point between 2004 and 2014, so the 2014 data includes them. Odds are good that they weren’t having a significant amount of building and not issuing permits, so permit issuance probably coincided with the increase in building, and therefore less may be omitted than it seems. Just be aware of it if you notice any oddities around universe update years – those ending in 4 – but don’t avoid looking at trends because of it.”

Fun Visualizations

The following visualizations are from the 2018 dataset.

Historic Data Availability

While there are historic data available, here’s a table that shows what years data are available for which geographies.

How Building Permit Data are Collected

• “Statistics on construction authorized by building permits are based upon reports submitted by local building permit officials in response to a mail survey. They are obtained using Form C-404,”Report of Building or Zoning Permits Issued and Local Public Construction.” When a report is not received, missing residential data are either (1) obtained from the Survey of Use of Permits (SUP) or (2) imputed. All other missing data are imputed.”
• “The monthly statistics are based on data from a sample of 8,500 permit-issuing places, selected from a universe of approximately 19,000 places.”
• “The annual statistics are not based upon a sample, but are obtained by directly cumulating the data for the 19,000 permit-issuing places.”

More Details

• Methodology Notes: https://www.census.gov/construction/bps/sample/placeasc.pdf
• Geography Notes – Monthly: https://www.census.gov/construction/bps/xls/footnote202002.xls
• Geography Notes – Annual: https://www.census.gov/construction/bps/xls/footnote2018.xls
• Counties and Independent Cities with All Land Areas Covered by Building Permit Systems: https://www.census.gov/construction/bps/pdf/2018completelycoveredcounties.pdf

Other Datasets You Might be Interested in

• Historic Home Sales Data By Zip Code

Don’t Have Time to Pull This Data Yourself?

Whew! Still reading? I’m impressed.

So if you’ve gotten this far and you’re thinking, “Yeah, sure I could pull all of this data myself in a couple of days, but I have other important things to do and I really don’t want to sift through all of the data dictionaries, methodology statements and tool instructions to make sure that I have the most current data for my area of interest,” you are not alone. You sound just like our other clients at Cubit who depend on us to provide clean, accurate and easy-to-work-with data as well as human-to-human customer support. You can get building permit data, as well as hundreds of other data points, in a custom data pull. Prices start at $599 with a 3 business day turnaround. Tell me what data you need for what geography & I’ll get you a free quote & turnaround estimate.

Photo by Renee Fisher on Unsplash

Background

In 2009, my co-founder, Anthony, and I quit our day jobs to build a web app for environmental engineers to pull demographic data. But after spending over $10,000, months of development work and cold calling over 1,000 environmental engineers, we came to the painful realization that the engineers didn’t want the app that we had built.

So we ran a small experiment.

We’d been getting a lot of web traffic to some city and county landing pages that we’d put up on our website to attract engineers. We were trying to introduce ourselves, like new companies do, using our (data) expertise. We didn’t want to build something that no-one wanted again, so this time, we put up a Buy Now button connected to nothing so we could measure if our visitors were even interested in paying money for demographics.

And you guys clicked the button. A lot of times. 

So the next day, we added a price next to the button. Every time a person clicked on the Buy Now button, they’d get a “We’re Sorry” message. But you kept clicking. We added and changed small details until we figured out people were looking to purchase a report with demographics for a city or county. We called those reports Starter Reports, because they were a good start in looking at demographics for a geography.

In the beginning, delivering Starter Reports was a manual process. Whenever someone bought a report, Anthony would rush to his computer to generate the report and then manually email it out in 5 minutes. After he had enough of waking up the middle of the night to send reports, he automated the delivery of these reports. And there it was: Cubit’s first successful product. We’ve updated the Starter Reports with new Census data and sold them for almost 10 years.

I tell you this long story to explain why it was a difficult decision for Anthony and me to stop selling Starter Reports. It wasn’t because we made a lot of money from them recently. Years ago, we discovered far more of you guys need spreadsheets with demographics for lots geographies (e.g. all cities in Texas) rather than a report about 1 geography (e.g. the city of Houston).

You see, Starter Reports took us from dew-eyed start up founders to business owners. Yes, they were our first successful product and that initial success gave us permission and courage to keep developing other data products. But more importantly, that Buy Now button taught us HOW to develop products. That is, don’t go build the thing that you want. Instead figure out how to sell a product and what the product is at the same time by interacting with customers. Some better-marketers-than-us call this process Lean Product Development, and this is the book about it that I still carry around with me in my laptop bag today if you want to avoid our mistakes.

But in continuing to talk to you over the years, we’ve got new ideas for products that we’re excited to develop. As a tiny team, we have to say no to something (that is: Starter Report development, updates & support) to free up our time to pursue the biggest opportunities. You know, the ones that make us excited to get out of bed and get to work on Monday mornings.

So Starter Reports will no longer be available for purchase. To everyone who bought one in the past: thank you. Truly. Your vote of confidence was at least as valuable as the money that you paid for these reports. And if you check out the Census’ Narrative Profiles and they won’t replace Starter Reports for your purposes, email me back and we’ll brainstorm other options.