If you are in danger, call 911. If you need to exit this website quickly, click on the ESCAPE button. This button will immediately open a browser window for weather.com and replace your current window with google.com.

When you want to compare a rate to a goal or another rate, a statistical test will tell you if there’s a difference. If there’s only a small chance there’s no difference, that’s called a statistically significant difference. Most statisticians choose 5% for that small chance, which means 1 out of 20 tests will falsely say there’s a difference when there really isn’t.

Cautions Before Testing

Percentages

A percentage is the same as a rate. They’re usually “crude” rates, which mean they’re not adjusted. However, rates from the Behavioral Risk Factor Surveillance System (BRFSS) survey have been adjusted.

Enough Events

An adjusted rates should be based on 20 or more events. If it has less, it is unreliable and should not be compared. Consider the options described in Calculating Reliable Rates to make the rate more reliable.

Similarly, a crude rate should be based on 10 or more events.

Significance Versus Importance

These tests can identify a statistically significant difference, but “significant” has a different meaning here than in everyday conversations. A difference between rates is significant if it likely did not happen by chance. If the difference is not significant, that only means we cannot say if there is a difference. It does not mean there is no difference.

A significant difference may not be an important difference. If one county’s rate is 1% higher than another county’s rate, a statistical test could tell you if that difference is significant. But it will not tell you if it’s worth acting on.

Comparing Rates Against a Target Value

Sometimes rates are compared against a chosen target, such as a goal or limit.

First, calculate the 95% confidence interval for the observed rate. See Confidence Intervals for an Age-adjusted Rate (PDF) for details. If the target value is not in the confidence interval, then the difference is statistically significant.

Simple Comparison of Two Rates

The simplest way to compare two rates or percentages is to calculate the [95% confidence interval] for each. See Calculating Reliable Rates for details. If those two intervals do not overlap, then the difference is statistically significant.

This is a convservative test. That means the chance of falsely finding a difference is smaller than 5%. More accurate tests are described below.

Flowchart for comparing rates and percentages

Process for Choosing a Test

  1. What type of rates do you have?
  2. Choose a test:
  3. Are the rates independent? See below for the definition of independence.

Dependent Versus Indepedent

Two rates are dependent if the value of one affects the value of the other. A common case of dependent rates is when some events are included in both. For example, a county rate and the state rate are dependent, because all cases in that county were used for both.