The myths and realities of racial profiling

Over the years, I heard many claims about racial profiling in traffic stops by police. But how much of racial profiling actually exists now? As it turns out, racial profiling in traffic stops is either small or nonexistent at the level of police departments, although a very small proportion of individual officers do engage in profiling. It is also interesting how different statistical techniques can be used to come to very different conclusions in a highly charged political debate.
The recent findings on this topic are described in a refreshingly deep presentation: Racial Profiling Analysis in a Post-Beer Summit World (video, 1 hour, of which the last 30 min is the optional Q&A session). It is given by Greg Ridgeway (Director, RAND Corporation Safety and Justice Program; Director, RAND Center on Quality Policing) at the Google TechTalks series.
As it turns out, it is easy to use poor statistical methodology to show that racial profiling exists. All you have to do is to show that the proportion of blacks among all people pulled over by the police is higher that the proportion of blacks in the population. But this does not address many confounding variables, such as the rate of traffic violations by race, the rate of policing in different neighborhoods, and others. Greg Ridgeway and colleagues used proper statistical methods to control for these variables, and found interesting results.

In the first study, the researchers measured race distribution of the drivers stopped by police in the hours before sunset (when you can see the race of the driver) and after sunset (when you cannot). Because the researchers found no difference, they concluded that there was no evidence of racial profiling in the data. This study, Testing for Racial Profiling in Traffic Stops From Behind a Veil of Darkness, used the police data from Oakland, CA. It was later replicated in Cincinnati, OH.

In the second study, the researchers analyzed pedestrian stops by police officers. When the stops were matched by most of the important variables, such as the neighborhood, time, nature of the assignment, etc, racial profiling disappeared. That is, except for 15 out of 3,000 officers, who were flagged for follow up internal investigation. The police actually prefer finding out about these cases early, before they hit the six o'clock news. This study, Doubly Robust Internal Benchmarking and False Discovery Rates for Detecting Racial Bias in Police Stops, used the police data from New York Police Department.

In the third study, the researchers analyzed the stop time. It had been widely reported that when police stop black drivers, the stops last longer than those for white drivers. However, the distribution of other confounding variables was different among the races, for example, black drivers had a much higher incidence of not having a valid drivers license. When the stops of black and white drivers were matched for most of these variables, the majority (80%) of the difference in stop duration disappeared.

This case of disappearing racial profiling shows how important proper data analysis is in political discourse.

Footnotes:
You can find more on this topic in The decline of racial profiling (an article co-authored by Greg Ridgeway).
You can find more info on Greg Ridgeway's employer, the RAND Corporation, in Soldiers of Reason: The RAND Corporation and the Rise of the American Empire by Alex Abella and in other sources.