Archives
-
Does Proactive Policing Really Increase Major Crime? A Replication Study of Sullivan and O’Keeffe (Nature Human Behaviour, 2017)
Vol. 3 No. 6 (2024)In December 2014 and January 2015, police officers in New York City engaged in an organized
slowdown of police work to protest the murder of two police officers who were targeted by a
gunman while sitting in their patrol car. An influential 2017 article in Nature Human Behaviour
studies the effect of the NYPD’s work slowdown on major crimes and concludes that the slowdown
led to a significant improvement in public safety. Contrary to the remainder of the literature, the
authors conclude that proactive policing can cause an increase in crime. We re-evaluate this claim
and point out several fatal weaknesses in the authors’ analysis — which purports to be a difference in-
differences analysis but isn’t — that call this finding into question. In particular, we note that
there was considerable variation in the intensity of the slowdown across NYC communities and
that the communities which experienced a more pronounced reduction in police proactivity did not
experience the largest reductions in major crime. The authors’ analysis constitutes a quintessential
fallacy in statistical reasoning, a logical miscalculation in which inferences from aggregated data are mistakenly applied to a more granular phenomenon. We raise several additional and equally
compelling concerns regarding the tests presented in the paper and conclude that there is little
evidence that the slowdown led to short-term changes in major crimes in either direction.
2 -
Drivers of Youth Outsiderness in European Labour Markets. A Comment on Marques and Salavisa (Socio-Economic Review, 2017)
Vol. 4 No. 3 (2025)Marques and Salavisa (2017) use fuzzy set qualitative comparative analysis (fsQCA) to analyze age-based labour market dualization in Southern European, Anglo-Saxon and a few Nordic countries. They argue that segmentation at the expense of young outsiders is driven by several factors in non-linear ways: deindustrialization, labour market coordination, employment protection, and liberalization can lead to youth outsiderness. We are able to replicate their analysis in technical terms, but argue that the analysis and the interpretation of its results are subject to technical misunderstandings. When correcting for these, we must call into question the study's results. To underpin our argument, we provide a hands-on discussion of how two measures of fit in fsQCA - the consistency and PRI scores of the sufficiency solution terms - are calculated. A good understanding of these allows the researcher to understand which cases and configurations drive the results, and thus facilitates technically correct decisions during the analysis and a better understanding of the results. We conjecture that the original paper cannot present conclusive evidence on the hypotheses it sought to test for a lack of sufficient variation in the empirical data sample.
-
Public Infrastructure and Economic Development: Evidence from Postal Systems A Replication Study of Rogowski et al. (American Journal of Political Science, 2022)
Vol. 4 No. 5 (2025)Rogowski et al. (2022) use secondary data to study the impact of historic postal infrastructure on
economic development, both cross-country and within the US. Their results suggest a large positive
effect of post offices on economic development that is robust across various sensitivity checks. We
successfully computationally reproduce all results. In a robustness assessment, we find the results to
be robust to simple changes in the analysis but observe some sensitivity to accounting for spatial
trends in the long-term cross-country analysis. Despite this, we find the results to be overall robust
given the numerous analyses and robustness checks in the original paper.
