Week 3: RCTs and Their Critics

Instructions

Critics of RCTs voice concerns about the ethics of ‘randomizing policies’ and raise doubts about the ability of RCTs to inform policy making. We look at both of these critics this week. After watching Nancy Cartwright’s 5-minute teaser video, read her paper with noble prize winner Angus Deaton. What is their main concern with RCTs? Next, read the paper by Dawn Langan Teele on the ethnics of RCTs. Are their concerns the same as those of Chris Pottas when he critized an RCT in Nairobi’s slums by Coville, Galiani, Gertler, and Yoshida?

Required readings

• Deaton, A. and N. Cartwright. “Understanding and Misunderstanding Randomized Controlled Trials”. In: Social Science & Medicine 210 (2018), pp. 2-21. DOI: 10.1016/j.socscimed.2017.12.005.

• Teele, D. L. “Reflections on the Ethics of Field Experiments”. In: Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences. Ed. by D. L. Teele. Yale University Press, 2014. DOI: 10.12987/9780300199307-007.

• Coville, A., S. Galiani, P. Gertler, et al. Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums. Working Paper 27569. National Bureau of Economic Research, 2020. DOI: 10.3386/w27569.

  • The authors. A Comment on the Ethical Issues Twitter Discussion in Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums.
  • Prottas, Chris. Commentary on Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums.

Further reading (Ethics)

• Phillips, T. “Ethics of Field Experiments”. In: Annual Review of Political Science 24 (2021), pp. 277-300. DOI: 10.1146/annurev-polisci-041719-101956.
• Johnson, J. B. “Protecting the Community: Lessons From the Montana Flyer Project”. In: PS: Political Science & Politics 51.3 (2018), pp. 615-619. DOI: 10.1017/s1049096518000021.
• Kramer, A. D., J. E. Guillory, and J. T. Hancock. “Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks”. In: Proceedings of the National Academy of Sciences 111.24 (2014), pp. 8788-8790. DOI: 10.1073/pnas.1320040111.
• Bond, R. M., C. Fariss, J. J. Jones, et al. “A 61-Million-Person Experiment in Social Influence and Political Mobilization”. In: Nature 489 (2012), pp. 295-298. DOI: 10.1038/nature11421.

Further reading (Generalizability)

• Cartwright, N. and J. Hardie. Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press, 2012. DOI: 10.5860/choice.50-5831.

• Dunning, T., G. Grossman, M. Humphreys, et al. Information, Accountability, and Cumulative Learning: Lessons from Metaketa I. Cambridge: Cambridge University Press, 2019. DOI: 10.1017/9781108381390.

• Various autors. 2018. Commentary on Deaton and Cartwright. Social Science & Medicine, 210:26–85, 2018.

Suggested media

• A Generation of Bad Blood, The Atlantic, June 17, 2016

Acknowledgments

Photo credit: https://unsplash.com/photos/n1QlAgX3zZ8.