The Democratic Virtues of Randomized Trials

References

Administrative Procedure Act, 5 U.S.C § 500 et seq. Also available at: https://www.law.cornell.edu/uscode/text/5/part-I.Search in Google Scholar

Amar, A. R. 1984. “Choosing Representatives by Lottery Voting.” The Yale Law Journal 93 (7): 1283–308. https://doi.org/10.2307/796258.Search in Google Scholar

American Economic Association (AEA). 2021. RCT Registry: Registry for Randomized Controlled Trials. Also available at: https://www.aeaweb.org/journals/policies/rct-registry.Search in Google Scholar

Backmann, M. 2017. “What’s in a Gold Standard? In Defense of Randomized Controlled Trials.” Medical Health Care and Philosophy 20: 513–23. https://doi.org/10.1007/s11019-017-9773-2.Search in Google Scholar

Banerjee, A., D. Karlan, and J. Zinman. 2015. “Six Randomized Evaluations of Microcredit: Introduction and Further Steps.” American Economic Journal: Applied Economics 7 (1): 1–21. https://doi.org/10.1257/app.20140287.Search in Google Scholar

Bedock, C., and J.-B. Pilet. 2021. “Who Supports Citizens Selected by Lot to Be Main Policymakers? A Study of French Citizens.” Government and Opposition 56: 485–504. https://doi.org/10.1017/gov.2020.1.Search in Google Scholar

Bijwaard, G. E., and G. Ridder. 2005. “Correcting for Selective Compliance in a Re-employment Bonus Experiment.” Journal of Econometrics 125 (1–2): 77–111. https://doi.org/10.1016/j.jeconom.2004.04.004.Search in Google Scholar

Bloom, H. S., C. Michalopoulos, C. J. Hill, and Y. Lei. 2002. Can Nonexperimental Comparison Group Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-To-Work Programs? In MDRC Working Papers on Research Methodology.Search in Google Scholar

Braybrooke, D., and C. E. Lindblom. 1963. A Strategy for Decision. New York: Free Press.Search in Google Scholar

Burnheim, J. 1985. Is Democracy Possible? The Alternative to Electoral Democracy. Cambridge: Polity Press.Search in Google Scholar

Burtless, G. 1995. “The Case for Randomized Field Trials in Economic and Policy Research.” The Journal of Economic Perspectives 9 (2): 63–84. https://doi.org/10.1257/jep.9.2.63.Search in Google Scholar

Callenbach, E., and M. Phillips. 1985. A Citizen Legislature. Berkeley: Banyan Tree books.Search in Google Scholar

Campbell, D. T. 1969. “Reforms as Experiments.” American Psychologist 24 (4): 409–29. https://doi.org/10.1037/h0027982.Search in Google Scholar

Campbell, D. T. 1998. “The Experimenting Society.” In The Experimenting Society: Essays in Honor of Donald, T. Campbell, Policy Studies Review Annual, Vol. 11, edited by W. Dunn, 35–68. New Brunswick: Transaction Publishers.10.4324/9781003578192-2Search in Google Scholar

Cavalcanti, A. B., F. G. Zampieri, R. G. Rosa, L. C. Azevedo, V. C. Veiga, A. Avezum, L. P. Damiani, A. Marcadenti, L. Kawano-Dourado, T. Lisboa, D. L. Junqueira. 2020. “Hydroxychloroquine with or without Azithromycin in Mild-To-Moderate Covid-19.” New England Journal of Medicine 383 (21): 2041–52. https://doi.org/10.1056/nejmoa2019014.Search in Google Scholar

Chang, E. H., K. L. Milkman, D. M. Gromet, R. W. Rebele, C. Massey, A. L. Duckworth, and A. M. Grant. 2019. “The Mixed Effects of Online Diversity Training.” Proceedings of the National Academy of Sciences 116 (16): 7778–83. https://doi.org/10.1073/pnas.1816076116.Search in Google Scholar

Chetty, R., N. Hendren, and L. F. Katz. 2016. “The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment.” The American Economic Review 106 (4): 855–902. https://doi.org/10.1257/aer.20150572.Search in Google Scholar

Cohen, J., and P. Dupas. 2010. “Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment.” Quarterly Journal of Economics 125: 1–45. https://doi.org/10.1162/qjec.2010.125.1.1.Search in Google Scholar

Corduneanu-Huci, C., M. T. Dorsch, and P. Maarek. 2021. “The Politics of Experimentation: Political Competition and Randomized Controlled Trials.” Journal of Comparative Economics 49 (1): 1–21. https://doi.org/10.1016/j.jce.2020.09.002.Search in Google Scholar

Deaton, A., and N. Cartwright. 2018. “Understanding and Misunderstanding Randomized Controlled Trials.” Social Science and Medicine 210: 2–21. https://doi.org/10.1016/j.socscimed.2017.12.005.Search in Google Scholar

Dewey, J. 2012. The Public and its Problems. University Park: Pennsylvania University Press.Search in Google Scholar

DiCenso, A., G. Guyatt, A. Willan, and L. Griffith. 2002. “Interventions to Reduce Unintended Pregnancies Among Adolescents: Systematic Review of Randomized Controlled Trials.” British Medical Journal 324 (7351): 1426. https://doi.org/10.1136/bmj.324.7351.1426.Search in Google Scholar

Dupas, P. 2009. “What Matters (And What Does Not) in Households’ Decision to Invest in Malaria Prevention?” The American Economic Review: Papers and Proceedings 99 (2): 224–30. https://doi.org/10.1257/aer.99.2.224.Search in Google Scholar

Dupas, P. 2014. “Short-run Subsidies and Long-Run Adoption of New Health Products: Evidence from a Field Experiment.” Econometrica 82 (1): 197–228. https://doi.org/10.3982/ECTA9508.Search in Google Scholar

Easterly, W. 2006. The White Man’s Burden: Why the West’s Efforts to Aid the Rest Have Done So Much Ill and So Little Good. New York: Penguin.Search in Google Scholar

Evidence Based Medicine Working Group. 1992. “Evidence-based Medicine: A New Approach to Teaching the Practice of Medicine.” The Journal of the American Medical Association 268: 2420–5. https://doi.org/10.1001/jama.1992.03490170092032.Search in Google Scholar

Ezrahi, Y. 1990. The Descent of Icarus: Science and the Transformation of Contemporary Democracy. Cambridge: Harvard University Press.Search in Google Scholar

Fuller, S. 1997. Science. Buckingham: Open University Press.Search in Google Scholar

Feinberg, J. 1987. Harm to Others: The Moral Limits of the Criminal Law. New York: Oxford University Press.10.1093/0195046641.001.0001Search in Google Scholar

Fiorina, M. P. 1981. Retrospective Voting in American National Elections. New Haven: Yale University Press.Search in Google Scholar

Finkelstein, A., S. Taubman, B. Wright, M. Bernstein, J. Gruber, J. P. Newhouse, H. Allen, and K. Baicker, and Oregon Health Study Group. 2012. “The Oregon Health Insurance Experiment: Evidence from the First Year.” Quarterly Journal of Economics 127 (3): 1057–106, https://doi.org/10.1093/qje/qjs020.Search in Google Scholar

Fishkin, J. S. 2009. When the People Speak. Oxford: Oxford University Press.Search in Google Scholar

Garfinkel, I., C. F. Manski, and C. Michalopoulos. 1992. “Micro Experiments and Macro Effects.” In Evaluating Welfare and Training Programs, edited by C. F. Manski, and I. Garfinkel, 253–76. Cambridge: Harvard University Press.Search in Google Scholar

Garry, J., J. Pow, J. Coakley, D. Farrell, B. O’Leary, and J. Tilley. 2022. “The Perception of the Legitimacy of Citizens’ Assemblies in Deeply Divided Places?” Evidence of Public and Elite Opinion from Consociational Northern Ireland.” Government and Opposition 57 (3): 532–51, https://doi.org/10.1017/gov.2021.4.Search in Google Scholar

Gautret, P., J. C. Lagier, P. Parola, L. Meddeb, M. Mailhe, B. Doudier, J. Courjon, V. Giordanengo, V. E. Vieira, H. T. Dupont, and S. Honoré. 2020. “Hydroxychloroquine and Azithromycin as a Treatment of COVID-19: Results of an Open-Label Non-randomized Clinical Trial.” International Journal of Antimicrobial Agents 56 (1): 105949. https://doi.org/10.1016/j.ijantimicag.2020.105949.Search in Google Scholar

Gleason, P. M., A. M. Resch, and J. A. Berk. 2012. Replicating Experimental Impact Estimates Using a Regression Discontinuity Approach. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. NCEE Reference Report 2012–4025.Search in Google Scholar

Glewwe, P., and K. Muralidharan. 2016. “Improving Education Outcomes in Developing Countries—Evidence, Knowledge Gaps, and Policy Implications.” In Handbook of the Economics of Education, Vol. 5, edited by E. Hanushek, S. Machin, and L. Woessman, 653–744. Amsterdam: North Holland.10.1016/B978-0-444-63459-7.00010-5Search in Google Scholar

Global Commission on Evidence to Address Societal Challenges. 2022. The Evidence Commission report: A wake-up call and path forward for decisionmakers, evidence intermediaries, and impact-oriented evidence producers. Hamilton: McMaster Health Forum.Search in Google Scholar

Goldberg, S., and A. Bächtiger. 2021. “Catching the ‘Deliberative Wave’? How (Disaffected) Citizens Assess Deliberative Citizen forums.” Draft Manuscript.10.1017/S0007123422000059Search in Google Scholar

Goodin, R. E. 1982. Political Theory and Public Policy. Chicago: University of Chicago Press.Search in Google Scholar

Goodin, R. E. 1995. Utilitarianism as a Public Philosophy. Cambridge: Cambridge University Press.10.1017/CBO9780511625053Search in Google Scholar

Goodin, R. E. 2004. “Support with Strings: Workfare as an ‘Impermissible Condition.” Journal of Applied Philosophy 21 (4): 297–308. https://doi.org/10.1111/j.0264-3758.2004.00283.x.Search in Google Scholar

Gurr, T. R. 1974. “Persistence and Change in Political Systems, 1800–1971.” American Political Science Review 68 (4): 1482–504. https://doi.org/10.2307/1959937.Search in Google Scholar

Hand, B. L. 1944. The Spirit of Liberty. Speech Delivered in Central Park. New York. (accessed May 21, 1944).Search in Google Scholar

International Committee of Medical Journal Editors (ICMJE). 2021. Clinical trials. Also available at: www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html.Search in Google Scholar

James-Burdumy, S., M. Dynarski, and J. Deke. 2008. “After-school Program Effects on Behavior: Results from the 21st Century Community Learning Centers Program National Evaluation.” Economic Inquiry 46 (1): 13–8. https://doi.org/10.1111/j.1465-7295.2007.00074.x.Search in Google Scholar

James-Burdumy, S., M. Dynarski, M. Moore, L. Rosenberg, J. Deke, W. Mansfield, and E. Warner. 2004. When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program: Final Report. US Department of Education, National Center for Education Evaluation and Regional Assistance.Search in Google Scholar

Jones, D., D. Molitor, and J. Reif. 2019. “What Do Workplace Wellness Programs Do? Evidence from the Illinois Workplace Wellness Study.” Quarterly Journal of Economics 134 (4): 1747–91. https://doi.org/10.1093/qje/qjz023.Search in Google Scholar

Joyce, T. J., S. Crockett, D. A. Jaeger, O. Altindag, and S. D. O’Connell. 2014. Does Classroom Time Matter? A Randomized Field Experiment of Hybrid and Traditional Lecture Formats in Economics. In NBER Working Paper 20006. Cambridge: National Bureau of Economic Research.10.3386/w20006Search in Google Scholar

Key, V. O. 1966. The Responsible Electorate: Rationality in Presidential Voting, 1936-1960. Cambridge: Harvard University Press.10.4159/harvard.9780674497764Search in Google Scholar

King, A., and I. Crewe. 2013. The Blunders of Our Governments. London: Oneworld.Search in Google Scholar

Kuhn, T. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.Search in Google Scholar

Lafont, C. 2019. Democracy without Shortcuts. Oxford: Oxford University Press.10.1093/oso/9780198848189.001.0001Search in Google Scholar

Landemore, H. 2013. “Deliberation, Cognitive Diversity, and Democratic Inclusiveness: An Epistemic Argument for the Random Selection of Representatives.” Synthese 190 (7): 1209–31. https://doi.org/10.1007/s11229-012-0062-6.Search in Google Scholar

Leigh, A. 2018. Randomistas: How Radical Researchers Are Changing Our World. New Haven: Yale University Press.10.12987/9780300240115Search in Google Scholar

McKee, M. 2007. “Cochrane on Communism: The Influence of Ideology on the Search for Evidence.” International Journal of Epidemiology 36 (2): 269–73. https://doi.org/10.1093/ije/dym002.Search in Google Scholar

McKenzie, D., S. Stillman, and J. Gibson. 2010. “How Important Is Selection? Experimental vs. Non-experimental Measures of the Income Gains from Migration.” Journal of the European Economic Association 8 (4): 913–45. https://doi.org/10.1111/j.1542-4774.2010.tb00544.x.Search in Google Scholar

Meldrum, M. 1998. “‘A Calculated Risk’: The Salk Polio Vaccine Field Trials of 1954.” British Medical Journal 317 (7167): 1233–6. https://doi.org/10.1136/bmj.317.7167.1233.Search in Google Scholar

Mill, J. S. 1859. On Liberty. London: John W. Parker and Son.Search in Google Scholar

Moffitt, R. 1992. “Evaluation Methods for Program Entry Effects.” In Evaluating Welfare and Training Programs, edited by C. F. Mansky, and I. Garfinkel, 231–52. Cambridge: Harvard University Press.Search in Google Scholar

Ober, J. 2017. Demopolis: Democracy Before Liberalism in Theory and Practice. Cambridge: Cambridge University Press.10.1017/9781108226790Search in Google Scholar

OECD. 2019. Health at a Glance 2019: OECD Indicators. Paris: OECD Publishing.Search in Google Scholar

Omotilewa, O., and J. Ricker-Gilbert. 2019. “Impact Evaluation of Large-Scale Extension Activities on Agricultural Technology Adoption: Experimental vs. Non-experimental Estimates.” In 6th International Conference of African Association of Agricultural Economists (AAAE), Abuja, Nigeria, September 23–26, 2019.10.1257/rct.4895-1.0Search in Google Scholar

Pearce, W., and S. Raman. 2014. “The New Randomised Controlled Trials (RCT) Movement in Public Policy: Challenges of Epistemic Governance.” Policy Sciences 47: 387–402. https://doi.org/10.1007/s11077-014-9208-3.Search in Google Scholar

Petrosino, A., and C. Turpin-Petrosino. 2002. “Scared Straight’ and Other Juvenile Awareness Programs for Preventing Juvenile Delinquency.” (Updated C2 Review). In Reviews of Intervention and Policy Evaluations, (C2-RIPE). Philadelphia: Campbell Collaboration.10.1002/14651858.CD002796Search in Google Scholar

Popper, K. R. 1980. The Open Society and its Enemies, Vol. I. London: Routledge & Kegan Paul.Search in Google Scholar

Pow, J., L. van Dijk, and S. Marien. 2020. “It’s Not Just the Taking Part that Counts: ‘Like Me’ Perceptions Connect the Wider Public to Mini-Publics.” Journal of Deliberative Democracy 16 (2): 43–55. https://doi.org/10.16997/jdd.368.Search in Google Scholar

Przeworski, A.. 2015. “Acquiring the Habit of Changing Governments through Elections.” Comparative Political Studies 48: 101–29. https://doi.org/10.1177/0010414014543614.Search in Google Scholar

Rawls, J. 2005. Political Liberalism. New York: Columbia University Press.Search in Google Scholar

Rivlin, A. M. 1971. Systematic Thinking for Social Action. Washington: Brookings Institution.Search in Google Scholar

Sachs, J. 2009. Good News on Malaria Control. Scientific American.10.1038/scientificamerican0809-38Search in Google Scholar

Sachs, J. 2014. “The Case for Aid.” Foreign Policy (21 January).Search in Google Scholar

Sauermann, J., and A. Stenberg. 2020. “Assessing Selection Bias in Non-experimental Estimates of the Returns to Workplace Training.” In IZA Discussion Paper 13789. Bonn: IZA.10.2139/ssrn.3713630Search in Google Scholar

Scanlon, T. M. 1998. What We Owe to Each Other: Harvard University Press.Search in Google Scholar

Schmeck, H. M. 1995. Dr. Jonas Salk Whose Vaccine Turned Tide on Polio, Dies at 80, 24 June, 1. New York Times.Search in Google Scholar

Schumpeter, J. A. 1950. Capitalism, Socialism and Democracy, 3rd ed. New York: Harper & Row.Search in Google Scholar

Sen, A. 2002. “Open and Closed Impartiality.” Journal of Philosophy 99 (9): 445–69. https://doi.org/10.2307/3655683.Search in Google Scholar

Skipper, C. P., K. A. Pastick, N. W. Engen, A. S. Bangdiwala, M. Abassi, S. M. Lofgren, D. A. Williams, E. C. Okafor, M. F. Pullen, M. R. Nicol, and A. A. Nascene. 2020. “Hydroxychloroquine in Nonhospitalized Adults with Early COVID-19: A Randomized Trial.” Annals of Internal Medicine 173 (8): 623–31. https://doi.org/10.7326/m20-4207.Search in Google Scholar

St.Clair, T., T. D. Cook, and K. Hallberg. 2014. “Examining the Internal Validity and Statistical Precision of the Comparative Interrupted Time Series Design by Comparison with a Randomized Experiment.” American Journal of Evaluation 35 (3): 311–27. https://doi.org/10.1177/1098214014527337.Search in Google Scholar

Stone, P. 2007. “Why Lotteries Are Just.” The Journal of Political Philosophy 15 (3): 276–95. https://doi.org/10.1111/j.1467-9760.2006.00274.x.Search in Google Scholar

Stone, P. 2011. The Luck of the Draw: The Role of Lotteries in Decision Making. Oxford: Oxford University Press.10.1093/acprof:oso/9780199756100.003.0001Search in Google Scholar

Sunstein, C. R. 2017. “Cost-benefit Analysis and Arbitrariness Review.” Harvard Environmental Law Review 41 (1): 1–41.Search in Google Scholar

Taleb, N. 2007. The Black Swan: The Impact of the Highly Improbable. New York: Random House.Search in Google Scholar

Thaler, R. H., and C. R. Sunstein. 2009. Nudge. New York: Penguin.Search in Google Scholar

Underhill, K., P. Montgomery, and D. Operario. 2007. “Sexual Abstinence Only Programmes to Prevent HIV Infection in High Income Countries: Systematic Review.” British Medical Journal 335 (7613): 248–52. https://doi.org/10.1136/bmj.39245.446586.be.Search in Google Scholar

Vinkers, C. H., H. J. Lamberink, J. K. Tijdink, P. Heus, L. Bouter, P. Glasziou, D. Moher, J. A. Damen, L. Hooft, and W. M. Otte. 2021. “The Methodological Quality of 176,620 Randomized Controlled Trials Published between 1966 and 2018 Reveals a Positive Trend but Also an Urgent Need for Improvement.” PLoS Biology 19 (4): e3001162. https://doi.org/10.1371/journal.pbio.3001162.Search in Google Scholar

Warren, M. E. 2008. “Citizen Representatives.” In Designing Deliberative Democracy: The British Columbia Citizens’ Assembly. 50–69, edited by M. E. Warren, and H. Pearse. Cambridge: Cambridge University Press.10.1017/CBO9780511491177.004Search in Google Scholar

Wartolowska, K., A. Judge, S. Hopewell, G. S. Collins, B. J. Dean, I. Rombach, D. Brindley, J. Savulescu, D. J. Beard, and A. J. Carr. 2014. “Use of Placebo Controls in the Evaluation of Surgery: Systematic Review.” British Medical Journal 348: g3253. https://doi.org/10.1136/bmj.g3253.Search in Google Scholar

Werner, H., and S. Marien. 2022. “Process vs. Outcome? How to Evaluate the Effects of Participatory Processes on Legitimacy Perceptions.” British Journal of Political Science 51 (1): 429–36, https://doi.org/10.1017/S0007123420000459.Search in Google Scholar

Wildavsky, A. 1979. Speaking the Truth to Power. Boston: Little Brown & Co.Search in Google Scholar

White, H. 2019. “The Twenty-First Century Experimenting Society: The Four Waves of the Evidence Revolution.” Palgrave Communications 5 (1): 1–7. https://doi.org/10.1057/s41599-019-0253-6.Search in Google Scholar