Home Perceptions of Justice: Assessing the Perceived Effectiveness of Punishments by Artificial Intelligence versus Human Judges
Article
Licensed
Unlicensed Requires Authentication

Perceptions of Justice: Assessing the Perceived Effectiveness of Punishments by Artificial Intelligence versus Human Judges

  • Gilles Grolleau , Murat C. Mungan and Naoufel Mzoughi ORCID logo EMAIL logo
Published/Copyright: February 14, 2025
Become an author with De Gruyter Brill

Abstract

Using an original experimental survey, we analyze how people perceive punishments generated by artificial intelligence (AI) compared to the same punishments generated by a human judge. We use two vignettes pertaining to two different albeit relatively common illegal behaviors, namely not picking up one’s dog waste on public roads and setting fire in dry areas. In general, participants perceived AI judgements as having a larger deterrence effect compared to the those rendered by a judge. However, when we analyzed each scenario separately, we found that the differential effect of AI is only significant in the first scenario. We discuss the implications of these findings.

JEL Classification: C9; K49

Corresponding author: Naoufel Mzoughi, INRAE, Ecodéveloppement, Avignon, France, E-mail: 

Appendix A. Survey Translation

A.1 Anonymous Survey

In the following, we present a hypothetical scenario. We invite you to read it carefully and answer the questions. There are no true or false answers: we are only interested in your honest opinion.

Treatment 1: For health reasons, dog waste is prohibited on sidewalks and public roads. For refusing to pick up his/her dog’s waste, the judicial institution, through a judge who analyzed the offense committed, sanctioned the guilty person to carry out 2 h of community service (cleaning of public spaces damaged by dog waste and other dirt).

Treatment 2: For health reasons, dog waste is prohibited on sidewalks and public roads. For refusing to pick up his/her dog’s waste, the judicial institution, through an artificial intelligence (AI) program which analyzed the offense committed, sanctioned the guilty person to carry out 2 h of community service (cleaning of public spaces damaged by dog waste and other dirt).

Treatment 3: Last summer, a person was convicted by a court for setting fire to dry grass in the middle of summer, using a lighter. Because of this infraction, 10 ha of a century-old forest went up in smoke. The court, through a judge who analyzed this offense, sentenced this person to 1 year in prison.

Treatment 4: Last summer, a person was convicted by a court for setting fire to dry grass in the middle of summer, using a lighter. Because of this infraction, 10 ha of a century-old forest went up in smoke. The court, through artificial intelligence (AI) program which analyzed this offense, sentenced this person to 1 year in prison.

Please assess whether this sanction discourages individuals from leaving their pet’s waste on public roads (T3, T4: setting fire to dry grass in the middle of summer):

1 not discouraging at all 2 3 4 5 6 7 very discouraging

To what extent does this sanction seem (un)just and (un)fait to you?

1 completely unjust and unfair 2 3 4 5 6 7 completely just and fair

In my opinion, not picking up your pet’s waste on sidewalks and public spaces (T3, T4: deliberately setting fire to dry grass) is an act:

1 not serious at all 2 3 4 5 6 7 very serious

Generally speaking, people seek to justify themselves by finding excuses for their actions (e.g. other serious problems at the same time):

1 fully disagree 2 3 4 5 6 7 fully agree

Please complete the following information:

Appendix B. Mean Values for the Variables used in Estimations

Variables Scenario 1 (dog waste) Scenario 2 (setting fire)
Treatment 1 Treatment 2 Treatment 3 Treatment 4
(judge; N = 49) (AI; N = 44) (judge; N = 65) (AI; N = 51)
Deterrence 3.837 5.068 4.369 3.941
Fair 5.571 5.614 4.846 4.490
Seriousness 5.082 4.909 5.846 5.961
Justification 4.898 4.409 5.108 5.216
Age (continuous) 29.428 25.909 30.461 43.412
Gender (=1 if female) 0.571 0.682 0.415 0.725
Education Cat. 1 (ref) 0.041 0.045 0.061 0.059
Cat. 2 0.571 0.364 0.338 0.196
Cat. 3 0.388 0.477 0.477 0.647
Income Cat. 1 (ref) 0.306 0.409 0.169 0.333
Cat. 2 0.204 0.341 0.123 0.118
Cat. 3 0.449 0.136 0.385 0.216
Cat. 4 0.041 0.114 0.323 0.333
  1. Education categories: baccalaureate or less, 1–3 years of university studies, and 4 years or more of university studies. Income categories: ≤€800/month, €801–€1,300/month, €1,301–€2,300/month, and ≥€2,301/month.

Appendix C. Linear Regression Estimates of the Effect of AI on Deterrence by Treatment

Variables Model 1 (without interaction) Model 2 (with interaction)
Scenario 1 (setting fire) (dog waste) (setting fire)
(dog waste) (setting fire) (dog waste) (setting fire)
Treatment Judge (ref)
AI 1.131*** (0.315) −0.275 (0.394) 2.332 (1.616) −2.053 (1.710)
Fair 0.158 (0.104) 0.377*** (0.089) 0.144 (0.107) 0.380*** (0.089)
Seriousness 0.023 (0.118) 0.012 (0.128) 0.102 (0.211) −0.005 (0.173)
Justification 0.251*** (0.081) −0.048 (0.094) 0.327** (0.126) −0.177 (0.133)
Age (continuous) −0.011 (0.015) −0.001 (0.013) −0.009 (0.016) −0.003 (0.013)
Gender (=1 if female) 0.093 (0.305) −0.175 (0.331) −0.046 (0.312) −0.100 (0.335)
Education Cat. 1 (ref)
Cat. 2 −0.358 (0.565) 0.769 (0.480) −0.312 (0.572) 0.694 (0.495)
Cat. 3 −0.859 (0.536) 0.650 (0.449) −0.831 (0.541) 0.608 (0.449)
Income Cat. 1 (ref)
Cat. 2 0.371 (0.397) −0.422 (0.572) 0.405 (0.403) −0.465 (0.573)
Cat. 3 −0.345 (0.414) −0.340 (0.467) −0.374 (0.420) −0.402 (0.469)
Cat. 4 0.184 (0.634) 0.344 (0.573) 0.315 (0.657) 0.445 (0.577)
Treatment##Seriousness −0.110 (0.258) 0.063 (0.246)
Treatment##Justification −0.138 (0.177) 0.267 (0.190)
Constant 1.364 (1.283) 3.177** (1.522) 1.731 (1.654) 3.141** (1.272)
Observations 93 116 93 116
F 3.74*** 2.61*** 3.17*** 2.38***
R 2 0.3368 0.2163 0.3432 0.2325
  1. Education categories: baccalaureate or less, 1–3 years of university studies, and 4 years or more of university studies. Income categories: ≤€800/month, €801–€1,300/month, €1,301–€2,300/month, and ≥€2,301/month. *** and ** Refer to significance at the levels of 1 % and 5 % respectively. The values between brackets correspond to Standard Errors.

Appendix D. Testing the Effect of AI in Interaction with Age and Gender (Standard Errors Between Brackets)

Variables Interaction AI##Age Interaction AI##Gender
Treatment Judge (ref)
AI 1.236** (0.563) 0.616* (0.363)
Fair 0.317*** (0.065) 0.319*** (0.065)
Seriousness 0.004 (0.085) −0.008 (0.085)
Justification 0.095 (0.063) 0.083 (0.063)
Age (continuous) 0.000 (0.014) −0.015* (0.009)
Gender (=1 if female) −0.088 (0.231) −0.034 (0.298)
Education Cat. 1 (ref)
Cat. 2 0.218 (0.354) 0.253 (0.355)
Cat. 3 −0.078 (0.337) −0.048 (0.338)
Income Cat. 1 (ref)
Cat. 2 0.376 (0.330) 0.390 (0.332)
Cat. 3 −0.089 (0.313) −0.108 (0.315)
Cat. 4 0.826** (0.393) 0.823** (0.395)
Treatment##Age −0.024 (0.016)
Treatment##Gender −0.237 (0.463)
Constant 1.774** (0.838) 2.330*** (0.749)
Observations 209 209
F 3.70*** 3.50***
R 2 0.1846 0.1765
  1. Education categories: baccalaureate or less, 1–3 years of university studies, and 4 years or more of university studies. Income categories: ≤€800/month, €801–€1,300/month, €1,301–€2,300/month, and ≥€2,301/month. ***, ** and *Refer to significance at the levels of 1 %, 5 % and 10 % respectively.

Appendix E. Testing the Interaction Between Age and Fairness (Dep. Variable: Deterrence)

Variables Coefficients and significance
Treatment Judge (ref)
AI 0.522** (0.236)
Fair 0.035 (0.161)
Seriousness −0.033 (0.085)
Justification 0.094 (0.062)
Age (continuous) −0.058** (0.023)
Fair##Age 0.008* (0.004)
Gender (=1 if female) −0.107 (0.229)
Education Cat. 1 (ref)
Cat. 2 0.228 (0.351)
Cat. 3 −0.129 (0.337)
Income Cat. 1 (ref)
Cat. 2 0.386 (0.329)
Cat. 3 −0.139 (0.312)
Cat. 4 0.925** (0.395)
Constant 3.957*** (1.116)
Observation 209
F 3.85***
R 2 0.1908
  1. Education categories: baccalaureate or less, 1–3 years of university studies, and 4 years or more of university studies. Income categories: ≤€800/month, €801–€1,300/month, €1,301–€2,300/month, and ≥€2,301/month. ***, ** and *Refer to significance at the levels of 1 %, 5 % and 10 % respectively. The values between brackets correspond to Standard Errors.

References

Bagaric, M., and G. Wolf. 2017. “Sentencing by Computer: Enhancing Sentencing Transparency and Predictability and (Possibly) Bridging the Gap between Sentencing Knowledge and Practice.” George Mason Law Review 25 (3): 653–710.Search in Google Scholar

Bagaric, M., D. Hunter, and N. Stobbs. 2019. “Erasing the Bias against Using Artificial Intelligence to Predict Future Criminality: Algorithms Are Color Blind and Never Tire.” University of Cincinnati Law Review 88 (4): 1037–81.Search in Google Scholar

Bagaric, M., J. Svilar, M. Bull, D. Hunter, and N. Stobbs. 2022. “The Solution to the Pervasive Bias and Discrimination in the Criminal Justice System: Transparent and Fair Artificial Intelligence.” American Criminal Law Review 59 (1): 95–148.Search in Google Scholar

Bénabou, R., and J. Tirole. 2006. “Incentives and Prosocial Behavior.” The American Economic Review 96 (5): 1652–78. https://doi.org/10.1257/aer.96.5.1652.Search in Google Scholar

Boeri, M., and A. K. Lamonica. 2015. “Sampling Designs and Issues in Qualitative Criminology.” In The Routledge Handbook of Qualitative Criminology, edited by Heith Copes, and J. Mitchell Miller, 125–43. Routledge. Chapter 9.Search in Google Scholar

Camerer, C. F., and R. M. Hogarth. 1999. “The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework.” Journal of Risk and Uncertainty 19: 7–42. https://doi.org/10.1023/a:1007850605129.10.1023/A:1007850605129Search in Google Scholar

Chandran, R. 2022. As Malaysia Tests AI Court Sentencing, Some Lawyers Fear for Justice. Reuters. https://www.reuters.com/article/idUSL8N2HD3V7/ (accessed April 12, 2024).Search in Google Scholar

Chen, B. M., A. Stremitzer, and K. Tobia. 2022. “Having Your Day in Robot Court.” Harvard Journal of Law and Technology 36 (1): 127–69.Search in Google Scholar

El Harbi, S., I. Bekir, G. Grolleau, and A. Sutan. 2015. “Efficiency, Equality, Positionality: What Do People Maximize? Experimental vs. Hypothetical Evidence from Tunisia.” Journal of Economic Psychology 47: 77–84. https://doi.org/10.1016/j.joep.2015.01.007.Search in Google Scholar

Fluet, C., and M. C. Mungan. 2022. “Laws and Norms with (Un) Observable Actions.” European Economic Review 145: 104129. https://doi.org/10.1016/j.euroecorev.2022.104129.Search in Google Scholar

Fluet, C., and M. C. Mungan. 2024. “Informational Properties of Liability Regimes.” Journal of Legal Studies. https://www.ssrn.com/abstract=4718793.Search in Google Scholar

Granulo, A., C. Fuchs, and S. Puntoni. 2021. “Preference for Human (Vs. Robotic) Labor Is Stronger in Symbolic Consumption Contexts.” Journal of Consumer Psychology 31 (1): 72–80. https://doi.org/10.1002/jcpy.1181.Search in Google Scholar

Grolleau, G., M. C. Mungan, and N. Mzoughi. 2022. “Seemingly Irrelevant Information? The Impact of Legal Team Size on Third Party Perceptions.” International Review of Law and Economics 71: 106068. https://doi.org/10.1016/j.irle.2022.106068.Search in Google Scholar

Hutton, N. 1995. “Sentencing, Rationality, and Computer Technology.” Journal of Law and Society 22: 549. https://doi.org/10.2307/1410614.Search in Google Scholar

Kleinberg, J., J. Ludwig, S. Mullainathan, and C. R. Sunstein. 2018. “Discrimination in the Age of Algorithms.” Journal of Legal Analysis 10: 113–74. https://doi.org/10.1093/jla/laz001.Search in Google Scholar

Kocsis, R. 2002. “Arson: Exploring Motives and Possible Solutions.” Trends & Issues in Crime & Criminal Justice, Australian Institute of Criminology 236: 1–6.Search in Google Scholar

Krupka, E. L., and R. A. Weber. 2013. “Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary?” Journal of the European Economic Association 11 (3): 495–524. https://doi.org/10.1111/jeea.12006.Search in Google Scholar

Krupnikov, Y., H. H. Nam, H. Style, J. N. Druckman, and D. P. Green. 2021. “Convenience Samples in Political Science Experiments.” In Advances in Experimental Political Science, edited by James N. Druckman, and Donald P. Green, 165–83. Cambridge University Press. Chapter 9.10.1017/9781108777919.012Search in Google Scholar

Lando, H., and M. C. Mungan. 2018. “The Effect of Type-1 Error on Deterrence.” International Review of Law and Economics 53: 1–8. https://doi.org/10.1016/j.irle.2017.08.001.Search in Google Scholar

Maguire, E. R., B. V. Lowrey, and D. Johnson. 2017. “Evaluating the Relative Impact of Positive and Negative Encounters with Police: A Randomized Experiment.” Journal of Experimental Criminology 13: 367–91. https://doi.org/10.1007/s11292-016-9276-9.Search in Google Scholar

Malek, M. A. 2022. “Criminal Courts’ Artificial Intelligence: The Way it Reinforces Bias and Discrimination.” AI and Ethics 2 (1): 233–45. https://doi.org/10.1007/s43681-022-00137-9.Search in Google Scholar

Mentzakis, E., and J. Sadeh. 2021. “Experimental Evidence on the Effect of Incentives and Domain in Risk Aversion and Discounting Tasks.” Journal of Risk and Uncertainty 62: 203–24. https://doi.org/10.1007/s11166-021-09354-9.Search in Google Scholar

Mullinix, K. J., T. J. Leeper, J. N. Druckman, and J. Freese. 2015. “The Generalizability of Survey Experiments.” Journal of Experimental Political Science 2 (2): 109–38.10.1017/XPS.2015.19Search in Google Scholar

Mungan, M. C. 2017. “Reducing Crime through Expungements.” Journal of Economic Behavior & Organization 137: 398–409. https://doi.org/10.1016/j.jebo.2017.03.021.Search in Google Scholar

Mungan, M. C. 2018. “Statistical (And Racial) Discrimination, “Ban the Box”, and Crime Rates.” American Law and Economics Review 20 (2): 512–35. https://doi.org/10.1093/aler/ahy008.Search in Google Scholar

Mungan, M. C., and J. Klick. 2015. “Identifying Criminals’ Risk Preferences.” Indiana Law Journal 91: 791.10.2139/ssrn.2567048Search in Google Scholar

Paternoster, R., R. Brame, R. Bachman, L. W. Sherman, S. Law, S. Review, and L. W. Sherman. 1997. “Do Fair Procedures Matter? The Effect of Procedural Justice on Spouse Assault.” Law & Society Review 31 (1): 163–204. https://doi.org/10.2307/3054098.Search in Google Scholar

Pillsbury, S. H. 1988. “Emotional Justice: Moralizing the Passions of Criminal Punishment.” Cornell Law Review 74 (4): 655–710.Search in Google Scholar

Polinsky, A. M., and S. Shavell. 1999. “On the Disutility and Discounting of Imprisonment and the Theory of Deterrence.” Journal of Legal Studies 28 (1): 1–16. https://doi.org/10.1086/468044.Search in Google Scholar

Pratt, T. C., and J. J. Turanovic. 2018. “Celerity and Deterrence.” In In Deterrence, Choice, and Crime, Vol. 23, 187–210. Routledge.Search in Google Scholar

Prescott, J. J., and S. B. Starr. 2019. “Expungement of Criminal Convictions: An Empirical Study.” Harvard Law Review 133: 2460.10.2139/ssrn.3353620Search in Google Scholar

Rasmusen, E. 1996. “Stigma and Self-Fulfilling Expectations of Criminality.” The Journal of Law and Economics 39 (2): 519–43. https://doi.org/10.1086/467358.Search in Google Scholar

Rizer, A., and C. Watney. 2018. “Artificial Intelligence Can Make Our Jail System More Efficient, Equitable, and Just.” Texas Review of Law & Politics 23: 181.10.2139/ssrn.3129576Search in Google Scholar

Rubinstein, A. 2001. “A Theorist’s View of Experiments.” European Economic Review 45 (4–6): 615–28. https://doi.org/10.1016/s0014-2921(01)00104-0.Search in Google Scholar

Ryberg, J. 2024. “Criminal Justice and Artificial Intelligence: How Should We Assess the Performance of Sentencing Algorithms?” Philosophy & Technology 37 (1): 9. https://doi.org/10.1007/s13347-024-00694-3.Search in Google Scholar

Sabia, J. J., T. T. Nguyen, T. Mackay, and D. Dave. 2021. “The Unintended Effects of Ban-The-Box Laws on Crime.” The Journal of Law and Economics 64 (4): 783–820. https://doi.org/10.1086/715187.Search in Google Scholar

Sanghvi, H., J. S. W. Ling, E. S. Tay, and C. Y. Kuek. 2022. “Digitalisation of Judiciary in Malaysia: Application of Artificial Intelligence in the Sentencing Process.” In International Conference on Law and Digitalization (ICLD 2022), 91–7. Atlantis Press.10.2991/978-2-494069-59-6_9Search in Google Scholar

Sourdin, T. 2018. “Judge V Robot? Artificial Intelligence and Judicial Decision-Making.” The University of New South Wales Law Journal 41 (4): 1114–33. https://doi.org/10.53637/zgux2213.Search in Google Scholar

Stobbs, N., D. Hunter, and M. Bagaric. 2017. “Can Sentencing Be Enhanced by the Use of Artificial Intelligence?” Criminal Law Journal 41 (5): 261–77.Search in Google Scholar

Susskind, R. 2000. Transforming the Law: Essays on Technology, Justice and the Legal Marketplace. Oxford: Oxford University Press.10.1093/oso/9780198299226.001.0001Search in Google Scholar

Tashea, J. 2017. “Courts Are Using AI to Sentence Criminals. That Must Stop Now.” Wired. https://www.wired.com/2017/04/courts-using-ai-sentence-criminals-must-stop-now/ (accessed April 17, 2024).Search in Google Scholar

Tyler, T. R. 1988. “What Is Procedural Justice-Criteria Used by Citizens to Assess the Fairness of Legal Procedures.” Law & Society Review 22: 103–36. https://doi.org/10.2307/3053563.Search in Google Scholar

Tyler, T. 2008. “Psychology and Institutional Design.” Review of Law & Economics 4 (3): 801–87. https://doi.org/10.2202/1555-5879.1233.Search in Google Scholar

Underhill, K. 2019. “Price and Prejudice: An Empirical Test of Financial Incentives, Altruism, and Racial Bias.” Journal of Legal Studies 48: 245–74. https://doi.org/10.1086/707010.Search in Google Scholar

Verboon, P., and M. van Dijke. 2011. “When Do Severe Sanctions Enhance Compliance? The Role of Procedural Fairness.” Journal of Economic Psychology 32 (1): 120–30. https://doi.org/10.1016/j.joep.2010.09.007.Search in Google Scholar

van Wingerden, S., and M. Plesničar. 2022. “Artificial Intelligence and Sentencing: Humans against Machines.” In Sentencing and Artificial Intelligence Studies in Penal Theory and Philosophy, edited by J. Ryberg, and J. Roberts, 230–51.10.1093/oso/9780197539538.003.0012Search in Google Scholar

Xu, Z. 2022. “Human Judges in the Era of Artificial Intelligence: Challenges and Opportunities.” Applied Artificial Intelligence 36 (1): 2013652. https://doi.org/10.1080/08839514.2021.2013652.Search in Google Scholar

Yasrebi-De Kom, F. M., A. J. Dirkzwager, P. H. Van Der Laan, and P. Nieuwbeerta. 2022. “The Effect of Sanction Severity and its Interaction with Procedural Justice.” Criminal Justice and Behavior 49 (2): 200–19. https://doi.org/10.1177/00938548211038358.Search in Google Scholar

Received: 2024-04-07
Accepted: 2024-12-22
Published Online: 2025-02-14

© 2025 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 21.10.2025 from https://www.degruyterbrill.com/document/doi/10.1515/rle-2024-0049/html
Scroll to top button