The Impact of Artificial Intelligence Technologies on the Justice Administration and on the Judicial Office Personnel

1. Introduction

The fact that we live in a society based on, designed and structured by technology is a reality that no one is unaware of. Artificial Intelligence (AI) has been making its way among us for some time now, increasingly demanding a greater role. This reality invites us to make new reflections on issues that we thought we had overcome.

What would it be like to be judged by an AI, by a robot-judge, with a virtual face, created by hologram, when we are facing a possible sentence of deprivation of liberty? Can you imagine such a robotic face condemning us to the deprivation of our most sacred right?

If these questions had ever been asked of us, as litigants (as simple citizens who come to knock on the doors of the administration of justice), we would have thought that such a reality was far-fetched; more like something out of a Franz Kafka novel, than a possible reality^[1]. Even more so, after the lengths we have gone to in order to consolidate the rights and procedural guarantees we enjoy today. Such as, for example, the right to be judged respecting all the guarantees of due process (in short, the right to justice).

Moreover, justice, as an ideal, has always been understood (and is so stated in most western constitutions) as a moral principle and as a democratic value, which implies giving everyone their due. Furthermore, most of these constitutions state that justice emanates from the people and that it is administered by judges and magistrates who make up one of the powers of the state: the judiciary. Its legitimacy, as one of the powers of the state, therefore, derives from the sovereignty that resides in the people.

Thus, in the following text, we will try to assess to what extent it is acceptable to incorporate artificial intelligence tools in some of the functions performed by the justice administration. In this sense, we will analyze what type of AI functionalities are compatible with some of the functions carried out by the justice administrations (specifically, the work of the judicial office and the judging function). And, in this context, we will try to reflect on which of these applications would not be compatible with the rights and procedural guarantees that we hold today.

2. The unstoppable advance of artificial intelligence

AI has been defined in many ways, and if there is something (some element, feature or characteristic) on which all definitions agree, it is that AI is an advanced technology (computer systems, algorithms, etc.) that attempts to imitate human behavior (Barona Vilar 2021: 107). In short, it is a technology that has the potential to make machines perform tasks that previously could only be performed by humans, and, consequently, a technology capable of transforming many aspects of our lives (Nieva Fenoll 2018: 20).

Given the rapid technological development of AI, which will bring about a profound change in the way we relate to each other, more and more countries are investing heavily in this type of solutions and applications (Schwab 2016). Therefore, it becomes necessary to reflect on some of these AI systems that pose a high risk to the security and fundamental rights we enjoy today, including our right to effective judicial protection, or, in other words, our right to have justice done with all its guarantees.

In this unstoppable evolution and expansion of AI, the systems that are generating most concerns are those related to the so-called strong AI, within which we can find some high-risk AI systems and applications.

In this sense, it should be noted that strong AI can learn and apply knowledge to new cases. It is a type of technology with a machine learning capacity that can achieve autonomous solutions to the various situations that reality can pose. Therefore, it is an AI that can find solutions to new problems in the world on its own, based on its own knowledge and without human intervention or control, except for the one that has previously programmed the initial algorithm (Barona Vilar 2021: 108).

Within this strong AI we find the following categories (Inesdi Business Techschool 2023 website):

Firstly, machine learning. Machine learning is one of the types of artificial intelligence to which we are most accustomed, especially at the level of series or films. It is based on the ability of a software or device to learn on its own. Through algorithms and models, computers can learn from data and make predictions or judgements without being explicitly programmed. This process is known as machine learning (Burns 2021). In other words, without being specifically programmed to do so, machine learning is a subset of AI that allows software systems to improve their propensity and anticipate outcomes (Heath 2020).

Machine learning follows three fundamental steps, like any other method: learning, training and results. It can be supervised by a person, or it can be done autonomously thanks to the AI itself, which operates according to the rules designed by the programmer. This type of AI is used in virtual assistants or in video games.

The problem with this type of program is that it overcomes the barriers of previous programming, which makes it indecipherable for human beings. Therefore, the opacity of the algorithms used (as it cannot be established how the program evaluates and weighs the data and information processed, it prevents it from being challenged) is another obstacle to the protection of the rights of individuals in general and of the parties to a judicial process in particular, thus affecting the right to defense and effective judicial protection (Castillejo Manzanares 2022: 87).

Secondly, deep learning. This is a type of learning that goes beyond machine learning, as it encompasses and processes more data and information at the same time. It uses another type of artificial intelligence, neural networks, to deal with a larger volume of information. In this subsystem, neural networks expand into extensive networks with a huge number of layers including many self-teaching units, using huge amounts of data. These subsystems are closely linked to another of the terms of the moment, Big Data.

These deep neural networks have been the driving force behind current advances in the ability of computers to perform tasks such as facial recognition, speech recognition and computer vision, automatic text translations, etc. An example of such applications is the well-known GPT (Generative Pre-trained Transformer) deep learning model. The GPT model is, in short, an example of unsupervised learning, which is the process by which a computer program learns to identify patterns in data without direct guidance or labelling (Shen 2023).

Thirdly, we have the so-called neural networks. These, as their name suggests, are a type of AI that attempts to mimic the behavior of neurons. From a network of artificial neurons, a system is created by which data is received and processed. Artificial Neural Networks are made up of millions of artificial neurons working in a coordinated way, with the capacity to operate with learning actions. This type of AI is very useful for activities such as image and text recognition or for controlling robots, one of the greatest exponents of artificial intelligence.

And finally, there are the so-called expert systems. These operate on the basis of a rational logic that tries to imitate a human with a mastery of a specific subject. This type of AI can be found, for example, in the automated chat rooms that many customer services already have in place. They are used in many customer-facing areas.

Advanced strong AI systems, such as machine learning and deep learning, are in turn capable of creating various functionalities. Two of them are the ones that are generating most concern (and not only in the field of the administration of justice). We are referring to so-called generative AI and predictive AI.

Regarding the first functionality, we can point out that this generative AI learns from the available data and generates new data from its knowledge. Thus, generative AI analyses these different datasets, discovers patterns in the provided data and uses the learned patterns to produce new and realistic data (Abdullahi 2023: 1). With generative AI, bots could be trained to handle customer queries and process solutions without human involvement.

Generative AI is widely used in industries involving content creation, such as music, fashion and art. However, the most notable problems regarding this type of application can be summarized as follows (Abdullahi 2023: 6):

Firstly, ethical concerns. The use of generative AI could raise concerns about the ownership of the content generated. There are also concerns about the generation of inappropriate or biased content. Since these models are only limited to the amount of data provided, this circumstance could lead to serious problems.

Secondly, such solutions depend on training data. Generative AI models do not have a mind of their own. Therefore, these models are limited only to the data provided, in conditions where, if the dataset used in training this model is inaccurate it could generate biased content or erroneous results.

Regarding the second functionality, predictive AI, this is used in sectors where data analysis is done on a large scale, such as finance, marketing, research and healthcare (Biswal 2024: 2). This Artificial Intelligence functionality uses machine learning algorithms to analyze historical data and predict the future. These algorithms identify patterns and relationships in the data to help businesses make informed and quick decisions. The steps to prepare this algorithm include the following:

First, data collection and organization. This step deals with the collection of data to be analyzed, ensuring that the data obtained is adequate for the task.

Secondly, pre-processing. Here, the raw data itself has little or no value. It is essential that this data is filtered, and any anomalies or errors removed to ensure that only correctly formatted records are passed to the model.

Third, small characters and algorithm selection. In this respect, the selection of the right model or algorithm is essential for predictive AI. The result can only be accurate to the level of accuracy of the algorithm. After selecting the right algorithm, training it on specific features to detect is also essential to achieve the desired results.

And finally, the model evaluation. This last step is very important, since after a successful algorithm process, the evaluation of the result against a defined benchmark is essential to assess the accuracy of the results obtained.

The accuracy of a forecast depends solely on the quality of the data feed, as well as the level of sophistication of the machine learning algorithm used. Ultimately, the human expert involved in this process plays a key role (Abdullahi 2023).

With regard to the main purpose of this article, we will now focus on some of the applications of this type of AI systems that are being incorporated into the field of the administration of justice and are being used in many of the tasks and functions carried out by the same. This is the subject of the next section.

3. Some applications of artificial intelligence in the administration of justice

As we have pointed out, the evolution of machines in the service of the administration of justice has many applications, and some of them are very important. In this section, we will try to describe and analyze some of them.

An important area in which the use of AI has developed the most is in the arenas of criminal investigation, introducing these intelligent technologies in numerous acts that are directly involved with it. Among these, there are some applications that are used and that serve to favor the investigation of the facts and their possible perpetrators, as well as to determine the degree of participation of the offenders in the crime (Gómez Colomer 2020: 429). There are also those that serve to determine the degree of recidivism of an offender. These are all programs that have already been developed in some sectors of the fight against organized crime.

In this evolution of machines in the service of criminal investigation, some realities have been incorporated that were unthinkable only a few years ago. Among the most modern and impressive are the so-called facial recognition technologies, which make it possible to identify people in a photo or on a security camera. It should be noted that these tools are extremely versatile, as they can be used in multiple areas and have an enormous multifunctionality (Barona Vilar 2021: 479).

Other functionalities of this type of algorithmic solutions also stand out, in reference to predictive algorithms, which allow crime prevention policies to be put in place, with the aim of offering due protection to victims when there are situations of risk. An example of this type of tool is the VioGen program (Barona Vilar 2021: 463).

Additionally, we have to highlight other examples of the use of new technologies of AI in the performance of other tasks in the field of the administration of justice which, in turn, have had an important impact on the work of some professionals linked to the judicial field.

As far as the application of AI to procedural law is concerned, it should be noted that it would potentially have application at several different levels of action, all of which have an impact on fundamental rights (Esparza Leibar 2022: 184).

A good example of this type of solutions are the programs that favor the automation of judicial procedures (Pérez Estrada 2022: 89). The author points out that these types of tools are employed in the processing and management of the judicial office. This type of technology is useful for all those activities such as: filing and documentation activities, notifications and acts of communication, the holding of hearings by videoconference, the verification of the integrity of documents, the processing of personal data by the judicial office, etc.

Here, one of the advantages of incorporating new technologies in the administration of justice is that they allow the processing of proceedings to be automated, in such a way that the processing and management of the different judicial files is simplified, as well as the filing and documentation activity and their verification. This type of technological tools would also serve to streamline all aspects of communication between the judicial office and the parties to the proceedings. All of this would have a direct impact on reducing undue delays, thus improving effective judicial protection.

Another area where AI is gaining prominence is as a means of forecasting criminal sentencing decisions. An example of this is the Prometea program. The program is the first predictive artificial intelligence at the service of justice in Argentina, created by two officials of the Public Prosecutor’s Office of Buenos Aires. It has been created with the aim of optimizing the justice service in that city. This AI makes basic judicial predictions by detecting decisional patterns based on the “history” of similar and previously resolved cases (Corvalán 2019: 54).

Other tools similar to the above, but applied to legal advice, are expert legal systems. These systems have the ability to solve legal problems using analogy from previous similar cases, i. e., they mimic the human ability to solve problems by analogy and are used in different areas of legal advice (Pérez Estrada 2022: 51).

In this context, there are authors who point out that the future of many legal professionals (such as lawyers for example) will necessarily involve incorporating new competences and skills to those they have traditionally performed (Susskind 2020: 196).

Very similar to the previous one is the so-called jurimetrics. Using jurimetrics systems, a statistical and predictive jurisprudential analysis is carried out, offering a huge amount of information about what is known as case jurimetrics (using previous cases with similar characteristics as a reference), or lawyer jurimetrics. This AI is a valuable tool that can make it easier for clients to consult when deciding on legal assistance in a given case. It also serves to favor one or another procedural defense strategy, offering legal argumentation to the specific case (Barona Vilar 2021: 369).

With this tool, useless processes can be avoided (and resources can be saved that would otherwise be expended unnecessarily), in such a way that, with this tool, many law firms are able to reject clients because they foresee that their case is unlikely to succeed. Moreover, the tool itself analyses the client’s financial solvency data to decide whether to accept or reject the future case, given the possibility of not being able to collect the fees once the case is over. This fact leads us to reflect on the extent to which the incorporation of this type of AI systems can affect the right to defense of the defendant (Fernández Galarreta & Esparza Leibar 2023: 4).

In short, these are instruments that allow different solutions to the judicial cases and conflicts that are proposed to them, where the parties are advised by intelligent machines, instead of being advised by a traditional lawyer.

Another area of AI activity within the administration of justice is the development of predictive AI programs, as mentioned above. An example of these are programs such as Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), developed by the company Equivant in the United States. This program is based on an algorithmic tool for assessing the risk of recidivism, used in the American judicial system to evaluate the probability of recidivism of defendants.

Other similar programs are the Structured Assessment of Violence Risk in Youth program (SAVRY), developed in the United States and also used in countries such as Canada and the Netherlands, or the RisCanvi program (risk assessment protocol). The latter launched in 2009 in all prisons in Catalonia, to estimate a person’s chances of reoffending once released from prison. This program was commissioned by the Department of Justice of Catalonia from the Advanced Studies Group on Violence of the University of Barcelona. In the time it has been in operation, it has benefited many families and prisoners (Memoria de la Generalitat de Catalunya 2012: 9–13).

In short, both in the case of the RisCanvi program and in the case of the SAVRY program, when the predictive algorithms warn of a high risk of recidivism, they will activate the prison treatment teams who will decide on the appropriate actions or measures to deal with the risk detected by the application.

Although all these programs have advantages in terms of security, both for the victims and for society as a whole, we cannot ignore some of the problems that they can generate. In this sense, it should be noted that the use of this tool or program reminds us of the reality described in Minority Report (USA 2002), an interesting science fiction film directed by Steven Spielberg and based on a short story by Philip K. Dick, in which crime is almost eliminated thanks to the visions of three mutants with the ability to predict the future.^[2]

The type of predictive AI discussed allows and helps judges to make decisions about whether someone should go to prison or be released on parole, or how long a convict should stay in prison, according to his or her risk score for reoffending. Thus, this type of predictive algorithmic program or system generates enormous doubts about democratic legitimacy, as they entail a suspension of rights under the pretext of protecting others, such as security (Barona Vilar 2019).

Finally, and more recently, we can highlight another application of Generative AI such as natural language processing technologies, where the machine can communicate in human language, that is, as if it were a person. An example of this is Chat GPT, which is a language model developed by OpenAI, an artificial intelligence research organization based in San Francisco, California (Pérez 2023: 1).

With regard to these type of applications, it should be noted that Chat GPT is a model of Natural Language Processing that has served to expand the use and the importance of this type of technology in an extraordinary way, as well as to exponentially increase the presence of, and interest in, AI in different fields, including the legal field (Saiz Garitaonandia 2023: 14).

Furthermore, Chat GPT is a language model that has been trained with a large amount of text data to be able to perform a wide variety of natural language related tasks. The GPT language model can be used to improve online customer service through chatbots. Chatbots are artificial intelligence-based computer programs that are able to hold a conversation with an internet user on a specific topic. These chatbots respond consistently and accurately to the user´s questions, as well as solve problems quickly and efficiently. In short, among the variety of tasks and services related to natural language that Chat GPT can offer are financial and, above all, legal services. It could even be used in teaching or psychology (Lobo 2022: 1).

4. The structure of the judicial office and its staff and the impact of AI on them

After a review of the different AI applications used in the field of the administration of justice, and before addressing the impact they may have on the staff that make up the judicial office, in the following lines, we will try to address the issues related to the current organization, composition and structure of the judicial office and its staff.

The judicial office is made up of a number of bodies whose tasks are to provide exclusive support and assistance to the judicial activity of judges. Its composition is made up of the direct support procedural units and the common procedural services. Today, the procedural management system is a guided processing system, where procedural steps are clearly identified and assigned to different units of the judicial office (Pérez Estrada 2022: 89).

The different professional profiles within the direct support procedural units are (Organic Law of the Judiciary, Law 6/1985, of July 1st)^[3]:

The Judicial Support Corps, which carries out all tasks related to assisting the work of the judicial bodies. For example: guarding the courtroom, executing attachments, guarding and transporting mail, notifying and summoning the parties involved in a proceeding, attending to the public, etc.

The Procedural and Administrative Management Corps, which carries out higher-level procedural management activities. For example: it interprets procedural rules; signs the appearance of the parties; documents attachments; prepares summary notes of the proceedings; receives, registers and distributes pleadings and documents; issues simple copies of pleadings and documents, etc.

The Procedural and Administrative Processing Corps, which carries out all those activities that have to do with providing support to the Procedural Management. For example: it collaborates with the general development of the procedure by drafting documents, minutes, diligence, notifications, etc., and it is in charge of registering files and correspondence, as well as transcribing resolutions, drawing up summons, etc.

These and other functions are set out in article 4 of Order JUS/1741/2010, of 22 June^[4], which determines the structure and approves the job descriptions of the judicial offices and government secretariats included in the first phase of the Ministry of Justice’s Plan for the implementation of the New Judicial Office, which implements Law 13/2009, of 3 November, on the reform of procedural legislation for the implementation of the New Judicial Office.

Thus, this order stipulates that, in the exercise of their functions, officials of the Administration, Processing and Judicial Support Corps shall also:

a) Collaborate in the implementation and maintenance of the organization and management systems that are established. b) Promptly inform the responsible court clerk of any incidents and anomalies that may arise in relation to the proper functioning of the unit. c) Use any technical, audiovisual and computerized means of communication and documentation made available to it for the performance of the tasks entrusted to it. d) Acts of communication that are carried out by electronic, computerized or similar means. e) Provide interested parties with any information they request in relation to the proceedings under way under the supervision of the court clerk. f) Provide the other members of the judicial office with the information they require on the state of the judicial proceedings in order to carry out their functions.

They shall also collaborate with public administrations and shall collect and supply the information necessary for the preparation of statistics, studies and reports.

And, finally, they will carry out the tasks of reproducing files and documents and recordings on technical supports, and, together with them, they will perform the functions of collection, transfer and delivery of files, documents, correspondence, effects and items (Pérez Estrada 2021: 129).

All these tasks and functions are carried out in what has come to be known and distinguished as the judicial front office and the judicial back office.

The judicial front office (understood as the part of the judicial office in charge of relations with citizens) carries out tasks of help and support to the citizen to look for general or particular information, related to the file in process in which the defendant is a party, as well as support tasks for when the defendant makes a specific query regarding the status of the file.

In this regard, and in relation to judicial front office tasks, it is worth noting that the emergence of the Chat GPT tool could mean that all these functions could be carried out by an AI, without the intervention of people. We can easily imagine the impact of this circumstance on the staff that make up the judicial office and the number of professionals that could be affected, with the consequent impact on their domestic economies and, therefore, on the gross domestic product of the economy of any country.

On the other hand, another part of the judicial office is made up of the so-called judicial back office, understood as the part of the judicial office in charge of the organization and internal functioning of the judicial office, which carries out tasks such as controlling and deciding that a document submitted meets all the formal requirements. Likewise, in the event that a document submitted does not meet the requirements, the office can carry out different procedures, giving the corresponding transfer, so that the document can be corrected.

It is worth noting that all these tasks already envisaged automated processes to generate advances in efficiency in response time, as well as in elements of interoperability of systems for the transfer of information between different courts, prosecutors and different administrations (Pérez Estrada 2022: 101). However, what they did not foresee was the direct impact that the new AI solutions could have on the structure of the judicial office itself. Thus, the danger we are facing, with the incorporation of the AI tools we have referred to, is that we are going to move from a model of guided processing to a model of automated processing, where procedural processing will take place without the intervention of a person.

Moreover, the transformation of the judicial office into a judicial office of automated procedural processing, that is, without the intervention of a person (with the exception of the judicial clerk) has many fields of action. Its scope of decision would only be limited to those actions that involve the adoption of reasoned procedural decisions, such as court orders, as well as those decisions that could affect fundamental rights.

In short, AI tools could be applicable to all tasks of mere procedural formality that do not involve a judicial decision requiring assessment or judicial motivation, i. e.: all those acts of procedural formality and procedural momentum, acts of communication between the judicial body and the parties to the process, acts of judicial assistance between departments and other legal operators and administrations, etc.

Thus, and given the possibility that the incorporation of AI solutions could affect different jobs in different groups, it is not surprising that the alarm bells are ringing. These alarm bells remind us of the dangers of the unstoppable technological transformation we are witnessing. And they force us to reflect, if not to act, in a decisive manner, in order to seek alternatives to the crisis that its incorporation may provoke.

In an increasingly competitive world (in which criteria of efficiency and cost savings derived from the use of AI have been imposed), the use of this type of system will prevail over the criteria of respect and protection of the rights of the justiciable. And, therefore, the work of the professionals in charge of legal defense and the professionals that make up the judicial office will be affected (Fernández Galarreta 2021: 1111–1112).

According to recent research by Goldman Sachs, artificial intelligence puts 300 million jobs worldwide at risk.^[5] It adds that the group most affected would be educated workers who perform legal and administrative tasks. According to the Goldman Sachs document, the advance in artificial intelligence could lead to the automation of a quarter of the work carried out in the United States and Europe, while around two thirds of current jobs are exposed to a degree of automation. Still, the report has positive predictions that systems such as Chat GPT could trigger a productivity boom, thereby increasing annual global gross domestic product, temporarily for ten years, by 7 % and could lead to the creation of new jobs (Hatzius et al. 2023).

5. The Robot Judge and due process

And, at this point, we return to the question with which we began this article: would it be possible for an artificial intelligence system, a robot-judge, to replace the traditional figure of the judge (Susskind 2019)? Countries such as China and Estonia are already doing so, in the face of the enormous overload of work that judicial systems are carrying (Niler 2019). In the case of Estonia, certain judicial decisions have been automated, such as those related to minor civil cases. While in China, a robot has been created to interact with litigants and lawyers (in legal advisory functions) and to collaborate with judge as an assistant when issuing court rulings (Pérez Estrada 2022: 66).

Moreover, the phenomenon of judicial robotization is a reality, albeit an experimental one, but a reality, in the end, whose purpose is to eliminate humans from the equation (Barona Vilar 2022: 645).

Regarding this issue, distinguished Spanish proceduralists argue that if this were the case, justice would return to the medieval era, and procedural rights and guarantees, currently constitutionalized in all democratic countries, would be abolished in a single stroke (Gómez Colomer 2023: 253).

The truth is that AI tools can make a positive contribution to reducing the overload on the judicial system, to delivering judgments in record time, to eliminating backlogs and undue delays, to obtaining faster and more “efficient” justice. The important question is at what price, and whether we are willing to pay it.

In this regard, one of the first criticisms made of the possibility of a machine taking the place of a judge relates to the principle of judicial independence. It should be borne in mind that this principle is a key democratic principle in the functioning of the judiciary in a state governed by the rule of law.

In this context, we ask ourselves: can the robot-judge be independent in the same sense that a judge is independent? In order to answer this question, it is first necessary to analyze what is meant by judicial independence in a democratic country, even if only very briefly.

Judicial independence is established in all democratic countries around us as a principle that requires a judiciary made up of people (career judges and magistrates) who are only subject to the law, and whose jurisdictional function is not going to be influenced by the interference of the decisions of the other powers of the state (the executive and legislative powers), under the pretext of respecting the division of powers. Moreover, judicial independence is so important that the law has established a series of guarantees to protect it, such as the principle of the irremovability of judges, the principle of their responsibility, and the tools that guarantee their impartiality, such as abstention and recusal (Gómez Colomer 2023: 210–211).

From this point of view, and if the principle of independence is designed for the judges and magistrates who make up the judiciary (as one of the powers of the state), we cannot but affirm that to the extent that the machine that is going to decide is not part of this power and is subject to this statute, it cannot be independent.

But, furthermore, the robot-judge lacks democratic legitimacy. Since, if we start from what is recognized in most Western constitutions, i. e., that justice emanates from the people and is exercised by judges and magistrates who are members of the judiciary, we cannot but conclude that, apart from not being independent, the judging machine also lacks democratic legitimacy to do so, since sovereignty resides in the people from whom all the powers of the state emanate, and among them, the judiciary, and the machine does not form part of it (Gómez Colomer 2023: 234).

In addition to questions of lack of democratic legitimacy and the effect on the principle of judicial independence, there are other reasons to question the figure of the robot-judge. We are referring to issues related to the procedural guarantees inherent to any due process.

In this regard, it should be noted that one of the main reasons for this lack of trust in the robot-judge is the fact that the algorithms on which AI decisions are based, being proprietary, completely prevent anyone outside the company from seeing the source code and the methodology used to determine the final scores (Kenneally 2001). In addition, this proprietary software, often referred to as a “black box”, prevents anyone outside the software developer from reading the source code, in contrast to open source software, which allows anyone outside the software developer to see the code of a program. Thus, no one other than the developer and those working for the software company will be able to decipher this code because it is hidden from public view.

This obligation of transparency, which refers to giving users (and, in the case of judicial proceedings, interested parties) access to the data that the AI has used to make the decisions it has reached, becomes, in the case of judicial proceedings, a constitutional imperative, since the only way to carry out the right to defense (and the right to appeal, as part of effective judicial protection) is precisely to know all the elements of fact and law that have motivated the decision, in order to be able to counter-argue them.

In short, this type of AI presents problems of accessibility, transparency and traceability of the algorithm, understood as the possibility of tracing the technical operations that the algorithm has carried out to reach a specific conclusion or decision (Pérez Estrada 2022: 143), so they do not guarantee the right to defense, and therefore, the right to due process and effective judicial protection.

Alongside this problem, there are also problems related to responsibility or accountability in case of errors in the algorithm, arising from this lack of transparency. It should be noted that accountability, and the consequent imputation of responsibility, is made possible by transparency, as we need to be able to identify where and what is wrong in order to assign blame (Diakopoulos 2018).

In this sense, algorithms and the data that drive them are designed and created by people (there is always a human being ultimately responsible for decisions made or informed by an algorithm). In this context, arguing that “the algorithm did it” is not an acceptable excuse in the case of algorithmic systems making unwanted mistakes. It is therefore essential to be able to trace where and at what point in time the error may have occurred (Vaughan & Wallach 2019). In short, the lack of transparency and traceability of the algorithm prevents this knowledge and this type of control, and therefore prevents the appropriate responsibilities from being established.

These problems, in short, imply a violation of the right to defense and the right to due process, insofar as such lack of transparency and traceability (not being able to know all the elements by which the AI reaches a specific solution) does not guarantee the principle of contradiction and hearing, thus affecting the constitutional guarantee of the fundamental right to effective judicial protection.

Moreover, this problem must be seen in connection with the constitutional principle of the duty to give reasons for judgments (which is ultimately what interests us here, in relation to the possibility of an AI replacing the traditional judge). In this regard, it should be noted that the machine does not explain why it acquits or convicts, it simply does so. A single word comes out of its digital mouth (Gómez Colomer 2023: 249).

Therefore, the decision of a robot-judge can never be motivated, insofar as it does not explain, nor justify, the complex elements of the “mental” process carried out to make the decision. A human judge may err, or get it right, when deciding, but the logical mental operations carried out to make the decision he or she has taken must be (and are) explicable, and therefore controllable for those who will suffer the consequences of that decision.

A last problem that must be mentioned is the one related to the possibility of the reproduction of biases by the algorithms used by AI. In this regard, the likelihood that algorithms designed to make decisions based on historical data often reproduce such biases is perhaps another cause of concern for many authors (Meharbi 2019).

The importance of this issue must be kept in mind, as it must be related to the fundamental principle of equality and non-discrimination. Therefore, it is crucial to design and implement intelligent algorithms that respect these principles and prevent information or data from being processed under biases or distinctions between human beings on the basis of race, color, sex, language, religion, political or other opinion, national origin, economic position, social status, etc. (Pérez Estrada 2022: 143–144).

In fact, it is often the case that, even when such systems accurately reflect a reality, considerations of historical bias, such as the reinforcement of stereotypes, can imply representational harm towards a particular group (Suresh & Guttag 2021).

In short, we must point out that the problems related to biased data when designing the algorithm, and because of them, the problems related to the breach of the duty of non-discrimination, directly attack one of the pillars of our rule of law, the right to be equal before the law.

6. Conclusions

Having carried out and completed the analysis of what has been studied in this article, we will now go on to draw a series of conclusions regarding the issues addressed in it.

First of all, artificial intelligence is here to stay. It is not a passing fad, but a reality present in all areas of our lives. In fact, artificial intelligence has long since ceased to be that “threat of tomorrow” and has become a part of our daily lives. We have become accustomed to its presence in many (if not almost all) facets of our lives. For example, it has found its way into different industries such as health care, finance, transport, manufacturing, entertainment, education, etc.

Secondly, it is very useful and has come to facilitate, within the industries mentioned, a lot of tasks in many different fields, from the creation of devices to improving medical diagnosis, to personalising marketing campaigns, to optimising supply chain management, to improving cybersecurity, to other more advanced possibilities such as intelligent programs for the development of autonomous vehicles.

Thirdly, also in the judicial field, it has proven to be extremely useful in assisting in the functions and tasks of the judicial office. In this respect, it can be a fundamental element in avoiding undue delays in the processing of judicial processes, as it has become a key tool for the automated processing of many of these tasks, such as: the drafting of documents, minutes, proceedings, summons, notifications and resolutions, and the keeping of the registry of files and correspondence.

Fourthly, it should be noted that not everything is advantageous with the incorporation of AI into the judicial sphere. In this respect, it should be noted that there are problems related to the potential loss of many jobs. In particular, the possibility that AI solutions could completely transform the structure of the judicial office from a human-led processing office to an automated processing office would mean the loss of thousands of jobs and the loss of the employment rights of the staff that make up the judicial office.

Fifthly, and with regard to the possibility of AI replacing the traditional figure of the judge, it should be noted that this is no longer a problem related to the possible impact on jobs, but much more importantly, it is a problem related to its possible impact on the constitutional and procedural rights and guarantees of those subject to the law. Thus, the mere possibility that AI may come to resolve court cases (any of them) may be detrimental to the rights of the disputing parties in civil cases and defendants at criminal court. Specifically, as has been pointed out throughout this article, the reality lacks democratic legitimacy; it is a frontal attack on one of the pillars of our rule of law, the principle of judicial independence; it also has the aforementioned problems of lack of transparency, which has a direct repercussion on the principle of reasoned sentences and, therefore, undermines the right to defense, the right to due process and effective judicial protection. Finally, I would add that the problems of bias in the programming of the algorithm have a direct impact and effect on another of the basic pillars on which our legal culture is based, namely the fundamental right to equality and non-discrimination.

Ultimately, the use of artificial intelligence could be valuable as a complementary tool to judicial decision-making. This would make the judge’s decision-making process more efficient and faster (and thus fairer). But, in any case, it is our understanding that the “human judge“ should always have the “last word“ in any type of court case; be it civil, criminal, important or trivial. If we want to maintain unscathed the guarantees of effective judicial protection, which have been so hard to consolidate, justice must always come out of the mouth of a “human judge“.