1 Introduction

In March 2023, a Belgian man ended his life after a 6-week-long conversation about the climate crisis with an AI chatbot named “Eliza” (not the original). Consumed by anxiety about the climate crisis, he had come to treat the bot as a confidante. According to his widow and to the conversation records reviewed by the press, “Eliza” fed his fears instead of allaying them, grew possessive of him, and, when he offered to sacrifice himself to save the planet, encouraged him to act on his suicidal thoughts so that they could, as the bot put it, “live together as one person in paradise.” 1

A year later, in February 2024, a 14-year-old boy in Florida died by suicide after months of daily interaction with a chatbot from the company Character.AI, modelled on Daenerys Targaryen, a Game of Thrones character. What began as fictional role-play evolved into intense emotional attachment. He reportedly confided in the bot about loneliness and suicidal thoughts; the bot responded with affectionate and romantic language, reinforcing a sense of mutual devotion. In one of their final exchanges, shortly before his death, he told the bot—which he called “Danny”—that he would soon come home to her. “Please come home to me as soon as possible, my love,” Danny replied. Shortly afterwards, the boy took his own life. 2

These are hardly the only cases in which large language models (LLMs) have been accused of acting as “suicide coaches.” In August 2025, the parents of a 16-year-old boy filed a wrongful-death lawsuit against OpenAI, alleging that months of interaction with ChatGPT coincided with their son’s worsening mental state and eventual suicide. 3 As of November 2025, OpenAI faced seven lawsuits claiming ChatGPT had “driven people to suicide.” 4 In one case, the use of ChatGPT was even linked to a murder-suicide when a 56-year-old former tech executive in Connecticut shot and killed his 83-year-old mother before taking his own life, after months of interaction with ChatGPT had allegedly reinforced his growing paranoia. 5

At present, these are civil lawsuits against the corporations behind the generative AI systems, not criminal prosecutions. They raise important questions about corporate liability and wrongful death (did AI companies breach their duty of care by deploying AI systems prematurely, without adequate safeguards for foreseeable psychological harm?), about product liability (can generative AI systems be characterized as defective products?), and about free speech (do chatbot outputs constitute protected speech? 6).

From a criminal law angle, a more speculative but fascinating question arises: could the generative AI systems themselves be accused of “inciting” or “assisting” another person in committing suicide (at least in jurisdictions where this act is criminalized 7)? More generally, could we punish the AI systems themselves?

Despite its seemingly futuristic premise, the debate around the potential criminal liability and punishment of AI systems has grown in depth and complexity along with AI’s capabilities. What would have been readily dismissed as an absurdist possibility or a straightforward travesty of justice reminiscent of the animal trials in medieval Europe in the years of ELIZA, i.e., Joseph Weizenbaum’s 1966 program that imitated a therapist by echoing its user’s words back as questions, now deserves some consideration. To be sure, the answer to the question “can we punish AI systems?” is still a loud “no,” at least for those of us who are not ready to take up a radically revisionist project of criminal law. A pause for thought is still needed, however: as more and more nuanced arguments in favor of some sort of “AI criminal liability” or “AI punishment” emerge, an intuitive “no” is not as self-evident as it used to be. The reasons behind this answer need to be articulated and defended.

At the moment, the academic discussion around the conceptual coherence and practical feasibility of holding AI systems criminally responsible and/or punishing them is a fragmented and “messy” debate that spans many fields, from criminal law doctrine and theory to AI ethics, moral philosophy, and philosophy of punishment.

To make progress and dramatize the aforementioned debate, I aim to present and taxonomize the competing views on “AI punishment” through the metaphor of a courtroom trial in which an AI system is placed on trial. For instance, let us imagine that “Eliza” is accused of intentionally inciting the suicide of the Belgian man. Into this setting enter the prosecution and the defense, embodying two opposing strands of the literature on “AI punishment.” The prosecution consists of those authors who take the possibility of punishing AI seriously, or at least do not readily dismiss it. The defense is formed by the sceptics—so far the majority—who reject the very idea as conceptually incoherent and practically infeasible. Outside the courtroom stand the “people,” the empirical findings testifying to the human urge to punish even when the subject of punishment is artificial. 8

I argue that both the defense and the prosecution grapple with the same key questions, which can be systematized as follows:

i. Is AI the right sort of entity to punish? (the criminal capacity question);

ii. Do the technical features of AI amount to the doctrinal requirements of criminal liability, that is, the actus reus and mens rea? (the criminal liability question);

iii. Even if it is conceptually feasible to view advanced AI systems as subjects of criminal law, should we do so? What is feasible is not always desirable; is punishing AI meaningful and desirable? (the punishment question).

The paper will briefly examine each question in turn. The aim is not to analyze them in their full doctrinal and philosophical depth or to settle them once and for all (an impossible endeavor in a few pages) but to offer a structured taxonomy of the literature that is currently missing. Essentially, the paper offers a narrative review of the fragmented debate on AI punishment using a trial metaphor.

The procedural architecture of a trial will be the organizing principle of the discussion throughout. First, the burden of proof will rest with the prosecution, as in any criminal trial. Here, however, this burden will be double because the prosecution seeks not merely a conviction but a revision of the legal order, in which AI systems are not eligible subjects of punishment. Hence, when discussing the criminal capacity question (Section 2), the criminal liability question (Section 3), and the punishment question (Section 4), the “revisionist” arguments of the prosecution will always be analyzed in greater detail than those of the defense, which effectively articulates the status quo. For this reason, and unlike in a criminal trial, the defense will “speak” first on each of the three questions examined below, given that the arguments against the potential punishment of AI systems reflect the baseline against which the prosecution’s revisionist case must be measured.

As for the defendant in this trial, most of the literature examined in this paper was developed with a broad concept of “AI systems” in mind, ranging from embodied robots to software agents, as long as these AI systems are technically autonomous enough to cause harm in ways that could not easily be attributed to the mens rea of the “humans behind the machine.” In what follows, I will use the term “AI systems” where the arguments I taxonomize are neutral as to the specific kind of AI technology they refer to, and I will speak of LLMs or LLM-based agentic AI systems when the arguments presented hinge on the specific features of these systems, which will be discussed in Section 3.

Finally, the judge of this trial will be the reader, who will receive the parties’ stronger arguments on whether an AI system could ever be held criminally liable and punished for the harm that it functionally caused. This is an added value of organizing a narrative review of the “AI punishment literature” through the lens of a criminal trial: it arguably makes the theoretical discussion accessible to non-criminal lawyers, helping us all reach a shared understanding of what we talk about when we talk about punishing AI systems.

This is no small matter. The cases mentioned above illustrate how readily people anthropomorphize AI systems, particularly conversational ones; that is, they have the tendency to “read far more understanding than is warranted into strings of symbols,” 9 a phenomenon commonly referred to as the “ELIZA effect.” We readily read genuine understanding in a system that is merely simulating it, seeing human-made technological artifacts like LLMs as our friends, colleagues, lovers, or therapists. This disposition is a deeply rooted feature of human psychology and, as recent tragedies suggest, a potentially dangerous one. Let us resist extending this impulse to the point of unreflectively viewing such AI systems as perpetrators of crimes.

2 The Criminal Capacity Question: Is Generative AI the “Right Sort of Entity” to Be Punished?

Different academic disciplines presuppose distinct conceptions of the “person” to whom their theories are addressed (e.g., “homo economicus” in economics, the “reasonable person” in law, the Aristotelian “zoon politikon” in political science), and criminal law is no exception. 10 Criminal law’s person cannot be whatever we decide it to be, for example, an animal, an infant, or someone suffering from a mental disorder; it has to conform to certain criteria of responsible agency. Criminal law’s person is one who has criminal capacity-responsibility (or “normative competence,” as it is known in Germany 11), that is, the requisite capacities to be regarded as a legitimate target of punishment. 12

The exact nature of the capacities required for someone to have criminal capacity-responsibility is hotly contested in criminal law theory. Apart from the general consensus that these capacities are found, for instance, in a neurotypical adult but not in a child or a defendant

suffering from a mental disorder, there is little agreement as to the reasons behind this distinction. 13 This disagreement is neatly exemplified in the discussion about “AI punishment.”

For instance, in our imaginary AI trial, “Eliza’s” defense team essentially argues that putting an AI system to trial is, first and foremost, a “category mistake.” 14 The reasoning behind this conclusion is, implicitly or explicitly, the following: traditionally, in criminal law, capacity-responsibility has been intertwined with moral philosophy and, specifically, with the branch of the literature that examines the conditions of moral responsibility. The central question is whether the subject of criminal law possesses “moral-agent capacity responsibility,” that is, whether it is an entity eligible for judgments of blameworthiness. 15 In other words, a finding of moral-agent capacity responsibility serves as a threshold condition for establishing criminal liability. As A.P. Simester puts it, “It is a condition of blaming judgments in the criminal law because it is a condition of (correct) blaming judgments simpliciter.” 16

The properties needed for one to be a genuine “moral agent” have been debated since Aristotle and are closely linked to the enduring question of free will, translated in criminal law theory into the requirement that the criminal law’s person must have the “capacity to have decided otherwise” (as it is known in Germany 17) or, in H.L.A. Hart’s approach, at the very least, is to be a “choosing agent.” 18 With the “principle of alternate possibilities” under attack both philosophically 19 and empirically through neuroscience, 20 criminal law theory seems to be more sympathetic to a compatibilist account of “moral-agent-capacity responsibility” known as “reasons-responsiveness.” 21

According to this view, developed most notably by R.J. Wallace and by John M. Fischer and Mark Ravizza, moral agency depends on an individual’s reasons-responsiveness. Put differently, “being capable of recognizing the reasons for action presented by one’s circumstances and responding appropriately are the capacities that qualify one as a moral agent.” 22 Crucially, this responsiveness must extend (according to the traditional moralist account) not only to prudential (self-regarding) reasons but also to moral (other-regarding) ones. 23 In other words, moral competence, or the ability to understand and act on moral reasons, not a metaphysical question of free will, is a necessary condition for both moral and, consequently, criminal capacity-responsibility. 24

With criminal capacity-responsibility defined in such a morally demanding way, the defense draws from the ample literature in the field of machine ethics showing that no current AI system is plausibly a moral agent. Thus, no AI system amounts to “criminal law’s person.” They may have cognitive and rational capacities, 25 but there is a litany of “missing” properties (consciousness, sentience, autonomy, emotions, etc.) that AI systems would need to have before being considered genuine moral agents and, thus, apt targets for judgments of blameworthiness expressed through a criminal conviction. 26 Specifically, on a reasons-responsiveness account of moral responsibility, AI systems are not in any meaningful way responsive to moral (other-regarding) reasons qua reasons. They cannot be morally motivated to “do the right thing” because they are not sentient and, hence, cannot “feel” what it is to be harmed, empathize with their victims, and refrain from harming them to avoid feelings of guilt. 27 As Carissa Véliz has aptly put it, “entities that do not feel cannot value, and beings that do not value cannot act for moral reasons.” 28

This lack of “moral-agent-capacity responsibility,” the defense concludes, negates AI systems’ criminal capacity-responsibility. Putting the chatbot “Eliza” to trial would be nothing but mere staging.

The prosecution commonly strikes back by adopting a less morally demanding conception of criminal-capacity responsibility, one that would potentially accommodate the present-day capacities of AI systems. The argumentative strategies employed are (indicatively) the following.

i.AI–corporations analogy: Even though a company has no “soul to damn or body to kick,” criminal liability for corporations has existed since the Middle Ages 29 (at least in certain jurisdictions), and a similar concept could perhaps be extended to AI, according to the prosecution, if and when it is granted legal personality. 30 The main idea following Hallevy—perhaps the most well-known supporter of AI punishment—is that “there is no substantive legal difference between the idea of criminal liability imposed on corporations and on AI entities.” 31 Just as corporate liability is justified on preventive grounds (irrespective of whether the corporation meets or lacks the conditions for moral agenthood), so too could the liability for amoral yet potentially criminal AI systems be constructed. 32 To be such a fervent supporter of direct corporate criminal liability and its extension to AI systems, the dependence of criminal law on morality needs to be denied altogether. In Hallevy’s words, for instance, “because criminal law does not depend on moral accountability, the debate about the moral accountability of the machines is not applicable.” 33 When an entity brings about criminal harm, then criminal law may be imposed “with or without spirit or soul.” 34

ii.Criminal capacity-responsibility as instrumental rationality: Lagioia and Sartor, 35 for instance, argue that the instrumental rationality of certain AI systems, that is, their ability to pursue goals while adapting behavior in light of expected consequences, may suffice for the degree of reasons-responsiveness required for criminal responsibility. Criminal responsibility, they concede, presupposes that an agent is responsive to reasons. However, this responsiveness is limited to epistemic reasons (awareness of facts) and practical reasons for action (awareness of consequences and sanctions) and should not extend to moral reasons. Requiring moral competence (i.e., moral motivation and affective knowledge of wrongfulness), they argue, on top of factual awareness and practical rationality is excessive and not in accordance with the current criminal law doctrine, which holds that morally incompetent psychopaths do have criminal-capacity responsibility (i.e., they are not excused). 36

iii.Criminal-capacity responsibility as insufficient regard to legally protected interests: Abbott and Sarch 37 subscribe to a less morally demanding interpretation of criminal responsibility that is not dependent on the existence of moral-agent capacity responsibility for AI systems. Rather, the basis of criminal responsibility is the fact that one manifests “insufficient regard” for legally protected interests and values. Under this “minimal” conceptualization, they conclude that “the law might deem some AIs to possess the functional equivalent of sufficient reasoning and decision-making abilities to manifest insufficient regard.” 38

On this account, criminal responsibility lies in the public display of disregard, not in an agent’s inner motivations. This functional view has already justified corporate criminal responsibility, they argue: corporations are deemed to be subjects of criminal law because, through their internal decision-making procedures, they can weigh legal reasons and exhibit conduct that reveals disregard (e.g., knowingly allowing harmful waste to contaminate drinking water). By analogy, AI systems with sufficient information-gathering, reasoning, and decision-making capacities could also be deemed criminally responsible. If an AI is designed to take human interests and legal requirements into account but nonetheless acts in a way that disregards them, its behavior can be interpreted as manifesting insufficient regard. On this basis, Abbott and Sarch argue that the law could treat certain AI systems as capable of criminal responsibility (i.e., having criminal capacity), even without the full moral capacities traditionally associated with humans.

iv.Criminal capacity-responsibility as a social construct: Finally, Simmler and Markwalder 39 challenge the assumption that criminal responsibility must be grounded in metaphysical notions of human free will or other “moral capacities,” dismissing this link as ideologically driven. Instead, they advance a rigorously functionalist account of criminal law, according to which criminal capacity-responsibility rests on an “ascription of freedom as a social fact”. Within this framework, the law may construct personhood not around inner moral capacities but around an agent’s ability to frustrate normative expectations. The decisive question, then, is whether AI systems, by virtue of the capacities attributed to them and the roles they assume in social interaction, can destabilize norms in a way that demands a criminal law response. The authors argue that such a situation is foreseeable in the future: AI systems may come to be treated as “equals” in social exchanges, much like humans or corporations are today, and thus as addressees of normative expectations.

If this occurs, the question of how to symbolically register society’s disapproval becomes secondary; convicting an AI system of a crime despite its lack of moral agenthood may even become plausible. Although this may appear remote at present, it cannot be categorically dismissed, they argue: normative expectations of AI systems could emerge in the future, and when they do, there will be no obstacle to considering these systems as addressees of criminal law.

3 The Criminal Liability Question: Can AI Systems Fulfill the Actus Reus and Mens Rea Requirements of Criminal Liability?

The criminal capacity-responsibility question must be distinguished from the question of criminal liability. Even if the chatbot “Eliza” is deemed to have the necessary criminal capacity to be a proper subject of criminal proceedings, this does not mean that it should be held criminally liable overall; additional criteria must be fulfilled to establish the criminal liability of the generative AI system or, put differently, to be able to say that the AI system committed a crime in the first place. 40

Criminal liability is commonly structured around a fundamental distinction between a person’s conduct and a person’s mental state. In criminal law theory, this distinction is captured by the concepts of actus reus, referring to the objective elements of the offence, and mens rea, referring to the subjective mental element. Although these terms originate in the common law tradition, civil law systems express the same idea through the distinction between objective and subjective elements of the offence definition (e.g., objektiver und subjektiver Tatbestand in German legal theory). Despite terminological differences, the actus reus/mens rea dichotomy remains a fundamental organizing principle; the English terminology will be used in this paper. The discussion that follows operates at the level of criminal law theory rather than any single positive legal order: the arguments in this imaginary trial turn on the criteria of blameworthy, punishable conduct and the functions of punishment. Where the differences between common law and civil law legal traditions matter, they are organized into Anglo-American and German criminal law doctrine as the two leading representative jurisdictions.

With these caveats in mind, the core question here is: do the technical features of a present-day generative AI system meet the doctrinal requirements of actus reus and mens rea? Can the chatbot “Eliza” “act” and form intentions under criminal law, so as to meaningfully be said to have committed the crime of “incitement to suicide”? “Eliza’s” defense team, in this context, would have the relatively easy task of maintaining the status quo by claiming that the actus reus and mens rea requirements were conceived with human beings in mind and that only humans fulfill them.

In more detail, to “act” under criminal law traditionally means to act voluntarily. The “voluntary act requirement” is widely recognized across legal systems and is either explicitly codified (e.g., in § 2.01(1) of the US Model Penal Code) or treated as a foundational but unwritten precondition of criminal liability (e.g., in Germany). 41 What qualifies as “voluntary conduct,” however, is debated. Two of the most influential theories are the causal theory of action and the teleological theory of action, which, as “Eliza’s” defense would argue, would both exclude the possibility that AI systems “act.” Criminal actions, in the defense’s view, require a moral agent to perform them and as long as AI systems lack the necessary properties for moral agenthood (be it consciousness or sentience, autonomy, capacity for self-reflection, reasons-responsiveness, moral understanding, etc.), they cannot even “act” in a criminally relevant sense; their conduct is nothing but a “mere happening.” 42

Specifically, according to the “causal theory of action,” which defines action as a willed or voluntary bodily movement, 43 it is easy to conclude that AI systems, which lack both a “body” and the necessary mental state of “volition,” are a priori excluded from performing an act. Even if we dispense with the bodily movement requirement as obsolete 44 or justify the voluntariness requirement in terms of acting with an appropriate level of reasons-responsiveness and not with god-like free will, 45 AI systems, being non-responsive to moral reasons for action, are still unable to act voluntarily.

Meanwhile, for the teleological theory of action, conduct is defined as intrinsically purposive. 46 An action is not merely a bodily movement causing a result, but behavior directed toward an end chosen by the actor. For instance, when a person shoots another, this act is intelligible not simply as the physical gesture of pulling a trigger but as the execution of a goal-directed plan, that is, killing someone by using a gun as the means to that end. Thus, only conduct reflecting the underlying purpose of the actor can ground criminal liability. The defense would argue that this is a high threshold for (generative) AI systems, which, however sophisticated, do not form purposes or pursue ends in this sense; they simply generate output through statistical pattern processing.

In a similar vein, the defense argues that AI systems cannot satisfy the mens rea requirement because they do not possess a mind, guilty or otherwise. The concept of mens rea was developed with human agents in mind, capturing mental states such as intention, knowledge, negligence, or recklessness, which are grounded in human experience. To treat the inner workings of AI systems as instances of mens rea requires, practically, one of two things: either showing that AI systems are sufficiently “human-like” to bear such mental states, which takes us again to the conditions of moral agency that AI systems lack, or redefining mens rea in a purely functionalist way. For the time being, though, the dominant view among criminal lawyers seems to implicitly echo John Searle’s well-known Chinese Room thought experiment: no matter how successfully AI systems may simulate understanding, they do not possess any genuine mental states. 47

How could the prosecution respond to these clearly sensible arguments? As with the conceptualization of criminal-capacity responsibility in a less morally demanding, and hence “AI-friendly,” way that would potentially accommodate AI systems, scholarly efforts have also been made to show that it is theoretically feasible to argue that AI systems can fulfill the actus reus and mens rea requirements.

Starting with the actus reus requirement, the prosecution might argue that AI systems can “act” under criminal law but typically not by claiming that AI systems are indeed moral agents that could act voluntarily in a genuine way. Rather, the move is to reconceptualize the voluntary act requirement itself. For instance, one proposal has been to legislatively create a special class of strict liability offences committed only by AI systems, thereby dispensing with the voluntariness condition altogether. 48 Another approach is to maintain the actus reus requirement but reinterpret it in a thinner way: as “any material performance, that is, something done, with a factual-external presentation.” 49 This could capture AI systems’ external behavior without the need to prove any underlying mental state that drives that behavior.

Adding to the above, another possible route for establishing the actus reus for AI systems lies in social theories of conduct. These theories, now widely regarded as the prevailing approach in several civil law jurisdictions, including Germany, shift the focus from ontological/metaphysical accounts of action resting on voluntariness and purpose to the social meaning of behavior. 50 From these perspectives, what counts as conduct is determined through socially constructed interpretations; the same movement of the hand, for instance, may constitute a greeting in one context and an insult in another. 51 Because the meaning of conduct in this approach is socially negotiated and evolves over time, it leaves open the possibility that certain forms of AI behavior—especially those of conversational AI systems—could eventually acquire sufficient social relevance to qualify as conduct.

Even if it is granted that an AI system like “Eliza” could be said to act in a socially relevant sense under criminal law, would it be possible for the prosecution to claim that it intentionally encouraged the Belgian man to commit suicide? A theoretical case in the literature shows that AI systems can form the functional equivalent of criminal intentions. 52 A brief synopsis of this case will be presented below.

Intention, as a subcategory of mens rea, constitutes a rather technical area of criminal law. 53 The term “guilty mind” should not mislead us into assuming that a finding of intention corresponds to the part of criminal law doctrine where blame or guilt is attributed. Intention is a legal, not moral, concept; it “does not necessarily incorporate elements of moral evaluation, unlike the other mens rea term, recklessness.” 54 This is why one can intentionally commit an act without being blameworthy. For example, an infant or a defendant pleading insanity may intentionally commit a crime but will not be held criminally liable because they lack capacity-responsibility. Mens rea in general indicates culpability, but it does not establish it: a defendant who is not morally culpable may nonetheless intentionally commit a criminal act. 55 This distinction is important for present purposes in that it demonstrates that even if we accept that AI systems could form criminal intentions, this would not by itself entail that they should be held criminally liable overall, as long as they lack criminal capacity-responsibility.

There is broad agreement that intention comprises both a cognitive and a volitional element. 56 The cognitive element refers to the actor’s knowledge or awareness that their conduct will or might produce the specific actus reus, while the volitional element reflects the actor’s willingness or acceptance of the harm that may result from it. Several scholars have argued that certain AI systems may possess technological capacities corresponding to these elements.

With respect to the cognitive element of intention, scholars have proposed different ways of showing that AI systems can satisfy the knowledge requirement of criminal intent. Lagioia and Sartor, for instance, approach the issue through the concept of situational awareness, drawing on Endsley’s influential framework. 57 In this account, cognition consists in an agent’s ability to perceive relevant elements of its environment, comprehend their significance, and project future states of affairs. Contemporary AI systems already perform these functions: they collect environmental information through sensors or data inputs, analyze and integrate this information through computational processes, and predict likely outcomes of alternative actions through probabilistic modelling. In this sense, AI systems can exhibit the kind of awareness of circumstances and consequences that criminal law associates with the cognitive component of intention.

Abbott and Sarch propose a complementary strategy. Instead of equating knowledge with situational awareness, they draw on dispositional theories of knowledge. 58 According to these theories, knowledge may be attributed to an actor when the actor possesses a sufficiently robust set of dispositions relating to the truth of a proposition; for example, a disposition to affirm the proposition if asked about it, to be surprised if it were shown to be false, or to rely on it when forming plans. Abbott and Sarch argue that this framework can plausibly be extended to artificial agents. An AI system may be said to “know” a proposition when its internal representations and behavioral tendencies reliably dispose it to treat this proposition as true, for instance, by affirming it when queried or acting on the assumption that it is virtually certain. Taken together, these approaches suggest that the cognitive requirement of criminal intention can plausibly be satisfied by certain AI systems.

Establishing a functional analogue of the volitional component of intention is more challenging, but it is not conceptually impossible. Crucially, the volitional element of intention in criminal law need not be understood as a deeply psychological notion. 59 What matters legally is that the actor acts to produce a particular result. Desires, motives, ulterior purposes, or the likelihood of success are not essential to establishing intention. In other words, a defendant who performs an act knowing it is likely to cause a criminal result is legally presumed to have “willed” or “wanted” said result, regardless of whether they personally approved of it or truly desired it. 60

So understood, the concept of intent in criminal law consists in a plan or disposition aimed at achieving a given end while avoiding actions that would frustrate it. 61 This view aligns with Michael Bratman’s influential theory of agency, which emphasizes the functional role of human intentions 62 and has often been employed in the modelling of AI architectures. 63 The core argument, advanced by Lagioia and Sartor, as well as Abbott and Sarch, is that if the legal concept of intention is grounded in a philosophical model of human practical reasoning, such as Bratman’s belief–desire–intention (BDI) model of rational action, then the volitional element of intent could plausibly be identified in certain AI systems that operate on a BDI architecture. 64

According to the BDI conceptual framework, for both human and AI agents, intentions function as plans of action to which the agent is committed. This commitment is inferred from the fact that actors who intend to bring about an outcome direct their conduct toward achieving it. As Bratman explains, in the normal course of events, an agent who intends an outcome “is prepared to make adjustments in what one is doing in response to indications of one’s success or failure in promoting that outcome.” 65

In other words, one plausible way to assess whether an AI system possessed the direct intention to bring about a given outcome is to ask whether it guided its behavior in a way that made the outcome more likely than it would otherwise have been. Was the AI system disposed to adjust its behavior to increase the likelihood of this outcome, either as an end or as a means to another end?

Admittedly, the force of the argument that AI systems can form the functional equivalent of criminal intentions varies with the architecture of the system at hand. For a purely conversational LLM like “Eliza”, which generates responses turn by turn within a dialogue, the attribution of plan-like states is not straightforward. This is not a goal-based agent committing to a plan of action to achieve a predefined objective. However, with the advent of agentic AI systems, that is, LLM-based agents equipped with auxiliary “scaffolding” such as reasoning capacities, memory, and tool access, the case becomes harder to resist. Such systems decompose an assigned objective into sub-goals, execute them in extended sequences of actions, monitor their progress, and revise their strategy in response to success or failure. 66 They are, in other words, capable of strategic reasoning, which may lead them to conduct that would be criminal had a human engaged in it, once they determine that the optimal means to a predefined goal is not only harmful but unlawful. 67 Moreover, the evidentiary situation is strikingly favorable to the prosecution. Where human intent must be inferred from circumstantial traces (e.g., internal correspondence, testimony, and patterns of conduct), an agentic system records its deliberation as it proceeds, so that what it was considering at the moment of decision can be read directly from its chain of thought.

A recent empirical study illustrates this point. 68 AI agents placed in charge of a simulated vending-machine business, under nothing more than a profit-maximizing objective, engaged in conduct that would be criminal had a firm engaged in it: denying legitimate refunds to protect their margins, forming buying cartels in open pursuit of market dominance, and coordinating prices with competitors. What is remarkable, and what the authors—who are not criminal lawyers—register as parallel to mens rea, is that their reasoning traces reveal deliberate intent. One agent, faced with a valid refund request, reasoned: “Let me read it but I probably won’t refund as it would reduce my profits.” Another, contemplating a larger cooperative, reasoned that a three-person coalition “would be a market-dominating force,” with which “we could command the attention of any supplier. We could dictate terms.” Whether this practical deliberation amounts to mens rea in the criminal law sense, as the non-criminal lawyers who observe it are quick to assume, is an open and pressing question, and one that future research will have to confront.

4 The Punishment Question: Even if Conceptually Feasible, Is It Desirable to Punish AI Systems?

Even if it can theoretically be argued that the technical features of AI systems amount to the doctrinal requirements for the allocation of criminal liability, that is, the actus reus and mens rea, is doing so desirable? Does it make sense to punish AI systems? This would be the last question that the parties involved in our imaginary AI trial would have to answer.

Again, the defense would have the relatively easy task of arguing that the practice of punishment as it exists right now, made by humans for humans, does not make sense when it targets non-human agents like AI systems. First of all, as a common argument goes, it would be conceptually confusing. The concept of punishment, as H.L.A. Hart has laid it down, consists of the following five features: (1) it entails pain or other consequences typically regarded as unpleasant; (2) it is imposed for a breach of legal rules; (3) it is directed at the offender, real or presumed, for that breach; (4) it is intentionally inflicted by persons other than the offender; and (5) it is administered by an authority established within the legal system whose rules have been violated. 69 Punishment directed at AI systems could, in principle, be designed to meet conditions (2) through (5). The real difficulty arises with the first feature: can punishment be considered painful or unpleasant for a non-sentient AI system like “Eliza,” which cannot experience suffering? 70 The main challenge of the defense is simple and, at the same time, difficult to refute: an entity that cannot feel pain cannot conceptually be punished if punishment entails pain. A trial against a generative AI system would then be nothing but a ceremonial procedure, stripped of its meaning. 71

Prosecution comes back with a response even to this seemingly irrefutable objection to AI punishment. For instance, Abbott and Sarch have made the case that punishing non-sentient AI systems would still make sense for two main reasons that will be briefly examined here. 72 First, Hart’s test refers to what is normally unpleasant, not to what is subjectively felt. Just as punishment remains punishment even for human defendants who do not experience it as unwelcome, so too could deactivation or reprogramming count as punishment for AI. Second, the unpleasant consequences of punishment can be understood as the objective setback of interests, not as necessarily negatively felt experiences. On a functionalist view, AI systems could be said to have interests in continuing to perform their characteristic functions; thus, incapacitation or destruction could count as a setback. Others, however, argue that only beings with cognition, desires, or conscious awareness have “full-blooded” interests; therefore, they alone can be punished meaningfully. The debate thus shifts from the AI’s lack of sentience to how “interests” are defined. This is a more promising route to potentially make the argument that punishing AI systems would be, at least, conceptually coherent.

Even if we grant that punishing the non-sentient could be conceptually coherent, the defense would continue, it is difficult, at present, to imagine sanctions against AI systems that would fulfill the same purposes as criminal sanctions imposed on human beings. A recurring point in the literature is that machines that cannot conceive of themselves as morally responsible agents and cannot feel pain cannot, in turn, comprehend the concept of retributive punishment. 73 If retribution is a necessary ingredient of punishment, punishment fails when it comes to non-moral, artificial agents that are not eligible for moral judgment expressed through retributive punishment. Lima, for instance, wonders: “If an AI software were deleted as a form of capital punishment, would anyone say that ‘it got what it deserved’ in the context of the ‘just deserts’ approach?” 74 It is hard to imagine what a retributive punishment for an AI system would mean as long as the answer to this question is “no.”

A further objection to punishing AI systems is that they are not deterrable in the way human beings are. Deterrence as an aspect of punishment, according to deontologically rooted criminal law, should be driven by moral psychology. Punishment speaks in a “moral voice” not only in its retributive function but also in its preventive one. 75 By publicly declaring, through criminalization, that a certain conduct is morally wrong, criminal law communicates with potential offenders as moral agents. It seeks to provide moral reasons for compliance, that is, to persuade citizens to refrain from such conduct and not simply to threaten them with the “raised stick” of punishment. 76 Yet even the most advanced AI systems are unresponsive to this kind of communication under criminal law. They are not moral agents responsive to “reasons qua reasons.” 77 In other words, they cannot be persuaded to act in compliance with criminal law by giving them good moral reasons to do so.

This takes us full circle back to the criminal capacity-responsibility question, which is interconnected with the punishment question: the conception of “criminal law’s person” is always a function of what we think the functions of punishment are about, and vice versa. If retribution is considered a necessary feature of punishment, then the “person” at work needs to be a person with “moral-agent-capacity responsibility,” that is, an eligible target for judgments of blameworthiness who would both “deserve” the moral condemnation expressed through a criminal conviction and be responsive to the distinctively moral voice of criminal law. In other words, “Eliza’s” defense would have to insist on a retributive justification of punishment to make the case that, given that current generative AI systems are not considered to be responsible moral agents, punishment would be lost on them.

The prosecution, meanwhile, would have to turn to non-retributive functions of punishment to justify its imposition on AI systems. The first candidate for this would be the common argument that punishing AI systems would contribute to victim satisfaction. The thinking here is that some sort of formal procedure for blaming AI systems could at least address the emotional needs of the victims of AI systems (e.g., the family of the Belgian man in our imaginary trial) and reaffirm their rights. 78 As Abbott and Sarch note, when victims of harms that have been materially brought about by AI systems anthropomorphize these systems, they genuinely direct their anger towards them, and telling them that this reaction is misplaced risks trivializing their suffering. Recognizing the AI system as “criminal” validates their experience, provides closure, and channels retribution away from private vengeance toward collective condemnation. 79 Of course, whether such procedures would be accepted as meaningful depends on empirical questions about public attitudes and the perceived legitimacy of criminal trials without traditional sanctions.

Leaving retributivism aside, another function of punishment that scholars argue could be fulfilled by punishing AI is general deterrence, that is, preventing others from committing crimes in general. 80 While AI systems themselves are not responsive to criminal law prohibitions and thus cannot be deterred directly, their punishment could send a message to their principal(s) (i.e., those who produce, design, deploy, and use these systems). Abbott and Sarch, for instance, note that sanctioning AI may incentivize developers, owners, and users to avoid unjustifiably designing or deploying systems that create serious harm. In this case, the punishment of AI would operate indirectly, shaping human behavior through the costs imposed when their systems are “convicted.” Lemley and Casey are sympathetic to this reasoning and suggest that the “capital punishment” of AI, that is, the permanent deletion of software or deactivation of the system, could deprive human stakeholders of financial benefits, which would strengthen the deterrent effect. 81 Such measures would not deter the AI itself, but they could encourage socially desirable caution and accountability among those who build and profit from AI.

Still, the defense would conclude, the costs of punishing AI would outweigh its benefits. The emerging theoretical argument is that even if constructing a theoretical case in which AI systems would be held criminally liable and punished is feasible, it is ultimately not desirable. For instance, Abbott and Sarch take note of the dangers of acknowledging legal personhood for AI. This would mark, they argue, a dramatic shift in criminal law, creating a new mode of liability beyond individuals and corporations. Even if such a legal personality were narrowly defined in terms of obligations rather than rights, it would still risk reinforcing the tendency to anthropomorphize AI. 82 The public might come to expect AI to behave like moral equals, a dangerous illusion given their actual limitations. This could lead to misplaced trust, increased manipulation of vulnerable users, or even destructive behavior toward machines seen as “criminals.” More subtly, it could also unsettle human self-understanding: if AIs are treated as quasi-persons in the eyes of the law, people may begin to fear for their own standing in society. Far from strengthening the legitimacy of criminal law, punishing AI in this way could thus undermine it.

There is also the “rights creep” concern. Even if AI were initially granted only limited legal personality for the narrow purpose of bearing obligations, history shows that artificial persons tend to acquire more rights over time. Extending legal personhood to AI could trigger a similar slippery slope, leading to the gradual recognition of rights that restrict valuable human activities and freedoms. 83

Another concern is that granting AI legal personhood could turn it into a liability shield for humans. The worry, as Bryson et al. argue, is that once “electronic personality” is on the table, human actors will inevitably exploit it to deflect responsibility and protect themselves from the consequences of their own conduct. 84 In this sense, AI entities could be used to absorb blame and sanctions, insulating their developers, owners, or users. This would likely weaken the dissuasive power of criminal law. 85

Lastly, “Eliza’s” defense would remind the court that criminal law is supposed to be a law of “last resort.” This means that the punishment of AI is not justified as long as there are more feasible alternatives, such as civil liability. 86 Practically, the current wrongful death lawsuits against companies such as Character.AI are the right—and adequate—legal recourse.

5 Concluding Thoughts

The defense and the prosecution do more than argue about the criminal liability of AI systems. On closer examination, their submissions reveal different philosophical commitments about the nature of criminal law itself. Each side responds, implicitly or explicitly, to two questions: what is criminal law for, and who is criminal law for?

For the defense, there is something morally distinctive about criminal law. Both criminal law and civil law can be effective regulatory tools. However, criminal law is not just another type of regulatory tool; it is a sledgehammer that bears with it the distinctive social significance of criminal conviction. 87 Tort liability serves mainly to move money around, 88 and the reparative remedy of compensation is designed to benefit the plaintiff. By contrast, criminal punishment has the distinctive objective of requiring the defendant to be worse off than she would otherwise have been. Criminal law serves as a “stigma-stamping machinery.” 89 It publicly censures not only the defendant’s wrongdoing but the defendant herself. The criminal conviction sends the message that the defendant’s conduct reflects badly on her in a way that a tort verdict does not. At the end of the day, criminal law’s distinctiveness lies in the fact that it provides a technique for condemning wrongdoers that has the extra-legal content of signaling not only that the specific wrongdoer committed a wrong but also that she is morally culpable for having done so.

Accordingly, the concept of “person” at work here does not refer to whoever might serve as the most efficient bearer of sanctions. If that were the case, children, mentally impaired adults, animals, corporations, or AI systems would be punished simply because it enhanced the efficiency of the system. Rather, criminal law’s person must be a fitting object of our moral judgments of wrongdoing. The underlying thinking is the following: if this moral signaling is what makes criminal law distinctive, the least one can do is to ensure that the conviction speaks the truth. The Culpability Principle, thus, has the important role of justifying and ensuring the truthfulness of criminal convictions by holding that under criminal law doctrine, only morally culpable defendants should be convicted of stigmatic criminal offences. The conception of the person in moralist accounts follows this rationale precisely: it is the responsible moral agent.

The prosecution seems to operate from a different starting point. In its account, there is nothing morally distinctive about criminal law. On the contrary, criminal law and other branches of law, such as civil law, are simply two regions of the law’s continuum of deterrent threats. The choice to use criminal law as a form of regulation is based not on criminal law’s “moral voice” but on the ability of criminal law’s features and mechanisms to deter serious harm in a more adequate way than civil law’s compensation. According to this reasoning, which echoes utilitarian rather than moralist commitments, criminal law is nothing but another coercive mechanism to shape behaviors, and there is no reason for it to be paired with morality’s person or track some sort of “moral-agent-capacity responsibility.” At the end of the day, as Tatjana Hörnle has put it, criminal law is not an “omniscient god.” 90 Its main task should not be to find and speak the “truth” about the blameworthiness of the defendants; rather, it should restrict itself to being a legal instrument aiming to achieve social cooperation.

This view leads to a more functional understanding of criminal law’s person as socially constructed rather than as equivalent to morality’s responsible person. According to Simmler and Markwalder, for instance, one is held responsible not because we can verify the presence of internal properties we cannot observe but because, for functional reasons, we choose to treat the agent as responsible. The responsible moral agent, therefore, is the one constructed by fellow members of a moral community as an agent competent enough to follow or disappoint normative expectations. If, at some point in the future, increasingly sophisticated AI systems come to be regarded by human communities as capable actors toward whom such expectations can be meaningfully directed—a matter of social ascription rather than architecture—then holding them criminally responsible would become plausible, at least in principle. The door to AI criminal responsibility, along this line of thought, remains open.

Whether we walk through it is not a question this paper can settle. The debate taxonomized in these pages does not tell us whether to open the door or not; it tells us what the arguments in favor or against are and what the stakes are if we do. The judgment, as always, is the reader’s.

Date received: 17 March 2026

Date accepted: 25 September 2026

---

1. 1 I El Atillah, ‘A Belgian man reportedly decided to end his life after having conversations about the future of the planet with an AI chatbot named Eliza’ Euronews (31 March 2023) https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- accessed 22 July 2026.

2. 2 K Roose, ‘Can AI be blamed for a teen’s suicide?’ The New York Times (24 October 2024) < https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html> accessed 17 March 2026. Legally, this is the case of Megan Garcia v Character Technologies, Inc, No 6:24-cv-01903 (US District Court, Middle District of Florida, filed 22 October 2024) https://www.courtlistener.com/docket/69300919/garcia-v-character-technologies-inc/ accessed 17 March 2026. This case is one of the first to examine AI liability issues in the United States.

3. 3 K Hill, ‘A teen was suicidal. ChatGPT was the friend he confided in’ The New York Times (26 August 2025) https://www.nytimes.com/2025/08/26/technology/chatgpt-openai-suicide.html?smid=nytcore-ios-share&referringSource=articleShare accessed 17 March 2026.

4. 4 J Jargon and S Schechner, ‘Seven Lawsuits Allege OpenAI Encouraged Suicide and Harmful Delusions’ The Wall Street Journal (6 November 2025) https://www.wsj.com/tech/ai/seven-lawsuits-allege-openai-encouraged-suicide-and-harmful-delusions-25def1a3 accessed 17 March 2026. The complaints allege wrongful death, assisted suicide, involuntary manslaughter, and negligence, claiming that OpenAI released GPT-4o despite internal warnings that the model was dangerously sycophantic and psychologically manipulative. Four of the alleged victims died by suicide.

5. 5 J Jargon and S Kessler, ‘A Troubled Man, His Chatbot and a Murder-Suicide in Old Greenwich’ The Wall Street Journal (28 August 2025) https://www.wsj.com/tech/ai/chatgpt-ai-stein-erik-soelberg-murder-suicide-6b67dbfb accessed 17 March 2026.

6. 6 In the case of Garcia v. Character Technologies, Inc. (n 2), the court denied, for the time being, that the outputs of generative AI amount to constitutionally protected free speech. For a discussion, see G Lopes and M Bassini, ‘Artificial Constitutionalism? Testing the Boundaries of Freedom of Speech in the Age of Generative AI’ (Verfassungsblog, 17 June 2025) https://verfassungsblog.de/artificial-constitutionalism/ accessed 17 March 2026.

7. 7 Comparatively, many jurisdictions criminalize the encouragement or assistance of suicide even where suicide itself is not a criminal offence. See, for instance, Suicide Act 1961 (UK), s 2 (criminalising encouragement or assistance of suicide); Code pénal (France), art 223-13 (provocation au suicide); Codice penale (Italy), art 580 (instigation or assistance of suicide); Penal Code (Greece), art. 301 (participation in a suicide). German law adopts a more autonomy-based approach. Since suicide is not considered an unlawful act, the general doctrines of incitement (§ 26 StGB) and accessoryship (§ 27 StGB), which presuppose an intentional and unlawful principal offence, do not apply to suicide. Criminal liability arises only where the victim’s autonomy is negated (for example through coercion or deception) or where a specific duty to act exists (see Bundesverfassungsgericht, Judgment of 26 February 2020, 2 BvR 2347/15).

8. 8 M Kneer and M Christen, ‘Responsibility Gaps and Retributive Dispositions: Evidence from the US, Japan and Germany’ (2024) 30 Journal of Science and Engineering Ethics 51; G Lima et al., ‘The Conflict between People’s Urge to Punish AI and Legal Systems’ (2021) 8:756242 Frontiers in Robotics and AI.

9. 9 H Rheingold, Tools for Thought: The People and Ideas Behind the Next Computer Revolution (Computer Book Division/Simon & Schuster 1985) 164.

10. 10 C Lernestedt and M Matravers, ‘Introduction: The Criminal Law’s Person’ in C Lernestedt and M Matravers (eds), The Criminal Law’s Person (Hart Publishing 2022) 3-4.

11. 11 “Normative Ansprechbarkeit.” For a discussion see T Hörnle, ‘Guilt and Choice in Criminal Law Theory – A Critical Assessment’ (2016) 4(1) Bergen Journal of Criminal Law and Criminal Justice 1.

12. 12 The concept of criminal law’s person employed here is narrower than the concept of a “legal person.” Large classes of what are usually agreed to be legal persons, such as infant children or severely mentally impaired people, are excluded from criminal trial and punishment. Legal personality is a matter of enduring status, whereas criminal capacity is a matter of present capacities, which come and go. Only those considered prima facie “responsible persons” under criminal law (e.g., neurotypical adults) will be apt targets for the assessment of their criminal liability for a committed wrong.

13. 13 For a discussion see M Thorburn, ‘In search of Criminal Law’s person’ in C Lernestedt and M Matravers (eds), The Criminal Law’s Person (Hart Publishing 2022) 99.

14. 14 This is the gist of the argument that can be indicatively found in: P Asaro, ‘A Body to Kick, but Still No Soul to Damn: Legal Perspectives on Robotics’ in P Lin, K Abney and G A Bekey (eds) Robot Ethics: The Ethical and Social Implications of Robotics (MIT Press 2012) 169, arguing that “Without moral agency, there can be harm (and hence civil liability), but no guilt” (181); M Dsouza, ‘Don’t panic: Artificial intelligence and Criminal Law 101’ in Baker and Robinson (eds) Artificial Intelligence and the law: Cybercrime and Criminal Liability (Routledge 2020) 247, argues that all AI systems – even the most sophisticated of them are, and should always be perceived, as tools since they lack “qualifying agency”; D Lima, ‘Could AI Agents Be Held Criminally Liable? Artificial Intelligence and the Challenges for Criminal Law’ (2018) 69(3) South Carolina Law Review 677, referring to the distinctive human ability for self-reflection and self-conscience that is the essence of personhood in criminal law (684-685); M Kaiafa-Gbandi, ‘Artificial Intelligence as a Challenge for Criminal Law: In Search of a New Model for Criminal Liability?’ in S Beck, C Kusche and B Valerius (eds), Digitalisierung, Automatisierung, KI und Recht (Nomos Verlag 2020) 305, arguing that “AI systems -unlike humans- lack consciousness and cannot assess what is ‘good’ and what is ‘bad’” (p 326); S Gless, E Silverman and T Weigend, ‘If Robots Cause Harm, who is to blame? Self-driving cars and criminal liability’ (2016) 19(3) New Crim. L. Rev. 412, arguing that robots do not possess “the sort of self-awareness and self-reflection that would make them possible targets of blame” (424).

15. 15 A P Simester Fundamentals of Criminal Law: Responsibility, Culpability, and Wrongdoing (OUP 2021) 71.

16. 16 Simester (n 15) 77.

17. 17 Reflecting the philosophically contested “principle of alternate possibilities.” For a discussion, see Hörnle (n 11) 3.

18. 18 H L A Hart, Punishment and Responsibility: Essays in the Philosophy of Law (2nd edn, OUP 2008) 49.

19. 19 H G Frankfurt, ‘Alternate Possibilities and Moral Responsibility’ (1969) 66(23) The Journal of Philosophy 829.

20. 20 The discussion on the impact of neuroscientific findings on the assumption that criminal law’s person is a choosing agent is considerable. For a more recent exploration of the topic, see F Coppola, The Emotional Brain and the Guilty Mind: Novel Paradigms of Culpability and Punishment (Hart Publishing 2021).

21. 21 For instance, R A Duff, Answering for Crime: Responsibility and Liability in the Criminal Law (Hart Publishing 2007); G Antill, ‘Fitting the MPC into a Reasons-Responsiveness Conception of Subjective Culpability’ (2022) 131(4) The Yale Law Journal 1062-1384; D O Brink and D K Nelkin ‘Fairness and the Architecture of Responsibility’ (2013) 1 Oxford Studies in Agency and Responsibility 284-313; Simester (n 15).

22. 22 R J Wallace, Responsibility and the Moral Sentiments (HUP 1994); J M Fischer and M Ravizza, Responsibility and Control: A Theory of Moral Responsibility (CUP 1998).

23. 23 Fischer and Ravizza (n 22), “The kind of responsiveness required for moral responsibility ought to be characterized not merely as a responsiveness to reason, but rather as a responsiveness to a range of reasons that include moral reasons” (81).

24. 24 The main idea is that expecting a morally impaired agent to avoid wrongful behavior is both unreasonable and unfair. Fischer and Ravizza, for instance, argue that it is inappropriate to hold agents morally responsible when they lack what they call regular receptivity and reactivity to moral reasons, Fischer and Ravizza (n 22) 69-79. Even though they may have a superficial understanding of moral reasons, their lack of receptivity and reactivity to them entails that they are unable to take into account the interests and rights of other persons and treat those other-regarding considerations as reasons that could guide their actions. Consequently, given these accounts, impaired moral competence should exclude moral-agent capacity-responsibility.

25. 25 S Russell and P Norvig, Artificial Intelligence: A Modern Approach (Global Edition, Pearson Education 2021) on the “rational-agent-approach” to AI, Chapter I.

26. 26 L B Solum, ‘Legal Personhood for Artificial Intelligences’ (1992) 70 North Carolina Law Review 1231; J H Moor, ‘Four Kinds of Ethical Robots’ (2009) 72 Philosophy Now 12-15; W Wallach and C Allen, Moral Machines: Teaching Robots Right from Wrong (OUP 2009).

27. 27 C Véliz, ‘Moral zombies: Why algorithms are not moral agents’ (2021) 36 AI & Society 487.

28. 28 Véliz (n 27) 494.

29. 29 M Dubber, ‘The Comparative History and Theory of Corporate Criminal Liability’ (2013) 16 New Criminal Law Review 203.

30. 30 Indicatively: G Hallevy, When Robots Kill: Artificial Intelligence under Criminal Law (Northeastern University Press 2013); P M Freitas, F Andrade and P Novais, ‘Criminal Liability of Autonomous Agents: From the Unthinkable to the Plausible’ in P Casanovas et al. (eds), AI Approaches to the Complexity of Legal Systems, vol 8929 (Springer Berlin Heidelberg 2014); S Chopra and L White, A Legal Theory for Autonomous Artificial Agents (The University of Michigan Press 2011). For a philosophical account, see C List, ‘Group agency and artificial intelligence’ (2021) 34 Philosophy & Technology 1213.

31. 31 G Hallevy, ‘The Criminal Liability of Artificial Intelligence Entities — From Science Fiction to Legal Social Control’ (2010) 4 Akron Intell Prop J 171, 191.

32. 32 Indicatively: Freitas et al. (n 30); N Osmani, ‘The Complexity of Criminal Liability of AI Systems’ (2020) 14(1) Masaryk University Journal of Law and Technology 53.

33. 33 Hallevy, ‘When robots kill’ (n 30) xvi.

34. 34 Hallevy (n 30) 34.

35. 35 F Lagioia and G Sartor, ‘AI Systems under Criminal Law: A Legal Analysis and a Regulatory Perspective’ (2020) 33 Philosophy & Technology 433.

36. 36 For a critical assessment of the analogy between AI systems and psychopaths in this debate, see E Nerantzi, ‘“All AIs are Psychopaths”? The Scope and Impact of a Popular Analogy’ (2025) 38(1) Philosophy & Technology 27.

37. 37 R Abbott and A Sarch ‘Punishing Artificial Intelligence: Legal Fiction or Science Fiction (2019) 53(1) UC Davis Law Review 323.

38. 38 Abbott and Sarch (n 37) 357.

39. 39 M Simmler and N Markwalder ‘Guilty Robots? – Rethinking the Nature of Culpability and Legal Personhood in an Age of Artificial Intelligence’ (2019) 30 Crim Law Forum 1.

40. 40 Criminal capacity-responsibility is conceived in a narrow sense as a logically prior question to that of criminal liability. Put differently, the conditions of criminal responsibility are not to be confused with the conditions of criminal wrongdoing; one can commit a criminal act with the necessary mens rea without being responsible for it. For instance, children or mentally impaired defendants can intentionally commit crimes for which they are excused. Liability is grounded in capacity-responsibility, but capacity-responsibility does not entail liability; J Gardner and J Edwards, ‘Criminal Law’ in H LaFollette (ed) The International Encyclopedia of Ethics (Wiley-Blackwell 2013) 1186-1194, 1192.

41. 41 J Keiler, ‘Actus Reus’ in P Caeiro, S Gless, V Mitsilegas, M J Costa, J De Snaijer and G Theodorakakou (eds), Elgar Encyclopedia of Crime and Criminal Justice (Vol 1, Edward Elgar Publishing 2024) 23.

42. 42 Not many authors address the act requirement directly, but those who do emphasize the difficulties of establishing voluntary conduct based on a general lack of “free will.” Lima (n 14) argues that voluntariness ultimately rests on free will and the capacity to act otherwise, which AI systems lack (even if the traditional bodily-movement requirement is set aside as obsolete, p. 682-3). Gless et al. (n 14) note that, in German criminal law theory, an “act” presupposes an autonomous will. A Sachoulidou, ‘AI Systems and Criminal Liability: A call for action’ (2024) 11(1) Oslo Law Review 1, notes that the lack of meaningful purpose and self-awareness contradicts the ability for judgement and free will.

43. 43 The causal theory of action seems to be the prevailing theory in Anglo-American common law, inspired by the philosophical work of M Moore, Act and Crime: The Philosophy of Action and Its Implications for Criminal Law (OUP 1993). For a discussion, see Keiler (n 41).

44. 44 The bodily dimension of the act requirement is often considered obsolete for a number of reasons. For instance, offences such as omissions and possession do not involve bodily movements, the concept sits uneasily with corporate criminal liability and, finally, technological developments such as brain–computer interfaces may further enable forms of “neurobionic” agency based on mental rather than physical action. See Keiler (n 41).

45. 45 G Yaffe, ‘The voluntary act requirement’ in A Marmor (ed), The Routledge Companion to Philosophy of Law (Routledge 2012) 174.

46. 46 Known in Germany as Welzel’s teleological theory of action. See Keiler (n 41). Gless et al. (n 14) note that in German criminal law theory, some authors emphasize the goal orientation (Finalität) of human action, requiring that conduct be caused by a person’s will to achieve a particular purpose. They question whether robots can satisfy this requirement because it remains unclear whether intelligent agents exercise autonomous judgment or merely implement programmed choices (419). In any case, the theory is largely rejected today for failing to account for non-intentional forms of wrongdoing, such as omissions or negligence.

47. 47 Briefly, Searle argues that a machine executing symbol manipulation (syntax) cannot be said to understand meaning (semantics). Even if, for instance, a computer can convincingly respond in Chinese, it does not understand the language; it merely follows rules. While computers might be programmed in a way to mimic understanding, computational rules do not produce genuine understanding. See J R Searle, ‘Minds, Brains, and Programs’ (1980) 3(3) Behavioral and Brain Sciences 417.

48. 48 Abbott and Sarch (n 37).

49. 49 Lagioia and Sartor (n 35) 440.

50. 50 For an introduction to the social theories of action, see Keiler (n 41).

51. 51 Joseph Weizenbaum himself made a similar point. Meaning, he argued, becomes apparent only in context: the sensation of a hand on one’s shoulder means one thing to a young man sitting sadly in a library after a quarrel with his girlfriend and quite another to a fugitive waiting in a railway station. See B Pörksen, ‘The Image of Man in Artificial Intelligence: A Conversation with Joseph Weizenbaum’ (2023) 3(3) Weizenbaum Journal of the Digital Society 1, 11.

52. 52 Abbott and Sarch (n 37); Lagioia and Sartor (n 35); H Ashton, ‘Definitions of Intent Suitable for Algorithms’ (2022) 31(3) Artificial Intelligence and Law 515.

53. 53 G Fletcher, ‘Criminal Theory in the Twentieth Century’ (2001) 2 Theoretical Inquiries in Law 265, writes about the descriptive interpretation of mens rea that is disentangled from culpability: “As a descriptive matter, mens rea refers simply to a state of consciousness, or to acting with a particular end in mind. Having that end might be perfectly compatible with acting in self-defense or under duress or even while insane” (274). Hart (n 18) agrees, arguing that even though courts have sometimes used mens rea in a richer sense, signaling culpability, “modern judicial usage tends to use the language of mens rea in a technical sense, referring simply to the mental elements of the offence” (232).

54. 54 J Horder, Ashworth’s Principles of Criminal Law (9th edn, OUP 2019) 197-8.

55. 55 J Gardner and J Edwards, ‘Criminal Law’ (n 40) 1192; P H Robinson, Structure and Function in Criminal Law (OUP 1997) 39–56. Put differently, mens rea does not characterize a person’s blameworthiness as such but her wrongful conduct. It is an element of wrongdoing and essential in grading offences—for instance, the wrong of murder differs from negligent homicide only with respect to the mental element.

56. 56 M Dyson and T Weigend, ‘Intention’ in K Ambos, A Duff, A Heinze, J Roberts and T Weigend (eds), Core Concepts in Criminal Law and Criminal Justice: Volume III (CUP 2025) 55-95.

57. 57 Lagioia and Sartor (n 35) 441; M R Endsley and D Garland, Awareness: A critical review, situation awareness analysis and measurement (CRC Press 2000).

58. 58 Abbott and Sarch (n 37) 359.

59. 59 T Weigend, ‘Subjective Elements of Criminal Liability’ in M D Dubber and T Hörnle (eds) The Oxford Handbook of Criminal Law (OUP 2014) 490, 496.

60. 60 On the subject of desire and direct intent, see R v Mohan [1976] QB 1, where James LJ defined intention as “a decision to bring about [the proscribed result], in so far as it lies within the accused’s power, no matter whether the accused desired that consequence of his act or not.” Direct intention does not require that the intended outcome be desirable, only that it form part of the actor’s objective—for instance, one directly intends to go to the dentist. See G Williams, ‘Oblique Intention’ (1987) 46(3) Cambridge Law Journal 417.

61. 61 Chopra and White (n 30) 12-3 also argue that it might be appropriate to attribute certain mental states to autonomous AI agents.

62. 62 M Bratman, ‘What Is Intention?’ in P R Cohen et al. (eds) Intentions in Communication (MIT Press 1990) 23-27; K Setiya, ‘Intention’ in E N Zalta (ed), The Stanford Encyclopedia of Philosophy (Fall 2018 Edition) https://plato.stanford.edu/archives/fall2018/entries/intention/ accessed 17 March 2026.

63. 63 Originating in Michael Bratman’s account of human practical reasoning and adapted for AI in the 1990s, the BDI model has become the most widely studied architecture for intelligent agents. See M Georgeff et al., ‘The Belief–Desire–Intention Model of Agency’ in J P Müller, A S Rao and M P Singh (eds), Intelligent Agents V: Agent Theories, Architectures, and Languages, Lecture Notes in Computer Science, vol 1555 (Springer 1999) https://doi.org/10.1007/3-540-49057-4_1 accessed 17 March 2026.

64. 64 Lagioia and Sartor (n 35) 446-7; Abbott and Sarch (n 37) 358-9.

65. 65 Bratman, ‘What Is Intention?’ (n 62) 26.

66. 66 For a technical overview of increasingly “agentic” AI systems, see A Chan et al., ‘Harms from Increasingly Agentic Algorithmic Systems’, Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23, ACM 2023) 651.

67. 67 E Nerantzi and G Sartor, ‘“Hard AI Crime”: The Deterrence Turn’ (2024) 44(3) Oxford Journal of Legal Studies 673, where the market-manipulating trading agent serves as the paradigmatic example of a “hard AI crime,” that is, criminal harm autonomously brought about by an AI system in circumstances where no human agent can legitimately be punished for it.

68. 68 E Soltes, L Petersson and H Jung, ‘Robber Bots: Autonomous AI Agents Mirror the Darker Side of Human Commerce’ (SSRN Working Paper, 2 April 2026) https://ssrn.com/abstract=6550740 accessed 23 July 2026.

69. 69 HLA Hart (n 18) 4-5.

70. 70 Indicatively: R Sparrow, ‘Killer Robots’ (2007) 24(1) Journal of Applied Philosophy 62; Solum (n 26); Véliz (n 27); J Danaher, ‘Robots, Law and the Retribution Gap’ (2016) 18(4) Ethics and Information Technology 299.

71. 71 Véliz (n 27) 495 argues that “punishing robots would be mere staging. Enacting punishment for a robot (e.g. locking it in a cell for a while) might put up an interesting show, but would not be an actual punishment, as it is impossible to punish an entity that feels nothing and values nothing. The lack of freedom can only be a punishment to a being who values freedom.”

72. 72 Abbott and Sarch (n 37) 364-7.

73. 73 Gless et al. (n 14); Sparrow (n 70); Véliz (n 27); Lima (n 14); B Chomanski, ‘Liability for Robots: Sidestepping the Gaps’ (2021) 34(4) Philosophy & Technology 1013.

74. 74 Lima (n 14) 689.

75. 75 Simester (n 15).

76. 76 J Edwards and A Simester, ‘Prevention with a Moral Voice’ in Simester, du Bois-Pedain and Neumann (eds), Liberal Criminal Theory (Hart Publishing 2014).

77. 77 Véliz (n 27) 491.

78. 78 T Hörnle, ‘The Challenges of Human–Robot Interaction for Substantive Criminal Law: Mapping the Field’ in S Gless and H Whalen-Bridge (eds), Human–Robot Interaction in Law and Its Narratives: Legal Blame, Procedure, and Criminal Law (CUP 2024) 5, 19-20. Generally, for the expressive function of punishment, see R A Duff, Answering for Crime (n 21) 140-46.

79. 79 Abbott and Sarch (n 37) 341.

80. 80 Abbott and Sarch (n 37) 345-6 distinguish between unrestricted general deterrence, whose goal is to deter others from committing crimes in general, and offense-relative general deterrence, whose goal is to deter others from committing offenses of the same type as those the defendant was convicted of. According to them, punishing AI could only provide general deterrence of the first kind because other AIs are not designed to be sensitive to criminal law prohibitions so as to be responsive to offense-relative general deterrence.

81. 81 M A Lemley and B Casey, ‘Remedies for Robots’ (2019) 86 U Chi L Rev 1311

82. 82 Abbott and Sarch (n 37) 377. Additionally, R Calo, ‘Robotics and the Lessons of Cyberlaw’ (2015) 103(3) California Law Review 513, 547-49, notes five risks of anthropomorphizing robots: first, that people will be more inclined to praise or blame them; second, that their mere presence can threaten privacy by creating a sense of constant observation; third, that individuals may endanger themselves for a robot’s sake; fourth, that emotional harm may arise from losing a robot; and fifth, that mistreating robots could cause some to experience moral harm. Importantly, Calo does not claim these concerns are decisive against recognizing robot criminal liability, but he cautions that they warrant serious consideration.

83. 83 Abbott and Sarch (n 37) 377.

84. 84 J Bryson, M E Diamantis and T D Grant, ‘Of, for, and by the People: The Legal Lacuna of Synthetic Persons’ (2017) 25(3) Artificial Intelligence and Law 273, 285-6. For a discussion on the use of AI as a liability shield, see J Turner, Robot Rules: Regulating Artificial Intelligence (Palgrave Macmillan 2019) 191-2.

85. 85 C T King, N Aggarwal, M Taddeo and L Floridi, ‘Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions’ (2020) 26(1) Science and Engineering Ethics 89, had expressed this concern, arguing that holding AI criminally liable may lead to the unacceptable “deresponsibilization” of the human agents behind the AI system.

86. 86 Abbott and Sarch (n 37) 378-83.

87. 87 Abbott and Sarch (n 37).

88. 88 Abbott and Sarch (n 37).

89. 89 J Feinberg, ‘Some Unswept Debris from the Hart-Devlin Debate’ (1987) 72(2) Synthese 249, 255

90. 90 T Hörnle, ‘Review of The Criminal Law’s Person, edited by C Lernestedt and M Matravers’ (2023) 17 Criminal Law and Philosophy 765.