1 Introduction

This essay revisits Joseph Weizenbaum’s reflections on the limits of ELIZA, the first chatbot, intended as a provocative probe into human-machine communication (Weizenbaum, 1966; 1976), and my own critical analysis of the same problem (Suchman, 1987, 2007). The latter builds on studies of face-to-face human interaction to detail our shared capacity for generative (in the old sense of the term) co-construction of mutual intelligibility in and through the contingencies of the open worlds that we inhabit together. Crucially, these capacities are irreducible to digits, the sensorial prerequisite for computation. Grounded in an associated critique of the project of artificial intelligence, I argue that across significant shifts in technologies and techniques, the differences that matter between humans (and other beings) and machines (of our making) persist. 1 My project has always been less to define these differences in principle than to closely track their persistence in practice, resist their rhetorical elision, and insist on the artifactual nature of machines. I begin with a review of the capacities that human interaction presupposes and that escape incorporation into the computational sensorium, then consider the question of whether the turn to large language models (LLMs) obviates continuing attention to those incomputable relations. The point, in the end, is not to diminish the capacities of computational systems but rather to demystify their operations, acknowledge their limits, and foreground the human agencies and political economies responsible for their consequences.

2 The Incomputability of Conversation

A commonly held view of conversation is that spekers and listeners, pursuing some topic according to individual predispositions and agendas, engage in an alternating sequence of talk and response. For students of human cognition and language, conversation generally has been treated as epiphenomenal with respect to the central concerns of their fields. Cognitively, conversation is just the meeting ground of individual psychologies, whereas linguistically it is the noisy, real-world occasion for the exercise of basic language abilities. In either view, the additional constraints imposed by situated language use are a complication that obscures the underlying structure of cognitive or linguistic competence.

Since the 1960s, interactionist studies of conversation have moved away from more traditional linguistic analyses of languages as formal systems toward talk-in-interaction as constitutive of and inseparable from social life (Atkinson & Heritage, 1984). Close analyses of face-to-face communication indicate that conversation is not so much an alternating series of actions and reactions between individuals as a joint activity accomplished through the participants’ continuous engagement in speaking and listening (Goodwin, 1981). Viewed as deeply skilled practice, the organization of conversation appears to be closer to improvisation in the musical sense than to the common notion of speaker prompt and listener response. The conventional “rules” of conversation are neither strictly optional (their breach does have consequences) nor obligatory (they may be breached without a necessary loss of coherence). Each turn at talk is, however, inexorably meaningful.

The incomputability of meaning rests in the radical indexicality of reference. Reference is a contingent practice of sense-making, involving an open-ended horizon of associations based in lived experience within personal/biographical and collective/historical circumstances, rather than a predetermined association between word and context, or name and object. Mutually intelligible conversation is enacted through a nuanced choreography of practices deeply embodied and embedded in specific persons and relations, not through message passing or probabilistic associations of words. To be “situated” is to be implicated in co-constituting the significance and relevance of the worlds that we are articulating together, not just to be placed in some surrounding and enumerable conditions. The contingency of action on a complex world of objects, artifacts, and other actors located in space and time is not an extraneous problem with which the individual actor must contend, but rather is the essential resource that makes knowledge possible and gives action its sense.

3 Rendering Language Computable: What Difference Do LLMs and Generative AI Make?

In the mid-20th century, Claude Shannon’s (1949) translation of numerical analysis into The Mathematical Theory of Communication laid the foundations for a model of signal processing as the mechanism not only for computational exchange but also for human cognition and social relations. Turing’s famous test for machine intelligence, formulated the following year (1950) as an experiment in passing, 2 further established the imitation of conversational competencies as a benchmark for intelligence regardless of differences in what Turing conceptualized as the internal operations that produce the relevant effects. Turing’s proposal was that an experiment could be engineered in which participants would be asked to distinguish first between a man attempting to pass as a woman and a woman, then between a man and a machine, based only on questions and responses conveyed across a teletype, without the distorting influence of embodied appearance. Turing’s thought experiment disengages identity from embodiment and posits a normative communicative plane on which what are taken as fundamental categories—first of man and woman, then of man and machine—can be distinguished.

A decade later, Joseph Weizenbaum (1966) implemented a program that he named ELIZA on the MAC time-sharing system at the Massachusetts Institute of Technology. ELIZA is widely cited as the first computational system to pass the Turing test insofar as “the subject finds it credible to believe that the responses which appear on his typewriter are generated by a human sitting at a similar instrument in another room” (Weizenbaum, 1966, p. 42). The program inspects typed input for keywords then transforms it into a response according to rules associated with the keyword in a script. If no keyword is found, the system either produces a “content-free remark” or retrieves an earlier transformation (Weizenbaum, 1966, p. 37). DOCTOR was the first example of an ELIZA script, simulating a plausible conversation with a Rogerian psychotherapist. Most famously, what struck Weizenbaum about the ensuing “dialogues” between the system and human interlocutors was the latter’s extraordinary ability to find significance and relevance in the output that was generated.

Inspired by ELIZA, Garfinkel (1967) elaborated what he named the “documentary method of interpretation” in an experiment that shed further light on Weizenbaum’s observations regarding ELIZA and the DOCTOR program. 3 Set up as an interaction between a student and a counsellor in two different rooms communicating through a teletype, the experiment produced a series of “yes/no” responses driven by a table of random numbers. Garfinkel (1967) observed of students’ interpretations of the “yes/no” patterns in the human random answer generator that

The underlying pattern was elaborated and compounded over the series of exchanges and was accommodated to each present “answer” so as to maintain the “course of advice,” to elaborate what had “really been advised” previously, and to motivate the new possibilities as emerging features of the problem. (p. 90)

Six decades later, the projection of interactional capacities onto computational systems has become commonplace, elaborated first through the rule-based manipulation of scripted text and now through statistical operations performed by billion-parameter LLMs. The advent of LLMs and the further development of so-called chat interfaces for these models, marked by the release of ChatGPT in 2022 (applying techniques of “generative pre-training”), has renewed the proposition that machines have the capacity to engage in conversation with us.

LLMs are predictive mathematical models of the statistical distribution of tokens in a corpus of human-generated text (Shanahan, 2024, p. 70). In the case of foundation models, the corpora are vast datasets scraped from the public internet (largely without consent). Given a sequence of tokens (words, parts of words, punctuation, emojis, etc.), the system calculates the next token based on a conditional probability distribution derived from its reference dataset. In contemporary usage, Shanahan et al. (2023) note that “the term ‘large language model’ tends to be reserved for transformer-based models that have billions of parameters and are trained on trillions of tokens” (p. 493). Model-generated text “prompted” by a string of tokens is then interleaved with further user-supplied text, the former being read by the user as a response to the latter. “Alignment” with normative conventions of what comprises an acceptable response in a given case is “fine-tuned” through so-called “reinforcement learning through human feedback” or through feedback from another LLM trained to align with “constitutionally” encoded prescriptive or proscriptive dialogic sequences.

Importantly for this analysis, every LLM “has a highly specific, well-defined function, which can be described in precise mathematical and engineering terms” (Shanahan, 2024, p. 70). With enough data, computational power, and fine-tuning, this basic function is able to generate an extraordinary range of intelligible output that seems appropriate to us as readers. Yet, crucially, as Shanahan (2024) points out, “knowing that the word ‘Burundi’ is likely to succeed the words ‘The country to the south of Rwanda is’ is not the same as knowing that Burundi is to the south of Rwanda” (p. 72), a statement moreover that we understand differently depending on our relations to the place that is referenced. Learning about Burundi or Rwanda is not simply a matter of updating the distributional frequency of words in the English language. The distinctive capacity of LLMs is that they can generate text that is coherent “in real time,” in the absence of understanding.

4 Why Play the Imitation Game

The effective consequences of the scaling of language (re)generation machines are the subject of widespread public debate, from the utility of so-called “assistants” to the injurious seductions of artificial “companions,” with their respective implications for mis/disinformation and greater social isolation. The lessons drawn by Joseph Weizenbaum from his experiments with the ELIZA program and the DOCTOR script concerned the generative structures of language when inscribed as computable code and the extraordinary ability of human interlocutors to read computer-generated text as meaningfully responsive to their prompts. Arguably, the absence of worldly knowledge on the part of the computational partner holds as much for contemporary LLM-based “chat” as it did for ELIZA. Or rather, the knowledge at play is that which informs the corpora that feed the LLM and the design of the prompt, as much as it did the design of the DOCTOR script. To assume otherwise requires conflating language inscribed as computationally tractable tokens with language as a collectively generated resource for the articulation of lived experience and relations. Perhaps the greatest difference between ELIZA and today’s LLMs is that while the workings of Weizenbaum’s machine could be easily explained (the task that he takes on in his 1966 Communication), the project of granting LLMs “mechanistical interpretability” continues to be elusive (Levy, 2024).

The idea that the body comprises not the fundamental medium of knowing but an encumbrance to it is assumed in the Turing test, where the teletype becomes the medium of exchange. Although the gendered and cultural specificity of the Imitation Game is largely erased in its figuration as a universal test of machine intelligence, recovering the game’s specificity helps us think more critically about projects to design humanlike machines initiated by Turing and his kin. To whom, we might ask, is the project compelling? What progress has been made, and how might we understand the problems that remain unsolved by looking more closely at the figure of the human that is being imitated? Even successful imitation has turned out to be a much more difficult undertaking than Turing anticipated, notwithstanding the extraordinary increase in computing power, the proliferation of so-called chatbots and software agents, and the continual promise of robotic servitude and companionship.

We don’t need to take a stand on whether or not a machine might in principle pass the Turing test to understand the investments (in both senses of the word) that the humanlike machine animates. Discourses of software agency at once naturalize the desirability of greater speed, efficiency, and service provision and further obscure the historical, political, and economic specificity of those desiderata and their associated injuries (Atanasoski & Vora, 2019). So, what might it mean to resist the invitation to play the Imitation Game? A growing movement of scholar/activists is questioning the costs of LLMs and their interfaces for the quality of learning, labour, social justice, and planetary sustainability. Refusing to play the game might then be one small but generative contribution to the larger project of reaffirming continuing human responsibility for each other and for the artifacts of our making.

References

Atanasoski, N., & Vora, K. (2019). Surrogate humanity: Race, robots, and the politics of technological futures. Duke University Press.

Atkinson, J. M., & Heritage, J. (1984). Structures of social action: Studies in conversation analysis. Cambridge University Press.

Dreyfus, H. (1992). What computers still can’t do: A critique of artificial reason. MIT Press.

Eisenmann, C., Mlynář, J., Turowetz, J., & Rawls, A. W. (2024). “Machine Down”: making sense of human–computer interaction—Garfinkel’s research on ELIZA and LYRIC from 1967 to 1969 and its contemporary relevance. AI & Society, 39, 2715–2733. https://doi.org/10.1007/s00146-023-01793-z

Garfinkel, H. (1967). Studies in ethnomethodology. Prentice-Hall.

Goodwin, C. (1981). Conversational organization: Interaction between speakers and hearers. Academic Press.

Levy, S. (2024, May 21). AI is a black box. Anthropic figured out a way to look inside. Wired. https://www.wired.com/story/anthropic-black-box-ai-research-neurons-features/

Shanahan, M. (2024). Talking about large language models. Communications of the ACM, 67(2), 68–79. https://doi.org/10.1145/3624724

Shanahan, M., McDonell, K., & Reynolds, L. (2023). Role play with large language models. Nature, 623, 493–498. https://doi.org/10.1038/s41586-023-06647-8

Shannon, C. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

Suchman, L. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge University Press.

Suchman, L. (2007). Human-machine reconfigurations: Plans and situated actions (Rev. ed.). Cambridge.

Suchman, L. (2019). Demystifying the intelligent machine. In T. Heffernan (Ed.), Cyborg futures: Social and cultural studies of robots and AI (pp. 35–61). Palgrave Macmillan.

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59, 433–460. http://www.jstor.org/stable/2251299

Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://dl.acm.org/doi/pdf/10.1145/365153.365168

Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman.

Date accepted: 18 August 2026

---

1. 1 While space limitations prevent me from engaging with his argument, the title of this essay is an homage to Hubert Dreyfus’ (1992) groundbreaking phenomenological critique of artificial intelligence, which has been an inspiration for my own. While taking a more anthropological, interactionist, and radically relational approach, my work embraces the premise that lived experience in an open world is fundamental to human beings and to our mutual intelligibility. Computational machines, whether they encode representational models of the world or models “trained” on data generated by human interventions and activities in the world, require closure or containment to operate successfully.

2. 2 I discuss feminist commentary on the neglected gender dimension of Turing’s imagined test in Suchman (2019).

3. 3 For an extended discussion of Garfinkel’s experiments with ELIZA based on unpublished archival materials and of the groundbreaking contribution of ethnomethodology and conversation analysis to studies of human-computer interaction, see Eisenmann et al. (2024).