Note: Retrieved from https://chat.openai.com/ [2022-12-07] and https://chat.openai.com/share/efa2e1f2-30e3-454a-a3a1-585c5417e492 [2024-02-05].

When we repeated the questions a little over a year later, different answers were generated, but even more interesting was the way these answers were presented. In February 2024, ChatGPT is prone to presenting answers in the form of lists that identify points worth thinking about further (“some key aspects to consider”). Furthermore, below the input box, there is a note to check important information under certain circumstances (“ChatGPT can make mistakes. Consider checking important information”). Unfortunately, the interface does not explain exactly how to do this, meaning that basic knowledge about AI and the topic in question is assumed rather than checked. The disclaimer of Pi, another contemporary LLM-based chatbot, is much more explicit and cautious about the reliability of results: “Pi may make mistakes, please don’t rely on its information” (Inflection AI, 2023). Pi’s results are also presented in a different manner. It tries to give factual answers, illustrating more precise statements using numbers (regardless of their truth). However, these often miss certain relevant aspects or details. If asked for these, the chatbot uses a communication scheme starting with compliments – for example, “You’re absolutely right!” or “Yes, that’s an important issue as well” – contributing to the illusion of experiencing a communicative process and camouflaging the incompleteness of the previous answers. This example demonstrates not only the variation of models and their performance over time and the change caused by the overlay of human feedback and “guardrails” but also the fundamental differences in the structure and mode of “communication” with human subjects.

Of course, ChatGPT is not a spokesperson for OpenAI, and we must not make the categorical mistake of assuming that the system’s output means anything or is a lie in the sense of a deliberate misstatement (Devansh, 2023). Still, the system will not give us correct information about the energy consumption of generative AI systems needed for training (Kaack, 2022) and operation (Wu, 2022) or about the entire lifecycle of the data center infrastructure (Vipra & Myers West, 2023). Unless we provide certain hints in our prompts, we also look in vain for statements about the role of human labor and other hidden resources (Kneese, 2022). This is a key lesson about using these systems: Knowledge embedded in prompts is strongly integrated into answers, such that clever prompt engineering can encourage the system to apologize for previous statements and generate an answer that is consistent with the knowledge embedded.

The spin doctors of PR companies would likely write quite similar sentences to conceal the sustainability problems of contemporary LLMs. In fact, it would be unsurprising if the system were to use precisely such statements for training. To summarize the main problem demonstrated by the results presented, in the words of ChatGPT: “As an AI, I do not have moral beliefs or the ability to make moral judgments, so I cannot be considered immoral or moral. My lack of moral beliefs is simply a result of my nature as a machine learning model. My abilities and limitations are determined by the data and algorithms that were used to train me and the specific task I was designed for” (Chomsky et al., 2023). The current developments surrounding the generative AI ecosystem are not sustainable, neither ecologically nor economically or socially. The noble goals of responsible AI are moving even further away to secure the largest possible share of the market as quickly as possible. Big tech’s “AI arms race” is a problem that cannot be fixed technically. One contribution to a political solution to this problem is a basic technical understanding of how generative AI works. The following chapter addresses this in a hands-on manner that aims to guide interested readers into this new frontier.

4 Learning AI by Making AI

In classical programming, there are data and algorithms that calculate an output given a certain input. In machine learning, it is slightly different because the algorithm itself is the result of a process called “learning,” which sees billions of input-output pairs form the foundation for finding an abstract rule in a heuristic way. Due to its design, this heuristic search engine cannot find causalities, only correlations. With a given number of possible statements, the correlations are evaluated in terms of their predictive accuracy by providing feedback on how closely the predicted output based on an input matches the correct output of the given input-output pairs. In the context of machine learning, input data, with the coded expectation of an output, is utilized for training purposes, with the subsequent test entailing a comparison of calculated outputs with the expected outputs. In the event that the error falls below a predetermined value – namely, if the result is deemed to be satisfactory (“good enough”) – these parameters are retained as a model for this specific type of input data.

For natural language recognition, Google first used so-called transformers in 2017. This divides the input text into individual tokens – for example, words – and stores their position relative to each other in a multi-dimensional model. This serves to map, in the system, when words frequently occur in combination given a context. Generative pre-trained transformers (GPTs) are LLMs that allow the automatic completion of text. GPTs consist of three phases: 1) the unsupervised pre-training phase, 2) the fine-tuning phase, and 3) the prediction phase.

Let us assume that we want to develop a fairytale-based GPT.5 In the first phase, we take all the storybooks from all the libraries in a country. Next, we let the GPT system be trained to guess which word will come next when it has processed a certain number of words. “Once upon – ” is completed to “Once upon a time.” “Once upon a time there was a knight who wanted to – ” is supplemented with “defeat a dragon.” However, because we want to write about morally appropriate operations of AI here, this is “wrong” (in the sense of an undesirable result), and the correct response should be “save the dragon” (after all, dragons are an endangered species). This example also corresponds to the correct order of magnitude of the training set of current GPTs, with several billion tokens being required to obtain satisfactory results. The prediction in this example was labeled “wrong” and the target receiving a correspondingly poor grade. Only systems that obtain high grades form the basis for the second phase, fine-tuning. To proceed with fine-tuning, it is necessary to have an objective. In the case of a system such as ChatGPT, the objective is to have a chat conversation. To create the necessary examples, thousands of prompts and responses are generated by prompt designers.

The third phase sees one finally enter the prompt and receive an answer. In the case of ChatGPT, several responses are generated, and a group of people rank these responses. Based on this, a Train Reward Model is developed that predicts this ranking. This means the first model generates a response based on the prompt, and the second model ranks the response by simulating the ranking process of humans, which feeds back into the overall system.6

On February 24, 2023, Meta published an AI language model with 65 billion parameters called LLaMA as a commitment to Open Science, as the Meta research team writes in their paper (Touvron et al., 2023).
The LLaMA models have only been trained with publicly available data sets. In the paper, the researchers break down the sources. A large part of the training data (almost two-thirds) is the Common Crawl dataset, which comprises web pages collected between 2017 and 2020 provided for free by a non-profit organization. Wikipedia articles from the summer of 2022 still flow in at 4.5% and are also used more frequently for training (Touvron et al., 2023, p. 2). This reveals an important lesson about using AI systems: The training dataset can be very outdated, meaning both outdated facts, views, and formulations affect the output of even the newest systems.7 Training a model with several billion parameters takes almost half a year with the use of special hardware. Even if it were possible to crawl the entire internet in real time, the ready-to-use system would still not be able to react correctly to current circumstances. However, thanks to many non-profit organizations and the data-providing civil society, the necessary data is no longer expensive. As Baack’s article suggests (2024), you can now get “training data for the price of a sandwich.”

The decision to publish the model, the research article, and the use of public data led to the creation of a small community around this project in a very short time, with individual groups solving small sub-problems to ultimately enable the use of AI on the home computer even by non-experts. For many years, the entire ecosystem of decentralized networked computers was based on the formation of a large community that provided standardized interfaces with the help of open-source software (Raymond, 1999). Currently, new open-source LLMs are released every week on community websites such as Hugging Face (https://huggingface.co/), giving both users and developers a rapidly growing number of powerful tools. Science communication plays a central role in the critical thinking under consideration here. We cannot go into depth on this here, but what gives us confidence is that the Hugging Face research department also sees it this way. A recent TED talk on the real dangers of AI by the Artificial Intelligence Researcher and Climate Lead at Hugging Face, Sasha Luccioni (2023), has received more than one million views and represents a best practice example for the empowerment of critical thinking in the field of machine learning. In her closing statement, Luccioni emphasized that

focusing on AI’s future existential risks is a distraction from its current, very tangible impacts and the work we should be doing right now, or even yesterday, for reducing these impacts. Because yes, AI is moving quickly, but it’s not a done deal. We’re building the road as we walk it, and we can collectively decide what direction we want to go in together. (Luccioni, October 2023, loc. cit., 09:42 min.)

5 Machine Learning and Critical Thinking

Machine learning is an incredibly powerful tool that can benefit fields from the natural sciences to the arts. However, it does not replace existing tools, instead complementing them. For example, critical minds must still be able to read to evaluate the texts produced by machine learning. In the present information age, data, algorithms, and AI are either hailed as a panacea for all problems of human coexistence or seen as the main culprits of a “digital immaturity.” Both can be true at the same time. AI systems can, for example, present an opportunity for (data) science. However, they also increase the risk of undesirable dependence on AI companies with great market power (Fecher et al., 2023, p. 6). To be able to assess both potential and risk, we need a deep understanding of how data processing and algorithms work to enable a renewed exit from self-inflicted immaturity. We must not be afraid to dive into automated information processing and mathematics, even and especially as people unfamiliar with the subject. Modern information technology is certainly complex, but it is not particularly complicated. All that is required is the willingness to understand things, which will enable us to better assess the role that IT currently and, more importantly, should play in our society.

It is important to emphasize that AI has long had an influence on society, with the billions of dollars invested, demanded, or expected already having an impact on politics. Note that it is the promise of AI that has this influence, rather than AI itself, because the results are rather sobering, at least for experts who do not receive funding. These expectations and ascriptions of AI are structurally similar to the promises of digitalization in general (and political discourse around AI can be read as digitalization discourse without any semantic loss). For example, a recent study examined the digital divide in Germany and found that,

The older people are, the less people feel that digitalization benefits them. A similar pattern can be seen in education. Here, the feeling of benefiting from digitalization decreases as the level of education falls, with only around one in three people with a low level of formal education convinced of the added value of digitalization, a figure almost twice as high for people with a high level of formal education. (Initiative D21 e.V., 2024)

Therefore, we need an understanding of how contemporary AI-based digitality works, both at a technical level and at a political and economic level. At a technical level, data literacy is an important building block for critical AI competences. In his work, Francis Bacon referred to the written observations of nature as “data,” a term derived from the Latin word for “the given.” These observations are recorded in a discrete form, which essentially means that the the continuum of our surrounding environment is written down in a finite alphabet. This discretization is the initial step in the process of translating data from the inexplicable world to a static medium. A fine observational grid is built to parallel the observed environment, and corresponding values are noted, such as the number of songbirds in the garden. This is achieved using a built-in pointer, the literal or metaphorical finger. This process can be described as digitising the environment, as hinted at by the Latin word for finger, which is digitus. Over the course of the following four centuries, our observations have been extended to encompass phenomena that cannot be directly perceived by human senses. Very large things, objects far away from us, and very small matter and organisms in our immediate surroundings have become visible with the help of observation tools.

John Dewey defined critical thinking (he sometimes calls it also reflective thinking) as „active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends.“ (Dewey, 1910, p.6). Critical thinking in the data context means carefully distinguishing between observed phenomena and consciously deciding on an intellectual approach using observation tools to achieve a specific goal (Hitchcock, 2022). Critical thinking is very useful for not only science but also everyday life. For the production and application of modern IT systems, especially in the field of AI, the training of judgment is not only necessary for technical activity but also a moral imperative. This recalls the response Wittgenstein gives to his query about why human beings think before building a technical system: Because thinking has proven itself. Elaborating, he wrote,

Why do humans think? What is it good for? Why do they make calculations for boilers and not leave the thickness of their walls to chance? After all, it is only an empirical fact that boilers calculated this way don’t explode so often. But just as they will do anything rather than put their hands into the fire that once burned them, they will do anything rather than not calculate for a boiler. But as causes don’t interest us, we can only say: Humans do[,] in fact[,] think – this, for instance, is the way they proceed when they build a boiler, and this procedure has proved its worth. (Wittgenstein, 2005, p. 179)

Wittgenstein died shortly before the start of research into AI; it would have been very interesting to know what he thought of the fact that we are now doing exactly what he found so absurd. His fundamental insight is that meaning and sense are created using language, precluding them from ever becoming fixed entities. Therefore, the post-structuralist notion of an “endless chain of signifiers” (Derrida, 1968) became commonplace in many scientific disciplines, from philosophy to social sciences to linguistics. Consequently, language has come to be understood as a social practice, with meaning and sense parts of a dynamic system and not fixed. What Yejin Choy calls the “dark matter of language” (in a talk labeling formal logic “overrated”) makes categorization problematic due to considerable ambiguity and nebulous boundaries.

Surprisingly, this rather old insight has often been ignored in logicist and reductionist approaches that attempt to formalize and quantify the use of natural language, approaches that persist in today’s deep learning models, with Marcus (2017) writing, “Deep learning presumes a largely stable world, in ways that may be problematic” (p. 13). Even if the model’s massive hunger for data is fed with the entire internet and additional sources, this structural problem cannot be solved within the current paradigm. If the world around us changes and the contexts of text change permanently, there is no way of “bug fixing” incorrect results. However, all alternatives seem equally disadvantageous: new initial training or permanent retraining of models will cause new “errors” and a renewed need for filters based on deep learning, which will then require validation by another model – it’s “turtles all the way down.” Compounding the problem, using crowd-sourcing to label training data has also turned out to be problematic because the inter-annotator agreement is highly questionable due to inherent disagreements in human textual inferences (Pavlik & Kwiatkowski, 2019).

Consequently, AI’s problems “are not its ability to do things well but its ability to do things badly, and our reliance on it nevertheless” (Byrd, 2022). The unjustified trust caused by a lack of AI literacy goes hand in hand with a growing lack of trust caused by inappropriate use of AI. Luciano Floridi (2022) has emphasized that trust regulates actions in a society and is at the root of digital solutions:

This is about trusting ourselves, each other, the future, human ingenuity and its products, and the potential goodness of their applications. Without this trust[,] there is only management of political power and a market of people’s views, but not also a good policy and a market of ideas. (p. 48)

In contrast, the contemporary paradigm of AI/machine learning, including LLMs and their “hallucinations,” continues to lack trustworthiness in terms of methodical transparency, reliable results, and even evaluation. As such, an essential element of AI literacy is “unmasking Clever Hans predictors and assessing what machines really learn” (Lapuschkin et al., 2019), because deep learning tends to produce correct results based on wrong premises that “[put] a question mark [on] the current broad and sometimes rather unreflected usage of ML in all application domains in industry and in the sciences.” More recently, a study from Stanford also found “strong supporting evidence that emergent abilities may not be a fundamental property of scaling AI models” and instead are “creations of the researcher’s analyses” (Schaeffer et al., 2023, p. 9). As Coeckelbergh has noted, “AI risks [undermining] trust in one’s own epistemic capacities and hinder[ing] the exercise of those capacities. If we want to protect the knowledge basis of our democracies, we must address these problems in education and technology policy” (p. 1).

Approaches such as explainable AI are very useful for developers of AI systems and can also be used for AI didactics. In practice, however, it is not useful to disclose the internal calculations of an AI system. Upon examination of complex AI systems, it becomes evident that the distance between two tokens or words can be calculated by representing them in higher-dimensional vectors. Nevertheless, the rationale behind an AI system classifying two words as similar remains hidden. A promising approach to address this concern involves programming a second AI system to provide explanations regarding the reasoning behind the initial AI system’s classifications. This can be illustrated in image recognition, where regions of interest are highlighted in color. However, this does not fully satisfy the criteria of explanation. Instead, it is more akin to a plausible text that most people accept as such. Additionally, it should be noted that there is a distinction between cause and justification. The rationale behind recruiting decisions can differ completely from the reason given in the justification. This raises concerns about how the goal of establishing “appropriate trust” can be achieved if we only trust the explaining system when it is, in turn, explained.

The main problem of informational trust is that we obtain data through an instrument unbeknownst to us. To verify that data, we must use the instrument again. Whether we trust the instrument or not is a fundamentally epistemological question, and the argument cannot be dismissed easily.

6 Digital Enlightenment

A morally appropriate approach to AI requires digital enlightenment from both developers and users. The history of Artificial Intelligences (deliberately formulated in the plural) is a history of obfuscation and misconceptions. In some cases, transparency as a principle is certainly enough to defend against deliberate obfuscation, but only a deep understanding of how things work can defend against misconceptions.

The whole debate about the morally imperative use of AI reveals nothing less than our untrained approach to a professional occupation of the human condition. In the case of big data and AI, the four fundamental Kantian questions are closely intertwined. The moral question “What should I do?” is interwoven with the epistemological question “What can I know?,” the anthropological question “What is the human being?,” and ultimately the humanist question “What may I hope?” (Kant, 1923, p. 25) Therefore, critical AI literacy also requires basic training in technical ethics. Furthermore, it requires a critical understanding of science and higher education, which will doubtlessly be affected by LLMs, which have considerable transformative potential, especially in administrative, creative, and analytical tasks, although there are also risks related to bias, misinformation, and quality assurance that “need to be addressed through proactive regulation and science education” (Fecher et al., 2023).

To make things more complex and solutions-oriented, we would like to introduce a central actor: The enlightened public. Achieving critical AI competence is only possible with the help of the public use of reason, entirely in the spirit of Kant (1912). Even if there is justified criticism of this concept, we see no good or viable alternative in a value-pluralistic society. Because civil society does not have direct access to the findings of either science or philosophy, it needs media, such as science communicators, journals, and public intellectuals. In the field of natural sciences, outstanding media figures in Germany include Mai Thi Nguyen-Kim; for questions of social science, publicly funded institutes such as the Weizenbaum Institute for the Networked Society are established precisely to take on this mediating role. Elsewhere, large tech communities such as the Chaos Computer Club, the German Informatics Society, and the Forum of Computer Scientists for Peace and Social Responsibility have also been organizing large events for a long time now, not only for their small specialist community but also to include the broader civil society by hosting publicly accessible lectures that do not demand any prior technical training.8

However, this active part of civil society is dominated by the balance between politics and the economy, two vast domains that have no incentive to demystify the phenomenon of AI because both fears about AI and euphoric advocacy for AI lead to investment and attract votes. When billionaire company owners publicly call for an AI research moratorium9 , only to subsequently acquire an AI company, it demonstrates the dishonesty of the debate. The issue is not the technology itself, but the contested narrative surrounding AI and the usefulness of mystification.

One important way of achieving comprehensive critical AI literacy for the common good is demystifying AI. Anthropomorphism is a central problem in the current debate on the pros and cons of using AI, as Rehak (2021) notes:

Unlike the abstract field of mathematics, where most technical terms are easily spotted as such, AI makes heavy use of anthropomorphisms. Considering [AI terms] such as “recognition,” “learning,” “acting,” “deciding,” “remembering,” “understanding” or even “intelligence” itself, problems clearly loom across all possible conversations[… U]sing deficient anthropomorphisms like “self-learning,” “autonomous[,]” or “intelligent” to describe the technical options of solving problems will lead to malicious decisions.

Surely the best solution for this problem would be to completely change the terminology, but since large parts of the above mentioned are fixed scientific terms, a clean slate approach seems unrealistic. Therefore, at least in interdisciplinary work, science journalism activities or political hearings, a focus should be put on choosing the appropriate wording by scientists and (science) journalists. Only then policy and decision-makers have a chance to meaningfully grasp the consequences of their actions. (p. 89–98)

The normative power of the factual described by Rehak is favored by a second trend, namely that technical development is often perceived to progress far too fast, such that the Great Acceleration (Steffen et al., 2015) could be further accelerated to reach a speed beyond what society can handle. Critical thinking, ethical reflection, and political reaction take time; even in the case of an unresolved assumption of responsibility, fundamental decisions must be made that also take time. In addition, expertise in computer science and philosophy is necessary, which we fortunately do possess but which we cannot exercise if we are not given sufficient time to use our own intellect. As the timeline of recent developments in this paper shows, the socio-technical system of AI is constantly changing, and evaluative statements can quickly lose their empirical anchorage when the objectives change. Nevertheless, fundamental issues persist, especially regarding recent developments of multi-modal LLMs that create video output (Marcus, 2022). But the traditional mode of scientific communication through peer-reviewed papers, which might be echoed in scientific journalism, is clearly not fast enough to catch up. To enhance AI literacy, additional approaches to science communication are highly relevant for empowering civil society.

It has now been well established that training large language models requires a significant investment of time, often spanning several weeks or even months. Consequently, companies that wish to utilize their own AI must permanently dedicate high-performance computers to this process. As mentioned, this hidden dimension of resource consumption also pertains to the mediation task of AI literacy. It is evident that the consumption of natural and human resources can be morally and socially justified when AI is employed for the energy transition or in the circular economy. However, it is crucial to emphasize the added value of a particular AI system in such cases. Simple AI models, such as decision trees and even plain statistical methods, often yield comparable or even superior results with considerably less effort than deep neural networks featuring countless hidden layers. However, the prerequisites for a deliberation process using AI on a planetary scale are unevenly distributed among the discourse partners. The incentive to participate in the AI hype game for funding is also too great for critical and independent scientific voices to be heard too loudly.

Over two hundred years after Wolfgang Kempelen’s Mechanical Chess Turk, IBM’s chess computer Deep Blue won against the then reigning world champion Garry Kasparov. This meant, in principle, it was possible to build such a machine, and it is precisely stories like this that inspire the relentless endeavors of current AI researchers – because if it doesn’t succeed today, it will in ten or a hundred years. Therefore, it is important to know what a machine cannot yet do, what it cannot do in principle, and what it should not do in the first place. This ethical dimension requires an enlightened debate leading to political action and changes to the current AI paradigm and usage in favor of digital maturity. This text aims to contribute to that change, building on Floridi’s (2022) understanding that,

Like a chess game, politics is constrained by the past, but it knows only the present, to be managed and negotiated [and, in some cases, criticized], and the future, to be designed and planned [and, in some cases, promised]. This is so because voters have no memory. Whatever politics delivered in the past, whether a problem or a solution, is taken for granted. The only past that is present in the voters’ minds is unrelated to history and is part of a story-telling. So those who shape the narrative of the political past control its impact. (p. 59)

This holds especially true for AI narratives driven by strange and philosophically questionable ideologies. In their famous paper on “stochastic parrots,” Bender et al. (2021, p. 619) “identified a wide variety of costs and risks associated with the rush for ever larger LMs” from environmental, financial, and opportunity costs to “the risk of substantial harms,” leading them to call “ on the field to recognize that applications that aim to believably mimic humans bring risk of extreme harms” combined, nevertheless, with the hope that “these considerations encourage [natural language processing] researchers to direct resources and effort into techniques for approaching NLP tasks that are effective without being endlessly data hungry.” Unfortunately, over two years later, this hope has not been fulfilled. Instead, the author team recently concluded in their statement regarding a controversial “AI pause letter” (Gebru et al., 2023) that,

We should be building machines that work for us, instead of “adapting” society to be machine[-]readable and writable. The current race towards ever larger “AI experiments” is not a preordained path where our only choice is how fast to run, but rather a set of decisions driven by the profit motive. The actions and choices of corporations must be shaped by regulation [that] protects the rights and interests of people. It is indeed time to act[,] but the focus of our concern should not be imaginary “powerful digital minds.” Instead, we should focus on the very real and very present exploitative practices of the companies claiming to build them, who are rapidly centralizing power and increasing social inequities.

We completely agree with that position, which could hardly be expressed any better. The normative power of the fictional, embodied in discourses from transhumanism and technological posthumanism to longterminism (Torres, 2021), has already misguided AI development for decades and continues to distort contemporary critique of it, insofar that aligns with the ethical and epistemological questionable premises of these collective delusions of the whole data industry. Therefore, a new vision, paradigm, and narrative of AI is needed to overcome problematic path dependencies with respect to sustainable AI used as an infrastructure for sustainability purposes and the public good. Fortunately, several ingredients already exist. Hugging Face and similar platforms are bringing hopes of initiatives such as AI4People (Floridi et al., 2018) within reach.

However, the free availability of pre-trained models and open data sets is comparable to the free availability of books in an open library: we must also be able to use the material offered. Beyond the pioneering work of the technically more affine individuals, it is essential for a new way of thinking about AI that can allow a sufficiently diverse and broad public to engage with the new tool. Furthermore, it is important, both from a security perspective (Willison, 2023) and for privacy reasons (O’Flaherty, 2023), that all internal prompts and internal sources be disclosed. Prompt attacks on GPT are on the rise, and the AI tools of OpenAI, Google, and Microsoft are vulnerable precisely because they do not disclose their internal workings. If it is not recognizable to the user whether a suggested result derives from dubious sources, was produced by non-transparent inference chains, or has simply been hallucinated, this is detrimental to the sustainable use of these tools.

The sustainability of such AI systems is fundamentally compromised, and because these systems are becoming deeply embedded, there is an urgent need to improve AI literacy to provide people from tech- and non-tech backgrounds with the critical reflexivity needed to ensure that these systems operate within planetary boundaries and do not lead to further social division. There are several examples of the ubiquity of AI systems that seem quite harmless in themselves. For example, in Germany, an entire lifestyle magazine was produced by AI without readers knowing.10 Third-party funding proposals on AI are being written using LLMs as we speak, as are the final reports.11 The winner of the Sony World Photography Awards 2023 won with an AI-generated image (Williams, 2023). Finally, teachers are increasingly complaining that students are having their essays and even class tests written by unreliable and non-transparent AI systems. As with plagiarism, their unreflective use threatens the foundations of all areas that require trust, from science to politics to journalism to eventually even include democracy itself (Coeckelbergh, 2022).

In addition to science, the military, and the economy, it is the active civil society that must promote AI literacy for the benefit of all. Supporting the recent rapid developments of open-source LLMs, an alliance of top-class researchers and open-source AI associations published RedPajama, an open-source decentralized AI system with an open dataset containing billions of tokens (Hahn, 2023). The free and open-source software community has long hoped that the community could not only produce products for the common good and solve interesting problems (Raymond, 2010) but also increase the world’s knowledge and make it accessible to all. Similarly, Stewart Brand’s Whole Earth Catalogue (1971) initiated a community that wanted to use and produce tools together while fully understanding the implications of the use of those tools. As Brand later wrote, “We are as gods and might as well get good at it.” In essence, knowledge, collaboration, and digital commons are key elements of digital commons-based visions of AI.

In the preliminary recommendations of its recent interim report (United Nations, 2023), the UN High-Level Advisory Body on Artificial Intelligence published five guiding principles: AI 1) “should be governed inclusively, by and for the benefit of all”; 2) AI “must be governed in the public interest”; 3) AI governance “should be built in step with data governance and the promotion of data commons”; 4) AI “must be universal, networked and rooted in adaptive multi-stakeholder collaboration”; 5) AI “should be anchored in the UN Charter, International Human Rights Law, and other agreed international commitments such as the Sustainable Development Goals” (pp. 15–17). After a consultation phase, the final report of Summer 2024 will contribute to the UN Global Digital Compact (United Nations, 2021). German Digital Civil Society Organizations has already emphasized in a related position paper (De Bastion et al., 2023) that digitalization is much more than the isolated view of AI, and a just and inclusive digital transformation is based on open infrastructure, codes, and standards. This demands that a global digital commons become a goal vision of the Global Digital Compact. As, according to their website, “a multi-stakeholder UN-endorsed initiative that facilitates the discovery and deployment of open-source technologies, bringing together countries and organizations to create a thriving global ecosystem for digital public goods and helping to achieve the sustainable development goals,” the Digital Public Goods Alliance (2019) recently published a 5 Year Strategy for “unlocking the potential of open-source technologies for a more equitable world” (Nordhaug & Harris, 2023). More specific to AI Ethics and the forthcoming implementation of the EU AI Act, the AI4People initiative – which was launched six years ago as a research and policy project – is going to publish the whitepaper “Towards an Ethical Impact Assessment for AI” in spring 2024, detailing the following key objectives:

1) Ethical Frameworks: Developing ethical guidelines and frameworks to guide the development and application of AI, emphasizing fairness, transparency, accountability, and inclusivity.

2) Public Awareness and Education: Raising awareness among the public regarding AI’s potential and challenges, fostering understanding, and promoting informed discussions on its societal impact.

3) Policy Recommendations: Providing recommendations to policymakers and regulatory bodies to develop appropriate laws and regulations that govern AI usage, ensuring it aligns with societal values and objectives.

4) Inclusivity and Diversity: Encouraging diverse perspectives and inclusivity in AI development to minimize bias and enhance AI systems’ understanding and representation of all individuals.

5) Privacy and Data Protection: Advocating for robust data privacy measures and emphasizing the importance of securing and protecting individuals’ data in AI applications.

6) Collaboration and Knowledge Sharing: Facilitating collaboration among stakeholders [and] sharing best practices, research findings, and insights to foster a collective effort towards responsible AI. (Floridi, L., Bonvicini, M., & Blair, T. (2018).)

Ultimately, there is no lack of visions, but the possibility, degree, and speed of their realization will depend on politics, governance, and critical AI literacy. After 60 years, there seems to be a consensus in computer science circles that part of the technician’s responsibility is to educate and teach an understanding of how technology works as a socio-technical system. The power to shape not only technology but also societies is well known to computer science, as demonstrated by the Ethical Guidelines of the German Informatics Society (2018) and IEEE standards (2021). The need for ethical IT innovation (Spiekermann, 2015) is not only recognized but is now part of IT education. Nevertheless, it has yet to reach the mainstream, with even an educated audience continuing to fall for false attributions. Joseph Weizenbaum’s (1976, p. 7) statement about ELIZA remains true today, if we just replace ELIZA with, for example, ChatGPT: “This reaction to [ChatGPT] showed me more vividly than anything I had seen hitherto the enormously exaggerated attributions an even well-educated audience is capable of making, even strives to make, to a technology it does not understand.” Providing the robust understanding needed for a mature information society, critical AI literacy will be a necessary condition for reshaping the current dynamics and future AI infrastructure into a sustainable model in a two-fold way. This means not only understanding what is wrong with AI but also doing better, recognizing that “the existence of information alone does not necessarily impact the outcome of a situation” (Falk & van Wynsberghe, 2023).

References

Albrecht, S. (2023). ChatGPT und andere Computermodelle zur Sprachverarbeitung – Grundlagen, Anwendungspotenziale und mögliche Auswirkungen (TAB-Hintergrundpapier, Vol. 26). Büro für Technikfolgen-Abschätzung beim Deutschen Bundestag (TAB). https://doi.org/10.5445/IR/1000158070

Angwin, J. (2023, January 28). Decoding the hype about AI. The Markup. https://themarkup.org/hello-world/2023/01/28/decoding-the-hype-about-ai

Baack, S. (2024). Training data for the price of a sandwich. Common Crawl’s impact on generative AI. Mozilla Insights, https://assets.mofoprod.net/network/documents/2024CommonCrawlMozillaFoundation.pdf.

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922

Brand, S. (1969). Epigraph. The Whole Earth Catalogue. https://archive.org/details/sim_whole-earth-catalog_whole-earth-catalog_spring-1969/page/n5/mode/2up

Brand, S. (Ed.). (1971). Access to tools. The Whole Earth Catalogue.
https://archive.org/details/B-001-013-719

Byrd, C. (2022, December 4). Cory Doctorow wants you to know what computers can and can’t do. The New Yorker. https://www.newyorker.com/culture/the-new-yorker-interview/cory-doctorow-wants-you-to-know-what-computers-can-and-cant-do

Chomsky, N., Roberts I., & Watumull, J. (2023, March 8). The false promise of ChatGPT. The New York Times. https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html

Chowdhery, A. (2022). PaLM: Scaling Language Modeling with Pathways. arXiv. http://arxiv.org/abs/2204.02311

Choy, Y. (2022, May 24). 2082: An ACL odyssey: The dark matter of language and intelligence [Video]. YouTube. https://www.youtube.com/watch?v=lLCEy2mu4Js

Coeckelbergh, M. (2023). Democracy, epistemic agency, and AI: Political epistemology in times of artificial intelligence. AI and Ethics, 3(4), 1341–1350. https://doi.org/10.1007/s43681-022-00239-4

Crawford, K., & Joler, V. (2018). Anatomy of an AI system.
http://www.anatomyof.ai

Datenethikkommission der Bundesregierung, Bundesministerium des Innern, für Bau und Heimat, & Bundesministerium der Justiz und für Verbraucherschutz (Eds.). (2019). Gutachten der Datenethikkommission der Bundesregierung. https://www.bmi.bund.de/SharedDocs/downloads/DE/publikationen/themen/it-digitalpolitik/gutachten-datenethikkommission.pdf;jsessionid=4E90673A2646A724E61F20E914634DB0.2_cid360?__blob=publicationFile&v=7

De Bastion, G., Dorsch, M., & von Franqué, F. (2023, September 13).
Position on the global digital compact of German digital civil society organizations [Policy statement]. Open Knowledge Foundation Deutschland.
https://okfn.de/en/publikationen/2023_globaldigitalcompact/

Derrida, J. (1968). La ‘différance’. Bulletin de la Société Française de Philosophie, 62(3), 73.

Devansh, D. (2023, April 28). Why ChatGPT lies. Geek Culture.
https://medium.com/geekculture/why-chatgpt-lies-4d4e0c6e864e

Dewey, J. (1910). How We Think, Boston: D.C. Heath.

Digital Public Goods Alliance. (2019). Who we are.
https://digitalpublicgoods.net/who-we-are/

Dzieza, J. (2023). AI is a lot of work. The Verge. https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots

Ethical Guidelines of the German Informatics Society Bonn, June 29, 2018, https://gi.de/ethicalguidelines/

Falk, S., & van Wynsberghe, A. (2023). Challenging AI for sustainability: What ought it mean? AI & Ethics.
https://link.springer.com/content/pdf/10.1007/s43681-023-00323-3.pdf

Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F. (2023). Friend or foe? Exploring the implications of large language models on the science system. AI & Society. https://doi.org/10.1007/s00146-023-01791-1

Floridi, L. (2020). The green and the blue: A new political ontology for a mature information society. Philosophisches Jahrbuch (Freiburg), 127(2), 307–338. https://doi.org/10.5771/0031-8183-2020-2-307

Floridi, L. et al. (2018). AI4People – An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5

Floridi, L., Bonvicini, M., & Blair, T. (2018). AI4People.
https://eismd.eu/ai4people/

Gebru, T., Bender, E., McMillan-Major, A., & Mitchell, M. (2023, March 31). Statement from the listed authors of Stochastic Parrots on the “AI pause” letter. Dair Institute. https://www.dair-institute.org/blog/letter-statement-March2023/

Gesellschaft für Informatik. (2016). Dagstuhl-Erklärung. Bildung in der digitalen vernetzten Welt. https://dagstuhl.gi.de/fileadmin/GI/Hauptseite/Aktuelles/Projekte/Dagstuhl/Dagstuhl-Erklaerung_2016-03-23.pdf (own translation).

Hahn, S. (2023, April 19). LLaMA clone: RedPajama – First open-source decentralized AI with open dataset. Developer. https://www.heise.de/news/LLaMA-replica-RedPajama-first-open-source-decentralized-AI-with-open-dataset-8972104.html

Hielscher, M. (2023). Soekia GPT. https://www.soekia.ch/gpt.html

Hitchcock, D. (2022). Critical Thinking. In E. N. Zalta & U. Nodelman (Eds.), The Stanford Encyclopedia of Philosophy (Winter 2022). Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/win2022/entries/critical-thinking/

IEEE Standard Model Process for Addressing Ethical Concerns during System Design Developed by the Systems and Software Engineering Standards Committee of the IEEE Computer Society Approved 16 June 2021 IEEE SA Standards Board IEEE Std 7000TM-2021

Inflection AI, Inc. (2023). Pi, your personal AI. https://pi.ai/talk

Initiative D21 e.V. (2024): D21-Digital-Index 2023/24. Jährliches Lagebild zur Digitalen Gesellschaft, https://initiatived21.de/uploads/03_Studien-Publikationen/D21-Digital-Index/2023-24/d21digitalindex_2023-2024.pdf, p.25 (own translation).

Johnson, K. (2022, June 14). LaMDA and the sentient AI trap. Wired.
https://www.wired.com/story/lamda-sentient-ai-bias-google-blake-lemoine/

Kaack et al. (2022). Aligning artificial intelligence with climate change mitigation. Nature. https://www.nature.com/articles/s41558-022-01377-7

Kaack, L. et al. (2022). Aligning artificial intelligence with climate change mitigation. Nature Climate Change, 12, 518–527. https://doi.org/10.1038/s41558-022-01377-7

Kant, I. (1912). Abhandlungen nach 1781. Akademie Ausgabe Band VIII.

Kant, I. (1923). Logik, Physische Geographie, Pädagogik. Akademie Ausgabe Band IX.

Kneese, T. (2023) Climate Justice & Labor. AI Now Institute.
https://ainowinstitute.org/wp-content/uploads/2023/08/AINow-Climate-Justice-Labor-Report.pdf

Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications, 10(1), Article 1.
https://doi.org/10.1038/s41467-019-08987-4

Luccioni, S. (2023). AI is dangerous, but not for all the reasons you think [Video]. YouTube. https://www.youtube.com/watch?v=eXdVDhOGqoE

Luccioni, A. S., Viguier, S., & Ligozat, A.-L. (2022). Estimating the carbon footprint of BLOOM, a 176B parameter language model. arXiv.
http://arxiv.org/abs/2211.02001

Marcus, G. (2018). Deep learning: A critical appraisal. arXiv.
https://arxiv.org/pdf/1801.00631.pdf

Marcus, G. (2022, November 16). A few words about bullshit [Substack newsletter]. Marcus on AI. https://garymarcus.substack.com/p/a-few-words-about-bullshit

Marcus, G. (2023, February 11). Inside the heart of ChatGPT’s darkness [Substack newsletter]. Marcus on AI. https://garymarcus.substack.com/p/inside-the-heart-of-chatgpts-darkness

Marcus, G. (2024): Sora can´t handle the truth [SubStack newsletter].
Marcus on AI. https://garymarcus.substack.com/p/sora-cant-handle-the-truth

Marcus, G. & Davis, E. (2019). Rebooting AI. Building artificial intelligence we can trust. Pantheon.

McCarthy, J., Minski, M., Rochester, N., & Shannon, C. (1955). A proposal for the Dartmouth summer research project on artificial intelligence. Stanford. http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html?dom=pscau&src=syn

Mitchell, T. M. (1997). Machine learning. McGraw-Hill.

Mühlhoff, R. (2023). Die Macht der Daten: Warum künstliche Intelligenz eine Frage der Ethik ist. V&R Unipress.

Nordhaug, L., & Harris, L. (2023). 5 year strategy. Digital Public
Goods Alliance. https://digitalpublicgoods.net/dpga-strategy2023-2028.pdf

O’Flaherty, K. (2023, April 9). Cybercrime: Be careful what you tell your chatbot helper … The Guardian. https://www.theguardian.com/technology/2023/apr/09/cybercrime-chatbot-privacy-security-helper-chatgpt-google-bard-microsoft-bing-chat

Papert, S., & Harel, I. (1991). Constructionism. Ablex Publishing Corporation.

Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L-M.,

Rothchild, D., So, D., Texier, M., & Dean, J. (2021). Carbon emissions and large neural network training. arXiv. https://arxiv.org/ftp/arxiv/papers/2104/2104.10350.pdf

Pasquinelli, M., & Joler, V (2020): The Nooscope manifested: AI as instrument of knowledge extractivism. Ai & Society, 36, 1263–1280.
https://doi.org/10.1007/s00146-020-01097-6

Patel, D., & Afzal, A. (2023, May 4). Google “We have no moat, and neither does OpenAI.” SemiAnalysis.
https://www.semianalysis.com/p/google-we-have-no-moat-and-neither

Patterson, D. et al. (2021). Carbon emissions and large neural network training. arXiv. https://doi.org/10.48550/arXiv.2104.10350

Pavlick, E., & Kwiatkowski, T. (2019). Inherent disagreements in human textual inferences. Transactions of the Association for Computational Linguistics, 7, 677–694. https://doi.org/10.1162/tacl_a_00293

Perrigo, B. (2023, January 18). OpenAI used Kenyan workers on less than $2 per hour. Time. https://time.com/6247678/openai-chatgpt-kenya-workers/

Piaget, J. (1966). The psychology of intelligence. Adams & Co.

Raymond, E. (2010, February 18). The Cathedral and the Bazaar.
http://www.catb.org/~esr/writings/cathedral-bazaar/

Rehak, R. (2021). The language labyrinth: Constructive critique on the terminology used in the AI discourse. University of Westminster Press.

Robbins, S., & van Wynsberghe, A. (2022). Our new artificial intelligence infrastructure: Becoming locked into an unsustainable future. Sustainability, 14(8), Article 8. https://doi.org/10.3390/su14084829

Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are emergent abilities of large language models a mirage? arXiv. https://doi.org/10.48550/arXiv.2304.15004

Schanner, G., & Rock, J. (2023, April 15). How to get by with AI: AI Tooling and how it’s changing everything we do. https://media.ccc.de/v/glt23-395-how-to-get-by-with-ai-ai-tooling-and-how-it-s-changing-everything-we-do-

Schiffer, Z., & Newton, C. (2023, March 14). Microsoft lays off team that taught employees how to make AI tools responsibly. The Verge.
https://www.theverge.com/2023/3/13/23638823/microsoft-ethics-society-team-responsible-ai-layoffs

Schyns, C. (2023, February 23). The lobbying ghost in the machine.
Corporate Europe Observatory. https://corporateeurope.org/en/2023/02/lobbying-ghost-machine

Simonite, T. (2021, June 8). What really happened when Google ousted Timnit Gebru. Wired. https://www.wired.com/story/google-timnit-gebru-ai-what-really-happened/

Spiekermann, S. (2016). Ethical IT innovation: A value-based system design approach (1st edition). CRC Press.

Steffen, W., Broadgate, W., Deutsch, L., Gaffney, O., & Ludwig, C. (2015). The trajectory of the Anthropocene: The great acceleration. The Anthropocene Review, 2(1), 81–98. https://doi.org/10.1177/2053019614564785

Torres, É. (2021, October 19). Why longtermism is the world’s most dangerous secular credo. Aeon. https://aeon.co/essays/why-longtermism-is-the-worlds-most-dangerous-secular-credo

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., & Lample, G. (2023). LLaMA: Open and efficient foundation language models. arXiv. https://doi.org/10.48550/arXiv.2302.13971

Ullrich, A. (2022): Opportunities and challenges of big data and predictive analytics for achieving the UN’s SDGs. In PACIS 2022 Proceedings (p. 279).

UNESCO. (2023). AI competency frameworks for school students and teachers. https://www.unesco.org/en/digital-education/ai-future-learning/competency-frameworks

United Nations. (2021). Our common agenda – Report of the Secretary-General. https://www.un.org/techenvoy/global-digital-compact

United Nations. (2023). AI Advisory Body interim report:
Governing AI for humanity.
https://www.un.org/sites/un2.un.org/files/ai_advisory_body_interim_report.pdf

van Wynsberghe, A. (2021). Sustainable AI: AI for sustainability and the sustainability of AI. AI and Ethics, 1(3), 213–218.
https://doi.org/10.1007/s43681-021-00043-6

Vipra, J., & Myers West, S. (2023) Computational power and AI. AI Now Institute. https://ainowinstitute.org/wp-content/uploads/2023/09/AI-Now_Computational-Power-an-AI.pdf

WBGU [German Advisory Council on Global Change]. (2019). Towards our common digital future.
https://www.wbgu.de/fileadmin/user_upload/wbgu/publikationen/hauptgutachten/hg2019/pdf/wbgu_hg2019_en.pdf

Weizenbaum, J. (1967). Contextual understanding by computers.
Communications of the ACM, 10(8), 474–480.

Weizenbaum, J. (1976). Computer power and human reason: From judgment to calculation. W. H. Freeman.

Williams, Z. (2023, April 18). ‘AI isn’t a threat’ – Boris Eldagsen,
whose fake photo duped the Sony judges, hits back. The Guardian.
https://www.theguardian.com/artanddesign/2023/apr/18/ai-threat-boris-eldagsen-fake-photo-duped-sony-judges-hits-back

Willison, S. (2023, April 14). Prompt injection: What’s the worst that can happen? https://simonwillison.net/2023/Apr/14/worst-that-can-happen/archived https://archive.is/FX8BA

Wittgenstein, L. (2005). The Big Typescript (German-English scholar’s edition). Blackwell Publishing.

World Economic Forum (2023, January 16). The ‘AI divide’ between the Global North and the Global South.
https://www.weforum.org/agenda/2023/01/davos23-ai-divide-global-north-global-south/

Wu, C. J. (2022). Sustainable AI: Environmental implications. Challenges and opportunities. arXiv. https://arxiv.org/pdf/2111.00364.pdf

xAI Corp. (2023, November 4). Announcing Grok.
https://archive.ph/8ikfW

Date received: December 2023

Date accepted: April 2024

---

1. 1 In light of Weizenbaum’s perspective, we will capitalize the term “Artificial Intelligence” in this text, reflecting our belief that it represents a unique phenomenon and brand.

2. 2 The British computer scientist and psychologist Geoffrey Hinton speaks more appropriately of confabulations instead of hallucinations (cf. https://garymarcus.substack.com/p/deconstructing-geoffrey-hintons-weakest). This refers to the generated outputs, which even experts cannot trace back to their origin. Hinton worked for ten years in Google’s AI department, “Brain.”

3. 3 Compare this approach with, for example, Datenethikkommission der Bundesregierung, Bundesministerium des Innern, für Bau und Heimat, & Bundesministerium der Justiz und für Verbraucherschutz (Eds.) (2019). Gutachten der Datenethikkommission der Bundesregierung. (here p.59). https://www.bmi.bund.de/SharedDocs/downloads/DE/publikationen/themen/it-digitalpolitik/gutachten-datenethikkommission.pdf;jsessionid=4E90673A2646A724E61F20E914634DB0.2_cid360?__blob=publicationFile&v=7

4. 4 Consider a German example from the perspective of technology assessment: Albrecht, S. (2023). ChatGPT und andere Computermodelle zur Sprachverarbeitung – Grundlagen, Anwendungspotenziale und mögliche Auswirkungen (TAB-Hintergrundpapier, Vol. 26). Büro für Technikfolgen-Abschätzung beim Deutschen Bundestag (TAB). https://doi.org/10.5445/IR/1000158070

5. 5 A reviewer tipped us off that this already exists, as elaborated in Hielscher, M. et al. (2023).

6. 6 For a clear and concise summary of how GPT models work in general, see Schanner and Rock (2023).

7. 7 Critical articles about AI systems are also quickly outdated. The first draft of the text was written more than one year ago.

8. 8 Disclosure: The authors are members of the Forum of Computer Scientists for Peace and Social Responsibility and
   the German Informatics Society.

9. 9 Pause Giant AI Experiments: An Open Letter, March 22, 2023. https://futureoflife.org/open-letter/pause-giant-ai-experiments/

10. 10 It concerned a recipe magazine from the Burda publishing house published earlier this year, which was sold for 2.99 euros, and which did not specifically mark that it was “created with the help of ChatGPT and Midjourney,” according to the publisher. For more context, see the statement by the Bavarian Journalists’ Association: https://www.bjv.de/pressemitteilungen/detail/keine-experimente-mit-der-glaubwuerdigkeit/

11. 11 That’s a lie, of course – there will always be enough underpaid grad students and overworked post-docs at hand to write these reports.