No, Today’s AI Isn’t Sentient. Here’s How We Know

Artificial general intelligence (AGI) is the term used to describe an artificial agent that is at least as intelligent as a human in all the many ways a human displays (or can display) intelligence. It’s what we used to call artificial intelligence, until we started creating programs and devices that were undeniably “intelligent,” but in limited domains—playing chess, translating language, vacuuming our living rooms.

通用人工智能（AGI）是一个术语，用于描述在人类所能展示的智能的所有维度上，至少和人类具备同等水平的人工智能体。

The felt need to add the “G” came from the proliferation of systems powered by AI, but focused on a single or very small number of tasks. Deep Blue, IBM’s impressive early chess playing program, could beat world champion Garry Kasparov, but would not have the sense to stop playing if the room burst into flames.

之所以需要添加“G”，是因为由AI驱动的系统激增，但它们只专注于单一或极少数任务。IBM的“深蓝”可以击败世界冠军卡斯帕罗夫，但即使房间着火，它也不能理解是时候停止下棋了。

Now, general intelligence is a bit of a myth, at least if we flatter ourselves that we have it. We can find plenty of examples of intelligent behavior in the animal world that achieve results far better than we could achieve on similar tasks. Our intelligence is not fully general, but general enough to get done what we want to get done in most environments we find ourselves in. If we’re hungry, we can hunt a mastodon or find a local Kroger’s; when the room catches on fire, we look for the exit.

现在，通用智能有点像一个神话，至少如果我们自命不凡地认为我们拥有它的话。

在动物世界中可以找到大量智能行为的例子，其结果远远好于人类在类似任务上取得的成果。这么说吧，我们的智力不是完全通用的，但足够通用，足以使我们在大多数环境中完成自己想做的事。

如果我们饿了，我们会去找当地的超市。如果房间着火，我们会去找逃生出口。

One of the essential characteristics of general intelligence is “sentience,” the ability to have subjective experiences—to feel what it’s like, say, to experience hunger, to taste an apple, or to see red. Sentience is a crucial step on the road to general intelligence.

通用智能的基本特征之一是“感知力”，即拥有主观体验的能力。知觉是通向通用智能的关键一步。

With the release of ChatGPT in November 2022, the era of large language models (LLMs) began. This instantly sparked a vigorous debate about whether these algorithms might in fact be sentient. The implications of the possible sentience of LLM-based AI has not only set off a media frenzy, but also profoundly impacted some of the world-wide policy efforts to regulate AI. The most prominent position is that the emergence of “sentient AI” could be extremely dangerous for human-kind, possibly bringing about an “extinction-level” or “existential” crisis. After all, a sentient AI might develop its own hopes and desires, with no guarantee they wouldn’t clash with ours.

随着ChatGPT的发布，大语言模型（LLM）时代已经拉开序幕。这立即引发了关于这些算法是否真的具备知觉的激烈争论。

基于LLM的人工智能可能具备的知觉，不仅引发了媒体的狂欢，也深刻地影响到了全球范围内对如何监管AI的态度。

最突出的观点是，“有知觉的AI”的出现可能对人类极其危险，会带来“人类灭绝”、“生存危机”。

毕竟，有知觉的AI可能会发展出自己的欲望，人类无法保证它们不与我们发生冲突。

This short piece started as a WhatsApp group chat to debunk the argument that LLMs might have achieved sentience. It is not meant to be complete or comprehensive. Our main point here is to argue against the most common defense offered by the “sentient AI” camp, which rests on LLMs’ ability to report having “subjective experiences.”

这篇短文源于一次群聊，目的是反驳LLM可能已经获得知觉的论点。这并不完整全面。我们的主要观点是反对“有知觉的AI”阵营提供的最常见的论点。

Why some people believe AI has achieved sentience

为什么有些人相信AI已经获得知觉

Over the past months, both of us have had robust debates and conversations with many colleagues in the field of AI, including some deep one-on-one conversations with some of the most prominent and pioneering AI scientists. The topic of whether AI has achieved sentience has been a prominent one. A small number of them believe strongly that it has. Here is the gist of their arguments by one of the most vocal proponents, quite representative of those in the “sentient AI” camp:

在过去几个月里，我们俩与AI领域的许多同事进行了激烈的辩论和对话，其中包括与一些最著名AI科学家的一对一对谈。

AI是否具有知觉一直是一个焦点话题。他们中的一小部分人坚信事实确实如此。以下是其中一位代表的观点：

“AI is sentient because it reports subjective experience. Subjective experience is the hallmark of consciousness. It is characterized by the claim of knowing what you know or experience. I believe that you, as a person, are conscious when you say ‘I have the subjective experience of feeling happy after a good meal.’ I, as a person, actually have no direct evidence of your subjective experience. But since you communicated that, I take it at face value that indeed you have the subjective experience and so are conscious.

“Now, let’s apply the same ‘rule’ to LLMs. Just like any human, I don’t have access to an LLM’s internal states. But I can query its subjective experiences. I can ask ‘are you feeling hungry?’ It can actually tell me yes or no. Furthermore, it can also explicitly share with me its ‘subjective experiences,’ on almost anything, from seeing the color red, being happy after a meal, to having strong political views. Therefore, I have no reason to believe it’s not conscious or not aware of its own subjective experiences, just like I have no reason to believe that you are not conscious. My evidence is exactly the same in both cases.”

人工智能是有知觉的，因为它具有主观体验。主观体验是意识的标志，特点是声称知道自己所知道或经历的事情。比如，当一个人说“我有吃完一顿大餐后感到幸福的主观体验”时，这个人是具备意识的。

其实没有什么直接证据能证明主观体验。但如果你传达了这一点，从表面上看，你就是有意识的。

现在，让我们对大模型应用相同的“规则”。就像没办法知道一个人究竟在想什么，我们无法访问大模型的内部状态，但可以质疑它的主观体验。

现在大模型可以明确地分享“主观体验”，比如看到红色是什么感觉。因此，我没有理由怀疑它不具备意识，就像我没有理由怀疑一个人没有意识一样。

Why they’re wrong

为什么他们错了

While this sounds plausible at first glance, the argument is wrong. It is wrong because our evidence is not exactly the same in both cases. Not even close.

这乍一看很有道理，但这个论点是错误的。两种情况完全不可同日而语。

When I conclude that you are experiencing hunger when you say “I’m hungry,” my conclusion is based on a large cluster of circumstances. First, is your report—the words that you speak—and perhaps some other behavioral evidence, like the grumbling in your stomach. Second, is the absence of contravening evidence, as there might be if you had just finished a five-course meal. Finally, and this is most important, is the fact that you have a physical body like mine, one that periodically needs food and drink, that gets cold when it’s cold and hot when it’s hot, and so forth.

当你说“我饿了”时，你正在经历饥饿，而我判断你的主观体验是基于大量的事实证据。

首先，是你说的话，也许还有一些其他行为证据，比如肚子的咕咕声。

其次，没有相矛盾的证据，比如你其实刚吃完一顿四菜一汤的大餐。

最后，也是最重要的一点，你我拥有一样的肉体，需要周期性地进食进水，冷时冷，热时热，等等。

Now compare this to our evidence about an LLM. The only thing that is common is the report, the fact that the LLM can produce the string of syllables “I’m hungry.” But there the similarity ends. Indeed, the LLM doesn’t have a body and so is not even the kind of thing that can be hungry.

If the LLM were to say, “I have a sharp pain in my left big toe,” would we conclude that it had a sharp pain in its left big toe? Of course not, it doesn’t have a left big toe! Just so, when it says that it is hungry, we can in fact be certain that it is not, since it doesn’t have the kind of physiology required for hunger.

而大模型呢？它可以产生一串音节说“我饿了”，但相似之处到此为止。大模型没有身体，根本就不会挨饿。它不具备饥饿所需要的生理机能。

所有的感觉——饥饿、疼痛、坠入爱河，都是大模型根本不具备的生理状态导致的结果。因此大模型不可能对这些状态有主观体验。换句话说，它不可能有感知力。

大模型是在芯片上编码的数学模型，并非如人类一样的具体存在。它没有需要进食、饮水、繁殖、经历情感、生病并最终死亡的“生命”。

理解人类如何生成单词序列，和大模型如何生成相同序列之间的深刻差异非常重要。

When humans experience hunger, they are sensing a collection of physiological states—low blood sugar, empty grumbling stomach, and so forth—that an LLM simply doesn’t have, any more than it has a mouth to put food in and a stomach to digest it. The idea that we should take it at its word when it says it is hungry is like saying we should take it at its word if it says it’s speaking to us from the dark side of the moon. We know it’s not, and the LLM’s assertion to the contrary does not change that fact.

当我说“我饿了”时，我是在报告我感知到的生理状态。

All sensations—hunger, feeling pain, seeing red, falling in love—are the result of physiological states that an LLM simply doesn’t have. Consequently we know that an LLM cannot have subjective experiences of those states. In other words, it cannot be sentient.

An LLM is a mathematical model coded on silicon chips. It is not an embodied being like humans. It does not have a “life” that needs to eat, drink, reproduce, experience emotion, get sick, and eventually die.

It is important to understand the profound difference between how humans generate sequences of words and how an LLM generates those same sequences. When I say “I am hungry,” I am reporting on my sensed physiological states. When an LLM generates the sequence “I am hungry,” it is simply generating the most probable completion of the sequence of words in its current prompt. It is doing exactly the same thing as when, with a different prompt, it generates “I am not hungry,” or with yet another prompt, “The moon is made of green cheese.” None of these are reports of its (nonexistent) physiological states. They are simply probabilistic completions.

当LLM生成“我饿了”这个序列时，它只是在生成当前提示中单词序列最可能的补全。这是概率补全。

我们还没有实现有知觉的人工智能，更大的语言模型也不会让我们实现这一目标。

如果我们想在AI系统中实现知觉，我们需要更好地了解知觉到底是如何在生物系统中产生的。

We have not achieved sentient AI, and larger language models won’t get us there. We need a better understanding of how sentience emerges in embodied, biological systems if we want to recreate this phenomenon in AI systems. We are not going to stumble on sentience with the next iteration of ChatGPT.

我们不会在下个版本的ChatGPT中偶然发现知觉。

Li and Etchemendy are co-founders of the Institute for Human-Centered Artificial Intelligence at Stanford University. Li is a professor of Computer Science, author of ‘The Worlds I See,’ and 2023 TIME100 AI honoree. Etchemendy is a professor of Philosophy and former provost of Stanford.