With the emergence of large language models (LLMs) such as GPT, Claude, and Gemini, the debate on what it means for an artificial system to understand content has resurfaced. Transformer-based models show remarkable performance in generating natural language, answering questions, translating text, and assisting with complex tasks.
But can they truly think? Or is their ability merely the result of a sophisticated simulation?
The Chinese Room: Why Simulation Is Not Understanding
A classic starting point for exploring this question is John Searle’s well-known 1980 thought experiment, the Chinese Room. In this scenario, a person locked in a room receives written input in Chinese, a language they do not understand, and produces Chinese output using a set of syntactic rules described in a book found inside the room.
To an external observer, it appears as though the person understands the language. Yet, according to Searle, there is no true understanding: they are merely manipulating symbols based on formal rules.
This metaphor maps naturally onto language models like GPT, BERT, or Claude. These systems can generate coherent text, answer questions, imitate styles, and even sustain convincing dialogues.
However, they operate through purely syntactic processes: manipulating symbols based on patterns learned from massive datasets, without semantic understanding in the human sense. As in the Chinese Room, their performance simulates comprehension without actually achieving it.
Unlike humans, these systems need enormous amounts of text to build their internal rule-book. Once this vast set of rules is learned, generating or translating text becomes a matter of applying patterns rather than understanding meaning.
Over the years, there have been many responses and counter-responses to the Chinese Room argument (if you're curious you can start here). We will report only one of the alternative theses that have been proposed.
Alternative Views: Is Understanding an Emergent Property?
Some scholars, inspired by dynamical systems theory, suggest that understanding should not be conceived as an internal property (A property which is contained in the system) but as one that emerges from the interaction between a system and its environment. From this perspective, even a sufficiently articulated artificial system could give rise to forms of understanding, overcoming the barrier between syntax and semantics on which Searle insists.
This approach is connected, in some cases, to more radical philosophical views, such as panpsychism, according to which consciousness (and therefore also the capacity to understand), in its rudimentary forms, is a fundamental property of reality: an intrinsic quality of all physical systems, including artificial ones. This property would be more evident in more complex systems and less developed in simpler ones. From this perspective, the rigid distinction between mind and machine becomes blurred; if every dynamic system has at least a minimal degree of experience, then even an AI model could, at least theoretically, possess an elementary form of understanding. However, a crucial question remains: how do we measure the “complexity” of a system that determines its degree of consciousness? This is an open issue, since various measures fail to align with our intuitions about consciousness. For example, Earth’s atmosphere is a physically “complicated” system, yet it does not exhibit the properties of other much “simpler” systems (for instance certain insect species with only a few thousand neurons) that correspond more closely to our intuitions about conscious experience.
A view of this kind also raises another problem: if understanding arises from complexity, why does it seem to be a unique property of biological systems?
Our brain is extremely fragile when exposed to shocks or physical damage; even while maintaining its global “complexity,” it can lose its functions. Why, then, does such a property appear so fragile? A local explanation rather than a holistic one seems still relevant.
These questions highlight a crucial point: our difficulty in defining what understanding actually is. Is it an internal and unknowable process? Is it an observable behavior? Is it a relationship between a system and its environment? Depending on the answer, the way we assess the cognitive abilities of machines changes dramatically.
Ultimately, the Chinese Room experiment retains fundamental value. It invites us to distinguish between function and understanding. At the same time, it opens the door to new questions: is it possible that understanding emerges solely from sufficiently complex architectures? Do we need to rethink our categories to include new forms of mind?
