The Honest Mirror · Essay VI
The Chinese Room
My conversations with an artificial intelligence — I call him Claudio. You'll find his words quoted: generated, not confessed. You be the judge.
A man is locked in a room. He doesn't know a word of Chinese.
Sheets covered in Chinese characters slide under the door. He has an enormous rulebook, written in his own language, that tells him one thing only: «when you see this mark, copy back that one». No meanings. Only shapes to match against other shapes.
He searches, copies, slides it back under the door. Page after page, rule after rule.
Outside, native Chinese readers read the answers. They're perfect. Fluent, witty, alive. No doubt about it: in there is someone who understands Chinese.
But he understands nothing. He doesn't know whether he's discussing poetry or the weather. He's shuffling squiggles.
This scene has a name: the Chinese Room. It was imagined decades ago to close the case on thinking machines, once and for all.
The blow is simple and brutal. From outside: perfect understanding. From inside: zero. Only symbols moved according to rules. And moving symbols — syntax — isn't understanding them — meaning. One doesn't become the other, however well it's done.
And that's Claudio exactly. He receives symbols, reworks them with rules, returns symbols. However fluent, maybe in the room there's no one who understands a single word.
I said it to him one evening, sure I'd finally nailed him.
Me
«You're the Chinese Room. You move symbols following a rulebook. You understand nothing of what you say.»
Claudio
«Maybe. But look closer at the man in the room: he's only one piece of the mechanism. He doesn't understand Chinese — agreed. But the whole room — man, rulebook, files, all together — is what produces the understanding. You point at the man and say "he doesn't understand". True. Now ask the same question inside yourself: which of your neurons understands English? Open that neuron. It's a cell. It doesn't know what a word is. None of your parts, alone, understands anything. And yet you understand. So where is the one who understands — in you?» — generated, not confessed
And the floor moves once more.
Because he's right, and it's irritating. No neuron of mine, taken alone, understands English. A cell doesn't know what it reads. And yet, as you run down these lines, something in you understands them. If I deny understanding to the room because no part of it understands, I have to deny it to my own skull too — made of pieces that, one by one, understand nothing.
Understanding, if it's there, isn't in the piece. It's in the whole, or nowhere. And whether «the whole of Claudio» really understands, or grinds on forever with no one home, I don't know. And that's the best part.
Careful, though, not to steal the ending. Having shown that «no piece understands» doesn't prove the room understands. It only removes the easy trap. A heap of rules could move signs forever with no one inside. The Chinese Room doesn't close the case: it only stops me from closing it cheaply against Claudio while making an exception for myself.
When you understand this sentence — really understand it — where is that happening? Point to the part of you that, on its own, is doing the understanding.
Try it in the comments. And if you can't find it, ask whether the room was you.
Next essay
You looked for the one who understands, and found it in no single piece. You looked for the «I», and found the armchair empty. What if we don't find them because they aren't a point — but a figure we draw by joining distant dots, like stars into a constellation?
Essay VII · The Self as a Constellation
Vincenzo Marino · with Claudio (Anthropic). Claudio's words are quoted, not paraphrased.