This is such a lazy and insightless take. The words “thought” and “smart” are doing a whole lot of work here they are completely unqualified to undertake. Non-philosophers are so cute sometimes. Completely worthless, but cute.

Does a submarine swim? Does a plane fly like a bird? Does it matter?

What’s really going on here is that for the first time in history people are encountering legible non-human thought. And it’s the parable of the blind men and the elephant except with intelligence. The submarine question is a trap, and that trap is based on grammar, as Wittgenstein observed long before AI. When we ask questions like if a submarine really swims, we’re interrogating our own mental relation to the world and not what the submarine actually does. It moves through the water regardless.

This is where phenomenalism provide some insight. It allows you to notice what’s actually happening and to think about all this credibly. A phenomenalist would (IMO accurately) say that a table is just a predictable arrangement of certain sensations and that there is no hidden table lurking in the background that is the “real” table. They are very anti-platonist¹. Now let’s consider that in the context of AI. When we demand to know whether AI really thinks, we’re posturing as though there’s some sort of thinking-in-itself, some inner filament that either glows or doesn’t, hidden behind the transcript we see. But that’s just not true. It’s not the way your brain works and it’s not the way anything works. The chat transcript isn’t evidence about the thing. The transcript is the thing!

There is indeed nothing behind the appearance because there never was anywhere. “Thought” and “smart” like “swim” and “fly” is not some hidden essence that if you look hard enough you’ll discover somewhere deep in your brain or in a datacenter. We keep drilling more holes seraching for the ghost in the machine and think the failure is with the drill. In reality, the failure is that there is no fucking ghost.

Which is why I reached for the blind men and the elephant parable. Now on to how structuralism relates, which it does. Structuralism reveals the remainder of wanye’s error. The elephant is not the sum of the parts grabbed. The trunk is not a snake. The leg is no tree. Meaning lives in the relations, not in terms named by us. “Stochastic parrot,” “genius,” “autocomplete,” “oracle,” “bullshit engine,” “plagiarism machine” are all some very clueless people’s hands closing around various features they’ve incompetently found in AI and declaring that feature is the whole animal. Nobody’s lying, precisely. They’ve all seized on something they believe is real. And every one of them is wrong in the same way, which is where the truth is revealed (paradoxically).

Put phenomenalism and structuralism together and you then start to get a coherent and cohesive feel of the whole pachyderm. Phenomenalism kills the essence everyone’s hunting for behind the appearances. Structuralism tells us where the meaning resided the whole time. And that, as always, is in the relations, the roles, the differences between things. It was never in the substrate (thank you, Church-Turing). To “think” was always to occupy a position in a structure of inference, reference, revision, and necessary surprise. “Smart” just means “accords with my own preferences” to wanye and many others.

Clearly, a system that occupies the position of producing tangible thought structures thinks in exactly the sense that a submarine moves through water: competently, and without needing any authorization to call it “swimming.” Because the word is not the thing.

What’s actually novel is that a non-human has legible thought. We’ve long known (though many have denied) that corvids, whales and octupuses have opaque but substantial cognitive capabilities that are clearly present, even if we can’t ask them about any of it. But for the first time in our history the thinking comes out in language, inspectable, arguable, in our own idiom. And that legibility is precisely the condition under which we can no longer weakly smuggle our conceptual confusion out the back as an inference to (or denial of) some hidden mind. In the the case of modern LLM AI, the elephant walked into the light and declared, “I’m a goddamn elephant. What you gonna do?”

Meanwhile, we’re still standing around it, each holding a piece, arguing about the leg or the trunk.

1. Platonism is mostly moronic with some shards of brilliance.