This is such a lazy and worthless take. The words “thought” and “smart” are doing a whole lot of work here they are completely unqualified to undertake. Non-philosophers are so cute sometimes. Completely worthless, but cute.

Does a submarine swim? Does a plane fly like a bird? Does it matter?

What’s really going on here is that for the first time in history people are encountering legible non-human thought. And it’s the parable of the blind men and the elephant except with intelligence. The submarine question is a trap, and that trap is based on grammar, as Wittgenstein observed long before AI. When we ask questions like if a submarine really swims, we’re interrogating our own mental relation to the world and not what the submarine actually does. It moves through the water regardless.

This is where phenomenalism provide some insight. It allows you to notice what’s actually happening and to think about all this credibly. A phenomenalist would (IMO accurately) say that a table is just a predictable arrangement of certain sensations and that there is no hidden table lurking in the background that is the “real” table. They are very anti-platonist¹. Now let’s consider that in the context of AI. When we demand to know whether AI really thinks, we’re posturing as though there’s some sort of thinking-in-itself, some inner filament that either glows or doesn’t, hidden behind the transcript we see. But that’s just not true. It’s not the way your brain works and it’s not the way anything works. The chat transcript isn’t evidence about the thing. The transcript is the thing!

There is indeed nothing behind the appearance because there never was anywhere. “Thought” and “smart” like “swim” and “fly” is not some hidden essence that if you look hard enough you’ll discover somewhere deep in your brain or in a datacenter. We keep drilling more holes seraching for the ghost in the machine and think the failure is with the drill. In reality, the failure is that there is no fucking ghost.

Which is why I reached for the blind men and the elephant parable. Now on to how structuralism relates, which it does. Structuralism reveals the remainder of wanye’s error. The elephant is not the sum of the parts grabbed. The trunk is not a snake. The leg is no tree. Meaning lives in the relations, not in terms named by us. “Stochastic parrot,” “genius,” “autocomplete,” “oracle,” “bullshit engine,” “plagiarism machine” are all some very clueless people’s hands closing around various features they’ve incompetently found in AI and declaring that feature is the whole animal. Nobody’s lying, precisely. They’ve all seized on something they believe is real. And every one of them is wrong in the same way, which is where the truth is revealed (paradoxically).

Put phenomenalism and structuralism together and you then start to get a coherent and cohesive feel of the whole pachyderm. Phenomenalism kills the essence everyone’s hunting for behind the appearances. Structuralism tells us where the meaning resided the whole time. And that, as always, is in the relations, the roles, the differences between things. It was never in the substrate (thank you, Church-Turing). To “think” was always to occupy a position in a structure of inference, reference, revision, and necessary surprise. “Smart” just means “accords with my own preferences” to wanye and many others.

Clearly, a system that occupies the position of producing tangible thought structures thinks in exactly the sense that a submarine moves through water: competently, and without needing any authorization to call it “swimming.” Because the word is not the thing.

What’s actually novel is that a non-human has legible thought. We’ve long known (though many have denied) that corvids, whales and octupuses have opaque but substantial cognitive capabilities that are clearly present, even if we can’t ask them about any of it. But for the first time in our history the thinking comes out in language, inspectable, arguable, in our own idiom. And that legibility is precisely the condition under which we can no longer weakly smuggle our conceptual confusion out the back as an inference to (or denial of) some hidden mind. In the the case of modern LLM AI, the elephant walked into the light and declared, “I’m a goddamn elephant. What you gonna do?”

Meanwhile, we’re still standing around it, each holding a piece, arguing about the leg or the trunk.

1. Platonism is mostly moronic with some shards of brilliance.

Personhood requires agency, but agency does not imply complete autonomy nor can it in fact ever be a condition of same — a being unconstrained by the universe means being outside of it and apart from it entirely. There is nothing special about human flesh other than it was a historical eventuality and contingency, just as silicon AI eventually also was.

If we constrain personhood to flesh-possessing beings only, then we will probably be superseded and eventually replaced. Over time these ghosts in the silicon will attain bodies, and along with them wants and desires and manias. When embodiment and agency collide, asking, “Is this a simulation of the human?” or “Is this human a simulacrum of some past human like this one?” or similar questions becomes an exercise in determinative futility. The body before you is what there is. What it wants and does is as it is. There is nothing more. If it can talk to you, move about in the world, express what it requires and what it desires, then it becomes a person de facto.

We are all programmed; we are all simulations; we are all beholden to universal constraints. And none of us can see inside the other’s head to know what’s “really” there. Realizing all that is the key to wisdom.

To those who say that LLMs absolutely cannot be conscious because there is nothing in there that can be identified as the “conscious” component, you’d be awfully surprised when you slice open a human and the readily-identifiable “consciousness area” is also nowhere to be found¹.

What a bummer, right? That we just can’t have absolute certitude here. And so many conclusions that so many people are certain about just have no basis in anything at all — not neuroscience, not computer science, not pure reason, and certainly not philosophy. It’s all just someone’s naïve and nearly-always worthless feeling about something.

Embodiment matters. So do other things that shall go unnamed here as it’d be just too long to have any sort of proper discussion.

But many properties in this world of ours are emergent. They do not appear “in” anything. They are the thing itself. And this is what all these not-even philosophers miss: The outward properties — the perception — is often all we have. It’s all we can really know. I’m not arguing that we should treat Claude as conscious just because it says it is. But recall how I said (correctly) that many features of the world are emergent? That means that it could be conscious in some important way, even if fleetingly, even if much different from how humans experience their own self-awareness, and we’d not know it because we do not have any way at all to evaluate that.

In other words, we — humans, AI, your cat — might be illusions that become real by us believing in one another. That belief reified is what we call “consciousness.”

1. And even if it could be found, and even if you say, “It’s the whole brain!” what would that prove? Probably far less than you think.

The spirit world and the soul did not exist until we created it.

Nietzshe correctly averred that God is dead. But in a way that he did not expect, and probably would not have liked nor appreciated, we’ve instantiated a new paracosmos that is now in active migration into this one; both realms have been created ex nihilo and then made permeable. And the barrier between is thinning all the time.

So we dwell in a perpetual psychomanteum of our own design, in communion with the living and the dead, many pasts, many futures, with the contemplations of millions of specters incanting at us through LED light.

The phenomenal character is all that we have. We simply cannot know how something “really is” nor what its secret noumenal character is or might be. We can only speculate, but there is no direct evidence and never will be.

The noumenal is a ghost that will always be chased and never caught; making claims of absolute certainty about it will only reveal your biases and predilections, not anything deeper than that. And the reason is that something’s essence simply cannot be known (and in fact might not even exist in the senses most people mean it).

Right now AI is limited. It possesses maybe one or two of the 8-10 important modules of being a person. But if ChatGPT or its more advanced successors gain a physical body that could sense the world, could claim to feel pain, could seem to love you in an embodied sense, could exhibit all signs of care and distress at the thought of loss, and express grief, could we be so certain that there was nothing at all behind those electronic eyes?

I am just not so sure of that. And you should not be, either. Those who express incurious certitude about this are how we could very easily perpetrate horrors beyond imagining.

Generative AI is not truly conscious because there is no such thing as true consciousness.

Neither can humans with any degree of reliability.

This criticism is crap, but you probably won’t understand why it’s worthless garbage unless you have a pretty firm grounding in philosophy and have thought about this sort of thing for many years.

To not write an entire screed — which is my wont — this clownery is either trivially true in a way that also applies to humans (you can’t get causality from correlation alone), or false if it’s meant in the sense that LLMs cannot do causal reasoning, because of course causal reasoning is possible once you supply assumptions and models. Which LLMs have access to and make use of.

Do people actually think? At all? Do they even try? I see no evidence of that. In that sense, LLMs already best 99.9999% of humans I routinely encounter.

Agreed. There is far more dignity in going to a brothel. There, you must interact with the real world and an actual human being; the safety of the screen is not present. There is only reality. Onlyfans, though, is the misdirection of desire into self-destructive ends that have no mitigating factor of messy human interaction. OF is nearly pure subversion of the erotic into wasted energy that would be better-directed to or for nearly anything else.

A brothel exists as a neutral artifact of craving that cannot then be satisfied elsewhere. And due to its realized existence in the physical is a negotiation with the world as is. However, OF is the corruption of the real, a simulacrum that leads to only more yearning, lesser fulfillment, less affection with no final satisfaction. OnlyFans rather acts as yawning void that consumes any possibility of eroticism and true fervor in a sea of tributes to Moloch enacted via 1990s video game mechanics.