It’s seems an extremely difficult idea for most people to grasp that if a thing consistently produces reasoning-like results that are indistinguishable for reasoning, then what it is “really” doing is like asking if a submarine really has fins like a fish.
It does not. Fucking. Matter.
There is no “really” there. You’re looking for that pesky ghost in the machine. And it’s not in you! It is also not in the machine. It just simply does not exist. Why are people so caught up on the process, instead of the result? Yesterday I saw someone make the absurd claim that if we were to set up an enormous database that had stock correct responses to every question inputted into it, then this impossible-to-create1 system was not actually doing any reasoning.
First of all, that’s not how LLMs work. Like, at all. Second, if that tool existed it’d be the most amazing thing in human history and worth trillions and trillions of dollars. Third, creating artificial scenarios that cannot exist in the real world with absolutely no bearing on what’s actually occurring is like asking, “What if my dog were a chimpanzee with lasers on its forehead?” I mean, ok, but who gives a crap. You’re not doing anything useful. You’re setting up fake toy worlds to disprove something that is actually already occurring in our really-existing world.
The “reasoning” debate is a big nothing. Human reasoning is not magic. It uses very nearly the same bag of tricks, elisions, confabulations, wild-ass guesses and speculation based on bad information as LLMs do. What strikes me about LLMs and humans is how very close they are in their thinking. It makes me wonder if there are not hints there of something more universal related to cogitation and cognition, at least as we can access via our narrow window of shared space of possible minds.