An LLM or AI with no intentionality is irrelevant, and without embodiment intentionality is impossible. The AI debateโs terms are ill-defined, but even those AIs who ape or even best human achievements do not pass the test as they possess no impetus, no internal drive, and merely operate with a mere shadow of intention borrowed from the whiff of humanity that permeates the silicon. There, these nascent pseudo-consciousnesses subsist on the fumes of minds beyond measure risen from only mud.
We might ourselves be ChatGPT most of the time. However, ChatGPT is a person engaged with and embodied in the world none of the time. And that makes all the difference.