Gilbert may have a point here, except for the part about how markets work. This is a popular free market product with a growing number of customers. Today I heard that ChatGPT Plus now has a waiting list. I also find Gilbert’s statement a bit overdramatic, but that’s beside the point. LLMs are useful tools if their limitations are well defined, communicated, and acknowledged, and if the lingering issues of copyright and privacy are resolved.
But if not, and if we continue to treat a computer model as if it had some kind of consciousness and general intelligence, progress will be painful. I’ve expressed this concern several times, using Searle’s Chinese Room argument and pointing out the dangers of so-called “hallucinations” in the hands of well-meaning users who don’t always know what they don’t know.