Sentence completion is a predictive task for the language model, not a causal one. It works as just another data model – it doesn’t need a world model, that is, unless a revolution is at stake.
World models are causal representations of the environment to the extent required by the tasks to be performed (as discussed here and there).
World models guide actions by making predictions based on this causal representation. So while not all data models need to be causal, all world models do.
LLM agents as world modelers?
LLMs are data models, so they are useful simplifications of the world. How well LLM agents can move from one useful simplification to another will determine the business use cases for which the agents will be useful. We’re about to find out.
* Image courtesy of xkcd.com.