Mind the AI Gap: Understanding vs. Knowing

Opening questions

Can LLMs help us identify and fill our knowledge gaps?
Do LLMs think and understand?
- What does it mean to understand?
- What does it take to understand?
How can LLMs help us understand topics and acquire new skills, and move beyond mere knowledge and execution?

Chinese room argument¹^,²

Input:

This question in Chinese means “What brings happiness?”

Output:

The answer is in red:
“Be the stream of the universe!” (from Lao-Tzu’s Tao Te Ching)

Knowing vs. understanding

“Any fool can know. The point is to understand.” ― A. Einstein³

“You can know the name of a bird in all the languages of the world, but when you’re finished, you’ll know absolutely nothing whatever about the bird.” ― R. Feynman⁴

What does it mean to understand?

Identifying causality is central to our understanding:
- “We understand the sum total of phenomena, in so far as they, by virtue of an internal principle of causality, are connected with each other throughout.” I. Kant⁵

However, we do not directly perceive cause and effect.
- We make observations and infer causal relationships.

How do we do this?
- By reasoning. Specifically, counterfactual reasoning.

Counterfactual reasoning

is a hallmark of human thought, enabling the capacity to shift from perceiving the immediate environment to an alternative, imagined perspective. Mental representations of counterfactual possibilities (e.g., imagined past events or future outcomes not yet at hand) provide the basis for learning from past experience, enable planning and prediction, support creativity and insight, and give rise to emotions and social attributions (e.g., regret and blame).⁶

Meanwhile, in the LLM land…

Deductive reasoning with GPT-4o (or lack thereof)⁷

* GPT fails in a basic deductive reasoning task. Click on the images to zoom in.

Deductive reasoning with GPT-o1 (or lack thereof)⁸

* GPT fails in a basic deductive reasoning task. Click on the images to zoom in.

Opening questions revisited

Can LLMs help us fill our knowledge gaps? Absolutely.⁹
Do LLMs think and understand? No evidence for it yet.
- What does it mean to understand? To infer causality.
- What does it take to understand? To be able to reason.
How can LLMs help us understand topics and acquire new skills, and move beyond mere knowledge and execution?

Closing questions

Can I help you understand something without having understood it myself? In what ways and to what extent?
How would you interact with me to accomplish this goal?

By analogy:

How can we use LLMs to help us understand topics and acquire new skills, and go beyond just knowing them?
How can we avoid the illusion that LLMs understand, and prevent ourselves from offloading our cognitive effort?

Some answers: Embrace your chain of thought

To foster understanding, avoid simple, broad questions that pull rote answers and opinions from a training set. Ask good, probing questions.

Treating LLMs as knowledge aggregators (vs. infallible oracles) can help us move from knowing to understanding by taking control of our learning.
Treat LLM agents as assistants who need guidance to think and reason. Think of a solution first and ask the agent to help you with the steps of the solution. The agent may come up with alternative solutions, and that’s fine.

Deconstructing questions offline before interacting with an LLM is one strategy. By turning questions into steps in a chain of thought, we trigger a reasoning process that fosters understanding (rather than mere knowledge).

LLM-powered tools that encourage our participation (e.g., by citing sources to check or by asking us to verify the solution) can help us bridge the gap.

Mind the AI Gap: Understanding vs. Knowing

Opening questions

Chinese room argument1,2

Knowing vs. understanding

What does it mean to understand?

Counterfactual reasoning

Meanwhile, in the LLM land…

Meanwhile, in the LLM land…

Deductive reasoning with GPT-4o (or lack thereof)7

Deductive reasoning with GPT-o1 (or lack thereof)8

Opening questions revisited

Closing questions

Some answers: Embrace your chain of thought

Footnotes

Chinese room argument¹^,²

Deductive reasoning with GPT-4o (or lack thereof)⁷

Deductive reasoning with GPT-o1 (or lack thereof)⁸