Mind the AI Gap: Understanding vs. Knowing

(Updated in December, 2024)

Gorkem Turgut (G.T.) Ozer

Paul College of Business and Economics

May, 2024

Opening questions

 

  1. Can LLMs help fill in the knowledge gaps?
  2. Can LLMs understand?
    • What does it mean to understand?
    • What does it take to understand?
  3. How can LLMs be used to help learners understand?
    (beyond providing information and facilitating knowledge)

Chinese room argument1,2

Input:



This question in Chinese means “What brings happiness?”

Output:

The answer is in red:
“Be the stream of the universe!” (from Lao-Tzu’s Tao Te Ching)

Knowing vs. understanding

 

“Any fool can know. The point is to understand.” ― A. Einstein3

 

“You can know the name of a bird in all the languages of the world, but when you’re finished, you’ll know absolutely nothing whatever about the bird.” ― R. Feynman4

What does it mean to understand?

  • Identifying causality is central to our understanding:
    • “We understand the sum total of phenomena, in so far as they, by virtue of an internal principle of causality, are connected with each other throughout.” I. Kant5
  • However, we do not directly perceive cause and effect.
    • We make observations and infer causal relationships.
  • How do we do this?
    • By reasoning. Specifically, counterfactual reasoning.

Counterfactual reasoning

is a hallmark of human thought, enabling the capacity to shift from perceiving the immediate environment to an alternative, imagined perspective. Mental representations of counterfactual possibilities (e.g., imagined past events or future outcomes not yet at hand) provide the basis for learning from past experience, enable planning and prediction, support creativity and insight, and give rise to emotions and social attributions (e.g., regret and blame).6

Meanwhile, in the LLM land…

Deductive reasoning with GPT-4o (or lack thereof)7

Deductive reasoning with GPT-o1 (or lack thereof)8

Opening questions revisited

 

  1. Can LLMs help fill in the knowledge gaps? Absolutely.9
  2. Can LLMs understand? No, not at the moment.
    • What does it mean to understand? To infer causality.
    • What does it take to understand? To be able to reason.
  3. How can LLMs be used to help learners understand?(beyond providing information and facilitating knowledge)

Closing questions

  1. Can I help you understand something without having understood it myself? In what way and to what extent?
  2. How would you interact with me to accomplish this?

By analogy:

  1. How can we use large language models in education to help learners understand (in addition to knowing)?

  2. How can we help learners avoid the illusion of a thinking machine that understands, so as not to lend their cognition?

Footnotes

  1. Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417-424.

  2. Anderson, D. L., Stufflebeam, R., & Cox, K. (2018). Searle’s Chinese Room Argument. Illinois State University.

  3. Christian, James Lee. 1990. Philosophy: An Introduction to the Art of Wondering. Holt, Rinehart & Winston.

  4. Feynman, R. P. (1988). What Do You Care What Other People Think? W. W. Norton & Company.

  5. Kant, I. (1908). Critique of pure reason. 1781. Modern Classical Philosophers, Cambridge, MA: Houghton Mifflin, 370-456.

  6. Van Hoeck, N., Watson, P. D., & Barbey, A. K. (2015). Cognitive neuroscience of human counterfactual reasoning. Frontiers in Human Neuroscience, 9, 420.

  7. Queries were executed using gpt-4o-2024-05-13 on May 22, 2024. Sample query adapted from Jingjing Li and Reza Mousavi.

  8. Queries were executed using gpt-o1-preview on September 13, 2024.

  9. This does not imply unbiased or complete knowledge. The issues of bias and ethics are outside the scope of this discussion.