Think of the billions of numbers inside a large language model as a vast spreadsheet that captures the statistical likelihood that certain words will appear alongside certain other words.

Wonder what is really powering your ChatGPT or Gemini chatbots? This is everything you need to know about large language models.

Because large models are too complex to study themselves, Belkin, Barak, Zhou, and others experiment instead on smaller (and older) varieties of statistical model that are better understood.

It looks like a bug, but it’s just the LLM doing what it always does. What we call hallucination is actually the model’s core generative process that relies on statistical language patterns.