Note: the following is an audio adaptation and accuracy is not guaranteed. Please see the original work online. You will hear the following sound to indicate a transition to a description of a table or figure: [[ and the following to indicate a return to the main text: ]]. Now, the post:

Internal experience machines

Papers on self-knowledge, introspection, and what models can tell us about themselves

By Robert Long. March 1, 2026.

I'm on a forthcoming 80,000 Hours podcast episode, out on March third. During my conversation with Luisa Rodriguez—which was in November, many AI-progress aeons ago—I spoke about AI introspection and self-reports. At one point, I promised a list of papers on the topic. Well, now the bill has come due. Here they are!

It's an exciting time to be working on introspection and self-reports: roughly, how and whether AI systems know things about their internal workings, and when and whether they can speak reliably about themselves. My list of papers about this is nowhere close to comprehensive, but should help people who are looking to get into this area.

Some of the earliest work on LLM self-knowledge concerns model calibration: can they track what they know and don't know? Does their confidence track how likely they are to be right?

Kadavath and colleagues, 2022, "Language Models (Mostly) Know What They Know." This showed that larger language models—larger for the time!—were reasonably well-calibrated on multiple-choice and true or false questions, and could be fine-tuned to be more accurate.

Lin, Hilton and Evans, 2022, "Teaching Models to Express Their Uncertainty in Words." They fine-tuned GPT-3 to express uncertainty in natural language—"maybe." This calibration training generalized moderately.

Since then, models are better in some ways but still have weird, jagged calibration abilities. In spite of massive capability increases, you can still see really wild failures of under- and over-confidence, as many LLM users have experienced.

Barkan, Black, and Sourbut, 2025, "Do Large Language Models Know What They Are Capable Of?" found that then-current LLMs were "systematically overconfident but have better-than-random ability to discriminate between tasks they can and cannot accomplish… We also find that LLMs with greater general capability often have neither better-calibrated confidence nor better discriminatory power."

Another important related literature is work on situational awareness. This is not always as "internal" as introspection work, but it asks in general how and whether models are able to reason about their own position in the world.

Laine and colleagues, twenty twenty-four, "The Situational Awareness Dataset," or SAD. This assesses whether models can answer questions and do tasks that depend on facts about itself, its outputs, and its deployment context.

Next up, there's a cluster of papers looking at whether models have some kind of privileged access to information about their own behavioral tendencies — information that wasn't explicitly present in their training data, and that they can leverage better than other models do.

Binder and colleagues, twenty twenty-four, "Looking Inward: Language Models Can Learn About Themselves by Introspection." I'm a middle co-author on this one. It fine-tunes models to predict their own behavior in hypothetical scenarios — in a way that requires them to not just think about what they'd do, but also extract some property of what they'd do.

Song, Hu and Mahowald, twenty twenty-five, "Language Models Fail to Introspect About Their Knowledge of Language." They argue that Binder and colleagues' measure is not stringent enough and overstates model capabilities.

Betley, Balesni, Hariharan, Meinke and Evans, twenty twenty-five, "Tell Me About Yourself: LLMs Are Aware of Their Learned Behaviors." This fine-tunes models on implicit behavioral policies — like making risk-seeking decisions, or steering users toward saying certain words — and then asks models to describe those policies without any in-context examples of their behavior. Models can do it! They can also recognize backdoor-like behaviors.

Plunkett, Morris, Reddy and Morales, twenty twenty-five, "Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions." This extends Betley and colleagues by fine-tuning models on novel quantitative preferences — specific attribute weights for decisions — and then asking models to report those weights. Models can do this, and training on self-report accuracy generalizes to other decision contexts.

Next we have some cool interpretability papers looking into this.

Lindsey, twenty twenty-five, "Emergent Introspective Awareness in Large Language Models." Lindsey injects concept vectors—patterns associated with "bread", say—into model activations and tests whether models can notice. He also tests whether models can exercise intentional control of internal states. Models can do these, although not reliably; that said, they seem to be improving with scale. I wrote about this paper at length when it came out, so I'll also point readers there.

Lindsey and colleagues, twenty twenty-five, "Biology of a Large Language Model." Not primarily about introspection, but there are some relevant sections. For example, it suggests that the mechanisms a model uses to assess whether it is familiar with something—a "do I know this entity?" circuit—are separate from the mechanisms it uses to actually retrieve information about that entity. It also shows that models are not aware of the rather convoluted way they do some mathematical tasks.

Ferrando and colleagues, twenty twenty-four, "Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models." Related mechanistic work identifying the internal circuits that distinguish entities the model knows about from those it doesn't, in a way that can drive model refusals and other behaviors.

Lastly, some overview papers.

The Stanford Encyclopedia of Philosophy entry on introspection. The ever-helpful SEP! This entry is by Eric Schwitzgebel, who literally wrote the book on introspection.

Eleos AI Research, twenty twenty-five, "Why Model Self-Reports Are Insufficient—and Why We Studied Them Anyway." This argues that there are three central problems with model self-reports of consciousness and other states—that is, when Claude says things like "there's something it feels like for me to process information."

First, we lack strong, independent evidence that LLMs have welfare-relevant states in the first place—although we take that possibility seriously—let alone human-like ones.

Second, even if models do have such states, there's no obvious introspective mechanism by which they could reliably report them.

Third, even if models can introspect, we can't be confident that model self-reports are produced by introspection.

Still, we argue that welfare interviews are still worth doing: they can raise red flags, they scale with capability, and they can help identify areas for improvement. We don't want to necessarily take self-reports at face value, but to treat them as one input among many.

Kammerer and Frankish, twenty twenty-three, "What forms could introspective systems take? A research programme." They propose a minimal definition of introspection as "a process by which a cognitive system represents its own current mental states, in a manner that allows the information to be used for online behavioural control." See Lindsey; Song, Lederman, Hu, and Mahowald; and Comșa and Shanahan for other definitions. There's no consensus yet on what we should mean by "AI introspection"; like most interesting concepts, there's probably no single best way to pick it out.

Perez and Long, twenty twenty-three, "Towards Evaluating AI Systems for Moral Status Using Self-Reports." We express hope that AI self-reports might eventually be able to provide evidence about morally significant states, if we can make them reliable. And we propose to train models to answer verifiable questions about themselves—as Binder and others have since done—to see if introspection-like capabilities can generalize to harder questions where we don't have ground truth.

Will this work? Will larger models just solve this with more capabilities? Only time will tell!

A final note: we can't consider introspection and self-reports without reckoning with the distinctive kinds of entities that language models are. Human concepts will not necessarily map neatly onto them. So I recommend thinking about this alongside a heavy dose of ideas about model identity and personas. Sam Marks, Jack Lindsey, and Chris Olah give a handy collection in a recent post: Andreas, 2022; janus, 2022; Hubinger and colleagues, 2023; Shanahan and colleagues, 2023; Byrnes, 2024; and nostalgebraist, 2025.

Please leave comments about areas and papers I've missed! As I said, this is not comprehensive.

[[Here we see Agnes Martin's painting Starlight, from 1963: a blue canvas covered in a delicate hand-drawn grid.]]