*Priyanka Kargupta, Shuyue Stella Li, Haocheng Wang, Jinu Lee, Shan Chen, Orevaoghene Ahia, Dean Light, Thomas L. Griffiths, Max Kleiman-Weiner, Jiawei Han, Asli Celikyilmaz, Yulia Tsvetkov*

** Equal contribution in alphabetical order.*

Paper: https://arxiv.org/abs/2511.16660

Github: https://github.com/pkargupta/cognitive_foundations/

Data: https://huggingface.co/collections/stellalisy/cognitive-foundations

TL;DR

<aside> 📌

We built a comprehensive framework grounded in cognitive science to understand how LLMs actually reason. By analyzing 192K+ reasoning traces from 18 models alongside human think-aloud traces, we found that:

• Models use fundamentally different reasoning strategies than humans — relying on shallow sequential processing instead of hierarchical planning and metacognitive monitoring 🤖 vs 👥
• Models deploy behaviors inversely to what success requires — they narrow their behavioral repertoire precisely when they should expand it 📉
• Latent capabilities exist but aren't spontaneously expressed — test-time guidance unlocks up to 72% performance gains 🚀
• Research focuses on easily measured behaviors — while neglecting meta-cognitive controls that correlate with success 📊

🤔 How can models solve complex problems while failing on simpler variants? What cognitive processes actually drive reasoning success?

Find the answers in our analysis below 👇

</aside>

🎯 The Measurement Crisis in LLM Reasoning

Large language models exhibit unprecedented cognitive dissonance: they master complex skills while lacking simpler prerequisite ones, solve challenging problems while failing on trivial variants, and achieve high benchmark performance without robust generalization.

Current training and evaluation paradigms reward reasoning outcomes without examining the cognitive processes that produce them. We lack the conceptual framework to characterize what reasoning behaviors should manifest and the empirical methods to assess whether they do.

📚 The Fragmentation Problem

Our meta-analysis of 1,598 arXiv papers on LLM reasoning reveals:

Distribution of cognitive element presence across 1,598 arXiv LLM Reasoning papers. Partially Present indicates that there is evidence that the behavior was considered in the design (motivation, method, evaluation) of the paper but was not the primary focus. Present indicates that there is evidence that the behavior was a conscious and significant design decision.

• 55% examine sequential organization, 60% study decomposition
• Only 8% study backward chaining, 16% self-awareness, 8% evaluation

Without a comprehensive framework, we risk optimizing what we measure rather than what matters. 🎯

🔬 A Unified Taxonomy of 28 Cognitive Reasoning Behaviors

We synthesized decades of cognitive science research into a framework spanning 28 reasoning elements across four dimensions: