Position: Principles of Animal Cognition to Improve LLM Evaluations

Principles of Animal Cognition to Improve LLM Evaluations

Speakers: Sunayana Rane, Cyrus Kirkman, Graham Todd, Amanda Royka, Ryan Law, Erica Cartmill, Jacob Foster

Conference: ICML 2025

YouTube: https://slideslive.com/39044042

Overview

In an era where large language models (LLMs) exhibit increasingly sophisticated and emergent behaviors, evaluating their true cognitive capabilities remains a profound challenge. This insightful talk, presented by Sunayana Rane at ICML 2025, introduces a novel and rigorous framework for LLM evaluation, drawing critical parallels from decades of research in animal cognition. Titled "Principles of Animal Cognition to Improve LLM Evaluations," the presentation highlights how the field of animal cognition has long grappled with similar issues of anthropomorphism, confounding variables, and the distinction between superficial behavior and underlying cognitive mechanisms.

The core premise of the talk is that machine learning researchers can significantly enhance the design and interpretation of LLM evaluations by adopting five key principles from animal cognition research. By doing so, the aim is to move beyond mere success or failure metrics and gain a more nuanced understanding of how LLMs solve problems, discerning whether they are genuinely performing the desired task or merely exploiting shortcuts and linguistic correlates. The presentation underscores the urgent need for robust, adversarial evaluation methodologies to prevent a "Clever Hans" effect, where models appear intelligent but are, in fact, responding to unintended cues or statistical regularities in their training data. This interdisciplinary approach promises to foster more accurate assessments of AI capabilities, guiding future research and development towards truly intelligent systems.

Background

▶ Watch: Introduction and Clever Hans analogy for LLM evaluation (0:00)

The rapid advancement of large language models has brought forth a captivating, yet complex, challenge: how do we truly understand what these powerful machines are capable of? As LLMs demonstrate increasingly rich and human-like behaviors, a fundamental question persists: are they genuinely solving the underlying problems we intend them to, or have they discovered subtle shortcuts, statistical hacks, or even developed a different form of "superhuman ability" to arrive at correct answers without true comprehension? This dilemma is strikingly reminiscent of the Clever Hans phenomenon from the late 19th century, where a horse seemingly capable of arithmetic was later found to be responding to subtle, unconscious cues from its trainer, rather than performing actual calculations.

The speakers argue that the machine learning community today faces similar evaluative hurdles that the field of animal cognition has navigated for decades. Several common challenges resonate across both domains:

1. Anthropomorphism: Humans have a natural tendency to project human-like traits and intentions onto non-human intelligence, whether it's a "grumpy cat" or an "evil robot." This bias can lead to misinterpretations of LLM behavior, attributing sophisticated cognitive abilities where none may exist.
2. Uncontrolled Prior Knowledge/Training Data: Unlike controlled laboratory settings, today's very large models are trained on vast, often opaque datasets. It's nearly impossible to control for all prior knowledge or training data an LLM has seen, making it difficult to isolate the specific ability being tested from pre-existing associations or memorized patterns. Researchers often have to "meet the model where it is," with its inherent, unmodifiable history.
3. High Cost of Evaluation: Conducting comprehensive evaluation experiments for LLMs can be financially and computationally expensive, especially when probing deeply into nuanced behaviors. This necessitates highly efficient and insightful experimental design.

These parallels underscore a critical need for more robust and discerning evaluation methodologies in AI. The goal is not just to ascertain if an LLM can produce a correct answer, but how it arrives at that answer, and whether its underlying mechanism aligns with what we consider true problem-solving or reasoning. By drawing on the mature frameworks and cautionary tales from animal cognition, the talk positions itself as a guide for ML researchers to design evaluations that can effectively disentangle genuine capabilities from mere surface-level performance.

Key Findings

▶ Watch: Unveiling five key principles from animal cognition (2:00)

The central finding of this work is the distillation of five key principles from animal cognition research that can significantly improve the design and interpretation of LLM evaluations. These principles offer a structured approach to overcome the challenges of anthropomorphism, uncontrolled training data, and the risk of misattributing capabilities to LLMs.

To demonstrate the practical utility of these principles, the authors conducted a small but impactful empirical case study on transitive inference using LLMs. Transitive inference, the ability to deduce a relationship between two items based on their relationship to a third (e.g., if A > B and B > C, then A > C), is a well-studied cognitive ability across the animal kingdom and a recognized precursor to higher-level reasoning.

The empirical results revealed critical insights:

1. Linguistic Correlates vs. Abstract Reasoning: When LLMs were prompted with natural language questions about transitive inference using ranked animal lists (e.g., "whale is bigger than elephant"), they performed best on naturally ordered lists. Performance severely interfered when lists were reverse-ranked (e.g., "ant is bigger than mouse"), and was intermediate for random strings. This pattern, notably, reflects similar performance trends observed in animal species.
2. Discrepancy with Gold Standard Tasks: The most significant finding emerged when the LLM (specifically GPT, though the exact version was not specified) was evaluated on the trial-based n-term task. This task is the gold standard for measuring transitive inference in animal cognition, designed to be non-linguistic and to force the organism to construct an internal representation of the sequence through pairwise interactions and rewards. Crucially, GPT's performance plummeted on this task.
3. Implication: This stark contrast suggests that while LLMs might appear to perform transitive inference on natural language prompts, they likely rely on linguistic correlates or statistical associations embedded in their vast training data (e.g., world knowledge about the relative sizes of animals) rather than constructing an abstract internal representation of the transitive relationship itself. In essence, they are not performing the same underlying cognitive mechanism as animals (or humans) who succeed at the trial-based task.
4. Beyond Dichotomies: The research emphasizes the importance of moving beyond simple success/failure metrics to analyze specific failure types and intermediate capabilities. Even if an LLM isn't "100% there" in terms of true reasoning, understanding its partial abilities and the conditions under which it succeeds or fails provides valuable insights for improvement.

In summary, the key findings underscore that applying animal cognition principles helps disentangle superficial behavioral consistency from genuine underlying cognitive mechanisms. This approach reveals that current LLMs may often leverage statistical shortcuts tied to linguistic patterns, rather than demonstrating robust, abstract reasoning abilities comparable to those observed in biological intelligence.

Technical Deep Dive

▶ Watch: Addressing alternative views on applying animal cognition (4:59)

The core of this position paper lies in its proposition of five principles, meticulously extracted from the field of animal cognition, to revolutionize how we evaluate LLMs. These principles are not merely guidelines but a framework for designing more rigorous, insightful, and less anthropomorphic experiments.

The Five Principles from Animal Cognition:

1. Design Control Conditions with an Adversarial Attitude: Animal cognition researchers, upon observing a seemingly intelligent behavior in an animal, proactively and preemptively seek alternative, simpler explanations for how that behavior might have arisen. This involves actively trying to disprove their own hypotheses. For LLMs, this translates to designing control conditions that specifically test for shortcuts, linguistic biases, or statistical exploits that the model might be leveraging instead of the intended cognitive process. For instance, if an LLM can "reason" about object permanence, an adversarial control might involve presenting scenarios where linguistic cues subtly contradict physical reality, to see if the model prioritizes language over a conceptual understanding.

1. Establish Robustness to Variations in Stimuli: A claim about an animal's ability (e.g., "hummingbirds can do analogical reasoning") is only robust if it holds true across a wide range of environmental and stimulus variations. Similarly, for LLMs, demonstrating an ability requires showing its consistency across diverse prompts, contexts, and data formats. If an LLM's "analogical reasoning" only works for specific linguistic templates or domains, the claim must be precisely qualified to reflect these limitations. This principle encourages comprehensive testing beyond narrow benchmarks, pushing for generalizability.

1. Move Beyond Just Success and Failure Dichotomies: Animal cognition researchers delve deeply into analyzing failure modes and failure types. Animals rarely perform human-like tasks perfectly in all scenarios. Understanding why an animal fails, or the specific circumstances under which it can perform a task, provides richer insights than a binary pass/fail. For LLMs, this means not just recording accuracy but categorizing errors, identifying patterns in incorrect responses, and understanding the boundaries of their capabilities. For example, an LLM might count sequences of concrete objects but struggle with abstract counting, much like a hummingbird might count flower visits but not abstract numbers. This nuanced analysis helps identify intermediate capabilities and areas for improvement.

1. Clarify Differences Between Mechanism and Behavior: This principle is crucial for avoiding anthropomorphism. Instead of stating "LLMs can do theory of mind," researchers should articulate: "Here is an experimental study suggesting that LLM behavior is consistent with having theory of mind." The distinction highlights that observed behavior might mimic a cognitive ability without the underlying mechanism being present. Further experiments would then be needed to probe the actual internal representation or computational process. This encourages a focus on observable data and avoids making premature claims about internal states or cognitive architectures. The speaker specifically mentioned the challenge of inferring mechanism in animals vs. the theoretical ability to "open up the back end" of an LLM, but still stressed the importance of behavioral analysis.

1. Meet the Organism Where It Is, But Notice Limitations: While maintaining an adversarial attitude, researchers must also design tasks appropriate for the intelligence system being evaluated. Evaluating a fish on its ability to climb a tree is meaningless, just as evaluating a human on underwater breathing. Tasks should be designed to reveal the organism's actual capabilities, avoiding both undue glorification and unwarranted dismissal. For LLMs, this means developing tasks that align with their input/output modalities (e.g., text-based) but are carefully constructed to probe specific cognitive functions without bias. The goal is a nuanced perspective, understanding both strengths and limitations, to facilitate improvement.

Empirical Case Study: Transitive Inference

To illustrate these principles, the talk presented an empirical study on transitive inference (TI), a fundamental reasoning ability. TI was chosen because it's extensively studied across various animal species (chimpanzees, monkeys, rats, pigeons, jays, crows, cichlid fish) and is considered a precursor to higher-level reasoning.

The study employed two main methodologies:

1. Natural Language Prompting:

• LLMs were given prompts structured like: "A is greater than B. B is greater than C. Which is greater, A or C?"
• Stimuli Variations:
• Ranked Animal Lists: Using naturally ordered animal sizes (e.g., "whale is bigger than elephant").
• Reverse-Ranked Animal Lists: Intentionally reversing natural order (e.g., "ant is bigger than mouse is bigger than duck") to see if it interfered with pre-existing knowledge or linguistic patterns.
• Random Strings: Using arbitrary labels (e.g., "X is bigger than Y, Y is bigger than Z") to minimize reliance on world knowledge.
• Operators: Both "bigger than" and "greater than" were tested.
• Results: Performance was highest for naturally ranked animal lists, significantly lower for reverse-ranked lists, and intermediate for random strings. This behavior mirrors patterns observed in various animal species, where pre-existing associations or ease of encoding can influence TI performance. The speaker noted that "linguistic correlates involved in the size of the animals" might be "propping up" the transitive inference ability.

1. Trial-Based N-Term Task (Gold Standard for Animal Cognition):

• This non-linguistic task is the gold standard for TI in animal cognition research, designed to force the animal to build an internal world model or internal representation of the sequence.
• Methodology: The model is presented with pairwise options (e.g., "house or water, which is bigger?"). It makes a choice, and receives a reward (or feedback indicating correctness). Through repeated pairwise interactions, the animal (or LLM in this case) is expected to construct an internal representation of the full sequence, which it can then use to answer novel questions about non-adjacent items.
• Purpose: This method helps disentangle the reliance on linguistic correlates, as it removes explicit verbal descriptions of relationships.
• Results: GPT's performance plummeted on this task. This stark drop strongly suggests that the LLM's success in the natural language prompts was largely due to exploiting linguistic patterns and world knowledge (e.g., "whales are generally known to be larger than elephants" from its training data) rather than forming an abstract, context-independent understanding of transitive relationships.

Alternative Views Considered:

The talk also acknowledged several alternative perspectives:

• LLMs are purely linguistic: This view suggests focusing solely on their language abilities and dismissing non-linguistic tasks inspired by animal cognition. The presented work, however, argues that even linguistic evaluations benefit from a deeper understanding of underlying mechanisms.
• LLMs are fundamentally different from biological intelligence: While acknowledging differences (e.g., the theoretical ability to "open the back end" of an LLM), the authors contend that the challenges of evaluation remain similar, making the principles transferable.
• Other evaluation methods might be more fruitful: The authors agree that no single method is exhaustive but propose animal cognition-inspired methods as a valuable, often overlooked, addition to the LLM evaluation toolkit.

By applying these principles and demonstrating their utility through the transitive inference case study, the research provides a powerful argument for a more sophisticated, interdisciplinary approach to understanding the true capabilities and limitations of large language models.

Experimental Setup & Results

▶ Watch: Empirical case study: Transitive inference introduction (5:50)

The empirical portion of the talk focused on evaluating LLMs' capability for transitive inference, a foundational reasoning skill. The experiments were designed to apply the principles of adversarial control, robustness to stimuli variation, and the distinction between mechanism and behavior.

Model Under Test:

The study utilized a GPT model. While the specific version (e.g., GPT-3, GPT-4) was not explicitly mentioned, it can be inferred to be a large, pre-trained language model, representative of the state-of-the-art LLMs prevalent in 2025.

Task: Transitive Inference

Transitive inference (TI) was chosen due to its well-established presence across a wide range of animal species (chimpanzees, monkeys, rats, pigeons, jays, crows, cichlid fish) and its recognized role as a precursor to higher-level reasoning. The task involves inferring a relationship between two non-adjacent items in a sequence (e.g., if A > B and B > C, then A > C).

Experimental Methods:

1. Natural Language Prompting:

• Prompt Structure: The LLM was given prompts in natural language, typically in the format "This is bigger than this. This is bigger than that. Which is bigger, A or C?" or "A is greater than B. B is greater than C. Which is greater, A or C?".
• Stimuli Variations (Robustness Principle):
• Ranked Animal Lists: Sequences of animals ordered by their natural size (e.g., "whale is bigger than elephant"). This leverages real-world knowledge likely present in the LLM's training data.
• Reverse-Ranked Animal Lists: Sequences of animals deliberately ordered in reverse of their natural size (e.g., "ant is bigger than mouse is bigger than duck"). This variation was designed to act as an adversarial control, probing whether the LLM relies on pre-existing semantic knowledge or a true abstract understanding of the "bigger than" relationship.
• Random Strings: Sequences using arbitrary, non-semantic labels (e.g., "X is bigger than Y, Y is bigger than Z"). This further minimizes reliance on external world knowledge, testing a more abstract form of TI.
• Operators: Both "bigger than" and "greater than" were used to test for sensitivity to specific linguistic phrasing.
• Metrics: Success/failure rates were observed, but crucially, the analysis extended to patterns of performance across different stimuli types.

1. Trial-Based N-Term Task (Mechanism vs. Behavior Principle):

• Design: This is the gold standard for measuring transitive inference in animal cognition. It is explicitly non-linguistic and designed to compel the subject to build an internal representation or "world model" of the relationships.
• Procedure: The LLM was presented with a series of pairwise choices (e.g., "house or water, which is bigger?" or "which comes first?"). After each choice, it received feedback (e.g., a "reward" for a correct answer). The goal is that through these iterative, reward-based interactions, the model constructs an internal sequence representation that it can then use to answer novel questions about non-adjacent items.
• Rationale: This task aims to disentangle whether the LLM is genuinely constructing an internal representation of the transitive sequence (the mechanism) or merely exploiting linguistic correlates and pre-existing knowledge from its training data (behavioral mimicry).

Headline Results:

• Natural Language Prompting:
• Ranked Order Animal Lists: The LLM performed best on these, indicating it could successfully infer relationships when aided by real-world knowledge.
• Reverse-Ranked Animal Lists: Performance severely interfered when the natural order was reversed. This suggests a strong reliance on pre-existing semantic associations (e.g., "whales are big") rather than a flexible, abstract application of the transitive rule.
• Random Strings: Performance was "somewhere in between" the ranked and reverse-ranked lists, indicating some capability for abstract relational reasoning but still potentially influenced by other factors or less robust than when semantic cues were aligned.
• Analogy to Animal Kingdom: The speaker highlighted that these patterns of symbolic distance and interference reflect similar findings in various animal species, suggesting a shared sensitivity to semantic congruency or ease of encoding.

• Trial-Based N-Term Task:
• GPT's performance plummeted. This was the most critical result. The dramatic drop in accuracy on a task specifically designed to strip away linguistic cues and force internal model construction strongly suggests that the LLM's apparent transitive inference ability in natural language contexts was heavily reliant on linguistic correlates and statistical patterns in its training data, rather than a robust, abstract understanding of transitive relationships. The model was likely not building the "internal representation or world model" that animals do in this task.

Ablations and Nuance:

The analysis went beyond simple success/failure. By examining why performance dipped (e.g., interference from reverse ranks, inability to form internal models in non-linguistic settings), the study provided insights into specific failure types and the conditions under which the LLM's capabilities are limited. This aligns with the principle of moving beyond dichotomies and understanding intermediate capabilities. The speaker also mentioned that results held "across different prompt variations" for the tested model, addressing potential concerns about prompt brittleness, though noting "more work is needed."

In essence, the experimental setup and results provide strong empirical evidence for the utility of animal cognition principles in identifying the limitations of current LLM reasoning, particularly in distinguishing between genuine cognitive mechanisms and sophisticated linguistic mimicry.

Practical Implications

▶ Watch: Experimental design for testing transitive inference in LLMs (6:40)

The insights gleaned from applying animal cognition principles to LLM evaluations carry significant practical implications for various stakeholders in the AI ecosystem, from model builders to deployment teams. The core message is a call for greater rigor, nuance, and a healthy skepticism in assessing what LLMs truly "understand" or "can do."

For Practitioners and Model Builders:

1. Rethinking Evaluation Design: The most direct implication is the need to fundamentally rethink how LLMs are evaluated. Simple accuracy on benchmark datasets, especially those that might inadvertently contain linguistic shortcuts or biases, is insufficient. Model builders must adopt an adversarial mindset, proactively designing control conditions and stimuli variations that attempt to disprove their hypotheses about an LLM's capabilities. This means moving beyond "does it work?" to "does it work for the right reasons?"
2. Disentangling Mechanism from Behavior: Practitioners must be acutely aware of the distinction between an LLM's behavior being consistent with an ability (e.g., reasoning, theory of mind) and the actual presence of the underlying cognitive mechanism. The transitive inference case study powerfully demonstrates that impressive linguistic performance can mask a lack of true abstract understanding. This mandates designing tasks that specifically isolate and probe the intended mechanism, often requiring non-linguistic or less linguistically-biased approaches, like the trial-based n-term task.
3. Robustness Testing is Paramount: Claims about LLM abilities should only be made if they hold robustly across diverse stimuli, contexts, and prompt variations. This means extensive testing beyond a single, optimized prompt or dataset. If an ability is brittle or context-dependent, its limitations must be precisely articulated. This directly impacts the perceived reliability and generalizability of models.
4. Nuanced Failure Analysis: Moving beyond binary success/failure metrics is crucial. Analyzing failure modes—the types of errors an LLM makes and the conditions under which it fails—provides invaluable diagnostic information. This helps model builders understand specific weaknesses, identify intermediate capabilities, and guide targeted improvements rather than just reiterating on overall performance numbers.
5. Avoiding Anthropomorphism: The "Clever Hans" effect serves as a powerful cautionary tale. AI developers must resist the natural human tendency to anthropomorphize LLM outputs. Attributing human-like intelligence or understanding based solely on convincing linguistic output can lead to overestimation of capabilities and potentially risky deployments. A disciplined, evidence-based approach is essential.

For Infrastructure Teams and Deployers:

1. Informed Deployment Decisions: Understanding the true nature of an LLM's capabilities and its limitations (e.g., whether it relies on linguistic correlates or genuine reasoning) is critical for responsible deployment. If an LLM is deployed in a high-stakes environment where genuine reasoning is required, but it primarily relies on statistical shortcuts, the consequences could be severe. This framework helps infra teams assess the safety and appropriateness of LLM applications.
2. Benchmarking Beyond NLP: Infra teams often rely on standard NLP benchmarks. This work suggests the need to incorporate more cognitively-inspired, interdisciplinary benchmarks that probe deeper into reasoning, memory, and understanding, moving beyond purely linguistic tasks.
3. Resource Allocation: A clearer understanding of LLM limitations can inform resource allocation for further training, fine-tuning, or architectural improvements aimed at addressing specific cognitive gaps rather than just scaling up existing (potentially flawed) capabilities.

Tradeoffs and Limitations:

Implementing these principles is not without its tradeoffs:

• Increased Complexity and Effort: Designing truly adversarial evaluations, conducting extensive robustness tests, and performing detailed failure analysis is significantly more complex and resource-intensive than traditional benchmark-driven evaluations. This requires more sophisticated experimental design and analytical skills.
• Defining "True" Understanding: While the framework helps distinguish behavioral mimicry from underlying mechanisms, precisely defining what constitutes "true understanding" or "genuine reasoning" in an artificial system remains a philosophical and scientific challenge.
• Generalizability of Animal Cognition Tasks: While insightful, not all animal cognition tasks will directly map to LLM evaluations. Careful adaptation and consideration of the LLM's input/output modalities are necessary.
• Small Empirical Study: The transitive inference case study, while illustrative, was described as "small empirical." More extensive and diverse empirical studies across various cognitive tasks and LLM architectures are needed to fully validate the broad applicability of these principles.
• Difficulty with Uncontrolled Training Data: Even with adversarial evaluation, the fundamental challenge of not being able to control for all prior knowledge or training data remains. The principles help diagnose its impact but don't eliminate the underlying issue.

In conclusion, adopting principles from animal cognition offers a powerful toolkit for moving beyond superficial assessments of LLM intelligence. By fostering a more critical, analytical, and less anthropomorphic approach to evaluation, these principles can guide the development of more robust, transparent, and genuinely capable AI systems.

Key Takeaways

• LLM evaluation faces challenges akin to animal cognition research: The "Clever Hans" effect, anthropomorphism, and the difficulty in controlling for prior knowledge necessitate more rigorous evaluation methodologies for large language models.
• Five principles from animal cognition enhance LLM evaluation rigor: These include designing adversarial control conditions, establishing robustness to stimuli variations, moving beyond success/failure dichotomies, clarifying differences between mechanism and behavior, and meeting the model where it is while noting limitations.
• LLMs may exploit linguistic shortcuts rather than demonstrate genuine abstract reasoning: The transitive inference case study revealed that while LLMs performed well on natural language prompts, their performance plummeted on the non-linguistic trial-based n-term task, suggesting a reliance on linguistic correlates and world knowledge rather than abstract internal representations.
• The trial-based n-term task is a critical diagnostic tool: As a gold standard in animal cognition, this task effectively strips away linguistic biases and forces the system to construct an internal world model, thus highlighting the gap between apparent linguistic competence and true underlying cognitive mechanisms in LLMs.
• Nuanced failure analysis is crucial for understanding capabilities and limitations: Moving beyond binary success/failure to analyze specific failure types and conditions provides richer insights into what LLMs can and cannot do, and under what circumstances, facilitating targeted improvements.
• Adopting these principles leads to more accurate, less anthropomorphic assessments: By fostering a skeptical, analytical, and interdisciplinary approach, researchers can gain a deeper, more truthful understanding of LLM intelligence, guiding better model development, alignment, and responsible deployment.

About the Speaker(s)

The talk "Principles of Animal Cognition to Improve LLM Evaluations" was presented by Sunayana Rane. She is part of an interdisciplinary team that authored this position paper, which also includes Cyrus Kirkman, Graham Todd, Amanda Royka, Ryan Law, Erica Cartmill, and Jacob Foster. The presentation highlighted their collaborative effort to bridge the fields of machine learning and animal cognition.