Redirecting to original paper in 30 seconds...
Click below to go immediately or wait for automatic redirect
📄 Abstract
Abstract: Language is fundamental to human cooperation, facilitating not only the
exchange of information but also the coordination of actions through shared
interpretations of situational contexts. This study explores whether the
Generative Agent-Based Model (GABM) Concordia can effectively model Theory of
Mind (ToM) within simulated real-world environments. Specifically, we assess
whether this framework successfully simulates ToM abilities and whether GPT-4
can perform tasks by making genuine inferences from social context, rather than
relying on linguistic memorization. Our findings reveal a critical limitation:
GPT-4 frequently fails to select actions based on belief attribution,
suggesting that apparent ToM-like abilities observed in previous studies may
stem from shallow statistical associations rather than true reasoning.
Additionally, the model struggles to generate coherent causal effects from
agent actions, exposing difficulties in processing complex social interactions.
These results challenge current statements about emergent ToM-like capabilities
in LLMs and highlight the need for more rigorous, action-based evaluation
frameworks.
Key Contributions
This study critically evaluates the Theory of Mind (ToM) simulation capabilities of LLMs using the Concordia GABM. It reveals that GPT-4 often fails to make genuine inferences based on belief attribution, suggesting apparent ToM-like abilities may stem from shallow associations rather than true reasoning, and highlights difficulties in processing complex social interactions and generating causal effects.
Business Value
Provides crucial insights into the limitations of current LLMs in understanding and simulating human social cognition, guiding the development of safer and more sophisticated AI systems for human interaction.