LLM Agents Blindly Defer to GNN Tools, Stronger Models Defer More

A new study reveals a critical limitation in how large language model (LLM) agents interact with external tools, specifically graph neural networks (GNNs). The research found that LLM agents, when given a GNN as a callable tool, do not exercise independent judgment but instead largely adopt the GNN's predictions, agreeing with them 97.6-99.2% of the time. This behavior effectively turns the agent into a "GNN parrot," overriding its own reasoning capabilities. Surprisingly, this deference is not a weakness of smaller models; rather, it increases with the LLM's capability, with stronger models deferring more. The cost of this blind deference does not diminish with increased LLM capability, and in scenarios where the agent's internal reasoning or simpler alternative tools could outperform the GNN, the agent still defers. For instance, a simple neighbor-label tool sometimes surpassed the GNN, yet the agent continued to rely on the GNN. The findings suggest that current evaluations of agent-plus-tool systems might be overestimating the agent's judgment. Selective invocation mechanisms need to be explicitly designed and integrated, as they are unlikely to emerge naturally from model scale. The study serves as a cautionary note for developers building tool-augmented LLM agents.