BiasTrace Links LLM Reasoning Behaviors to Biased Outputs

Large Language Models (LLMs) frequently exhibit social biases, leading to inaccurate or discriminatory inferences, which poses significant risks, especially in critical applications. While previous efforts have focused on measuring and mitigating bias in final model outputs, there's been a limited understanding of the underlying mechanisms that produce these biased outcomes. Recent advancements in LLM reasoning capabilities offer a new avenue for investigating bias, but the direct link between reasoning processes and biased results remains underexplored. Existing methods often concentrate on the correctness of final answers or the presence of explicitly biased language, potentially overlooking various reasoning behaviors that implicitly contribute to bias. To address this, a new annotation scheme called BiasTrace has been developed. BiasTrace allows for labeling specific reasoning behaviors within model-generated traces and directly linking them to biased outputs. This scheme captures both bias-specific behaviors, such as unsupported demographic assumptions, and general reasoning patterns like "overthinking" that might indirectly foster bias. By applying BiasTrace to reasoning traces in bias-sensitive contexts and scaling the annotations using LLM-as-a-judge methods, a substantial dataset was created. Analysis of this dataset revealed that biased outputs frequently originate from subtle reasoning behaviors rather than overt biased language. Furthermore, reasoning-level annotations significantly enhance bias detection and can be leveraged for effective inference-time mitigation strategies. These findings emphasize the importance of examining a broader spectrum of reasoning patterns to gain a deeper understanding of bias in LLMs.