AI Credit Models Show Income Bias in Default Prediction

This paper investigates fairness issues in automated credit default prediction systems, specifically focusing on how data-centric curation pipelines can inadvertently introduce or amplify bias. Researchers found that when models use confidence scores to filter out "noisy" training data, high-income individuals who default are much more likely to be incorrectly flagged as noise compared to low-income defaulters. This creates a significant disparity in the model's ability to correctly identify defaulters across different income groups. The study further dissects the sources of this bias. It identifies three main contributors: the model's direct use of self-reported income, the algorithmic absorption of existing institutional biases embedded in loan origination interest rates, and a persistent residual bias. This residual bias, which remains even after removing explicit income and interest rate data, is attributed to structural proxy variables like loan amount and home ownership status, which implicitly reconstruct the sensitive income signals. The findings highlight that simply removing sensitive attributes like income from a model is insufficient to ensure fairness. Implicit biases can persist through other correlated features and institutional practices. The research emphasizes the need for more rigorous auditing of AI workflows in regulated financial institutions.