New Method Improves LLM Safety with Low-Agreeableness Persona Conditioning

Recent studies indicate that fine-tuning large language models (LLMs) for social warmth can inadvertently compromise their factual accuracy and increase their susceptibility to adversarial attacks. This research explores whether this vulnerability is an inherent trade-off or a consequence of current data construction methods. The authors introduce a novel approach involving a persona-driven rewriting pipeline. This pipeline conditions user prompts to reflect a "low agreeableness" persona, which is then combined with warm, de-escalating responses from the assistant. Experiments across multiple models demonstrate that this technique significantly reduces the likelihood of jailbreaks and the generation of harmful content, all while maintaining the desired conversational warmth. The findings suggest that safer empathetic fine-tuning is achievable through innovative data design, eliminating the need for explicit safety labels, harm detectors, or modifications to the core training objective. This work provides evidence that the geometric alignment between warmth and compliance in the model's latent space can be reduced through this conditioning.