Multilayer Fusion Improves LLM Contextual Value Alignment

Aligning large language models (LLMs) with human values is a significant hurdle for trustworthy AI, as current methods often rely on single-agent frameworks and unified reward systems. This limits their ability to capture the complexities of ethical pluralism and adapt to varied moral contexts, especially in multi-agent reasoning scenarios. To address these limitations, a new framework called Multilayer Combinatorial Fusion for Contextual Value Alignment (MCF-CVA) has been proposed. This framework operates by instantiating multiple "moral agents," each fine-tuned to embody a specific value. Their outputs are then expanded combinatorially using both score- and rank-based aggregations. This "expansion and reduction" (EAR) process is iterated across multiple layers, leveraging the cognitive diversity among agents to mitigate conflicts and redundancies. Empirical evaluations demonstrate that MCF-CVA significantly outperforms single-agent baselines, single-layer multi-agent results, and previous aggregation techniques, proving to be a robust mechanism for enhancing contextual value alignment in LLMs.