New Framework Improves Multi-Agent LLM Reliability with Semantic Uncertainty

Researchers have developed HASSUM, a novel framework for orchestrating hierarchical multi-agent systems powered by large language models. The core innovation lies in its ability to manage uncertainty by evaluating the semantic reliability of intermediate reasoning steps, rather than relying on fixed interaction patterns or simple output probabilities. This approach aims to prevent the propagation of errors and hallucinations throughout the system. HASSUM employs semantic entropy and semantic density as key metrics to gauge the trustworthiness of an agent's output at a semantic level. These signals then inform adaptive orchestration decisions, such as verifying outputs, selectively reprompting agents, initiating further deliberation, or choosing responses based on confidence levels. The framework's independence from specific agent architectures makes it broadly applicable across various multi-agent system designs, demonstrating improved robustness and trustworthiness in complex reasoning tasks.