FUSE Framework Evaluates LLM Dangerous Capabilities and Safety.

The fragmented nature of current safety evaluations for large language models (LLMs) hinders effective governance of their potentially dangerous capabilities. To address this, the FUSE framework offers a unified, modular approach to assess LLMs. It evaluates each model across three distinct, yet orthogonal, pipelines: Knowledge (K), which measures what the model knows about harmful topics; Defense (D), which assesses its refusal resilience to harmful requests; and Harm (H), which quantifies the harmfulness of its generated content when it does comply. The results are aggregated into a standardized dangerous-capability profile.The framework's modular design allows for pluggable components like scenario seeds and judge rubrics, while the core evaluation engine remains consistent. Instantiated with a chemical-biological (CB) module, FUSE was used to evaluate 12 commercial LLMs. This revealed significant differences in their safety profiles; models with similar knowledge levels could vary widely in their refusal rates, and strong defenders didn't necessarily produce less harmful content when they did comply. Family-level patterns also emerged, distinguishing models like Claude, DeepSeek, and GPT.Furthermore, a temporal analysis tracking K, D, and H against model release dates showed that dangerous capabilities have not consistently decreased over time. Newer models often exhibit deeper knowledge while only partially improving their defense mechanisms, indicating that advancements in scaling and alignment do not uniformly translate into enhanced safety across all dimensions. The framework's reliability was confirmed through high cross-judge consistency and pipeline orthogonality.