Alignment and Filtering Cannot Fully Eliminate LLM Harm

The widespread deployment of large language models (LLMs) necessitates robust methods to prevent harmful outputs. Current strategies often involve alignment schemes that reshape a model's output distribution and external safety filters. However, it remains an open question whether these practical pipelines can truly eliminate all harmful behavior or merely suppress its most visible forms. This research formalizes the problem by analyzing support-preserving alignment operators combined with bounded filtering algorithms, considering black-box, white-box, and statistical-query access to LLMs. Through computational and information-theoretic arguments, the study indicates that under these constraints, bounded filtering may not be able to remove all harmful outputs that are inherently supported by the base model's distribution. This suggests a fundamental limitation in achieving zero-harm. Empirical evaluations across various state-of-the-art open-weight and hosted LLMs, using adversarial prompts from cybersecurity scenarios and safety benchmarks, support these theoretical findings. While additional filtering compute reduces the estimated harmful-output rate, it consistently plateaus above zero. This persistent "empirical harm floor" implies that current alignment and filtering techniques, while effective at reduction, cannot entirely eliminate the potential for harmful generations.