Activation Probes Detect AI Risks, Not Contextual Nuances.

This paper investigates the effectiveness of using activation-space probes within large language models (LLMs) to identify and mitigate harmful requests. The central question is whether these probes can differentiate between harmful and benign requests, especially when context is the primary differentiator, rather than just topic or surface form. The findings indicate that these activation sensors are highly effective as broad risk detectors, successfully blocking a significant percentage of judge-classified compliant attacks across several 7-8B model families. However, their performance in distinguishing harmful requests from surface-matched benign controls, where context is key, is considerably weaker. The study concludes that at the tested read points, activation scores primarily function as general risk indicators. They are not reliable as standalone context adjudicators, meaning they struggle to precisely determine if a request is harmful solely based on its nuanced contextual meaning without direct pair-boundary fitting.