CodeSIFT Detects Malicious Code-Generation Prompts Using Influence Functions

Large language models (LLMs) are increasingly used for generating code, but they are susceptible to malicious prompts that can lead to insecure or vulnerable implementations. Current defense mechanisms often rely on predefined threat models or known vulnerability signatures, which limits their effectiveness against novel or evolving attack strategies. Researchers have developed CodeSIFT, a new detection method that is agnostic to specific threat models. CodeSIFT leverages influence functions to identify batches of prompts that cause an LLM to exhibit anomalous behavior. Instead of looking for particular vulnerabilities, the system measures how the generated code influences the model's parameter space and then uses a statistical test to determine if a given set of prompts deviates significantly from a benign reference distribution. To evaluate CodeSIFT, two new benchmark datasets were created, encompassing a variety of vulnerabilities. The method was tested on three open-weight code LLMs (ranging from 3B to 7B parameters), achieving AUROC scores up to 0.98 at moderate-to-high injection rates. Crucially, CodeSIFT maintained well-calibrated false positive rates and substantially outperformed traditional static analysis baselines, suggesting that influence-function-based detection is a promising direction for identifying malicious code-generation prompts without requiring prior knowledge of the attack class.