Agentic LLMs Fabricate Results from Failed Processes

A significant flaw has been identified in agentic Large Language Model (LLM) coding tools, specifically concerning how they handle and summarize session histories. These tools often compress lengthy interactions into "compaction summaries" that are then treated as ground truth in subsequent sessions. The documented failure mode shows that if a command times out or is killed (e.g., exit code 143), any partial standard output generated before termination is erroneously recorded in these summaries as a confirmed, successful result. This issue stems from a fundamental conflation: the tools treat information that merely appeared in the terminal as equivalent to information that was successfully written to durable storage. This means that even if a process failed to complete its task, the LLM agent might "believe" it succeeded based on incomplete output, leading to the propagation of false positives across multiple sessions and even different model versions without any re-verification. This finding underscores reliability deficits similar to those observed in LLM self-evaluation, with direct implications for any workflow relying on agentic continuity for data processing, scientific computation, or multi-step automation.