Activity Frames: New Agent Memory System for User Activity.

Current AI agents struggle to recall user actions because their memory typically records what was said, not what was done. This research proposes "Activity Frames," a novel system designed to address this gap by compiling screen activity into a structured, agent-readable memory format. The system operates deterministically, meaning it uses no AI models in the compilation loop, ensuring byte-identical, cacheable, and auditable output. The Activity Frames compiler segments screen captures into "activity frames," which are bounded episodes containing details like application, timing, and input volume. This process dramatically reduces the raw data size; for instance, a day's capture can be condensed 86-fold into a prompt-ready block in milliseconds. Agents using this compiled memory achieve high accuracy (98.4%) in answering questions about user activity, outperforming traditional LLM summaries. The system also provides new metrics for agent cost modeling, such as Routine Overhead Ratio and routine recurrence, offering insights into the efficiency of delegable tasks.