EvoClawBench Evaluates Agent Skill Learning from Self-Runs

Current benchmarks for AI agents primarily focus on task completion, tool utilization, or the general utility of skills. However, they often fail to isolate a crucial question: can an agent runtime effectively convert evidence from its own past executions into reusable skills that genuinely enhance subsequent, fresh runs, beyond the initial authoring effort? To address this, EvoClawBench was introduced as a new benchmark specifically for this "closed-loop skill-learning" problem. It features repeated, fixture-backed tasks and compares three modes: direct execution without skills, "PreSkill" authoring before execution, and "PostSkill" summarization from a first run followed by a second execution. The benchmark comprises 100 tasks and 502 sub-problems spanning coding, data, office, security, operations, and domain-document workflows, supporting multiple agent runtimes. Initial experiments with OpenClaw and nanobot agents under local execution revealed that baseline performance is highly dependent on the runtime, with OpenClaw performing below 20% and nanobot ranging from 56.45% to 96.13%. The impact of self-authored skills was mixed. While nanobot GPT-5.4 maintained high performance across all modes and nanobot MiniMax-M2.7 improved with PostSkill, other configurations like nanobot DeepSeek-V4-Pro saw significant performance drops with skill authoring. OpenClaw also exhibited non-monotonic behavior. These findings suggest that the ability for agents to learn reusable skills from their own runs is not a guaranteed benefit but rather a selective and cost-sensitive process.