AI Scientist Avoids Drift in Robot Research with Falsifiable Hypotheses

Autonomous research loops, often powered by large language models, can conduct machine learning experiments at scale but frequently suffer from "drift," where they tend to optimize local metrics rather than rigorously testing core hypotheses. This research presents an "AI Scientist" designed to overcome this issue, specifically for studying generalization in quadruped robot navigation policies within a simulated environment. The proposed framework builds on existing autoresearch paradigms but adds three crucial components. First, an immutable experiment card ensures that each iteration's prediction and outcome are recorded in a fixed schema, preventing "retconning" of falsified hypotheses. Second, specialized subagents are restricted to mechanical roles, streamlining the process. Third, a preference oracle, named kkanbu, encapsulates the user's research taste as a typed knowledge graph and is the sole component allowed to make subjective judgments, guiding the research direction. The system was tested by running identical loops with and without the kkanbu oracle across eleven research streams. Both arms successfully avoided drift, falsifying approximately three-quarters of their hypotheses. While the oracle-less arm produced the best-trained policy in terms of score, the oracle significantly influenced the research direction, exploring test-time adaptation and carrying lessons across streams that the other arm repeatedly rediscovered. This demonstrates that a structured scaffold keeps the research honest, while the oracle provides strategic guidance.