Training AI Agents to Reveal State Through Observable Actions

Autonomous agents often operate under communication constraints, making it difficult to monitor their internal state or coordinate them effectively in multi-agent systems. While explicit communication might be absent, the actions an agent takes can serve as a valuable, albeit indirect, source of information about its state. This study investigates a novel approach: training control policies using reinforcement learning to make an agent's actions inherently more "observable." The objective is to develop policies that, while performing their primary task, also generate actions that simplify the problem of reconstructing the agent's relevant state through estimation. The researchers encourage policy observability by incorporating it into the training reward function. In an aircraft tracking simulation, they successfully developed a policy that significantly enhanced observability with minimal impact on the agent's core task performance. This suggests a promising avenue for improving monitoring and coordination in environments with limited communication.