SPEAR Simulator Boosts Photorealistic Embodied AI Research

Researchers have unveiled SPEAR, a novel simulator designed to advance photorealistic embodied AI research. This Python library establishes a deep connection with Unreal Engine (UE) applications, enabling programmatic control over a vast array of UE functions—over 14,000, which is a significant increase compared to existing simulators. SPEAR also boasts superior rendering performance, capable of generating 1920x1080 photorealistic images at 73 frames per second, an order of magnitude faster than current UE plugins. Beyond speed, it offers unique ground truth image modalities, such as non-diffuse intrinsic image decomposition and material IDs, which are crucial for detailed agent training. The simulator introduces an expressive high-level programming model, allowing users to define complex UE workflows with precise data dependencies and deterministic execution within a single frame. Its utility is demonstrated across diverse applications, including controlling multiple agent types (humans, cars, robots) in various UE projects, rendering city-scale environments, manipulating procedural content, and even co-simulating with physics engines like MuJoCo.