Tactile Gives AI Agents Reliable Desktop Interaction.

While computer-using AI agents are becoming more capable, their interaction with desktop applications often relies on a fragile "motor layer" that predicts screen coordinates for clicks. This approach combines target identification, action execution, and outcome verification into a single, ambiguous operation. A new open-source tool layer called Tactile aims to provide agents with more reliable "hands and feet" for desktop use. Tactile processes various UI evidence—including operating system accessibility semantics, OCR-grounded text, and visual fallback regions—to create action-grounded interface states. These states offer compact target candidates with clear labels, roles, geometry, executable affordances, and verification cues. Agents using Tactile operate through an observe-ground-act-verify loop, prioritizing native semantic actions and falling back to OCR-grounded coordinates when necessary, all while maintaining full provenance for debugging. This approach significantly boosts agent success rates on macOSWorld-style tasks, improving performance from 41.1% to 50.0% overall, demonstrating that robust computer use requires semantic action substrates, not just stronger models.