Persistent Discovery Context Boosts Data-Centric Agent Retrieval.

Data-centric AI agents frequently perform a "discovery" step to identify relevant data objects for a given task, but the successful outcomes of this process are typically discarded. This research proposes a novel concept called "persistent discovery context," which acts as a lightweight memory layer. This layer stores previously successful mappings between user intents and the data objects found to be relevant, then reuses these stored memories to augment and improve subsequent data retrieval operations.The effectiveness of persistent discovery context was evaluated across three distinct structured data environments. The results consistently showed an improvement in retrieval quality compared to methods relying solely on metadata search. The memory layer proved effective even when the memories themselves were automatically generated, highlighting its robustness.A significant finding was that in domains where lexical information is sparse, retrieval based purely on these stored memories could even surpass the performance of metadata-based retrieval. This suggests that the accumulated knowledge of past successful discoveries forms a valuable, reusable form of context, offering a way to enhance agent performance beyond the limitations of traditional context windows.