MO-DiT+HPPO Enhances Generative Retrieval for Pattern-Preserving Attributes

A new research paper presents MO-DiT+HPPO, a sophisticated framework designed to improve generative retrieval, particularly for "pattern-preserving attribute retrieval." This challenging task requires finding items that not only possess a specific target attribute but also adhere to a fine-grained pattern expressed by a seed set of items. Traditional embedding-based retrieval often struggles with this dual objective, as preserving the pattern can lead to low-attribute regions, while global attribute retrieval might drift to unrelated patterns. MO-DiT+HPPO tackles this by using a continuous generative retrieval approach, where a Diffusion Transformer model learns to generate query embeddings for nearest-neighbor search. The framework employs a staged training process: initial raw-sequence pretraining, followed by multi-domain metric-ordered continuation pretraining. This metric-ordered training converts sparse online retrieval labels into ordered trajectories, teaching the model to improve metrics across domains. The final stage involves "Hybrid-Policy Preference Optimization (HPPO)," which aligns the generated query distribution with the true online objective. HPPO uses a hybrid candidate pool labeled with an online intersection metric and applies reference-anchored preference optimization. A Pareto pair filter ensures that only winner pairs that do not compromise same-pattern purity are kept, thereby raising the attribute metric without sacrificing the pattern. Experiments across four attribute domains show significant gains in the intersection metric, demonstrating the effectiveness of metric-ordered DiT and HPPO.