Prompt Echoing Boosts Vision-Language Model Performance

Vision-language models (VLMs) often struggle when a question is presented before an image, a counter-intuitive problem known as the "question-first paradox." While placing the question first helps the model focus its attention on relevant parts of the image, the question itself can get lost among the numerous image tokens by the time the model needs to generate an answer. This leads to the answer token prioritizing image-driven responses over question-driven ones. Researchers have identified that the issue stems from a conflict between the VLM's initial perception steering and its later ability to access the question for answer generation. To address this, a training-free solution called "question echoing" has been developed. This technique involves restating the question both before and after the image within the prompt. This dual placement ensures that one instance of the question guides the model's visual attention, while the other is readily available for the answer generation phase. This approach significantly improves VLM performance across various benchmarks, closing the performance gap and even surpassing traditional prompting methods without requiring any model retraining or architectural changes.