SALA Improves LLM In-Context Learning for Complex Reasoning

Effective in-context learning (ICL) for complex reasoning tasks in large language models (LLMs) heavily depends on selecting the most appropriate demonstrations. Conventional retrieval methods, which often rely on surface-level similarity, frequently fail to capture the underlying problem-solving logic. While more recent logic-based approaches attempt to match predefined reasoning steps, their rigid rules and exact-match criteria struggle with the flexible and diverse nature of real-world reasoning processes. To overcome these limitations, researchers propose SALA, a Semantic-Aware Logical Alignment framework. Instead of using a fixed set of reasoning operations, SALA automatically identifies and learns task-specific reasoning operations. It then embeds these operations into a continuous semantic space. The framework employs dynamic time warping (DTW) to align the reasoning sequences, allowing for a soft and flexible matching of reasoning logic while maintaining high interpretability. Experiments conducted across four reasoning benchmarks and with three different LLMs demonstrate that SALA consistently outperforms existing demonstration selection methods. Further analysis confirms the crucial roles played by both the operation induction and the logical semantic alignment components within the SALA framework.