Survey on In-Context RL in Non-Stationary Environments

Recent advancements in areas like decision-pretrained transformers and meta-RL have reignited interest in In-Context Reinforcement Learning (ICRL). ICRL refers to an agent's ability to infer task rules and improve its behavior using only interaction context, without requiring explicit parameter updates during deployment. While existing ICRL surveys focus on aspects like architectures and pretraining, the critical challenge of non-stationary environments has been less explored. This paper provides a comprehensive survey specifically on ICRL in non-stationary settings. In such dynamic environments, accumulated context can become outdated or misleading as reward structures, transition dynamics, or observation channels shift. The survey defines non-stationary ICRL as the problem where a fixed-parameter policy must infer both the current decision rule and which parts of its historical context remain relevant. It systematically categorizes the literature by addressing what environmental aspects change, how these changes evolve, and the extent to which an agent can observe them.