Persistent Homology Reveals LLM Internal Representation Dynamics During Alignment.

Large Language Models (LLMs) are typically aligned through supervised fine-tuning to achieve desired behaviors. However, the internal changes within these models during this alignment process are not well understood. This research investigates how the internal representations of LLMs evolve by applying persistent homology, a topological data analysis technique, to track the topology of their activation spaces throughout fine-tuning. The study analyzed four transformer language models, ranging from 1 billion to 7 billion parameters, and three different alignment objectives (helpful, harmless, and mixed training data). A key finding was that the majority of the topological reorganization within the models' representations occurs during the initial stages of training. A detailed analysis of checkpoints revealed a transient peak in topological activity, followed by a rapid stabilization of the internal structure. Furthermore, the research demonstrated that different alignment objectives lead to distinguishable topological trajectories, and instruction-tuned models exhibit qualitatively different evolutionary patterns compared to their pretrained counterparts. These results suggest that persistent homology provides a valuable, complementary perspective on LLM alignment, uncovering representation-level changes that are not evident from behavioral metrics alone.