LLM Embeddings Boost Program Analysis and Optimization

This research explores using Large Language Models (LLMs) to create embeddings for program code, aiming to enhance analysis and optimization. The team utilized LLMCompiler, an LLM specifically trained on intermediate representation (IR) code, to generate these embeddings. Their method involves breaking down programs into smaller segments, embedding each segment independently using pretrained LLMs, and then combining these into a single program embedding. Experiments demonstrated notable improvements. By integrating both source and IR code embeddings, the system achieved a 1.54% error rate in algorithm classification, marking a 12% improvement over existing state-of-the-art methods. It also showed promising accuracy in mapping programs to different hardware devices. These findings suggest a significant potential for developing performance-aware LLMs specifically for embedding IR code. Such models could lead to new benchmarks in code optimization and analysis, offering more efficient ways to understand and improve software performance.