LLMs Optimize Neural Network Channel Widths in Closed-Loop Search

This research scales up a previous study on using large language models (LLMs) for closed-loop optimization of neural network channel configurations. The process involves LLMs generating executable code for network architectures, which are then evaluated for accuracy, with feedback used to refine subsequent generations. The expanded study evaluated 250 candidate networks per fine-tuning cycle, totaling 2000 generated candidates and 462 verified CIFAR-100 evaluations. The scaled experiment confirmed and strengthened initial findings: mean accuracy showed a positive linear trend, and the performance of the highest-performing models improved significantly. The best model achieved an accuracy of 0.3676 with only 11.8 million parameters, a substantial improvement in parameter efficiency compared to earlier high-performing models that required 166.5 million parameters for lower accuracy. Furthermore, the larger sample size allowed for the discovery of architectural regularities previously unobservable. It was found that non-power-of-two channel widths are common, appearing in over 40% of verified candidates. The strongest models exhibited structured channel allocation patterns, typically characterized by moderate widths in early layers followed by expansion in middle or later blocks. These results underscore the effectiveness of LLM-driven closed-loop search for neural architecture optimization.