PromptPack Scales LLM Annotation for Online Recommendations.

This paper introduces PromptPack, a novel and scalable LLM annotation agent designed to address the high costs associated with using Large Language Models (LLMs) for extracting structured features from ad creatives in online recommendation systems. While single-call LLM annotation agents have shown significant improvements in Click-Through Rate (CTR) in production, the per-creative prompting cost is prohibitive for large-scale deployment, with redundant system instructions accounting for 94% of billed input tokens. PromptPack overcomes this cost bottleneck through an innovative in-context batching approach. It combines a shared system prompt, a strict XML structural envelope for output, and an output correction layer to ensure deterministic and pipeline-ready feature extraction across multiple creatives simultaneously. Offline retrieval benchmarks, using a downstream logistic-regression ranker, evaluated PromptPack. The results demonstrate that at a batch size of 20, PromptPack reduces LLM costs by 89% and accelerates throughput by 2.5x, all while fully preserving the Area Under the Curve (AUC) performance of the generated features. The research also introduces Volume-Weighted Absolute Lift (VWAL) as a new metric to capture the signal quality of the generated features.