LLMs Enhance Public Sector Service Feedback Analysis for Emerging Topics.

Public sector organizations, particularly tax administrations, face increasing challenges in analyzing vast volumes of service feedback to identify emerging issues and potential disparities. Traditional methods often struggle with scalability and capturing complex patterns. This research proposes a new framework that combines fine-tuned, quantized large language models (LLMs) with statistical methods and human oversight. The methodology focuses on detecting new service quality topics and potential inequities in service delivery across diverse populations. By incorporating a human-in-the-loop approach, the system aims to reduce LLM "fabrication" and enhance the reliability of insights. Evaluations showed that this hybrid approach aligns more closely with expert judgments compared to existing models. The findings suggest a practical way to leverage LLMs and human expertise for scalable, evidence-based decision-making, ultimately improving service quality, fairness, and public trust.