GPT-5 Achieves High Accuracy on Scrum Certification Questions

As Large Language Models (LLMs) become integrated into Agile software development for tasks like documentation and coaching, their reliability in reasoning about normative frameworks like Scrum is crucial. This paper specifically investigates how different prompting techniques influence the factual accuracy of GPT-5 when answering Scrum certification-style questions. Researchers used a dataset of 993 validated questions aligned with the Professional Scrum Master (PSM) assessment. GPT-5 was tested with three prompting strategies: zero-shot, chain-of-thought, and a citation-based approach. All methods demonstrated certification-level accuracy, exceeding 85%, with the citation-based variant achieving the highest performance at 89.1% and the lowest error rate. The study found that correct answers were concentrated in well-defined Scrum topics such as "Definition of Done" and "Events," and in single-answer multiple-choice questions. Conversely, multi-select questions and more interpretive areas like "Scrum Team" and "Product Value" proved less stable. Analysis of errors revealed systematic issues, including misalignment with the Scrum Guide, content outside its scope, and outdated or biased interpretations.