Synthetic Data Boosts Small LLM Arithmetic Reasoning on Consumer Hardware

Small language models (SLMs) are appealing for local deployment due to their lower resource requirements, but they often struggle with complex multi-step arithmetic reasoning. This study investigates whether fine-tuning SLMs with specially designed structured synthetic reasoning data can enhance their arithmetic performance, even on consumer-grade hardware. Researchers generated a corpus of 21,250 grade-school arithmetic word problem variants using GPT-5-mini, incorporating natural-language solution traces, Socratic-style cues, structural variations, and irrelevant distractor context. This synthetic data was then used to fine-tune Qwen3-0.6B and Qwen3-1.7B models via LoRA on an Apple M4 with 16 GB RAM. The fine-tuned Qwen3-0.6B saw its exact-match accuracy on GSM8K improve from 36.5% to 49.1%, while Qwen3-1.7B improved from 53.5% to 66.5%. The Qwen3-1.7B model also showed strong transfer to other benchmarks, reaching 98.9% on MultiArith and 73.0% on SVAMP. Qualitative analysis suggests these gains stem from shorter reasoning traces, fewer errors, and better distractor handling, indicating the effectiveness of structured synthetic data design.