Empowering Small Language Models With Chain of Thought and Parameter Efficient Fine Tuning for Efficient Deep Reasoning

Abstract

Small Language Models (SLMs) run faster and fit on modest hardware, yet solving multi-step logic problems has traditionally been difficult for them. This work investigates a systematic, multi-model framework that combines Chain-of-Thought (CoT) prompting with Low-Rank Adaptation (LoRA) parameter-efficient fine-tuning, and measures the contribution of each component independently. We evaluate three SLMs (TinyLlama-1.1B, Phi-3-mini-4k-instruct, and Qwen2.5-1.5B) using 1,164 training samples drawn from GSM8K, Microsoft Orca-Math-Word-Problems-200k, and OpenAI HumanEval, and evaluate on 500 GSM8K test problems. Phi-3-mini-4k-instruct achieves the strongest overall gains, reaching 16.0% exact match and 61.7% step accuracy under +CoT+LoRA, a 95% improvement over its own baseline. The decline observed under CoT-only prompting and the subsequent recovery under CoT+LoRA were statistically significant (McNemar test, p<0.05). A notable finding is that CoT prompting applied without fine-tuning degrades Phi-3-mini’s exact match score by approximately 44% due to an instruction-format conflict that LoRA fine-tuning resolves. Our ablation shows that neither technique alone produces peak performance; their combination does. The framework trains fewer than 0.05% of model parameters and runs end-to-end on a single consumer-grade NVIDIA RTX 4500 Ada GPU with 24 GB VRAM, making capable SLM-based reasoning deployable without datacenter-scale infrastructure.