Parameter-Efficient Fine-Tuning of Open-Weights Language Models for Low-Resource Educational Contexts

Large Language Models (LLMs) have shown impressive skills in general problem-solving. However, these models are mostly tested on Western datasets, leaving a major gap when it comes to low-resource educational systems. Recent research by IrokoBench tested 16 LLMs across 17 African languages and found that highly capable open-weights models like Gemma 2 (27B) achieve only about 63% of the performance of GPT-4o. To explore how we can close this gap efficiently without relying on expensive proprietary systems, this paper presents a clear comparison of three open-weights models: Llama 3 (8B), Qwen 2.5 (7B), and Gemma 2 (9B-Instruct). We evaluate them specifically on the West African Senior School Certificate Examination (WASSCE) to represent a localized, low-resource educational context. We chose to focus strictly on Mathematics and Physics because these core STEM subjects require strict, multi-step problem solving and leave no room for grading ambiguity, making them perfect for testing true reasoning capabilities. We compare their zero-shot baseline performance against versions fine-tuned with Quantized Low-Rank Adaptation (QLoRA) using a dataset of 1,887 training questions and 472 test questions. Furthermore, we show how this training can be done on a single 15GB T4 GPU by working around memory limits, utilizing dynamic optimizers, and fixing hardware datatype conflicts. Our results show that Qwen 2.5 has the strongest baseline (54.24%), while Gemma 2 shows the best growth after fine-tuning (jumping from 48.52% to 52.97%). This study provides a rigorous, mathematical guide for building affordable and localized AI tutors for students in data-scarce environments.