A Framework for Synthesizing Arithmetical Puzzle Datasets for Large Language Models

This article introduces a novel arithmetical puzzle dataset designed to test and enhance AI reasoning capabilities. The puzzles involve manipulating integers through arithmetic operations to reach a target, with each number used exactly once. A data synthesis pipeline generates large-scale datasets, with controlled parameters for training, in-distribution testing, and out-of-distribution evaluation. Using the LLaMA architecture with LoRA fine-tuning, the study achieves efficient parameter reduction while benchmarking AI’s ability to generalize across numerical scales and abstract puzzle forms.