WizardMath:
Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct

Haipeng Luo 2,*, Qingfeng Sun1*, Can Xu1,‡, Pu Zhao1, Jianguang Lou1,
Chongyang Tao1, Xiubo Geng31, Qingwei Lin1, Shifeng Chen2,‡, Dongmei Zhang1
1Microsoft, 2Chinese Academy of Sciences
*Equal Contribution, ‡Corresponding author
qins@microsoft.com, caxu@microsoft.com, hp.luo@siat.ac.cn
🤗 Models Code arXiv

Abstract

Large language models (LLMs), such as GPT-4, have shown remarkable performance in natural language processing (NLP) tasks, including challenging mathematical reasoning. However, most existing open-source models are only pre-trained on large-scale internet data and without math-related optimization. In this paper, we present WizardMath, which enhances the mathematical reasoning abilities of Llama-2, by applying our proposed Reinforcement Learning from Evol-Instruct Feedback (RLEIF) method to the domain of math. Through extensive experiments on two mathematical reasoning benchmarks, namely GSM8k and MATH, we reveal the extraordinary capabilities of our model. WizardMath surpasses all other open-source LLMs by a substantial margin. Furthermore, our model even outperforms ChatGPT-3.5, Claude Instant-1, PaLM-2 and Minerva on GSM8k, simultaneously surpasses Text-davinci-002, PaLM-1 and GPT-3 on MATH.

Overview

wizardmath

Experiment Results

wizardmath

BibTeX

@article{luo2023wizardmath,
  title={WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct},
  author={Luo, Haipeng and Sun, Qingfeng and Xu, Can and Zhao, Pu and Lou, Jianguang and Tao, Chongyang and Geng, Xiubo and Lin, Qingwei and Chen, Shifeng and Zhang, Dongmei},
  journal={arXiv preprint arXiv:2308.09583},
  year={2023}
}