'%3e%3cpath%20id='Union_1'%20data-name='Union%201'%20d='M-5781,4536a7.008,7.008,0,0,1-7-7V4303a7.008,7.008,0,0,1,7-7h406a7.007,7.007,0,0,1,7,7v226a7.007,7.007,0,0,1-7,7Zm296.87-59.21h.429a34.709,34.709,0,0,0,13.062,12.363,41.318,41.318,0,0,0,19.833,4.623,44.184,44.184,0,0,0,18.991-4.085,41.906,41.906,0,0,0,14.738-11.663,54.459,54.459,0,0,0,9.538-18.357,81.868,81.868,0,0,0,3.389-24.161,76.7,76.7,0,0,0-3.524-23.615,56.05,56.05,0,0,0-9.833-18.417,44.205,44.205,0,0,0-15.033-11.965,43.653,43.653,0,0,0-19.125-4.268,41.589,41.589,0,0,0-10.4,1.253,33.377,33.377,0,0,0-8.442,3.451,35.836,35.836,0,0,0-11.906,11.636h-.645v-56.76H-5512.3V4490.55h28.165v-13.759h0Zm-86.973-77.544a16.939,16.939,0,0,1,5.783,2.644,11.2,11.2,0,0,1,3.483,4.136,12.285,12.285,0,0,1,1.165,5.4,7.957,7.957,0,0,1-1.6,5.124,11.352,11.352,0,0,1-4.881,3.289c-4.365,1.74-10.826,2.737-20.184,4.059-14.33,2.012-24.288,5.163-31.337,9.917a31.516,31.516,0,0,0-5.4,4.564,26.736,26.736,0,0,0-4.077,5.686,28.308,28.308,0,0,0-2.576,6.949,37.143,37.143,0,0,0-.9,8.358,35.863,35.863,0,0,0,2.84,14.526,28.821,28.821,0,0,0,7.937,10.576c6.757,5.661,16.339,8.653,27.709,8.653,8.826,0,15.771-1.4,21.231-4.272a27.467,27.467,0,0,0,6.464-4.706,35.968,35.968,0,0,0,4.769-5.857h.43c.566,4.336,1.621,9.981,3.44,12.255h28.166v-1.075c-2.6-1.952-3.87-6.946-3.87-15.266v-56.115a52.109,52.109,0,0,0-2.6-17.232,33.613,33.613,0,0,0-3.363-6.962,30,30,0,0,0-4.84-5.8,32.371,32.371,0,0,0-6.393-4.587,41.66,41.66,0,0,0-8.02-3.333,73.776,73.776,0,0,0-21.226-2.722,68.568,68.568,0,0,0-20.907,2.958,41.752,41.752,0,0,0-14.62,7.955,33.743,33.743,0,0,0-8.735,11.583,37.016,37.016,0,0,0-3.252,13.84h26.014c.436-5.33,2.334-9.154,5.806-11.691,3.489-2.55,8.553-3.79,15.479-3.79A31.85,31.85,0,0,1-5571.1,4399.246Zm-172.534,91.3h31.174v-63.64h66.866v-26.66h-66.866v-36.98h78.9v-26.446h-110.079Zm565.082,0V4336.825h31.175V4490.55Zm-47.3,0-10.319-32.68h-55.47l-10.105,32.68h-31.605l53.535-153.725h32.25l54.179,153.725Zm-44.72-100.835-13.76,44.29h40.635l-13.76-44.29c-2.58-8.169-6.234-23.649-6.234-23.649h-.43S-5267.994,4381.545-5270.574,4389.715Zm-322.508,82.258a15.1,15.1,0,0,1-5.428-2.385,10.732,10.732,0,0,1-3.494-4.32,15.912,15.912,0,0,1-1.237-6.538,14.009,14.009,0,0,1,1.415-6.557,12.537,12.537,0,0,1,4.256-4.622c3.37-2.3,8.331-4.04,17.119-6.021l1.478-.346.119-.028c6.9-1.614,14.042-3.283,18.828-6.076v16.555a18.766,18.766,0,0,1-1.707,7.828,20.2,20.2,0,0,1-5.012,6.739,24.3,24.3,0,0,1-8.156,4.724,33.506,33.506,0,0,1-11.14,1.779A30.49,30.49,0,0,1-5593.083,4471.973Zm116.477-11.683c-4.641-5.844-7.1-14.189-7.1-24.135,0-10.853,1.936-19.36,5.752-25.289a20.911,20.911,0,0,1,7.853-7.334,25.024,25.024,0,0,1,11.765-2.637,22.381,22.381,0,0,1,10.724,2.537,21.5,21.5,0,0,1,7.66,7.112,34.013,34.013,0,0,1,4.595,10.941,59.734,59.734,0,0,1,1.532,14.025,60.268,60.268,0,0,1-1.545,14.254,32.125,32.125,0,0,1-4.528,10.6,20.177,20.177,0,0,1-7.352,6.615,21.564,21.564,0,0,1-10.011,2.281C-5465.25,4469.265-5471.94,4466.162-5476.606,4460.29Z'%20transform='translate(5788%20-4296)'/%3e%3c/g%3e%3c/svg%3e)
CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?
CMATH is a dataset of 1,700 Chinese elementary school math word problems (grades 1-6) designed to benchmark LLMs' arithmetic and reasoning capabilities against human grade-level performance, evaluating models like GPT-4, ChatGPT, and others on their ability to solve age-appropriate math problems.
We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1.7k elementary school-level math word problems with detailed annotations, source from actual Chinese workbooks and exams. This dataset aims to provide a benchmark tool for assessing the following question: to what grade level of elementary school math do the abilities of popular large language models (LLMs) correspond? We evaluate a variety of popular LLMs, including both commercial and open-source options,