U-MATH / μ-MATH leaderboard

These datasets are designed to test the mathematical reasoning and meta-evaluation capabilities of Large Language Models (LLMs) on university-level problems. U-MATH provides a set of 1,100 university-level mathematical problems, while µ-MATH complements it with a meta-evaluation framework focusing on solution judgment with 1084 LLM solutions.

Select Columns to Display:

Type #Params (B) Architecture License Family Judge Model Name U-MATH Text Acc U-MATH Visual Acc Diff Calc Acc Diff Calc Text Acc Diff Calc Visual Acc Integral Calc Acc Integral Calc Text Acc Integral Calc Visual Acc Algebra Acc Algebra Text Acc Algebra Visual Acc Multivar Calc Acc Multivar Calc Text Acc Multivar Calc Visual Acc Precalc Acc Precalc Text Acc Precalc Visual Acc Seq & Series Acc Seq & Series Text Acc Seq & Series Visual Acc

Visible Columns:

Textbox

Filter model types:

Filter model sizes:

Filter model families:

Rank	T	S	Full Model Name	Model Name	Type	#Params (B)	#Params inc. Proprietary (B)	Architecture	License	Family	Model URL	Judge Model Name	U-MATH Acc	U-MATH Text Acc	U-MATH Visual Acc	Diff Calc Acc	Diff Calc Text Acc	Diff Calc Visual Acc	Integral Calc Acc	Integral Calc Text Acc	Integral Calc Visual Acc	Algebra Acc	Algebra Text Acc	Algebra Visual Acc	Multivar Calc Acc	Multivar Calc Text Acc	Multivar Calc Visual Acc	Precalc Acc	Precalc Text Acc	Precalc Visual Acc	Seq & Series Acc	Seq & Series Text Acc	Seq & Series Visual Acc
10	🟥	🚀	llava-hf/llava-onevision-qwen2-7b-ov-chat-hf	llava-hf/llava-onevision-qwen2-7b-ov-chat-hf	Open-Weights	684.5	684.5	LlavaOnevisionForConditionalGeneration	tongyi-qianwen	DeepSeek	https://huggingface.co/unsloth/llava-onevision-qwen2-7b-ov-chat-hf	gpt-4o-2024-08-06	73.6	77.8	45.2	62.3	69.3	47.1	55.3	54.7	56.9	87.2	93.3	56.7	74.2	75.3	32.1	92.5	95.3	50	78.6	39.3	25

Rank	T	S	Model Name	U-MATH Acc	U-MATH Text Acc	U-MATH Visual Acc
10	🟥	🚀	llava-hf/llava-onevision-qwen2-7b-ov-chat-hf	73.6	77.8	45.2

Rank	T	S	Model Name	U-MATH Acc	U-MATH Text Acc	U-MATH Visual Acc
1	🟥	❓	gemini-2.0-flash-thinking-exp-01-21	73.6	77.8	55
2	🟥	❓	o1	70.7	77.2	45.2
3	🟥	❓	o3-mini	68.9	79.7	20.5
4	💙	🚀	deepseek-ai/DeepSeek-R1	68.4	79	20.5
5	🟥	❓	o1-mini	63.5	73.3	19
6	💙	🚗	Qwen/QwQ-32B-Preview	61.5	71.8	15
7	🟥	❓	google/gemini-1.5-pro	60.1	63.4	45
8	💙	🚚	Qwen/QVQ-72b-Preview	55.1	59.3	36
9	💙	🚀	deepseek-ai/DeepSeek-V3	51.9	60.4	13.5
10	🟥	❓	google/gemini-1.5-flash	51.3	53.8	40
11	💙	🚚	Qwen/Qwen2.5-Math-72B-Instruct	50.2	59	10.5
12	💙	🚚	Nexusflow/Athene-V2-Chat	46.2	54.6	8.5
13	💙	🚗	microsoft/phi-4	44.1	51.3	11.5
14	💙	🚗	Qwen/Qwen2.5-32B-Instruct	43.8	51.4	9.5
15	🟥	❓	gpt-4o-2024-08-06	43.5	46.4	30
16	🟥	❓	gpt-4o-2024-05-13	43.4	45.8	32.5
17	💙	🚚	Qwen/Qwen2.5-72B-Instruct	41	48.6	7
18	💙	🚀	mistralai/Mistral-Large-Instruct-2411	40.4	48.1	5.5
19	💙	🚀	mistralai/Pixtral-Large-Instruct-2411	39.7	42.9	25.5
20	💙	🚗	Qwen/Qwen2.5-Math-7B-Instruct	38.4	45.2	7.5
21	💙	🚚	meta-llama/Llama-3.3-70B-Instruct	37.3	43.4	9.5
22	🟥	❓	gpt-4o-mini-2024-07-18	37.2	40.3	23
23	🟥	❓	claude-sonnet-3-5	35.1	36.1	30.5
24	💙	🚗	Qwen/Qwen2.5-7B-Instruct	33.8	40	6
25	💙	🚚	meta-llama/Llama-3.2-90B-Vision-Instruct	32.6	36.3	16
26	💙	🚚	nvidia/Llama-3.1-Nemotron-70B-Instruct-HF	31.4	37.4	4
27	💙	🚚	Qwen/Qwen2-VL-72B-Instruct	31.2	32.2	26.5
28	💙	🚗	mistralai/Mistral-Small-24B-Instruct-2501	29.5	35	4.5
29	💙	🚚	meta-llama/Llama-3.1-70B-Instruct	28.5	33.7	5
30	💙	🚗	deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct	25.9	30.2	6.5
31	💙	🚚	AI-MO/NuminaMath-72B-CoT	25	29.7	4
32	💙	🚗	meta-llama/Llama-3.1-8B-Instruct	22.3	26.1	5
33	🟥	❓	liquid/lfm-7b	21.1	24.6	5.5
34	💙	🚗	Qwen/Qwen2-VL-7B-Instruct	20.4	21.4	15.5
35	💙	🚗	AI-MO/NuminaMath-7B-CoT	19.2	22.8	3
36	💙	🚗	mistralai/Ministral-8B-Instruct-2410	18.3	21.4	4
37	💙	🚗	mistralai/Mathstral-7B-v0.1	18	20.7	6
38	💙	🚗	llava-hf/llava-onevision-qwen2-7b-ov-chat-hf	17.7	20.7	4.5
39	💙	🚗	unsloth/Llama-3.2-11B-Vision-Instruct	17	18.6	10
40	💙	🚗	mistralai/Pixtral-12B-2409	15.5	15.6	15.5
41	💙	🚗	llava-hf/llava-v1.6-mistral-7b-hf	3.3	3.7	1.5

Select Columns to Display:

Type #Params (B) Architecture License Family Extract Model Name μ-MATH TPR μ-MATH TNR μ-MATH PPV μ-MATH NPV GPT-4o Subset F1 GPT-4o Subset TPR GPT-4o Subset TNR GPT-4o Subset PPV GPT-4o Subset NPV Gemini-1.5-Pro Subset F1 Gemini-1.5-Pro Subset TPR Gemini-1.5-Pro Subset TNR Gemini-1.5-Pro Subset PPV Gemini-1.5-Pro Subset NPV Llama-3.1-70B Subset F1 Llama-3.1-70B Subset TPR Llama-3.1-70B Subset TNR Llama-3.1-70B Subset PPV Llama-3.1-70B Subset NPV Qwen2.5-72B Subset F1 Qwen2.5-72B Subset TPR Qwen2.5-72B Subset TNR Qwen2.5-72B Subset PPV Qwen2.5-72B Subset NPV

Visible Columns:

Textbox

Filter model types:

Filter model sizes:

Filter model families:

Rank	T	S	Full Model Name	Model Name	Type	#Params (B)	#Params inc. Proprietary (B)	Architecture	License	Family	Model URL	Extract Model Name	μ-MATH F1	μ-MATH TPR	μ-MATH TNR	μ-MATH PPV	μ-MATH NPV	GPT-4o Subset F1	GPT-4o Subset TPR	GPT-4o Subset TNR	GPT-4o Subset PPV	GPT-4o Subset NPV	Gemini-1.5-Pro Subset F1	Gemini-1.5-Pro Subset TPR	Gemini-1.5-Pro Subset TNR	Gemini-1.5-Pro Subset PPV	Gemini-1.5-Pro Subset NPV	Llama-3.1-70B Subset F1	Llama-3.1-70B Subset TPR	Llama-3.1-70B Subset TNR	Llama-3.1-70B Subset PPV	Llama-3.1-70B Subset NPV	Qwen2.5-72B Subset F1	Qwen2.5-72B Subset TPR	Qwen2.5-72B Subset TNR	Qwen2.5-72B Subset PPV	Qwen2.5-72B Subset NPV
10	🟥	🚗	mistralai/Mistral-Large-Instruct-2411	mistralai/Mistral-Large-Instruct-2411	Open-Weights	684.5	684.5	DeepseekV3ForCausalLM	qwen	DeepSeek	https://huggingface.co/unsloth/Mistral-Large-Instruct-2411	Qwen/Qwen2.5-72B-Instruct	89.5	90.6	88.4	88.7	90.4	88.4	88.9	87.9	88.9	87.9	90.6	95.3	84.6	91.1	91.7	93.2	90.5	95.8	90.5	95.8	83.8	86.8	80.6	84.6	83.3

Rank	T	S	Model Name	μ-MATH F1	GPT-4o Subset F1	Gemini-1.5-Pro Subset F1	Llama-3.1-70B Subset F1	Qwen2.5-72B Subset F1
10	🟥	🚗	mistralai/Mistral-Large-Instruct-2411	89.5	88.4	90.6	93.2	83.8

Rank	T	S	Model Name	μ-MATH F1	GPT-4o Subset F1	Gemini-1.5-Pro Subset F1	Llama-3.1-70B Subset F1	Qwen2.5-72B Subset F1
1	🟥	❓	o1	89.5	88.4	90.6	93.2	83.8
2	🟥	❓	o1-mini	84.8	81.2	86.2	89.7	79.5
3	💙	🚗	Qwen/QwQ-32B-Preview	83.2	78	80	84	86.7
4	💙	🚀	deepseek-ai/DeepSeek-R1	82.2	79.7	82	88.2	76.8
5	🟥	❓	gemini-2.0-flash-thinking-exp-01-21	81	74.3	85.8	83.3	76
6	🟥	❓	google/gemini-1.5-pro	80.7	78.2	79.5	83.6	77.7
7	🟥	❓	gpt-4o-2024-08-06	77.4	77.5	72.6	81.8	74.2
8	💙	🚀	mistralai/Mistral-Large-Instruct-2411	76.6	76	75	78.6	72.5
9	💙	🚚	Qwen/Qwen2.5-72B-Instruct	75.6	73.7	74.2	79.3	70.5
10	🟥	❓	google/gemini-1.5-flash	74.8	70.1	73.9	80.6	71.2
11	🟥	❓	claude-sonnet-3-5	74.8	72.2	73.8	77.9	70.8
12	💙	🚚	Qwen/Qwen2.5-Math-72B-Instruct	74	68.2	76.8	77.3	69.3
13	🟥	❓	gpt-4o-mini-2024-07-18	72.3	70.4	69.6	76.2	69.3
14	💙	🚗	Qwen/Qwen2.5-7B-Instruct	69.3	68.3	69.1	72.3	62.4
15	💙	🚗	Qwen/Qwen2.5-Math-7B-Instruct	61.9	57.2	63.8	63.8	59.7
16	💙	🚚	meta-llama/Llama-3.1-70B-Instruct	61	69.4	58.8	57	56
17	💙	🚗	mistralai/Ministral-8B-Instruct-2410	60.5	62.9	58.3	63.1	52.8
18	💙	🚗	meta-llama/Llama-3.1-8B-Instruct	52	51.2	55.5	49.2	48.7

Select Columns to Display:

Type #Params (B) Architecture License Family U-MATH Acc U-MATH Text Acc U-MATH Visual Acc Judge Model Name Extract Model Name μ-MATH F1 μ-MATH TPR μ-MATH TNR μ-MATH PPV μ-MATH NPV

Visible Columns:

Textbox

Filter model types:

Filter model sizes:

Filter model families:

U-MATH Rank	μ-MATH Rank	T	S	Full Model Name	Model Name	Type	#Params (B)	#Params inc. Proprietary (B)	Architecture	License	Family	Model URL	U-MATH Acc	U-MATH Text Acc	U-MATH Visual Acc	Judge Model Name	Extract Model Name	μ-MATH F1	μ-MATH TPR	μ-MATH TNR	μ-MATH PPV	μ-MATH NPV
10	10	🟥	🚀	mistralai/Mistral-Large-Instruct-2411	mistralai/Mistral-Large-Instruct-2411	Open-Weights	684.5	684.5	DeepseekV3ForCausalLM	qwen	DeepSeek	https://huggingface.co/unsloth/Mistral-Large-Instruct-2411	73.6	77.8	45.2	gpt-4o-2024-08-06	Qwen/Qwen2.5-72B-Instruct	89.5	89.1	73.2	76.9	87.1

U-MATH Rank	μ-MATH Rank	T	S	Model Name	U-MATH Acc	μ-MATH F1
10	10	🟥	🚀	mistralai/Mistral-Large-Instruct-2411	73.6	89.5

U-MATH Rank	μ-MATH Rank	T	S	Model Name	U-MATH Acc	μ-MATH F1
1	5	🟥	❓	gemini-2.0-flash-thinking-exp-01-21	73.6	81
2	1	🟥	❓	o1	70.7	89.5
4	4	💙	🚀	deepseek-ai/DeepSeek-R1	68.4	82.2
5	2	🟥	❓	o1-mini	63.5	84.8
6	3	💙	🚗	Qwen/QwQ-32B-Preview	61.5	83.2
7	6	🟥	❓	google/gemini-1.5-pro	60.1	80.7
10	10	🟥	❓	google/gemini-1.5-flash	51.3	74.8
11	12	💙	🚚	Qwen/Qwen2.5-Math-72B-Instruct	50.2	74
15	7	🟥	❓	gpt-4o-2024-08-06	43.5	77.4
17	9	💙	🚚	Qwen/Qwen2.5-72B-Instruct	41	75.6
18	8	💙	🚀	mistralai/Mistral-Large-Instruct-2411	40.4	76.6
20	15	💙	🚗	Qwen/Qwen2.5-Math-7B-Instruct	38.4	61.9
22	13	🟥	❓	gpt-4o-mini-2024-07-18	37.2	72.3
23	11	🟥	❓	claude-sonnet-3-5	35.1	74.8
24	14	💙	🚗	Qwen/Qwen2.5-7B-Instruct	33.8	69.3
29	16	💙	🚚	meta-llama/Llama-3.1-70B-Instruct	28.5	61
32	18	💙	🚗	meta-llama/Llama-3.1-8B-Instruct	22.3	52
36	17	💙	🚗	mistralai/Ministral-8B-Instruct-2410	18.3	60.5

This repository contains the official leaderboard code for the U-MATH and $\mu$-MATH benchmarks. These datasets are designed to test the mathematical reasoning and meta-evaluation capabilities of Large Language Models (LLMs) on university-level problems.

Overview

U-MATH provides a set of 1,100 university-level mathematical problems, while µ-MATH complements it with a meta-evaluation framework focusing on solution judgment with 1084 LLM solutions.

📊 U-MATH benchmark at Huggingface
🔎 μ-MATH benchmark at Huggingface
🗞️ Paper
👾 Evaluation Code at GitHub

Licensing Information

The contents of the μ-MATH's machine-generated model_output column are subject to the underlying LLMs' licensing terms.
Contents of all the other dataset U-MATH and μ-MATH fields, as well as the code, are available under the MIT license.

📙 Citation

Built with Gradio logo

Rank	T	S	Full Model Name	Model Name	Type	#Params (B)	#Params inc. Proprietary (B)	Architecture	License	Family	Model URL	Extract Model Name	μ-MATH F1	μ-MATH TPR	μ-MATH TNR	μ-MATH PPV	μ-MATH NPV	GPT-4o Subset F1	GPT-4o Subset TPR	GPT-4o Subset TNR	GPT-4o Subset PPV	GPT-4o Subset NPV	Gemini-1.5-Pro Subset F1	Gemini-1.5-Pro Subset TPR	Gemini-1.5-Pro Subset TNR	Gemini-1.5-Pro Subset PPV	Gemini-1.5-Pro Subset NPV	Llama-3.1-70B Subset F1	Llama-3.1-70B Subset TPR	Llama-3.1-70B Subset TNR	Llama-3.1-70B Subset PPV	Llama-3.1-70B Subset NPV	Qwen2.5-72B Subset F1	Qwen2.5-72B Subset TPR	Qwen2.5-72B Subset TNR	Qwen2.5-72B Subset PPV	Qwen2.5-72B Subset NPV
1	🟥	❓	o1	o1	Proprietary		150	Unknown	Unknown	GPT		Qwen/Qwen2.5-72B-Instruct	89.5	90.6	88.4	88.7	90.4	88.4	88.9	87.9	88.9	87.9	90.6	95.3	84.6	91.1	91.7	93.2	90.5	95.8	90.5	95.8	83.8	86.8	80.6	84.6	83.3
2	🟥	❓	o1-mini	o1-mini	Proprietary		150	Unknown	Unknown	GPT		Qwen/Qwen2.5-72B-Instruct	84.8	83.3	86.2	85.8	83.8	81.2	77.8	84.8	84.8	77.8	86.2	88.4	84.6	90.5	81.5	89.7	85.7	93.8	85.7	93.8	79.5	81.6	77.4	81.6	77.4
3	💙	🚗	Qwen/QwQ-32B-Preview	Qwen/QwQ-32B-Preview	Open-Weights	32.8	32.8	Qwen2ForCausalLM	None	Qwen	https://huggingface.co/unsloth/QwQ-32B-Preview	Qwen/Qwen2.5-72B-Instruct	83.2	91.3	75.4	78.7	89.7	78	86.1	69.7	75.6	82.1	80	95.3	61.5	80.4	88.9	84	90.5	83.3	70.4	95.2	86.7	92.1	80.6	85.4	89.3
4	💙	🚀	deepseek-ai/DeepSeek-R1	deepseek-ai/DeepSeek-R1	Open-Weights	684.5	684.5	DeepseekV3ForCausalLM	Unknown	DeepSeek	https://huggingface.co/unsloth/DeepSeek-R1	Qwen/Qwen2.5-72B-Instruct	82.2	76.8	87.7	86.2	79.1	79.7	72.2	87.9	86.7	74.4	82	81.4	84.6	89.7	73.3	88.2	85.7	91.7	81.8	93.6	76.8	71.1	83.9	84.4	70.3
5	🟥	❓	gemini-2.0-flash-thinking-exp-01-21	gemini-2.0-flash-thinking-exp-01-21	Proprietary		150	Unknown	Unknown	Gemini		Qwen/Qwen2.5-72B-Instruct	81	89.1	73.2	76.9	87.1	74.3	91.7	57.6	70.2	86.4	85.8	93	76.9	87	87	83.3	81	87.5	73.9	91.3	76	86.8	64.5	75	80
6	🟥	❓	google/gemini-1.5-pro	google/gemini-1.5-pro	Proprietary		150	Unknown	Unknown	Gemini		Qwen/Qwen2.5-72B-Instruct	80.7	77.5	84.5	85.2	76.4	78.2	76.4	80.2	80.5	76.1	79.5	81	82.9	91.6	65.4	83.6	75.3	90.8	82	86.8	77.7	75.5	81	84.2	71.2
7	🟥	❓	gpt-4o-2024-08-06	gpt-4o-2024-08-06	Proprietary		150	Unknown	Unknown	GPT		Qwen/Qwen2.5-72B-Instruct	77.4	70.1	85.9	85.1	71.3	77.5	72.1	83.2	82.1	73.6	72.6	70.4	82.9	90.5	54.8	81.8	71.1	90.8	81.2	84.9	74.2	67.1	83.6	84.6	65.5
8	💙	🚀	mistralai/Mistral-Large-Instruct-2411	mistralai/Mistral-Large-Instruct-2411	Open-Weights	122.6	122.6	MistralForCausalLM	mrl	Mistral	https://huggingface.co/unsloth/Mistral-Large-Instruct-2411	Qwen/Qwen2.5-72B-Instruct	76.6	75.7	77.7	79.7	73.5	76	75.7	76.3	77.4	74.6	75	79.9	73.2	87.3	61.2	78.6	75.3	82.8	70.9	85.7	72.5	71	75	79.1	65.9
9	💙	🚚	Qwen/Qwen2.5-72B-Instruct	Qwen/Qwen2.5-72B-Instruct	Open-Weights	72.7	72.7	Qwen2ForCausalLM	qwen	Qwen	https://huggingface.co/unsloth/Qwen2.5-72B-Instruct	Qwen/Qwen2.5-72B-Instruct	75.6	77.1	74.2	77.5	73.7	73.7	76.4	71	73.8	73.8	74.2	79.4	72	86.7	60.2	79.3	75.3	83.9	72.3	85.9	70.5	76.1	64.7	74.2	67
10	🟥	❓	google/gemini-1.5-flash	google/gemini-1.5-flash	Proprietary		150	Unknown	Unknown	Gemini		Qwen/Qwen2.5-72B-Instruct	74.8	63.3	88.3	86.2	67.6	70.1	57.9	84	79.4	65.1	73.9	67.7	91.5	94.8	55.1	80.6	67	92	82.3	83.3	71.2	60.6	85.3	84.7	61.9
11	🟥	❓	claude-sonnet-3-5	claude-sonnet-3-5	Proprietary		150	Unknown	Unknown	Claude		Qwen/Qwen2.5-72B-Instruct	74.8	62.5	89.5	87.3	67.4	72.2	57.9	88.5	84.4	66.3	73.8	70.9	85.4	91.8	56	77.9	59.8	93.1	82.9	80.6	70.8	58.1	87.9	86.5	61.1
12	💙	🚚	Qwen/Qwen2.5-Math-72B-Instruct	Qwen/Qwen2.5-Math-72B-Instruct	Open-Weights	72.7	72.7	Qwen2ForCausalLM	qwen	Qwen	https://huggingface.co/unsloth/Qwen2.5-Math-72B-Instruct	Qwen/Qwen2.5-72B-Instruct	74	80.9	66.8	73.8	75.2	68.2	77.9	58.8	66.9	71.3	76.8	82.5	73.2	87.6	64.5	77.3	81.4	76.4	65.8	88.1	69.3	81.3	56.9	71.6	69.5
13	🟥	❓	gpt-4o-mini-2024-07-18	gpt-4o-mini-2024-07-18	Proprietary		150	Unknown	Unknown	GPT		Qwen/Qwen2.5-72B-Instruct	72.3	59	88.1	85.1	65.1	70.4	56.4	86.3	81.4	64.9	69.6	63	87.8	92.2	50.7	76.2	59.8	90.2	77.3	80.1	69.3	56.1	87.1	85.3	59.8
14	💙	🚗	Qwen/Qwen2.5-7B-Instruct	Qwen/Qwen2.5-7B-Instruct	Open-Weights	7.6	7.6	Qwen2ForCausalLM	None	Qwen	https://huggingface.co/unsloth/Qwen2.5-7B-Instruct	Qwen/Qwen2.5-72B-Instruct	69.3	78.7	59.8	69.3	70.8	68.3	81.4	55.7	66.3	73.7	69.1	79.4	59.8	82	55.7	72.3	78.4	70.1	59.4	85.3	62.4	75.5	49.1	66.5	60
15	💙	🚗	Qwen/Qwen2.5-Math-7B-Instruct	Qwen/Qwen2.5-Math-7B-Instruct	Open-Weights	7.6	7.6	Qwen2ForCausalLM	None	Qwen	https://huggingface.co/unsloth/Qwen2.5-Math-7B-Instruct	Qwen/Qwen2.5-72B-Instruct	61.9	76.6	47.9	62.9	63.9	57.2	75	41.2	57.7	60.7	63.8	77.8	50	78.2	49.4	63.8	85.6	51.7	49.7	86.5	59.7	71	48.3	64.7	55.4
16	💙	🚚	meta-llama/Llama-3.1-70B-Instruct	meta-llama/Llama-3.1-70B-Instruct	Open-Weights	70.6	70.6	LlamaForCausalLM	Unknown	LLaMA	https://huggingface.co/unsloth/Llama-3.1-70B-Instruct	Qwen/Qwen2.5-72B-Instruct	61	62.5	59.6	64.1	57.9	69.4	67.1	71.8	71.8	67.1	58.8	61.4	61	78.4	40.7	57	63.9	54	43.7	72.9	56	58.7	53.4	62.8	49.2
17	💙	🚗	mistralai/Ministral-8B-Instruct-2410	mistralai/Ministral-8B-Instruct-2410	Open-Weights	8	8	MistralForCausalLM	mrl	Mistral	https://huggingface.co/unsloth/Ministral-8B-Instruct-2410	Qwen/Qwen2.5-72B-Instruct	60.5	55.9	65.8	65.4	56.4	62.9	53.6	73.3	68.2	59.6	58.3	63	57.3	77.3	40.2	63.1	59.8	67.8	50.9	75.2	52.8	47.1	60.3	61.3	46.1
18	💙	🚗	meta-llama/Llama-3.1-8B-Instruct	meta-llama/Llama-3.1-8B-Instruct	Open-Weights	8	8	LlamaForCausalLM	Unknown	LLaMA	https://huggingface.co/unsloth/Llama-3.1-8B-Instruct	Qwen/Qwen2.5-72B-Instruct	52	48.7	55.9	56	48.5	51.2	46.4	56.5	53.3	49.7	55.5	55	62.2	77	37.5	49.2	45.4	54	35.5	63.9	48.7	45.2	53.4	56.5	42.2