Google Gemini 1.5 Pro Achieves 91.1% Accuracy in Math Benchmark

Google announced that its Gemini 1.5 Pro model, after specialized training in mathematics, significantly improved its performance, solving problems from international math competitions. The Gemini 1.5 Pro was tested using the MATH benchmark, the American Mathematics Invitational Examination (AIME), and Google’s internal HiddenMath benchmark.

Data shows that the math-specific Gemini 1.5 Pro performed comparably to human experts, solving more AIME problems and scoring higher on other benchmarks than the standard version. Google provided examples demonstrating the model’s ability to recall and apply basic algebraic formulas to derive correct answers.

In the benchmarks, Gemini 1.5 Pro outperformed GPT-4 Turbo and Amazon’s Claude. The model achieved 80.6% accuracy on single MATH benchmark samples and 91.1% accuracy when sampling 256 solutions to select one candidate answer.

Google emphasizes the model’s superior performance across all five benchmark tests, highlighting its advancement in the field of mathematical problem-solving.

Keep visiting for more such awesome posts, internet tips, lifestyle tips, and remember we cover,
“Everything under the Sun!”

Follow Inspire2rise on Twitter. | Follow Inspire2rise on Facebook. | Follow Inspire2rise on YouTube

Aditya Nath Jha

Aditya Nath Jha is an Engineer from New Delhi, India. His areas of interest include Gadgets, WordPress, speed optimization & latest technology. When he is not busy blogging he loves to write poetry, compose his own songs and has a taste for music! Find him on Facebook, Twitter, Linked in, Instagram. And watch his videos on YouTube.