Mistral Large 4 on STEMBench: Strong in Engineering
Mistral Large 4 reached 2.501 on STEMBench, showing high marks in engineering and astronomy while placing sixth overall among ten frontier AI models.
In a 120-task scientific benchmark from Ulam AI, the French model averaged 2.501 on a 0-to-4 scale, placing sixth overall while standing out in engineering and astronomy.
Mistral AI released benchmark numbers for its flagship Mistral Large 4 on STEMBench, a rigorous scientific evaluation suite run by Ulam AI. The Paris-based team singled out two disciplines where the system performed best: engineering and astronomy. Full independent records from Ulam AI put that spotlight in context. Mistral Large 4 finished with an overall average of 2.501 on a 0-to-4 scale, placing sixth among ten frontier systems evaluated in a closed-book audit.
How STEMBench measures scientific work
Introduced on October 14, 2026, by Ulam AI under mathematician Przemek Chojecki, STEMBench120 covers 120 research problems designed at the graduate and post-doctoral level. The test splits tasks across six disciplines with 20 questions each: biology, chemistry, physics, earth sciences, astronomy, and engineering. Mathematics is tracked separately in a dedicated environment.
Unlike standard multiple-choice evaluations that reward lucky guesses, STEMBench grades partial execution. The scoring runs from 0 to 4. Zero means no usable output. A score of 1 indicates a valid methodology or initial protocol. Two marks meaningful partial progress on core equations. Three represents a substantial advance toward the solution. Four is reserved for a complete, autonomous answer without human intervention. Across the ten frontier systems evaluated in this run, not a single model scored a 4 on any task.
Engineering and astronomy results
Mistral highlighted its two highest marks for Large 4, an architecture with roughly 1.05 trillion parameters and 50 to 52 billion active per token across a 1-million-token window. The system posted 2.602 in engineering and 2.597 in astronomy. In 118 of the 120 assigned tasks, it generated structured, technically valid intermediate steps, bottoming out at zero only twice.
Those numbers explain why the model holds up well in technical workflows. It handles intermediate derivations, sets up governing equations, and sketches mechanical assemblies reliably. What it does not do is take an open-ended research problem and run with it from start to finish without human supervision.
Middle of the pack and real limits
While engineering and astronomy gave Mistral strong data points, the overall 2.501 average placed Large 4 in the lower half of the ten audited systems. The leaderboard is currently led by models like GPT-6 Astra in its xhigh configuration, which reached 2.809.
The results outline where AI stands in the hard sciences today. Top systems are dependable calculation partners for specific, well-bounded steps, but autonomous scientific problem solving remains an open challenge. For teams looking to audit these tests locally, Ulam AI keeps the 120 core problems private to prevent data contamination, but released a 60-task subset named STEMBench-small under an Apache-2.0 license.