A 7B model hit 88.89% on GPQA Diamond with zero gradient steps — they averaged four checkpoints

Ajouté : 2026-05-25

[00:00:00]A 7-billion model just hit 89% [music] on GPQA, diamond frontier territory, and the team didn't run a single gradient step.

[00:00:07]>> [music] >> They averaged four checkpoints. Naive averaging usually trashes both models.

[00:00:12]So, how do four good checkpoints become one better one without any training?

[00:00:16]Look at every parameter across all four checkpoints. Some are big and distinctive. Others are tiny and look the same [music] in every copy, basically dead weight. Darwin scores each one by magnitude [music] and rank, call it a trust score, then averages by trust. It can't invent a skill none of the parents had. [music] If all four are weak at math, the merge is weak at math.

[00:00:36]So, if you fine-tune the same [music] base model four times for four different jobs, you're already sitting on a bigger model. Hugging Face open-sourced [music] the Darwin family on May 14th. 89 on GPQA diamond, zero gradient steps.