Building a larger AI model can make its training unstable, and unstable training costs real money. It burns computing power, forces reruns and slows the path from an idea to a working model. DeepSeek’s method, mHC, is built to fix one specific cause of that instability, which lies in how information travels through the model.
Modern AI models process information through many stages stacked one after another. Alongside those stages runs a direct path, like an express lane on a highway that always stays open. However busy the rest of the road gets, the express lane keeps information moving, which is why it became a standard part of these models.
The next step was to widen that lane. An earlier design called HC split it into several lanes and learned how information should mix between them. The aim was more room inside the express lane and more ways to recombine information.
Widening brought a new problem. If the mixing has no limits, one lane can carry more and more of the traffic as the stages stack up, while the others carry less. Over many stages, that drift breaks the model’s structure, so training turns fragile and growing the model becomes unreliable.
mHC keeps the multiple lanes but adds a hard rule: each stage must mix its lanes in balance, so no lane can quietly take most of the traffic. Information can still switch lanes, but the totals do not drift into overload.
The balance comes from repeatedly rescaling a table of numbers until its rows and columns add up to set targets, a Sinkhorn-Knopp-style method. Applied to the lanes, the method turns the mixing into a controlled reshuffle instead of an amplifier. DeepSeek caps the procedure at 20 rounds of rescaling to keep it fast.
In DeepSeek’s tests, the stacked stages amplified information at most about 1.6 times with mHC, against nearly 3,000 times with HC. The company also says the added cost is negligible, thanks to engineering work on its underlying systems.
Stable training also cuts the time teams spend diagnosing instability that looks like poor model quality. It gives builders a second way to grow a model, by safely widening the express lane instead of only adding size and training time.
