Researchers had a stronger AI model coach a cheaper one through a set of tasks, then deleted the coach. Working alone, the cheaper model completed 23.3% of its test tasks before the coaching and 44.3% after, close to double. A team from Peking University, Google and the Hong Kong University of Science and Technology reported the result in a paper.
Usually, engineers make an AI model better at tasks by surrounding it with extra tools, rules, guardrails and checks. That gain lasts only while the setup stays attached, and removing it sends the model back to its old performance. The team set out to move the improvement into the model itself.
The cheaper model worked on three kinds of tasks: editing spreadsheets, using tools in simulated software apps, and solving chemistry problems. The stronger model checked each proposed move before it took effect. This coach worked from private guidance drawn from a specialized setup built for each kind of task, with no answer key.
Sound moves went through unchanged. When the cheaper model was about to make a mistake, the coach swapped in a better move, changing as little as possible. The cheaper model never saw the swap, because its record of the session showed only the corrected move.
The corrected sessions became training material. Retraining the cheaper model on them changed the model itself, so the lessons stayed when the team removed the coach and the specialized setup. Only a single tool for running commands remained.
All scores come from test tasks kept out of training. With the full specialized setup attached, the original model completed 41.7% of them. The retrained model, working alone, beat the same model with every added tool, rule and check. Chemistry was the exception, where the retrained model rose from 12.0% to 30.0% of tasks. The original model had reached 38.0% there with the full setup.
The authors tie that shortfall to the kind of knowledge involved. Step-by-step habits, such as checking a file before editing it, were easier to absorb than chemistry, which leans on deep subject knowledge.
Review by weaker models sometimes does harm, the authors report, so the method depends on a capable coach. The paper’s results also come from one round of training on three kinds of work. Coaching adds cost, because a second model reviews every move, but that cost ends once the coach is deleted.
