You may have already seen “AutoResearch” released by Andrej Karpathy yesterday. It is another interesting experiment: research agents training on a single-GPU implementation of nanoGPT.
In this context, “research” is mostly hyperparameter tuning, but the agent is fully autonomous. So it can modify the code as it sees fit without a human in the loop.
While checking it out, I saw a session report posted by the agent, making me smile:
Changing random seed from 42→137 improved by 0.0004. Seed 7 was worse. Make of that what you will.
Even though the agent knows that optimizing the seed is pointless, it does it anyway and then tosses the ball back to you. Do whatever you want with that information!