You may have already seen “AutoResearch” released by Andrej Karpathy yesterday. It is another interesting experiment: research agents training on a single-GPU implementation of nanoGPT.<\/p>\n
In this context, “research” is mostly hyperparameter tuning, but the agent is fully autonomous. So it can modify the code as it sees fit without a human in the loop.<\/p>\n
While checking it out, I saw a session report posted by the agent, making me smile:<\/p>\n
Changing random seed from 42\u2192137 improved by 0.0004. Seed 7 was worse. Make of that what you will.<\/p><\/blockquote>\n
Even though the agent knows that optimizing the seed is pointless, it does it anyway and then tosses the ball back to you. Do whatever you want with that information!<\/p>\n
Source 1<\/a> (Autoresearch repo) \u2013 Source 2<\/a> (Discussion link)<\/p>\n","protected":false},"excerpt":{"rendered":"
You may have already seen “AutoResearch” released by Andrej Karpathy yesterday. It is another interesting experiment: research agents training on a single-GPU implementation of nanoGPT. In this context, “research” is mostly hyperparameter tuning, but the agent is fully autonomous. So it can modify the code as it sees fit without a human in the loop. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-3128","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/3128","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/comments?post=3128"}],"version-history":[{"count":10,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/3128\/revisions"}],"predecessor-version":[{"id":3138,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/3128\/revisions\/3138"}],"wp:attachment":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/media?parent=3128"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/categories?post=3128"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/tags?post=3128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}