{"id":497,"date":"2024-03-28T15:30:19","date_gmt":"2024-03-28T20:30:19","guid":{"rendered":"https:\/\/ozer.gt\/log\/?p=497"},"modified":"2024-07-17T17:32:58","modified_gmt":"2024-07-17T22:32:58","slug":"how-do-language-models-represent-relations-between-entities","status":"publish","type":"post","link":"https:\/\/ozer.gt\/log\/2024\/03\/28\/how-do-language-models-represent-relations-between-entities\/","title":{"rendered":"How do language models represent relations between entities?"},"content":{"rendered":"<blockquote><p>This work shows that the complex nonlinear computation of LLMs for attribute extraction can be well-approximated with a simple linear function&#8230;<\/p><\/blockquote>\n<p>and more importantly, without a conceptual model.<\/p>\n<p>The study has two main findings:<br \/>\n1. Some of the implicit knowledge is represented in a simple, interpretable, and structured format.<br \/>\n2.. This representation is not universally used, and superficially similar facts can be encoded and extracted in very different ways.<\/p>\n<p>This is an interesting study that highlights the simplistic and associative nature of language models and the resulting randomness in their output.<\/p>\n<p><a href=\"https:\/\/arxiv.org\/abs\/2308.09124\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This work shows that the complex nonlinear computation of LLMs for attribute extraction can be well-approximated with a simple linear function&#8230; and more importantly, without a conceptual model. The study has two main findings: 1. Some of the implicit knowledge is represented in a simple, interpretable, and structured format. 2.. This representation is not universally [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-497","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/497","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/comments?post=497"}],"version-history":[{"count":1,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/497\/revisions"}],"predecessor-version":[{"id":498,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/497\/revisions\/498"}],"wp:attachment":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/media?parent=497"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/categories?post=497"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/tags?post=497"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}