{"id":167,"date":"2023-09-27T12:35:53","date_gmt":"2023-09-27T12:35:53","guid":{"rendered":"https:\/\/ozer.gt\/log\/?p=167"},"modified":"2024-01-29T10:45:32","modified_gmt":"2024-01-29T15:45:32","slug":"using-predictive-modeling-as-a-hammer-when-the-nail-needs-more-thinking","status":"publish","type":"post","link":"https:\/\/ozer.gt\/log\/2023\/09\/27\/using-predictive-modeling-as-a-hammer-when-the-nail-needs-more-thinking\/","title":{"rendered":"Using predictive modeling as a hammer when the nail needs more thinking"},"content":{"rendered":"

\"\"<\/p><\/blockquote>\n

The business problem is to put a lifeguard station on a beach to save some lives (i.e., find the best location for the lifeguard station). This is not really a predictive modeling problem. But that’s the hammer our data scientists have and they have access to fancy libraries. There is also some historical data: swimmers rescued and drowned at other beaches. It all checks out. Resistance to \ud835\ude31\ud835\ude2a\ud835\ude31 \ud835\ude2a\ud835\ude2f\ud835\ude34\ud835\ude35\ud835\ude22\ud835\ude2d\ud835\ude2d \ud835\ude31\ud835\ude33\ud835\ude30\ud835\ude31\ud835\ude29\ud835\ude26\ud835\ude35 is futile.<\/p>\n

Transforming the problem into an objective function could have signaled that this is an optimization problem (a prescriptive modeling problem), but that step was skipped. In the picture shown, we may need a solution:<\/p>\n

– minimizes distance => \ud835\udde6\ud835\uddfc\ud835\uddf9\ud835\ude03\ud835\uddf2\ud835\uddf1 \ud835\ude02\ud835\ude00\ud835\uddf6\ud835\uddfb\ud835\uddf4 \ud835\uddfd\ud835\uddf6\ud835\uddfd \ud835\uddf6\ud835\uddfb\ud835\ude00\ud835\ude01\ud835\uddee\ud835\uddf9\ud835\uddf9 \ud835\uddf3\ud835\uddee\ud835\uddfb\ud835\uddf0\ud835\ude06_\ud835\uddf9\ud835\uddf6\ud835\uddef\ud835\uddff\ud835\uddee\ud835\uddff\ud835\ude06
\nwhile also…
\n– minimizing time => \ud835\udde7\ud835\uddf5\ud835\uddf2 \ud835\uddf1\ud835\uddfc\ud835\uddfa\ud835\uddee\ud835\uddf6\ud835\uddfb \ud835\uddf2\ud835\ude05\ud835\uddfd\ud835\uddf2\ud835\uddff\ud835\ude01 \ud835\uddf2\ud835\uddfb\ud835\ude01\ud835\uddf2\ud835\uddff\ud835\ude00 \ud835\ude01\ud835\uddf5\ud835\uddf2 \ud835\uddff\ud835\uddfc\ud835\uddfc\ud835\uddfa
\n– minimizing swimming => \ud835\udde7\ud835\uddf5\ud835\uddf2 \ud835\uddf9\ud835\uddee\ud835\uddef\ud835\uddfc\ud835\uddff \ud835\ude02\ud835\uddfb\ud835\uddf6\ud835\uddfc\ud835\uddfb \ud835\uddf6\ud835\uddfb\ud835\ude01\ud835\uddf2\ud835\uddff\ud835\ude03\ud835\uddf2\ud835\uddfb\ud835\uddf2\ud835\ude00
\n– minimizing time to ice cream => \ud835\udde7\ud835\uddf5\ud835\uddf2 \ud835\uddf2\ud835\ude05\ud835\uddf2\ud835\uddf0\ud835\ude02\ud835\ude01\ud835\uddf6\ud835\ude03\ud835\uddf2 \ud835\uddf9\ud835\uddf2\ud835\uddee\ud835\uddf1\ud835\uddf2\ud835\uddff\ud835\ude00\ud835\uddf5\ud835\uddf6\ud835\uddfd \ud835\ude00\ud835\ude01\ud835\uddf2\ud835\uddfd\ud835\ude00 \ud835\uddf6\ud835\uddfb
\n– [not shown] minimizing walking on sand => \ud835\udde7\ud835\uddf5\ud835\uddf2 \ud835\uddd7\ud835\uddf2\ud835\uddfd\ud835\uddee\ud835\uddff\ud835\ude01\ud835\uddfa\ud835\uddf2\ud835\uddfb\ud835\ude01 \ud835\uddfc\ud835\uddf3 \ud835\udddf\ud835\uddee\ud835\uddef\ud835\uddfc\ud835\uddff \ud835\uddff\ud835\uddf2\ud835\uddfe\ud835\ude02\ud835\uddf6\ud835\uddff\ud835\uddf2\ud835\uddfa\ud835\uddf2\ud835\uddfb\ud835\ude01
\nand hopefully not…
\n– maximizing time => \ud835\uddd4 \ud835\uddf7\ud835\ude02\ud835\uddfb\ud835\uddf6\ud835\uddfc\ud835\uddff \ud835\uddf1\ud835\uddee\ud835\ude01\ud835\uddee \ud835\ude00\ud835\uddf0\ud835\uddf6\ud835\uddf2\ud835\uddfb\ud835\ude01\ud835\uddf6\ud835\ude00\ud835\ude01 \ud835\ude00\ud835\uddfc\ud835\uddf9\ud835\ude03\ud835\uddf2\ud835\ude00 \ud835\ude01\ud835\uddf5\ud835\uddf2 \ud835\uddfd\ud835\uddff\ud835\uddfc\ud835\uddef\ud835\uddf9\ud835\uddf2\ud835\uddfa<\/p>\n

So, the ideal solution requires more thinking about the problem. For example, maximizing the number of lives saved may actually require constraints on how to minimize time so that lifeguards don’t risk their lives during the rescue.<\/p>\n

The law of the instrument works a little too well in predictive modeling (and more generally in machine learning). Objective functions are often lost in translation when they should be an explicit step in the modeling process. Best practice tends to favor performance metrics, even though achieving the highest performance on the wrong function is clearly useless (and sometimes detrimental).<\/p>\n

More focus on objective functions and less obsession with “better performance” may be what we need. This would underline the importance of problem formulation and domain knowledge, and undermine the \ud835\ude31\ud835\ude2a\ud835\ude31 \ud835\ude2a\ud835\ude2f\ud835\ude34\ud835\ude35\ud835\ude22\ud835\ude2d\ud835\ude2d \ud835\ude31\ud835\ude33\ud835\ude30\ud835\ude31\ud835\ude29\ud835\ude26\ud835\ude35 solution.<\/p>\n

A combination of Warren Powell<\/a>‘s writing and the accompanying xkcd comic inspired this post (courtesy of xkcd.com<\/a>).<\/p>\n","protected":false},"excerpt":{"rendered":"

The business problem is to put a lifeguard station on a beach to save some lives (i.e., find the best location for the lifeguard station). This is not really a predictive modeling problem. But that’s the hammer our data scientists have and they have access to fancy libraries. There is also some historical data: swimmers […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-167","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/167","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/comments?post=167"}],"version-history":[{"count":2,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/167\/revisions"}],"predecessor-version":[{"id":257,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/posts\/167\/revisions\/257"}],"wp:attachment":[{"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/media?parent=167"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/categories?post=167"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ozer.gt\/log\/wp-json\/wp\/v2\/tags?post=167"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}