Otherwise, tree ensembles continue to outperform neural networks. The decision tree in the figure shows the winner among the top five methods.
Now, the background:
I explored the why of this question before, but didn’t get very far. This may be expected, given the black-box and data-driven nature of these methods.
This is another study, this time testing larger tabular datasets. By comparing 19 methods on 176 datasets, this paper shows that ๐ณ๐ผ๐ฟ ๐ฎ ๐น๐ฎ๐ฟ๐ด๐ฒ ๐ป๐๐บ๐ฏ๐ฒ๐ฟ ๐ผ๐ณ ๐ฑ๐ฎ๐๐ฎ๐๐ฒ๐๐, ๐ฒ๐ถ๐๐ต๐ฒ๐ฟ ๐ฎ ๐๐ถ๐บ๐ฝ๐น๐ฒ ๐ฏ๐ฎ๐๐ฒ๐น๐ถ๐ป๐ฒ ๐บ๐ฒ๐๐ต๐ผ๐ฑ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ ๐ฎ๐ ๐๐ฒ๐น๐น ๐ฎ๐ ๐ฎ๐ป๐ ๐ผ๐๐ต๐ฒ๐ฟ ๐บ๐ฒ๐๐ต๐ผ๐ฑ, ๐ผ๐ฟ ๐ฏ๐ฎ๐๐ถ๐ฐ ๐ต๐๐ฝ๐ฒ๐ฟ๐ฝ๐ฎ๐ฟ๐ฎ๐บ๐ฒ๐๐ฒ๐ฟ ๐๐๐ป๐ถ๐ป๐ด ๐ผ๐ป ๐ฎ ๐๐ฟ๐ฒ๐ฒ-๐ฏ๐ฎ๐๐ฒ๐ฑ ๐ฒ๐ป๐๐ฒ๐บ๐ฏ๐น๐ฒ ๐บ๐ฒ๐๐ต๐ผ๐ฑ ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐ ๐ฝ๐ฒ๐ฟ๐ณ๐ผ๐ฟ๐บ๐ฎ๐ป๐ฐ๐ฒ ๐บ๐ผ๐ฟ๐ฒ ๐๐ต๐ฎ๐ป ๐ฐ๐ต๐ผ๐ผ๐๐ถ๐ป๐ด ๐๐ต๐ฒ ๐ฏ๐ฒ๐๐ ๐ฎ๐น๐ด๐ผ๐ฟ๐ถ๐๐ต๐บ.
This project also comes with a great resource. This time it comes with a ready-to-use codebase and testbed along with the paper.