No Free Lunch
- Wolpert and Macready had a 1997 paper called No Free Lunch Theorems
for Optimization.
- This translates to machine learning and states that no algorithm
is better than others on all data sets.
- It actually goes further and states that no algorithm does better
than just guessing on the average data set.
- That includes statistics.
- So why does anyone learn statistics (or other types of machine
learning)?
- The key to all of these is that the learned model actually fits
the underlying data. If it does, then it's good.