Kriss Jessop About Me Projects Personal Projects RPGs

The No Free Lunch Theorem

Posted on September 29th, 2016

There are two dominant No Free Lunch theorem’s that relate to computing. One is focused towards search and optimisation, while the other is for supervised machine learning.

The Key Point

Eric Cai eloquently describes the No Free Lunch theorem in relation to Machine Learning. He describes the theorem as a series of simplifications and assumptions that apply to a problem and it’s solution. In turn, he also points out that the simplifications and assumptions that work for one problem and it’s solution will not nescessarily work on another problem, thus making the solution ineffective.

The “No Free Lunch” theorem states that there is no one model that works best for every problem

The idea that a solution cannot be picked up and applied to another problem without any work at all is most likely the origin of the name.

The original No Free Lunch Theorems for Optimization paper can be found here, if you’re into that kind of thing.