What is the black swan paradox?
The black swan paradox represents the event of something unexpected happening. In machine learning, this highlights the key difference between a frequentist and a Bayesian. More explicitly, it emphasizes the fact that a frequentist does not consider the prior in their probability computations whereas a Bayesian does.
The Black Swan Paradox Explained
A hunter wakes up in the morning, looks out the window, and sees a black swan. The hunter is baffled because this has never happened before! By the time he reaches for his gun, the black swan has disappeared. Fascinated by his discovery, he runs into town to tell everyone.
When the hunter explains this rare sighting, everyone begins to laugh at him and question his sanity. They call him crazy and confused because blacks swans do not exist! “They have never been seen before!” one villager said. “It’s impossible!” another exclaimed. The hunter knew no one would ever believe him unless he brought proof. Thus, he went to hunt the swan.
The hunter went out, found the black swan again, killed it, and brought it back to the village. Everyone was in shock because black swans are impossible.
This is the black swan paradox. The takeaway from this story is that we cannot rule out some hypothesis because we have never witnessed it before. This is why Bayesian statistics (in contrast to frequentism) has become so vital in machine learning today.