Decision Tree

Bootstrap Aggregating (Bagging)

Bagging = bootstrap sampling + aggregation

Bootstrap sampling

The idea is that averaging over $B$ independent datasets reduces variance by factor $B$. But this needs $B$ times more data. So we cheat by reusing parts of the same dataset (sampling with replacement). That is what bootstrap sampling does!