MapReduce
So this exists before CUDA times.
Solve Embarrassingly Parallel problems.
Hemal Shah is telling me that this was one of the revolutionary ideas that enabled us to train neural networks at scale. And the stuff that the Google Brain team was doing was super duper cool.