Horizontal Scaling and Vertical Scaling

Learned formally from SE464.

While horizontal scaling refers to adding additional nodes, vertical scaling describes adding more power to your current machines. For instance, if your server requires more processing power, vertical scaling would mean upgrading the CPUs. You can also vertically scale the memory, storage, or network speed.

Analogy:

• Think of the CPU vs. GPU race as vertical scaling vs. horizontal scaling. With GPUs, you have hundreds/thousands of cores, each core isn’t as powerful, but because it so much easier to scale horizontal, GPUs combined enable you to do things that can be easily parallelized

• Vertical scaling is upgrading your machine(s).

  • The simplest and most efficient way of scaling… but there is a ceiling.

• Horizontal scaling is adding more machines.

  • Coordinating a cluster of machines is complicated, but it’s necessary for global scale and massive throughput.

You can think of GPUs as horizontal scaling.