Horizontal Scaling and Vertical Scaling

While horizontal scaling refers to adding additional nodes, vertical scaling describes adding more power to your current machines. For instance, if your server requires more processing power, vertical scaling would mean upgrading the CPUs. You can also vertically scale the memory, storage, or network speed.

Analogy:

• Think of the CPU vs. GPU race as vertical scaling vs. horizontal scaling. With GPUs, you have hundreds/thousands of cores, each core isn’t as powerful, but because it so much easier to scale horizontal, GPUs combined enable you to do things that can be easily parallelized