Parallel Computing


There are multiple levels of parallelism:

  • Data-Level Parallelism


Difference between Multi-processing vs multi-threading?

  • multithreading = execute multiple threads concurrently at a processor level
  • multiprocessing = running multiple processors in parallel (each processor can run one or more threads)

Source: towardsdatascience

Use Joblib to run parallel tasks, I originally saw it from the fedden/poker_ai repository.

Parallel computing is a type of computational process where multiple processes are used to handle data at the same time. This usually allows us to compute things much faster as we are able to do many things at the same time instead of handling one task at a time. This is the opposite of serial/concurrent processing.




There are multiple types of parallelism:

Medium-Grained Parallelism

  • Parallel processing or multitasking within a single application
  • Threads usually interact frequently and share data requires synchronization
  • Scheduling decisions regarding one thread might affect the performance of the entire application