Parallel Computing

Multiprocessor

There are multiple levels of parallelism:

• Data-Level Parallelism

Multi-threading: https://www.tutorialspoint.com/python/python_multithreading.htm

Difference between Multi-processing vs multi-threading?

• multithreading = execute multiple threads concurrently at a processor level
• multiprocessing = running multiple processors in parallel (each processor can run one or more threads)

Source: towardsdatascience

https://www.youtube.com/watch?v=Dm4up8_zJdo&ab_channel=NeuralNine

Use Joblib to run parallel tasks, I originally saw it from the fedden/poker_ai repository.

• https://joblib.readthedocs.io/en/latest/parallel.html

Parallel computing is a type of computational process where multiple processes are used to handle data at the same time. This usually allows us to compute things much faster as we are able to do many things at the same time instead of handling one task at a time. This is the opposite of serial/concurrent processing.

Source: https://sebastianraschka.com/Articles/2014_multiprocessing.html

Links:

• https://www.machinelearningplus.com/python/parallel-processing-python/
• https://docs.dask.org/en/stable/

SE350

There are multiple types of parallelism:

Medium-Grained Parallelism

• Parallel processing or multitasking within a single application
• Threads usually interact frequently and share data $\to$ requires synchronization
• Scheduling decisions regarding one thread might affect the performance of the entire application

Related

• Daemon