1. EN.540.635 - Software Carpentry
Python – A Crash Course
Esoteric Sections
Parallelization

2. Central Processing Units (CPUs)
Intel i7 7700k
4 Cores
8 Threads
AMD Ryzen X
8 Cores
16 Threads

3. Cores and Threads Computers need to run code
All code is run serially in the Processing Unit of the CPU
With more Processing Units, we can effectively run them in parallel
NOTE! Still in serial on the Processing Unit, but in parallel in the CPU
CPU
Core

4. Cores and Threads
Multithreading is when a single core tries parallelizing code.
Recall that cores cannot truly do this!
Standard practice restricts multithreading to 2 threads per core
CPU
Core

5. Memory
Imagine the following pseudo-code:
What happens?

6. Locks / Mutex
To prevent multiple writes to the same location (leading to corruption), we lock the memory down
A mutex is a fancy word for a lock
How it works:
Code checks if lock exists
If yes, then wait
If no, then lock and continue
NOTE! An “atomic operation” must occur here

7. Locks / Mutex
What’s wrong with the following lock method:
Scope

8. Typical Outline of Parallel Code

9. Typical Outline of Parallel Code

10. Embarrassingly Parallel
What if there is little to no need for different cores to communicate?
Ex. Re-run a probabilistic code N times for an average
Much easier to accomplish!

11. Always Parallel
If embarrassingly parallel code is so easy to setup, why not always use it instead of a regular for loop?
A lot still needs to be done in the background!
Locks/Mutexs
Delegation of tasks to cores
Multithreading