EN.540.635 - Software Carpentry Python – A Crash Course Esoteric Sections Parallelization https://lammps.sandia.gov/movies.html

Central Processing Units (CPUs) Intel i7 7700k ----------------------- 4 Cores 8 Threads AMD Ryzen 7 1800X ----------------------- 8 Cores 16 Threads

Cores and Threads Computers need to run code All code is run serially in the Processing Unit of the CPU With more Processing Units, we can effectively run them in parallel NOTE! Still in serial on the Processing Unit, but in parallel in the CPU CPU Core

Cores and Threads Multithreading is when a single core tries parallelizing code. Recall that cores cannot truly do this! Standard practice restricts multithreading to 2 threads per core CPU Core

Memory Imagine the following pseudo-code: What happens?

Locks / Mutex To prevent multiple writes to the same location (leading to corruption), we lock the memory down A mutex is a fancy word for a lock How it works: Code checks if lock exists If yes, then wait If no, then lock and continue NOTE! An “atomic operation” must occur here

Locks / Mutex What’s wrong with the following lock method: Scope

Typical Outline of Parallel Code

Typical Outline of Parallel Code

Embarrassingly Parallel What if there is little to no need for different cores to communicate? Ex. Re-run a probabilistic code N times for an average Much easier to accomplish!

Always Parallel If embarrassingly parallel code is so easy to setup, why not always use it instead of a regular for loop? A lot still needs to be done in the background! Locks/Mutexs Delegation of tasks to cores Multithreading