© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 1 Concurrency in Programming Languages Matthew J. Sottile Timothy G. Mattson Craig E Rasmussen

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 2 Chapter 1 Objectives Introduce the topic of concurrency and the current relevance of the topic.

Outline Motivation Where does concurrency appear? Real-world concurrency Timeliness of the topic © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 3

Motivation Concurrency and parallelism have traditionally existed in niche applications. Multicore CPUs and novel architectures (such as GPUs) are bringing the topic to a wider audience. Learning how to deal with widespread and pervasive concurrency is critical for being able to program modern computers. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 4

Motivation Traditional languages and methods for concurrent programming are: –Awkward: explicit thread management –Error prone: easy to create correctness problems –Difficult for compilation: common language features prohibit automatic parallelization (e.g.: pointers) Languages that are concurrency-aware address these issues for programmers. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 5

Traditional areas Operating systems –The most common topic where concurrency is taught and addressed is in the context of operating systems design and implementation. Scientific and high-performance computing –For many decades, the HPC and scientific computing areas were the primary source of work on parallelism. Databases and servers –Server-side applications that needed to handle multiple concurrent users. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 6

Modern areas Any application that is run on a laptop or desktop system with multiple cores can use concurrency for performance gains. –Games –Desktop applications –Development tools –Image and data processing © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 7

Outline Motivation Where does concurrency appear? Real-world concurrency Timeliness of the topic © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 8

Operating systems Managing concurrently operating hardware devices. Multitasking support. Sharing of resources between multiple programs. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 9

Distributed systems Network-based systems. Servers providing services to multiple clients simultaneously. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 10

User interfaces Multiple programs running concurrently in a graphical environment. Applications must support both application logic at the same time as handling user interactions. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 11

Databases Supporting multiple simultaneous accesses to a shared data store. Transactional systems for providing atomic operations. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 12

Scientific computing Parallel computing: implementing algorithms that use many processors in concert to solve a single problem. Commonly focus on very large data sets that cannot fit in a single computer. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 13

Outline Motivation Where does concurrency appear? Real-world concurrency Timeliness of the topic © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 14

Is concurrency really “hard”? Concurrent and parallel computing has a reputation of being difficult or confusing. This is a misconception. –In our daily lives outside computers, we are very used to concurrency. We live in a world where multiple activities are always happening at the same time. –The reputation likely comes from the difficulty of traditional tools for dealing with concurrency in a programming context. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 15

Is concurrency really “hard”? © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 16 “There is therefore nothing new in the basic idea of parallel programming, but only in its application to computers.” S. Gill, 1958

Real world example Everyone has likely cooked a meal at some point. Very often the act of cooking even the simplest of meals involves some degree of concurrency. Let’s consider a simple Italian dish – pasta with sauce. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 17

Steps in cooking a pasta meal © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 18

Cooking a meal Concurrency is present in a simple cooking activity. Parts of the dish are prepared at the same time as others. –Boiling water and cooking pasta can take place while preparing a tomato sauce. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 19

Real-world concurrency We perform many day-to-day activities like cooking, where we “multitask” instead of doing everything in a single sequence of activities. It isn’t intimidating to think about multitasking real- world tasks. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 20

Real-world concurrency The challenge is how to translate this natural way of thinking concurrently into the context of building programs. Can we easily design programs that exploit the same degrees of concurrency as the real world activities that we are comfortable with? © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 21

Outline Motivation Where does concurrency appear? Real-world concurrency Timeliness of the topic © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 22

Concurrency is here Concurrency is unavoidable with modern computers. Nearly all laptops, desktops, and even handheld systems today are based on parallel computing hardware. –Programming them requires addressing issues of concurrency. Concurrency is no longer a niche topic – it touches nearly every area of computing today. © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 23

Approach This book approaches the problem of concurrency at the language level. How does concurrency impact the languages we use to build programs? © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 24

Approach Start with fundamentals: what are the core issues that arise when dealing with concurrency. –Concurrency control –Correctness issues Examine the state of the art Look at history and see how the topic of concurrency has been addressed –Hardware level –Software level © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 25

Approach Look deeper at how concurrency has impacted language development. Consider performance implications of concurrency. Examine design patterns for parallel programming in the context of concurrent programming languages. –In this text, we choose three specific languages to motivate the discussion. OpenMP extensions to C/C++/Fortran The Cilk derivative of C The declarative language Erlang © 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 26