Download presentation
Presentation is loading. Please wait.
1
Lightweight Concurrency in GHC KC Sivaramakrishnan Tim Harris Simon Marlow Simon Peyton Jones 1
2
GHC: Concurrency and Parallelism forkIO Bound threads Par Monad MVars STM Safe foreign calls Asynchronous exceptions 2
3
Concurrency landscape in GHC Capability 0 Capability N Haskell Code LWT Scheduler OS Thread pool MVar STM RTS (C Code) Black Holes Black Holes Safe FFI and more… preemptive, round-robin scheduler + work-sharing
4
Idea Haskell Code MVar+ STM+ OS Thread pool Concurrency Substrate RTS (C Code) LWT Scheduler+ Black Holes Black Holes Safe FFI Capability 0 Capability N 4 Capability 0 Capability N Haskell Code LWT Scheduler OS Thread pool MVar STM RTS (C Code) Black Holes Black Holes Safe FFI and more…
5
Capability 0 Capability N Capability 0 Capability N Contributions Haskell Code LWT Scheduler OS Thread pool MVar STM RTS (C Code) Haskell Code MVar+ STM+ OS Thread pool Concurrency Substrate Black Holes Black Holes Safe FFI Black Holes Black Holes Safe FFI and more… What should this be? How to unify these? Where do these live in the new design? 5 LWT Scheduler+ RTS (C Code)
6
Concurrency Substrate One-shot continuations (SCont) and primitive transactional memory (PTM) PTM is a bare-bones TM – Better composability than CAS ---------------- PTM ---------------- data PTM a data PVar a instance Monad PTM atomically :: PTM a -> IO a newPVar :: a -> PTM (PVar a) readPVar :: PVar a -> PTM a writePVar :: PVar a -> a -> PTM () ---------------- SCont -------------- data SCont -- Stack Continuations newSCont :: IO () -> IO SCont switch :: (SCont -> PTM SCont) -> IO () getCurrentSCont :: PTM SCont switchTo :: SCont -> PTM () ---------------- PTM ---------------- data PTM a data PVar a instance Monad PTM atomically :: PTM a -> IO a newPVar :: a -> PTM (PVar a) readPVar :: PVar a -> PTM a writePVar :: PVar a -> a -> PTM () ---------------- SCont -------------- data SCont -- Stack Continuations newSCont :: IO () -> IO SCont switch :: (SCont -> PTM SCont) -> IO () getCurrentSCont :: PTM SCont switchTo :: SCont -> PTM () 6
7
Switch switch :: (SCont -> PTM SCont) -> IO () 7 Current SCont SCont to switch to PTM!
8
Primitive scheduler actions – SCont {scheduleSContAction :: SCont -> PTM (), yieldControlAction :: PTM ()} – Expected from every user-level thread 8 Abstract Scheduler Interface Haskell Code MVar+ STM+ Concurrency Substrate Black Holes Black Holes Safe FFI How to unify these? LWT Scheduler+
9
Primitive Scheduler Actions (1) 9 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> …
![Primitive Scheduler Actions (1) 9 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> …](http://images.slideplayer.com./22/6365596/slides/slide_9.jpg "Primitive Scheduler Actions (1) 9 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> …")
10
Primitive Scheduler Actions (2) 10 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () Substrate Primitives
![Primitive Scheduler Actions (2) 10 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () Substrate Primitives](http://images.slideplayer.com./22/6365596/slides/slide_10.jpg "Primitive Scheduler Actions (2) 10 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () Substrate Primitives")
11
Primitive Scheduler Actions (3) 11 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getSSA = getScheduleSContAction setSSA = setScheduleScontAction getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () getYCA = getYieldControlAction setYCA = setYieldControlAction Substrate Primitives Helper functions
![Primitive Scheduler Actions (3) 11 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getSSA = getScheduleSContAction setSSA = setScheduleScontAction getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () getYCA = getYieldControlAction setYCA = setYieldControlAction Substrate Primitives Helper functions](http://images.slideplayer.com./22/6365596/slides/slide_11.jpg "Primitive Scheduler Actions (3) 11 scheduleSContAction :: SCont -> PTM () scheduleSContAction sc = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched writePVar $ contents ++ [sc] yieldControlAction :: PTM () yieldControlAction = do sched :: PVar [SCont] <- -- get sched contents :: [SCont] <- readPVar sched case contents of x:tail -> do { writePVar $ contents tail; switchTo x -- DOES NOT RETURN } otherwise -> … getScheduleSContAction :: SCont -> PTM (SCont -> PTM()) setScheduleSContAction :: SCont -> (SCont -> PTM()) -> PTM() getSSA = getScheduleSContAction setSSA = setScheduleScontAction getYieldControlAction :: SCont -> PTM (PTM ()) setScheduleSContAction :: SCont -> PTM () -> PTM () getYCA = getYieldControlAction setYCA = setYieldControlAction Substrate Primitives Helper functions")
12
Building Concurrency Primitives (1) 12 yield :: IO () yield = atomically $ do s :: SCont <- getCurrentSCont -- Add current SCont to scheduler ssa :: (SCont -> PTM ()) <- getSSA s enque :: PTM () <- ssa s enque -- Switch to next scont from scheduler switchToNext :: PTM () <- getYCA s switchToNext
13
Building Concurrency Primitives (2) 13 forkIO :: IO () -> IO SCont forkIO f = do ns <- newSCont f atomically $ do { s :: SCont <- getCurrentSCont; -- Initialize new sconts scheduler actions ssa :: (SCont -> PTM ()) <- getSSA s; setSSA ns ssa; yca :: PTM () <- getYCA s; setYCA ns yca; -- Add to new scont current scheduler enqueAct :: PTM () <- ssa ns; enqueAct } return ns
14
Building MVars 14 An MVar is either empty or full and has a single hole newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x', wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x' ts wakeup otherwise -> … atomically $ readPVar hole
![Building MVars 14 An MVar is either empty or full and has a single hole newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole](http://images.slideplayer.com./22/6365596/slides/slide_14.jpg "Building MVars 14 An MVar is either empty or full and has a single hole newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole")
15
Building MVars 15 An MVar is either empty or full and has a single hole Result will be here newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x', wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x' ts wakeup otherwise -> … atomically $ readPVar hole
![Building MVars 15 An MVar is either empty or full and has a single hole Result will be here newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole](http://images.slideplayer.com./22/6365596/slides/slide_15.jpg "Building MVars 15 An MVar is either empty or full and has a single hole Result will be here newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole")
16
Building MVars 16 An MVar is either empty or full and has a single hole Result will be here If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x', wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x' ts wakeup otherwise -> … atomically $ readPVar hole
![Building MVars 16 An MVar is either empty or full and has a single hole Result will be here If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole](http://images.slideplayer.com./22/6365596/slides/slide_16.jpg "Building MVars 16 An MVar is either empty or full and has a single hole Result will be here If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x , wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x ts wakeup otherwise -> … atomically $ readPVar hole")
17
Building MVars 17 An MVar is either empty or full and has a single hole Result will be here Wake up a pending writer, if any. wakeup is a PTM ()! MVar is scheduler agnostic! If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) If the mvar is empty (1)Append hole & wakeup info to mvar list (getSSA!) (2)Yield control to scheduler (getYCA!) newtype MVar a = MVar (PVar (ST a)) data ST a = Full a [(a, PTM())] | Empty [(PVar a, PTM())] takeMVar :: MVar a -> IO a takeMVar (MVar ref) = do hole <- atomically $ newPVar undefined atomically $ do st <- readPVar ref case st of Empty ts -> do s <- getCurrentSCont ssa :: (SCont -> PTM ()) <- getSSA s wakeup :: PTM () <- ssa s writePVar ref $ v where v = Empty $ ts++[(hole, wakeup)] switchToNext <- getYCA s switchToNext Full x ((x', wakeup :: PTM ()):ts) -> do writePVar hole x writePVar ref $ Full x' ts wakeup otherwise -> … atomically $ readPVar hole
18
Interaction of C RTS and User-level scheduler Many “Events” that necessitate actions on the scheduler become apparent only in the C part of the RTS 18 Haskell Code MVar+ STM+ Concurrency Substrate LWT Scheduler+ Safe FFI Black Hole Asynchronous exceptions Finalizers
19
Interaction of C RTS and User-level scheduler Many “Events” that necessitate actions on the scheduler become apparent only in the C part of the RTS 19 Haskell Code MVar+ STM+ Concurrency Substrate LWT Scheduler+ Safe FFI Black Hole Asynchronous exceptions Capability X UT Pending upcall queue :: [PTM ()] Upcall Thread Finalizers Re-use primitive scheduler actions!
![Interaction of C RTS and User-level scheduler Many Events that necessitate actions on the scheduler become apparent only in the C part of the RTS 19 Haskell Code MVar+ STM+ Concurrency Substrate LWT Scheduler+ Safe FFI Black Hole Asynchronous exceptions Capability X UT Pending upcall queue :: [PTM ()] Upcall Thread Finalizers Re-use primitive scheduler actions!](http://images.slideplayer.com./22/6365596/slides/slide_19.jpg "Interaction of C RTS and User-level scheduler Many Events that necessitate actions on the scheduler become apparent only in the C part of the RTS 19 Haskell Code MVar+ STM+ Concurrency Substrate LWT Scheduler+ Safe FFI Black Hole Asynchronous exceptions Capability X UT Pending upcall queue :: [PTM ()] Upcall Thread Finalizers Re-use primitive scheduler actions!")
20
Blackholes 20 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked Thunk evaluating..
21
Blackholes 21 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked BH thunk “blackholed”
22
Blackholes 22 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked BH enters blackhole
23
Blackholes 23 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked BH
24
Blackholes 24 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked BH Yield control action
25
Blackholes 25 T1 T2 T3 Capability 0Capability 1 T T T T T T Running Suspended Blocked V V Schedule SCont action finishes evaluation
26
Blackholes : The Problem 26 T2 BH T T T T T T Running Suspended Blocked T1 Capability 0 Switch $ \T1 -> do -- return T2
27
Blackholes : The Problem 27 T2 BH T T T T T T Running Suspended Blocked T1 Capability 0 Switch $ \T1 -> do -- return T2 enters blackhole In order to make progress, we need to resume to T2 But, in order to resume to T2, we need to resume T2 (Deadlocked!) – Can be resolved through runtime system tricks (Work in Progress!)
28
Conclusions Status – Mostly implemented (SConts, PTM, Simple schedulers, MVars, Safe FFI, bound threads, asynchronous exceptions, finalizers, etc.) – 2X to 3X slower on micro benchmarks (programs only doing synchronization work) To-do – Re-implement Control.Concurrent with LWC – Formal operational semantics – Building real-world programs Open questions – Hierarchical schedulers, Thread priority, load balancing, Fairness, etc. – STM on top of PTM – PTM on top of SpecTM – Integration with par/seq, evaluation strategies, etc. – and more… 28
Similar presentations
