Parallel Exact Stochastic Simulation in Biochemical Systems Jiangtian Li 7/9/2019 csc801 system research seminar

csc801 system research seminar Introduction Overview of parallel exact stochastic simulation My summer work at Oak Ridge National Lab Joint work with Mike McCollum, Nagiza Samatova One of two research projects this summer 7/9/2019 csc801 system research seminar

csc801 system research seminar Motivation for PESS Model large-scale biochemical systems such as gene regulatory network A single cell cycle of E.coli requires 1014~1016 executions of reactions Accelerate simulation by parallelizing executions while preserving accuracy of stochastic simulation 7/9/2019 csc801 system research seminar

Gillespie’s stochastic simulation Event-driven simulation of random process Modeling spatially uniform mixture of N species (state variables) through M reaction channels (events) Probability of reaction scheduled is characterized by propensity function Simulation ends when species populations converge 7/9/2019 csc801 system research seminar

Simulation Framework Simulation Application Simulation Executive • state variables • code modeling system behavior • I/O and user interface software State variables Species A=1000; Species B=500; Species C=100; … calls to schedule events calls to event handlers Event list (reactions) A + B -> C C + D -> E … Simulation Executive • event list management • managing advances in simulation time Global clock = 3:00 7/9/2019 csc801 system research seminar

Issues in Parallelization Inter-processor synchronization to avoid causality errors 12 17 23 35 affect processed event 14 26 42 unprocessed event 7/9/2019 csc801 system research seminar

Issues in Parallelization Model subdivision and processor mapping Reducing communications Load balancing 7/9/2019 csc801 system research seminar

Inter-processor Synchronization Optimistic synchronization protocol – Time Warp To ensure correct execution order in whole simulation To exploit more parallelism in situation where causality errors don’t occur natural and elegant implementation 7/9/2019 csc801 system research seminar

csc801 system research seminar Time Warp Roll back state variables when a straggler message is received record a log of the modified state variables restore the state variables straggler message 18 processed event 12 A=1 B=2 C=3 21 A=4 35 A=5 C=9 unprocessed event 41 state variables log A=0 B=0 C=0 A=1 A=4 C=3 A=1 A=4 C=3 A=1 B=2 C=3 A=5 B=2 C=9 rollback 7/9/2019 csc801 system research seminar

csc801 system research seminar Time Warp (cont.) Unsend messages use anti-message – logically identical copy of original message, with opposite sign anti-message stored in sender’s output queue and resent when rollback message/anti-message annihilated in receiver’s input queue 7/9/2019 csc801 system research seminar

Example: unsend message Input queue processed event 12 21 35 41 unprocessed event saved state State queue anti-message Anti-message queue 19 42 18 send anti-message Sender Straggler message, rollback 42 Anti-message arrives, rollback Annihilate message and anti-message Input queue 42 27 45 55 State queue Receiver Anti-message queue 33 57 send anti-message 7/9/2019 csc801 system research seminar

csc801 system research seminar Compute G.V.T Global virtual time Defined as minimum among unprocessed and partially processed events and anti-messages Serve as lower bound on time stamp of any future rollback Compute synchronous algorithm handle transient message – message acknowledgement 7/9/2019 csc801 system research seminar

csc801 system research seminar Model subdivision Goal - Efficient parallel execution Reduce communications Reduce rollbacks Maintain load balance Approach - Analyze interdependence among reactions 7/9/2019 csc801 system research seminar

Model subdivision (cont.) 7/9/2019 csc801 system research seminar

Model subdivision (cont.) Approach - Construct graph model vertex – reaction vertex weight – propensity function edge weight – sum of vertices weights 7/9/2019 csc801 system research seminar

Model subdivision (cont.) Partition graph – Objective - minimize messages communicated (cut-edge weights) Constraint – balance executions Metis – graph partition package University of Minnesota 7/9/2019 csc801 system research seminar

csc801 system research seminar Results Tested on SGI Altix (Ram) at ORNL 256 Intel Itanium2 processors running at 1.5 GHz 8 GB of memory per processor 7/9/2019 csc801 system research seminar

csc801 system research seminar Results 7/9/2019 csc801 system research seminar

csc801 system research seminar Results 7/9/2019 csc801 system research seminar

csc801 system research seminar Results 7/9/2019 csc801 system research seminar

csc801 system research seminar Results 7/9/2019 csc801 system research seminar

csc801 system research seminar Future work Dynamic model subdivision during simulation Trade-off – how much is overhead? 7/9/2019 csc801 system research seminar

csc801 system research seminar Conclusion Parallel Exact Stochastic Simulation has demonstrated to be efficient and effective in modeling biochemical systems Proper model subdivision significantly enhanced performance 7/9/2019 csc801 system research seminar

csc801 system research seminar Any question? 7/9/2019 csc801 system research seminar