AtomCaml: First-class Atomicity via Rollback Michael F. Ringenburg and Dan Grossman University of Washington International Conference on Functional Programming September 26-28, 2005

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 One New Line This work boils down to one new line in Objective Caml’s Thread module: This work boils down to one new line in Objective Caml’s Thread module: This talk: This talk: Design: Atomic execution and fair scheduling Design: Atomic execution and fair scheduling Implementation: Logging and Rollback Implementation: Logging and Rollback Evaluation: Experience and Benchmarks Evaluation: Experience and Benchmarks external atomic: (unit->’a)->’a = “atomic”

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 OCaml without atomic Shared memory programs in OCaml usually use locks. Shared memory programs in OCaml usually use locks. Typically used to form critical sections: Typically used to form critical sections: let critical lk th = try lock lk; let result = th () in unlock lk; result with e -> (unlock lk; raise e) unlocked output_ptr input_ptr

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 OCaml without atomic let critical lk th = try lock lk; let result = th () in unlock lk; result with e -> (unlock lk; raise e) locked output_ptr input_ptr Shared memory programs in OCaml usually use locks. Shared memory programs in OCaml usually use locks. Typically used to form critical sections: Typically used to form critical sections:

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 OCaml without atomic let critical lk th = try lock lk; let result = th () in unlock lk; result with e -> (unlock lk; raise e) locked output_ptr input_ptr Shared memory programs in OCaml usually use locks. Shared memory programs in OCaml usually use locks. Typically used to form critical sections: Typically used to form critical sections:

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 OCaml without atomic let critical lk th = try lock lk; let result = th () in unlock lk; result with e -> (unlock lk; raise e) unlocked output_ptr input_ptr Shared memory programs in OCaml usually use locks. Shared memory programs in OCaml usually use locks. Typically used to form critical sections: Typically used to form critical sections:

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Some Problems with Locks locked output_ptr input_ptr Race conditions Race conditions A thread fails to acquire the correct lock(s) before accessing a shared object. A thread fails to acquire the correct lock(s) before accessing a shared object. Deadlock Deadlock A thread fails to release a lock. A thread fails to release a lock.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Some Problems with Locks locked output_ptr input_ptr Race conditions Race conditions A thread fails to acquire the correct lock(s) before accessing a shared object. A thread fails to acquire the correct lock(s) before accessing a shared object. Deadlock Deadlock A thread fails to release a lock. A thread fails to release a lock.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 AtomCaml: OCaml w/ atomic output_ptr input_ptr atomic’s argument executes as a single step, without any interleaving. atomic’s argument executes as a single step, without any interleaving. No race conditions. No race conditions. No deadlock. No deadlock. let critical lk th = Thread.atomic th

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 AtomCaml: OCaml w/ atomic output_ptr input_ptr let critical th = Thread.atomic th atomic’s argument executes as a single step, without any interleaving. atomic’s argument executes as a single step, without any interleaving. No race conditions. No race conditions. No deadlock. No deadlock.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 AtomCaml: OCaml w/ atomic output_ptr input_ptr let critical th = Thread.atomic th atomic’s argument executes as a single step, without any interleaving. atomic’s argument executes as a single step, without any interleaving. No race conditions. No race conditions. No deadlock. No deadlock.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Outline Design of AtomCaml Design of AtomCaml Semantics of atomic and exceptions Semantics of atomic and exceptions The yield_r primitive The yield_r primitive Implementation Implementation Logging, Rollback, and Closures Logging, Rollback, and Closures Takes advantage of OCaml's uniprocessor restriction. Takes advantage of OCaml's uniprocessor restriction. Evaluation Evaluation Related and future work Related and future work

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Design: The atomic Primitive Takes a thunked block of code (the “atomic block”), and executes it with no interleaving. Takes a thunked block of code (the “atomic block”), and executes it with no interleaving. The atomic block may contain almost any valid Caml code, including: The atomic block may contain almost any valid Caml code, including: Function calls to external C code Function calls to external C code Exceptions Exceptions Buffered output Buffered output

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Design: Exceptions Two choices if an exception escapes an atomic block: keep the block’s effects or discard its effects. Two choices if an exception escapes an atomic block: keep the block’s effects or discard its effects. We chose to keep effects: We chose to keep effects: Exceptions are merely non-local control transfer. When control leaves atomic block, it commits. Exceptions are merely non-local control transfer. When control leaves atomic block, it commits. Atomic blocks should behave like sequential code. Atomic blocks should behave like sequential code. Exception may depend on effects. Exception may depend on effects. The alternative may cause unexpected errors: The alternative may cause unexpected errors: let x = ref 0 atomic (fun () -> x := 1; f () (* may raise exception *)) if !x = 0 then failwith “huh” else...

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Design: The yield_r Primitive atomic (fun () -> while (is_empty bbuf) do yield_r (bbuf.input_ptr) done;... (* remove item *)) --- output_ptr input_ptr Often wish to wait for a specific condition to hold before attempting a critical section. Often wish to wait for a specific condition to hold before attempting a critical section. E.g., wait until the buffer is non-empty before removing an item. E.g., wait until the buffer is non-empty before removing an item. Like guards and conditional critical regions Like guards and conditional critical regions We use the yield_r primitive to achieve a similar effect (see paper). We use the yield_r primitive to achieve a similar effect (see paper).

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Design: The yield_r Primitive output_ptr input_ptr atomic (fun () -> while (is_empty bbuf) do yield_r (bbuf.input_ptr) done;... (* remove item *)) Often wish to wait for a specific condition to hold before attempting a critical section. Often wish to wait for a specific condition to hold before attempting a critical section. E.g., wait until the buffer is non-empty before removing an item. E.g., wait until the buffer is non-empty before removing an item. Like guards and conditional critical regions Like guards and conditional critical regions We use the yield_r primitive to achieve a similar effect (see paper). We use the yield_r primitive to achieve a similar effect (see paper).

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Design: The yield_r Primitive --- output_ptr input_ptr atomic (fun () -> while (is_empty bbuf) do yield_r (bbuf.input_ptr) done;... (* remove item *)) Often wish to wait for a specific condition to hold before attempting a critical section. Often wish to wait for a specific condition to hold before attempting a critical section. E.g., wait until the buffer is non-empty before removing an item. E.g., wait until the buffer is non-empty before removing an item. Like guards and conditional critical regions Like guards and conditional critical regions We use the yield_r primitive to achieve a similar effect (see paper). We use the yield_r primitive to achieve a similar effect (see paper).

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Outline Design of AtomCaml Design of AtomCaml Semantics of atomic and exceptions Semantics of atomic and exceptions The yield_r primitive The yield_r primitive Implementation Implementation Logging, Rollback, and Closures Logging, Rollback, and Closures Takes advantage of OCaml's uniprocessor restriction. Takes advantage of OCaml's uniprocessor restriction. Evaluation Evaluation Related and future work Related and future work

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Overview Atomic blocks Atomic blocks If a thread executing an atomic block is preempted, rollback and retry later If a thread executing an atomic block is preempted, rollback and retry later Otherwise, the block executed with no interleaving (OCaml runs as a uniprocessor) Otherwise, the block executed with no interleaving (OCaml runs as a uniprocessor) Scheduling Scheduling Allow preemption, so fairness is ensured Allow preemption, so fairness is ensured Allocate extra time to threads with long atomic blocks, and skip rounds to maintain fairness Allocate extra time to threads with long atomic blocks, and skip rounds to maintain fairness

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Assumptions Most atomic blocks will be short, so rollbacks will be rare. Most atomic blocks will be short, so rollbacks will be rare. Optimize for the common case (no rollback) Optimize for the common case (no rollback) Only a small percentage of code will execute inside atomic blocks. Only a small percentage of code will execute inside atomic blocks. Add as little overhead as possible to non- atomic code Add as little overhead as possible to non- atomic code

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback Two choices for reversing effects: Two choices for reversing effects: Record effects in an undo log and reverse on rollback Record effects in an undo log and reverse on rollback Store effects in a commit log and execute on completion. Store effects in a commit log and execute on completion. Memory writes: undo log Memory writes: undo log Output: commit log (buffer) Output: commit log (buffer) 7338 xLog x := 6337

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback Memory writes: undo log Memory writes: undo log Makes reads in atomic faster - can execute directly, rather than checking log. Makes reads in atomic faster - can execute directly, rather than checking log. Only need to log writes to mutable variables Only need to log writes to mutable variables No need to log outside of atomic blocks No need to log outside of atomic blocks Output: commit log (buffer) Output: commit log (buffer) Can’t “reverse” already-viewed writes Can’t “reverse” already-viewed writes 7338 xLog x := 6337

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback 6337 xLog x := 4242 (&x,7338) Memory writes: undo log Memory writes: undo log Makes reads in atomic faster - can execute directly, rather than checking log. Makes reads in atomic faster - can execute directly, rather than checking log. Only need to log writes to mutable variables Only need to log writes to mutable variables No need to log outside of atomic blocks No need to log outside of atomic blocks Output: commit log (buffer) Output: commit log (buffer) Can’t “reverse” already-viewed writes Can’t “reverse” already-viewed writes

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback 4242 xLog Rollback (&x,7338) (&x,6337) Memory writes: undo log Memory writes: undo log Makes reads in atomic faster - can execute directly, rather than checking log. Makes reads in atomic faster - can execute directly, rather than checking log. Only need to log writes to mutable variables Only need to log writes to mutable variables No need to log outside of atomic blocks No need to log outside of atomic blocks Output: commit log (buffer) Output: commit log (buffer) Can’t “reverse” already-viewed writes Can’t “reverse” already-viewed writes

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback 6337 xLog Rollback (&x,7338) Memory writes: undo log Memory writes: undo log Makes reads in atomic faster - can execute directly, rather than checking log. Makes reads in atomic faster - can execute directly, rather than checking log. Only need to log writes to mutable variables Only need to log writes to mutable variables No need to log outside of atomic blocks No need to log outside of atomic blocks Output: commit log (buffer) Output: commit log (buffer) Can’t “reverse” already-viewed writes Can’t “reverse” already-viewed writes

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Rollback 7338 xLog Rollback Memory writes: undo log Memory writes: undo log Makes reads in atomic faster - can execute directly, rather than checking log. Makes reads in atomic faster - can execute directly, rather than checking log. Only need to log writes to mutable variables Only need to log writes to mutable variables No need to log outside of atomic blocks No need to log outside of atomic blocks Output: commit log (buffer) Output: commit log (buffer) Can’t “reverse” already-viewed writes Can’t “reverse” already-viewed writes

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Implementation: Functions Functions called in atomic blocks must log and buffer. We create two versions of each function: atomic (logs and buffers) and non-atomic. Closures contain pointers to both. We know at compile-time which pointer to follow, so we just insert the appropriate byte code. Atomic Code Pointer Free Variable Atomic Function Non-atomic Code Pointer Non-atomic Function

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Calling External C Functions external foo: type = “c_foo” The programmer must tell AtomCaml how to handle external calls in an atomic block. The programmer must tell AtomCaml how to handle external calls in an atomic block. If the function can be called safely as is (e.g., it only modifies local state): If the function can be called safely as is (e.g., it only modifies local state): If the programmer provides a separate, atomic- safe version: If the programmer provides a separate, atomic- safe version: If a function should never be called in an atomic: If a function should never be called in an atomic: external foo: type = “c_foo_non c_foo_atom” external foo: type = “c_foo raise_on_atomic”

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Calling External C Functions To construct atomic-safe versions of C functions, programmers must be able to: To construct atomic-safe versions of C functions, programmers must be able to: Specify actions that should occur if the block is rolled back (e.g., reversing write): Specify actions that should occur if the block is rolled back (e.g., reversing write): Specify actions that should be delayed until the block commits (e.g., output): Specify actions that should be delayed until the block commits (e.g., output): caml_register_rollback_action(func, env) caml_register_commit_action(func, env)

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Outline Design of AtomCaml Design of AtomCaml Semantics of atomic and exceptions Semantics of atomic and exceptions The yield_r primitive The yield_r primitive Implementation Implementation Logging, Rollback, and Closures Logging, Rollback, and Closures Takes advantage of OCaml's uniprocessor restriction. Takes advantage of OCaml's uniprocessor restriction. Evaluation Evaluation Related and future work Related and future work

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Evaluation: PLANet Active networking system written in OCaml. We ported it to AtomCaml (we replaced all locks). Active networking system written in OCaml. We ported it to AtomCaml (we replaced all locks). No significant structural changes were required. No significant structural changes were required. Had to create atomic safe versions of three C functions: Had to create atomic safe versions of three C functions: Two output functions (we buffered) Two output functions (we buffered) One killed a thread (we delayed) One killed a thread (we delayed) Moved three other C calls out of atomic Moved three other C calls out of atomic Converting to atomic fixed (at least) three concurrency bugs, without any effort. Converting to atomic fixed (at least) three concurrency bugs, without any effort.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Evaluation: Benchmarks BenchmarkOCamlAtomCamlOverhead Web Cache sec sec 0.8% PLANet ping sec sec 4.2% PLANet send sec sec (-9.4%) PLANet receive sec sec 7.0% Overhead of non-atomic, single-threaded benchmarks ≈ 2% Overhead of non-atomic, single-threaded benchmarks ≈ 2% Alternate closures (see paper) sped up sequential code, but slowed down concurrent code. Alternate closures (see paper) sped up sequential code, but slowed down concurrent code. Overhead of real concurrent applications: Overhead of real concurrent applications:

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Outline Design of AtomCaml Design of AtomCaml Semantics of atomic and exceptions Semantics of atomic and exceptions The yield_r primitive The yield_r primitive Implementation Implementation Logging, Rollback, and Closures Logging, Rollback, and Closures Takes advantage of OCaml's uniprocessor restriction. Takes advantage of OCaml's uniprocessor restriction. Evaluation Evaluation Related and future work Related and future work

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Related Work Harris et al.’s Haskell with transactions Harris et al.’s Haskell with transactions Uses an STM monad to provide atomicity Uses an STM monad to provide atomicity Also provides sequential and alternative composition Also provides sequential and alternative composition Many other atomicity implementations, e.g., Many other atomicity implementations, e.g., Harris and Fraser’s Java transactions Harris and Fraser’s Java transactions Manson et al.’s real-time Java Manson et al.’s real-time Java Concurrent ML Concurrent ML Asynchronous message-passing style concurrency Asynchronous message-passing style concurrency OCaml CML uses a master lock, which we replaced with atomic. OCaml CML uses a master lock, which we replaced with atomic.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Future Work More efficient logging and rollback More efficient logging and rollback Only generate dual versions of functions that might be called in atomic blocks Only generate dual versions of functions that might be called in atomic blocks May eliminate as much as 90% of code bloat May eliminate as much as 90% of code bloat Multiprocessor support: Already working on AtomJava Multiprocessor support: Already working on AtomJava

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Conclusions We designed, implemented an evaluated AtomCaml. We designed, implemented an evaluated AtomCaml. Extension to Objective Caml that supports atomic critical sections Extension to Objective Caml that supports atomic critical sections Using a logging-and-rollback approach to guarantee atomicity Using a logging-and-rollback approach to guarantee atomicity Ensures fair scheduling by allowing preemption Ensures fair scheduling by allowing preemption Low overhead on real applications Low overhead on real applications Atomicity offers a higher-level, stronger, easier- to-reason about synchronization guarantee than locks. Atomicity offers a higher-level, stronger, easier- to-reason about synchronization guarantee than locks.

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Evaluation: Benchmarks Logging and rollback overhead: Logging and rollback overhead: Locks vs. Atomic: Locks vs. Atomic:BlockLocksAtomicOverhead 0 Writes sec sec 115% 5 Writes sec sec 108% 10 Writes sec sec 82%

September 26, 2005 AtomCaml: First-Class Atomicity via Rollback ICFP 2005 Evaluation: Benchmarks Overhead of non-atomic code: Overhead of non-atomic code: BenchmarkOCamlAtomCamlOverhead Compile sec sec 2.7% Compile sec sec 1.6% CDS Interpret sec sec 1.3%