A Programming Language View of Transactional Memory Hagit Attiya, Technion Joint work with Sandeep Hans, Alexey Gotsman and Noam Rinetzky Published in PODC 2013 & DISC 2014 Supported by MC-ITN Transform and Cost EuroTM

Transactional Memory Consistency The possible histories that a TM can exhibit Equivalent to a history where transactions execute atomically (explanation) Similar to serializability in databases Which transactions to explain, and which to consider in the explanation? Create new stack node nd res = abort while (res == abort) do { res = atomic { nd.next = Top.read() Top = node }

TM Consistency Conditions: Opacity All transactions (including live and aborted ones, unlike serializability) have a single explanation … res = atomic { Z = 1/(X-Y) abort } while (res == abort) do { res = atomic { l = Y X = l X = l-1 } [Guerraoui & Kapalka 08]

TM Consistency Conditions: Opacity All transactions (including live and aborted ones, unlike serializability) have a single explanation Preserve real-time order of transactions wait until G == 1 … while (res == abort) do { res = atomic { … } … while (res == abort) do { res = atomic { … } G = 1 [Guerraoui & Kapalka 08]

TM Consistency Conditions: Opacity All transactions (including live and aborted ones, unlike serializability) have a single explanation Can observe local variables of aborted transactions wait until G ≠ 1 lnd = G while (res == abort) do { nd = Top.read() } res = atomic { nd = Top.read() Top = Top.next abort } G = nd

TM Consistency Conditions: TMS One explanation for all committed transactions BUT Each response has an explanation including a (carefully picked) subset of the transactions Cannot observe local variables of aborted transactions wait until G ≠ 1 lnd = G while (res == abort) do { nd = Top.read() } res = atomic { nd = Top.read() Top = Top.next abort } G = nd [Doherty, Groves, Luchangco, Moir 09]

TM Consistency Conditions: TMS One explanation for all committed transactions BUT Each response has an explanation including a (carefully picked) subset of the transactions Cannot observe local variables of aborted transactions wait until G ≠ 1 lnd = G while (res == abort) do { nd = Top.read() } res = atomic { nd = Top.read() Top = Top.next abort } G = nd [Doherty, Groves, Luchangco, Moir 09] Formally prove that Opacity / TMS allow to program with atomicity in mind (sufficiency)? Is one of them the minimal condition for that (necessity)? Formally prove that Opacity / TMS allow to program with atomicity in mind (sufficiency)? Is one of them the minimal condition for that (necessity)?

The Right Consistency Condition? Must take the program context into account: If local variables are not rolled back after a transaction aborts (e.g., ScalaTM), Opacity is sufficient and necessary If they are rolled back, TMS is sufficient and necessary

The Right Consistency Condition? Must take the program context into account: Local actions: access only local variables Interface actions: interact with the TM Global actions: access global variables (not of the TM) only outside atomic blocks

Histories History: Finite sequence of interface actions Transactional Memory (TM): set of histories – well-formed, prefix-closed TM Atomic : each transaction appears in whole, without interleaving res req

Histories and Traces History: Finite sequence of interface actions Transactional Memory (TM): set of histories – well-formed, prefix-closed Trace: also includes non-transactional (local and global) actions val=9 G=7 val=8 val=3

Trace Equivalence Two traces are observationally equivalent  ~  ’ if threads see the same sequence of local values Possibly except for local values inside aborted transactions (because these are rolled back) TM C observationally refines TM Atomic if every trace  with history in TM C has a trace  ’ ~  with history in TM Atomic [He, Hoare, Sanders 86] val=9 G=7 val=8 val=3

Why Observational Refinement? Prove properties when running with TM Atomic and deduce them also for TM C Traces with history in TM C Traces with history in TM Atomic  ’’ ~ TM C observationally refines TM Atomic if every trace  with history in TM C has a trace  ’ ~  with history in TM Atomic

1 st Main Result TM C is opaque  TM C observationally refines TM Atomic

Every trace  whose history is in TM C has an equivalent trace  ’ whose history is in TM Atomic TM C is opaque  TM C observationally refines TM Atomic

Every trace  whose history is in TM C has an equivalent trace  ’ whose history is in TM Atomic H is opaque  has a “related” sequential history S Get a trace  ’ ~  of TM Atomic whose history is S trace  history H  TM C sequential history S ~ trace  ’

Every trace  whose history is in TM C has an equivalent trace  ’ whose history is in TM Atomic  Permute the actions of  to get  ’, while preserving the order of same-thread and global actions (inductively) trace  history H  TM C sequential history S ~ trace  ’

Permuted prefix of  so it is a prefix of a trace in TM Atomic  Locate , the next interface action in S, and move it  Reordering  relative to earlier actions in  may violate trace equivalence Sufficiency: Inductive Step trace  history H  TM C sequential history ~   

 (action of thread t) is not the beginning of a transaction Only local actions by t before  Move these actions, followed by  followed by actions by other threads Inductive Step: Simpler Case trace  sequential history ~   

 is the beginning of a transaction by thread t No interface actions by t before  Move actions outside transactions, followed by  followed by actions inside transactions (none by t) Inductive Step: The Other Case trace  sequential history ~   

Opacity is Necessary A program P H for every opaque history H Ensure that the real-time order between transactions in every trace of P H agrees with the real-time order of the transactions in H, using global variables Local variables record these order relations TM C is opaque  TM C observationally refines TM Atomic

If Local Variables are Rolled Back? Can only remember that an aborted transaction existed, but not what it read Necessity proof does not work  Adapted notion of trace equivalence  TMS: one explanation for all committed transactions, and possibly different one for each response of a live transaction

T1T1 TMS: Pick a Past T4T4 T5T5 T3T3  T2T2 C A C CP G 2 =2 G 1 =1 G 1 ==1? -For a response  : take its transaction and some transactions that started to commit (possibly pending or even unsuccessful) -If transaction T is included, include also all the committed transactions that precede T

T4T4 T1T1 TMS: Commit the Past T5T5 T3T3  T2T2 C A C -Commit included commit-pending transactions and replace abort with commit for included aborted transactions G 2 =2 G 1 =1 G 1 ==1? CP C

T1T1 TMS: Commit the Past T4T4 T5T5 T3T3  T2T2 C A C CP C -Remove all other transactions -Find an equivalent sequential execution -Preserving the real-time order G 2 =2 G 1 =1 G 1 ==1?

2nd Main Result Under rollback semantics TM C is TMS  TM C observationally refines TM Atomic

Sufficiency of TMS Similar to opacity, but a lot more complicated Construct an equivalent trace for each response TM C is TMS  TM C observationally refines TM Atomic

-Consider the beginning of the last included transaction -For every thread t, take the trace until the latest of: -The last non-transactional action before this point -The last transactional action of t in H’ The Equivalent Trace: Example T1T1 T4T4 T5T5 T3T3  T2T2 C A C CP C G 2 =2 G 1 =1 G 1 ==1?

TMS is Necessary Like opacity, but with a separate program for each response action  H Get a completed possible past that explains the history H (with some of the transaction) Use global variables to force real-time order between transactions

What Else? Extend the results to handle nesting and privatization Weaker observations (e.g., no global variables) are preserved by VWC? Stronger observations (e.g., global accesses inside a transaction) are preserved by deferred-update opacity or TMS2?