Software Transactional Memory Yoav Cohen Seminar in Distributed Computing Spring 2007 Yoav Cohen Seminar in Distributed Computing Spring 2007

Agenda  Motivation behind STM  Intro to STM  Static STM Implemented by Shavit and Touitou  Dynamic STM Implemented by Herlihy et al.  Motivation behind STM  Intro to STM  Static STM Implemented by Shavit and Touitou  Dynamic STM Implemented by Herlihy et al.

Motivation  Material covered so far:  Mutual Exclusion (Netanel)  Specifications for concurrent objects (Liza)  Registers for concurrent access (Yaniv)  Replace locks with HTM (Royi and Merav)  What else do we need?!  Material covered so far:  Mutual Exclusion (Netanel)  Specifications for concurrent objects (Liza)  Registers for concurrent access (Yaniv)  Replace locks with HTM (Royi and Merav)  What else do we need?!

HTM Shortcomings  Blocking - Can still deadlock  A thread is killed without releasing a lock  A thread is interrupted with a Page Fault while holding the lock  It’s hardware  Suggested at 1993 (Herlihy and Moss) - Not yet implemented  Blocking - Can still deadlock  A thread is killed without releasing a lock  A thread is interrupted with a Page Fault while holding the lock  It’s hardware  Suggested at 1993 (Herlihy and Moss) - Not yet implemented

STM 101  Same ideas as HTM  A thread executes a transaction  The transaction either commits or aborts  Different from HTM  Non-blocking - The system makes progress  Implemented in software - Available today  Same ideas as HTM  A thread executes a transaction  The transaction either commits or aborts  Different from HTM  Non-blocking - The system makes progress  Implemented in software - Available today

STM 101  A thread wants to change a shared object 1)The thread announces it 2)The thread copies the object to its private memory 3)The thread changes its private copy 4)The thread updates the changes  A thread wants to change a shared object 1)The thread announces it 2)The thread copies the object to its private memory 3)The thread changes its private copy 4)The thread updates the changes

STM 101  Two ways to update an object  Selective update - Update selected locations  Two ways to update an object  Selective update - Update selected locations Local copy Original object

STM 101  Two ways to update an object  Replacing the object all together  Two ways to update an object  Replacing the object all together Local copy Original object

Software Transactional Memory Nir Shavit, Dan Touitou (1995) A non-blocking concurrency framework implemented in software Uses the selective update approach A non-blocking concurrency framework implemented in software Uses the selective update approach

Overview Thread 1 Thread 2

The System Model We assume that every shared memory location supports these 4 operations:  Write i (L,v) - thread i writes v to L  Read i (L,v) - thread i reads v from L  LL i (L,v) - thread i reads v from L and marks that L was read by I  SC i (L,v) - thread i writes v to L and returns success if L is marked as read by i. Otherwise it returns failure. We assume that every shared memory location supports these 4 operations:  Write i (L,v) - thread i writes v to L  Read i (L,v) - thread i reads v from L  LL i (L,v) - thread i reads v from L and marks that L was read by I  SC i (L,v) - thread i writes v to L and returns success if L is marked as read by i. Otherwise it returns failure.

Thread class Rec { boolean stable = false; boolean,int status= (false,0); //can have two values … boolean allWritten = false; int version = 0; int size = 0; int locs[] = {null}; int oldValues[] = {null}; } class Rec { boolean stable = false; boolean,int status= (false,0); //can have two values … boolean allWritten = false; int version = 0; int size = 0; int locs[] = {null}; int oldValues[] = {null}; } Each thread is defined by an instance of a Rec class (short for record). The Rec instance defines the current transaction the thread is executing (only one transaction at a time)

The STM Object Memory Ownerships status version size locs[] oldValues[] Rec 1 status version size locs[] oldValues[] Rec 2 status version size locs[] oldValues[] Rec n This is the shared memory Pointers to threads

Flow of a transaction startTransactionThread i initialize transaction acquire Ownerships agreeOldValues calcNewValues updateMemory release Ownerships release Ownerships isInitiator? ThreadsSTM (Failure,failed loc) FT Initiate helping transaction to failed loc (isInitiator:=F) (Null, 0) Success Failure

The STM Object public class STM { int memory[]; Rec ownerships[]; public boolean, int[] startTranscation(Rec rec, int[] dataSet){...}; private void initialize(Rec rec, int[] dataSet) private void transaction(Rec rec, int version, boolean isInitiator) {...}; private void acquireOwnerships(Rec rec, int version) {...}; private void releaseOwnershipd(Rec rec, int version) {...}; private void agreeOldValues(Rec rec, int version) {...}; private void updateMemory(Rec rec, int version, int[] newvalues) {...}; } public class STM { int memory[]; Rec ownerships[]; public boolean, int[] startTranscation(Rec rec, int[] dataSet){...}; private void initialize(Rec rec, int[] dataSet) private void transaction(Rec rec, int version, boolean isInitiator) {...}; private void acquireOwnerships(Rec rec, int version) {...}; private void releaseOwnershipd(Rec rec, int version) {...}; private void agreeOldValues(Rec rec, int version) {...}; private void updateMemory(Rec rec, int version, int[] newvalues) {...}; }

Implementation public boolean, int[] startTranscation(Rec rec, int[] dataSet) { initialize(rec, dataSet); rec.stable = true; transaction(rec, rec.version, true); rec.stable = false; rec.version++; if (rec.status) return (true, rec.oldValues); else return false; } public boolean, int[] startTranscation(Rec rec, int[] dataSet) { initialize(rec, dataSet); rec.stable = true; transaction(rec, rec.version, true); rec.stable = false; rec.version++; if (rec.status) return (true, rec.oldValues); else return false; } This notifies other threads that I can be helped rec – The thread that executes this transaction. dataSet – The location in memory it needs to own.

Implementation private void transaction(Rec rec, int version, boolean isInitiator) { acquireOwnerships(rec, version); // try to own locations (status, failedLoc) = LL(rec.status); if (status == null) { // success in acquireOwnerships if (versoin != rec.version) return; SC(rec.status, (true,0)); } (status, failedLoc) = LL(rec.status); if (status == true) { // execute the transaction agreeOldValues(rec, version); int[] newVals = calcNewVals(rec.oldvalues); updateMemory(rec, version); releaseOwnerships(rec, version); } else { // failed in acquireOwnerships releaseOwnerships(rec, version); if (isInitiator) { Rec failedTrans = ownerships[failedLoc]; if (failedTrans == null) return; else {// execute the transaction that owns the location you want int failedVer = failedTrans.version; if (failedTrans.stable) transaction(failedTrans, failedVer, false); } private void transaction(Rec rec, int version, boolean isInitiator) { acquireOwnerships(rec, version); // try to own locations (status, failedLoc) = LL(rec.status); if (status == null) { // success in acquireOwnerships if (versoin != rec.version) return; SC(rec.status, (true,0)); } (status, failedLoc) = LL(rec.status); if (status == true) { // execute the transaction agreeOldValues(rec, version); int[] newVals = calcNewVals(rec.oldvalues); updateMemory(rec, version); releaseOwnerships(rec, version); } else { // failed in acquireOwnerships releaseOwnerships(rec, version); if (isInitiator) { Rec failedTrans = ownerships[failedLoc]; if (failedTrans == null) return; else {// execute the transaction that owns the location you want int failedVer = failedTrans.version; if (failedTrans.stable) transaction(failedTrans, failedVer, false); } rec – The thread that executes this transaction. version – Serial number of the transaction. isInitiator – Am I the initiating thread or the helper? Another thread own the locations I need and it hasn’t finished its transaction yet. So I go out and execute its transaction in order to help it.

Implementation private void acquireOwnerships(Rec rec, int version) { for (int j=1; j<=rec.size; j++) { while (true) do { int loc = locs[j]; if LL(rec.status) != null return; // transaction completed by some other thread Rec owner = LL(ownerships[loc]); if (rec.version != version) return; if (owner == rec) break;// location is already mine if (owner == null) {// acquire location if ( SC(rec.status, (null, 0)) ) { if ( SC(ownerships[loc], rec) ) { break; } else {// location is taken by someone else if ( SC(rec.status, (false, j)) ) return; } private void acquireOwnerships(Rec rec, int version) { for (int j=1; j<=rec.size; j++) { while (true) do { int loc = locs[j]; if LL(rec.status) != null return; // transaction completed by some other thread Rec owner = LL(ownerships[loc]); if (rec.version != version) return; if (owner == rec) break;// location is already mine if (owner == null) {// acquire location if ( SC(rec.status, (null, 0)) ) { if ( SC(ownerships[loc], rec) ) { break; } else {// location is taken by someone else if ( SC(rec.status, (false, j)) ) return; } If I’m not the last one to read this field, it means that another thread is trying to execute this transaction. Try to loop until I succeed or until the other thread completes the transaction

Implementation private void agreeOldValues(Rec rec, int version) { for (int j=1; j<=rec.size; j++) { int loc = locs[j]; if ( LL(rec.oldvalues[loc]) != null ) { if (rec.version != version) return; SC(rec.oldvalues[loc], memory[loc]); } private void updateMemory(Rec rec, int version, int[] newvalues) { for (int j=1; j<=rec.size; j++) { int loc = locs[j]; int oldValue = LL(memory[loc]); if (rec.allWritten) return; // work is done if (rec.version != version) return; if (oldValue != newValues[j]) SC(memory[loc], newValues[j]); } if (! LL(rec.allWritten) ) { if (rec.version != version) SC(rec.allWritten, true); } private void agreeOldValues(Rec rec, int version) { for (int j=1; j<=rec.size; j++) { int loc = locs[j]; if ( LL(rec.oldvalues[loc]) != null ) { if (rec.version != version) return; SC(rec.oldvalues[loc], memory[loc]); } private void updateMemory(Rec rec, int version, int[] newvalues) { for (int j=1; j<=rec.size; j++) { int loc = locs[j]; int oldValue = LL(memory[loc]); if (rec.allWritten) return; // work is done if (rec.version != version) return; if (oldValue != newValues[j]) SC(memory[loc], newValues[j]); } if (! LL(rec.allWritten) ) { if (rec.version != version) SC(rec.allWritten, true); } Copy the dataSet to my private space Selectively update the shared memory

Proving the Non-blocking Property  Every failing transaction has a thread which writes Failure to its status field.  Intuition – Let’s show that a situation where the system is stuck can’t happen.  Proof outline – Assume the system is stuck and derive a contradiction.  Every failing transaction has a thread which writes Failure to its status field.  Intuition – Let’s show that a situation where the system is stuck can’t happen.  Proof outline – Assume the system is stuck and derive a contradiction.

Proving the Non-blocking Property  Claim (No proof): Given a failing transaction, in which the failing thread failed to acquire a location A, all threads executing it will never acquire ownership of a location which is higher than the A.

Proving the Non-blocking Property  The system is stuck – There are infinitely many transactions that do not succeed. 1)Number of failing transactions is finite – The other ones are stuck in a loop 2)Number of failing transactions is infinite  The system is stuck – There are infinitely many transactions that do not succeed. 1)Number of failing transactions is finite – The other ones are stuck in a loop 2)Number of failing transactions is infinite

Proving the Non-blocking Property  Number of failing transactions is finite  Other ones are stuck in the loop in acquireOwnerships.  This can only happen if some threads are trying to acquire the same location for the same transaction.  This state can’t be reached (on the board).  A contradiction.  Number of failing transactions is finite  Other ones are stuck in the loop in acquireOwnerships.  This can only happen if some threads are trying to acquire the same location for the same transaction.  This state can’t be reached (on the board).  A contradiction. T1 (Loop) T2 (Fail) T3 (Loop) … Tn (Loop)

Proving the Non-blocking Property  Number of failing transactions is infinite  Since the number of locations is finite, there exists at least one location which is a failing location infinitely often.  Choose A, the highest of those locations.  Intuition – If A is a failing location infinitely often, there are infinitely many transactions who acquired A and failed.  Contradiction – A is not the highest location.  Number of failing transactions is infinite  Since the number of locations is finite, there exists at least one location which is a failing location infinitely often.  Choose A, the highest of those locations.  Intuition – If A is a failing location infinitely often, there are infinitely many transactions who acquired A and failed.  Contradiction – A is not the highest location.

Limitations of STM  Static - Information about transactions is required beforehand:  Size  DataSet  A static software transactional memory is limited only to predefined transactions and data structures.  Static - Information about transactions is required beforehand:  Size  DataSet  A static software transactional memory is limited only to predefined transactions and data structures.

Performance of STM  In stable scenarios, it has a lower throughput than locks.  But, it is non-blocking, hence the system will always progress.  In stable scenarios, it has a lower throughput than locks.  But, it is non-blocking, hence the system will always progress.

Dynamic STM Herlihy, Moir, Luchangco, Scherer (2003) Suited for Dynamic-Sized Data Structures Uses the pointers swap technique Introduces a weaker non-blocking property called obstruction freedom Introduces Contention Managers Suited for Dynamic-Sized Data Structures Uses the pointers swap technique Introduces a weaker non-blocking property called obstruction freedom Introduces Contention Managers

Thread  A thread announces the start of a transaction  A thread executes a series of operations on shared objects  A thread tries to commit the transaction public class TMThread { public void beginTransaction(); public void abortTransaction(); public boolean commitTransaction(); }  A thread announces the start of a transaction  A thread executes a series of operations on shared objects  A thread tries to commit the transaction public class TMThread { public void beginTransaction(); public void abortTransaction(); public boolean commitTransaction(); }

Flow of a transaction TMThread i ThreadsConcurrent Dynamic-Sized Data Structure Shared Object someMethod() Thread.startTransaction() SharedObject.open(WRITE) … SharedObject.release() Thread.commitTransaction() end someMethod() Committing a transaction is done atomically!

A concurrent system View from above TMThread j Threads TMThread k TMThread i TMThread m

Opening a shared object Intuition:  When you open a shared object you get a clone.  You change the clone  When you commit the transaction the clone replaces the original object Intuition:  When you open a shared object you get a clone.  You change the clone  When you commit the transaction the clone replaces the original object

Structure of a shared object Locator transaction oldObject newObject Data TMObject ACTIVE Transaction

Implementation class Transaction { enum Status {ACTIVE, COMMITTED, ABORTED}; Status status = Status.ACTIVE; } class Transaction { enum Status {ACTIVE, COMMITTED, ABORTED}; Status status = Status.ACTIVE; } Initialized to ACTIVE

Implementation class TMObject { // internal classes enum AccessType {WRITE, READ}; class Locator { Transaction transaction; Object newObject; Object oldObject; } // data members Locator locator; // methods public void open(AccessType type) throws DeniedException {}; } class TMObject { // internal classes enum AccessType {WRITE, READ}; class Locator { Transaction transaction; Object newObject; Object oldObject; } // data members Locator locator; // methods public void open(AccessType type) throws DeniedException {}; } Ways to access the object Pointers to the object’s data Access the object

Opening a shared object for Writing public Object open(AccessType type) throws DeniedException { if (type==AccessType.WRITE) { Locator newLocator = new Locator(); newLocator.transaction = TMThread.getCurrentTransaction(); if (locator.transaction.status==Status.ACTIVE) { resolveConflict(); } else { if (locator.transaction.status==Status.COMMITTED) { newLocator.oldObject = locator.newObject; newLocator.newObject = locator.newObject.clone(); } else if (locator.transaction.status==Status.ABORTED) { newLocator.oldObject = locator.oldObject; newLocator.newObject = locator.oldObject.clone(); } validateTransaction(newLocator.transaction) return newLocator.newObject; } public Object open(AccessType type) throws DeniedException { if (type==AccessType.WRITE) { Locator newLocator = new Locator(); newLocator.transaction = TMThread.getCurrentTransaction(); if (locator.transaction.status==Status.ACTIVE) { resolveConflict(); } else { if (locator.transaction.status==Status.COMMITTED) { newLocator.oldObject = locator.newObject; newLocator.newObject = locator.newObject.clone(); } else if (locator.transaction.status==Status.ABORTED) { newLocator.oldObject = locator.oldObject; newLocator.newObject = locator.oldObject.clone(); } validateTransaction(newLocator.transaction) return newLocator.newObject; } Trying to access an already open object Make sure the transaction is still active and its read table is up to date

Opening a shared object for Writing COMMITTED locator transaction oldObject newObject Data newLocator transaction oldObject newObject Data clone() ACTIVE Because the last transaction committed we take its changes

Opening a shared object for Writing ABORTED locator transaction oldObject newObject Data newLocator transaction oldObject newObject Data clone() ACTIVE Because the last transaction aborted we discard its changes

Opening a shared object for Reading  Intuition: Just have to make sure threads read the most updated version  Practice:  We keep a Thread Local table of the objects we opened for read and their latest version  We keep a counter for each object to track number of open and release invocations  We increment the counter when open is called  We decrement the counter when release is called. If counter == 0 we remove the object from the table  Intuition: Just have to make sure threads read the most updated version  Practice:  We keep a Thread Local table of the objects we opened for read and their latest version  We keep a counter for each object to track number of open and release invocations  We increment the counter when open is called  We decrement the counter when release is called. If counter == 0 we remove the object from the table

Opening a shared object for Reading class ReadTable { class ReadTableItem { Object object; int counter; } Map readTable; void insert(int objID, Object objInst) { if ( readTable.containsKey(objID) ) { ReadTableItem item = readTable.get(objID); item.counter++; } else { ReadTableItem newItem = new ReadTableItem(objInst,1); readTable.put(objID, newItem); } void remove(int objID) {... } class ReadTable { class ReadTableItem { Object object; int counter; } Map readTable; void insert(int objID, Object objInst) { if ( readTable.containsKey(objID) ) { ReadTableItem item = readTable.get(objID); item.counter++; } else { ReadTableItem newItem = new ReadTableItem(objInst,1); readTable.put(objID, newItem); } void remove(int objID) {... } The object and its counter This is how we open an object for reading.

Opening a shared object for Reading public class TMThread { ThreadLocal readTable; public void beginTransaction(); public void abortTransaction(); public boolean commitTransaction(); } public class TMThread { ThreadLocal readTable; public void beginTransaction(); public void abortTransaction(); public boolean commitTransaction(); } Each thread has a read table

Committing a transaction Commiting a transaction requires 2 steps: 1.Validating the read table of the thread 2.Using Compare&Swap to change the transaction status from ACTIVE to COMMITED Commiting a transaction requires 2 steps: 1.Validating the read table of the thread 2.Using Compare&Swap to change the transaction status from ACTIVE to COMMITED

Example public class SomeDynamicSizedDataStructure { TMObject data; public boolean insert(Element elem) { TMThread thread = (TMThread) Thread.getCurrentThread(); while (true) { // loop until commited or aborted thread.beginTransaction(); boolean result; try { data.open(WRITE); /* Insert elem to data here */ data.release(); result = true; } catch (DeniedException e) { /* Could not open a shared object */ result = fasle; } if (thread.commitTransaction() == true) return result; } public class SomeDynamicSizedDataStructure { TMObject data; public boolean insert(Element elem) { TMThread thread = (TMThread) Thread.getCurrentThread(); while (true) { // loop until commited or aborted thread.beginTransaction(); boolean result; try { data.open(WRITE); /* Insert elem to data here */ data.release(); result = true; } catch (DeniedException e) { /* Could not open a shared object */ result = fasle; } if (thread.commitTransaction() == true) return result; }

Non-blocking  The DSTM implementation ensures a non- blocking property called obstruction- freedom.  It means that any thread that runs alone for a long enough time makes progress.  Weaker than lockout-freedom of STM  The DSTM implementation ensures a non- blocking property called obstruction- freedom.  It means that any thread that runs alone for a long enough time makes progress.  Weaker than lockout-freedom of STM

Contention Management  Contention Management policy - What does a thread do when it encounters a conflict?  The DSTM implementation has an extension mechanism to allow for different contention management policies.  These extensions are called Contention Managers.  Contention Management policy - What does a thread do when it encounters a conflict?  The DSTM implementation has an extension mechanism to allow for different contention management policies.  These extensions are called Contention Managers.

Contention Managers  Each thread has a reference to a Contention Manager.  Whenever the thread encounters a conflict, it advices with its Contention Manger to decide what to do.  When a conflict is encountered a thread can either abort itself, wait or abort the other transaction.  Each thread has a reference to a Contention Manager.  Whenever the thread encounters a conflict, it advices with its Contention Manger to decide what to do.  When a conflict is encountered a thread can either abort itself, wait or abort the other transaction.

Summary  Pros and Cons of STM  Pros and Cons of DSTM  Discussion – what should the future concurrency framework look like?  Pros and Cons of STM  Pros and Cons of DSTM  Discussion – what should the future concurrency framework look like?