xCalls: Safe I/O in Memory Transactions Haris Volos, Andres Jaan Tack, Neelam Goyal +, Michael Swift, Adam Welc § University of Wisconsin - Madison + §

CMP leads to more concurrency within programs Synchronizing access to shared data via locks is hard atomic construct simplifies synchronizing access to shared data atomic { x = x - c; y = y + c; } LOCK(L) x = x - c; y = y + c; UNLOCK(L) store(A) store(B) store(A) Transactional Memory (TM) Thread 1 atomic { A = A – 20; B = B + 20; } atomic { B = B – 10; A = A + 10; } Thread 2 Atomicity: PERFORM both updates to A and B or none Isolation: OBSERVE both red transaction’s updates to A and B or none ABORT conflicting transactions Abort blue Conflict 2

A challenging world… Real world programs frequently take actions outside of their own memory – Firefox: ~1% critical sections do system call [Baugh TRANSACT ’07] Most TM systems apply only to user-level memory Thread 1 atomic { item = procItem(queue); write (file, principal); write (file, item->header); } atomic { item = procItem(queue); write (file, principal); write (file, item->header); } memoryfile Thread 2 1a 2b 1a Abort Interleaved writes File writes not dropped Memory updates dropped 3 1a 1b 2a 2b

State of the art Defer Undo Global Lock Ignore failures Stop the world 4 atomic { item = procItem(queue); write (file, principal); write (file, item->header); send (socket, item->body); } LAN Internet NAS COMMIT Perform send Defer

Contribution 5 xCall programming interface – Exposes transactional semantics to programmer – Enables I/O within transactions w/o stopping the world – Exposes all failures to the program Transactional Program Transaction-unaware kernel System call interface xCall Library Legacy calls xCalls Runs in user mode

Outline Motivation xCall Design & Implementation Evaluation Conclusion 6

Design overview Principles 1.As early as possible but not earlier 2.Expose all failures Components – Atomic execution – Isolation – Error handling 7

Atomic execution Provide abort semantics for kernel data and I/O  Expose to programmer when action is performed 8 atomic { item = procItem(queue); x_write (file, principal); x_write (file, item->header); } file Abort buffers Can reverse action?Need result?ExecutionExample NoYesGlobal lock ioctl() No Defer x_write_pipe() Yes--In-place x_write()

Isolation Prevent conflicting changes to kernel data made within a transaction  Sentinels – Revocable user-level locks – Lock logical kernel state visible through system calls Thread 1 atomic { item = procItem(queue); x_write (file, principal); x_write (file, item->header); } atomic { item = procItem(queue); x_write (file, principal); x_write (file, item->header); } memoryfile Thread 2 Conflict 9

atomic { item = procItem(queue); x_write (file, principal, &err1); x_write (file, item->header, &err2); x_send (socket, item->body, &err3); } if (err1 || err2 || err3) { /* CLEANUP */ } Error handling Some errors might not happen until transaction commits or aborts  Inform programmer when failures happen – Handle errors after transaction completes LAN Internet Defer Deferred send: FAILED err3 = error COMMIT Perform send Handle error here 10

ssize_t x_write (int fd, void *buf, ssize_t nbytes ) { void *localbuf; ssize_t bytes; get_sentinel(fd); localbuf = alloc_local_buf(nbytes); read(fd, localbuf, nbytes); bytes = write(fd, buf, nbytes); if (bytes != -1) compensate(x_undo_write, fd, localbuf, bytes, result); } int x_undo_write (int fd, void *buf, ssize_t nbytes, int *result){ off_t ret1, ret2; lseek(fd, -nbytes, SEEK_CUR); if (ret1) pwrite(fd, buf, nbytes, ret1); if (ret1 == -1 || ret2 == -1) *result = errno; return (ret1 == -1 || ret2 == -1); } Example: file write, int *result Atomic execution Isolation Error handling ret2 = 11 ret1 =

Summary xCall API exposes transactional semantics – Atomicity – Isolation – Error handling Prototype implementation – Executes as user-mode library – Relies on Intel STM for transactional memory – Provides 27 xCalls including file handling, communication, threading 12

Outline Motivation xCall Design & Implementation Evaluation – Benefit of xCalls over global lock – Benefit of TM over locks Conclusion 13

Evaluation platform Transactified three large multithreaded apps – Berkeley DB – BIND – XMMS Configurations – Native: locks + system calls – STM: transactions + system calls + global lock – xCalls: transactions + xCalls Run on 4 quad-core 2 GHz AMD Barcelona 14

Performance: Berkeley DB (1/2) 15 xCalls scales better than STM with global lock TM worse than locks due to STM overhead Workload: TPC-C

Performance: Berkeley DB (2/2) 16 xCalls improve concurrency over coarse grain lock Global lock kills optimistic concurrency Workload: Lockscale

Performance: BIND 17 Transactions scale better than coarse grain locks xCalls enable additional concurrency Workload: QueryPerf

Performance summary 18 xCalls benefit programs with I/O concurrency TM benefits programs with contended but not conflicting critical sections

Conclusion xCall programming interface – Brings common OS services to TM programs – Requires no kernel modifications – Improves scalability over state of the art 19 Questions?