1. Problem Parallelize (serial) applications that use files. In general
Examples: compression tools, logging utilities, databases.
In general
applications that use files depend on sequential output,
serial append is the usual file I/O operation.
Goal:
perform file I/O operations in parallel,
keep the sequential, serial append of the file.
The problem about which I will be talking today is: how to parallelize serial applications that use files for input output. Examples of such files are: compression tools, logging utilities, and databases.
In general the applications that use files are strictly dependent on the sequential output of the the file operations, hence, they are difficult to parallelize. One usual operation that applications using files perform is serial append, namely appending data sequentially to a file.
Our goal is to perform the I/O operations, in particular the append operation, in parallel, while keeping the sequential, serial append of the file.

2. Results Cilk runtime-support for serial append with good scalability.
Three serial append schemes and implementations for Cilk:
ported Cheerio, previous parallel file I/O API (M. Debergalis),
simple prototype (with concurrent Linked Lists),
extension, more efficient data structure (concurrent double- linked Skip Lists).
Parallel bz2 using PLIO.
Results so far include ...

3. Single Processor Serial Append
FILE (serial append)
computation DAG 1 7 8 11 12
In order to make clear what the problem is I will give a short example of serial append on a single processor. This is equivalent with opening the file in append mode on most operating systems.
The file is represented as the gray rectangle, and we have the computation represented by the DAG. Each of the threads writes some bytes to the file. 2 5 6 9 10 3 4

4. Single Processor Serial Append
FILE (serial append) 1 2 3
computation DAG 1 7 8 11 12
Executing the program is equivalent with a depth first search traversal of the DAG. Data is being appended to the file sequentially, in this manner ... 2 5 6 9 10 3 4

5. Single Processor Serial Append
FILE (serial append) 1 2 3 4 5 6 7
computation DAG 1 7 8 11 12
... more data is being appended while execution progresses. 2 5 6 9 10 3 4

6. Single Processor Serial Append
FILE (serial append) 1 2 3 4 5 6 7 8 9 10 11 12
computation DAG 1 7 8 11 12
And the final output is ...
All the operations were sequential. We want all the file write operations to occurs in parallel and still have as final output the same result, roughly (conceptually) the same file. 2 5 6 9 10 3 4

7. Single Processor Serial Append
FILE (serial append) 1 2 3 4 5 6 7 8 9 10 11 12
Why not in parallel?!
computation DAG 1 7 8 11 12
And the final output is ...
All the operations were sequential. We want all the file write operations to occurs in parallel and still have as final output the same result, roughly (conceptually) the same file. 2 5 6 9 10 3 4

8. ParalleL file I/O (PLIO) support
Fast Serial Append
ParalleL file I/O (PLIO) support
for Serial Append in
Cilk
Alexandru Caracaş
My name is ... ad the title of my project and of this presentation is fast serial append, which is a method of performing parallel file I/O. In particular I will discuss this in terms of the Cilk multithreaded programming language.

9. Outline Example Semantics Algorithm Implementation Performance
single processor & multiprocessor
Semantics
view of Cilk Programmer
Algorithm
modification of Cilk runtime system
Implementation
Previous work
Performance
Comparison
We have already seen how appending data to a file works on a single processor.
I will continue with showing an example of what it looks when the a program is executed on multiple processors.
Subsequently I will discuss the semantics of serial append and the view of the Cilk programmer.
I will then show the algorithm idea and the algorithm itself.
Next I will talk about several of the implementations.
And finally I will show some experimental results for the execution time performance of the implementations.

10. Multiprocessor Serial Append
FILE (serial append)
computation DAG 1 7 8 11 12
this shows an example of serial append when executing on three processors: red, green, blue.
The file is again represented by the gray rectangle and the computation, or program by the DAG. 2 5 6 9 10 3 4

11. Multiprocessor Serial Append
FILE (serial append) 1 2 7
computation DAG 1 7 8 11 12
The first processor that starts executing is blue. It executes threads 1 and 2. At some point red comes and steal thread 7.
At this point the serial append file looks like this ...
We continue execution ... 2 5 6 9 10 3 4

12. Multiprocessor Serial Append
FILE (serial append) 1 2 3 5 7 8 9
computation DAG 1 7 8 11 12
After a while green steals thread 5 from blue. Red continues executing threads 8 and 9. And blue executed thread 3.
The serial append file looks like this ...
After some time ... 2 5 6 9 10 3 4

13. Multiprocessor Serial Append
FILE (serial append) 1 2 3 4 5 6 7 8 9 10
computation DAG 1 7 8 11 12
Processors keep executing and the write operations to the serial append file are correctly inserted.
At this point the serial append file looks like this ... 2 5 6 9 10 3 4

14. Multiprocessor Serial Append
FILE (serial append) 1 2 3 4 5 6 7 8 9 10 11 12
computation DAG 1 7 8 11 12
At the end of the computation, we have the same output result as the sequential execution. 2 5 6 9 10 3 4

15. File Operations open (FILE, mode) / close (FILE).
write (FILE, DATA, size)
processor writes to its PION.
read (FILE, BUFFER, size)
processor reads from PION.
Note: a seek operation may be required
seek (FILE, offset, whence)
processor searches for the right PION in the ordered data structure
This shows the API available and the respective actions performed by the Cilk runtime to support the respective function calls.
...

16. Semantics View of Cilk programmer: Write operations
preserve the sequential, serial append.
Read and Seek operations
can occur only after the file has been closed,
or on a newly opened file.
The semantics of serial append, and implicitly that of the semantics for the Cilk programmer are ...

17. Approach (for Cilk) Bookkeeping (to reconstruct serial append)
Divide execution of the computation,
Meta-Data (PIONs) about the execution of the computation.
Observation
In Cilk, steals need to be accounted for during execution.
Theorem
expected # of steals = O ( PT∞ ).
Corollary (see algorithm)
expected # of PIONs = O ( PT∞ ).
To keep the serial append of the file consistent we need to do booking in the forms of Meta-Data about the execution of the computation. One observation is that steals operations in Cilk affect the serial append of the file. Thus, our Meta-Data, which is kept in PIONs accounts for the steal operations.
We now have reduced the problem of maintaining a serial append of a file to properly creating PIONS and maintaining their order.
From class we know that the expected number of steals is ...
As we will see in the algorithm a PION is created on each steal. Hence we have the corollary that the expected number of PIONs that will be created is also ...

18. PION (Parallel I/O Node)
Definition: a PION represents all the write operations to a file performed by a processor in between 2 steals.
A PION contains:
# data bytes written,
victim processor ID,
pointer to written data.
We brake the problem of maintaining the serial append by diving the computation into parallel i/o nodes (PION). We define a PION to represent all the write operations to a file performed by a processor in between 2 steals operations. That is, each time a processor steals it creates a new PION.
A PION contains Meta-Data information: ...
If we look at the file from the previous example we have the following PIONs.
PION π1 π3 π2 π4
FILE 1 2 3 4 5 6 7 8 9 10 11 12

19. Algorithm All PIONSs are kept in an ordered data structure.
very simple Example: Linked List.
On each steal operation performed by processor Pi from processor Pj:
create a new PION πi,
attach πi immediately after πj, the PION of Pj in the order data structure.
you can think of the ordered data structure as being a simple linked list.
PIONs π1 πj πk

20. Algorithm All PIONSs are kept in an ordered data structure.
very simple Example: Linked List.
On each steal operation performed by processor Pi from processor Pj:
create a new PION πi,
attach πi immediately after πj, the PION of Pj in the order data structure. πi
PIONs π1 πj πk

21. Algorithm All PIONSs are kept in an ordered data structure.
very simple Example: Linked List.
On each steal operation performed by processor Pi from processor Pj:
create a new PION πi,
attach πi immediately after πj, the PION of Pj in the order data structure.
PIONs π1 πj πi πk

22. Implementation
Modified the Cilk runtime system to support desired operations.
implemented hooks on the steal operations.
Initial implementation:
concurrent Linked List (easier algorithms).
Final implementation:
concurrent double-linked Skip List.
Ported Cheerio to Cilk 5.4.

23. Details of Implementation
Each processor has a buffer for the data in its own PIONs
implemented as a file.
Data structure to maintain the order of PIONs:
Linked List, Skip List.
Meta-Data (order maintenance structure of PIONs)
kept in memory,
saved to a file when serial append file is closed.

24. Skip List Similar performance with search trees: O ( log (SIZE) ). NIL
This is how the underlying data structure that I mentioned throughout the presentation, and used in the more advanced implementation, looks like. It contains nodes linked through forward pointers, much like a normal linked list. However, every node can have multiple levels, and each such level has forward pointers. In this way a node has pointers that can skip over the nodes that have fewer levels.
This data structure has performance equivalent to search trees, balances or unbalanced, namely logarithmic time in the size of the list for updates, inserts, deletes and search operations.
All the operations have to be done concurrently in the Cilk runtime system!
NIL
NIL

25. Double-Linked Skip List
Based on Skip Lists (logarithmic performance).
Cilk runtime-support in advanced implementation of PLIO as rank order statistics.
NIL
NIL
The version that I am using is a little more complicated and contains among other things backward pointers. Also I do not use any absolute keys.
The structure used in the Cilk runtime is similar to an rank order statistics as described in CLRS. To maintain an order statistics I introduced the backward pointers. Essentially an rank-order statistics can answer the question of seeking to a certain position given its rank, or finding a location within the serial append file given an offset.
Again all the operations on the data structure are done in parallel by the Cilk workers.
NIL
NIL

26. PLIO Performance no I/O vs writing 100MB with PLIO (w/ linked list),
Tests were run on yggdrasil a 32 proc Origin machine.
Parallelism=32,
Legend:
black: no I/O,
red: PLIO.
As test case a simple program with parallelism 16 has been used.
One test included the cilk prohram with all the I/O calls stripped.
The other is the same Cilk program with I/O calls, more specific, it is writing 10MB of data in parallel to a serial append file.
The graph shows theexecution time of the simple program and that of the same program doing I/O.
Tests were conducted on ygg and on a a the falcon theory machine.
yggdrasil is a 32 processors SGI machine and gyrfalcon is a dual PIV processor ...

27. Improvements & Conclusion
Possible Improvements:
Optimization of algorithm:
delete PIONs with no data,
cache oblivious Skip List,
File system support,
Experiment with other order maintenance data structures:
B-Trees.
Conclusion:
Cilk runtime-support for parallel I/O
allows serial applications dependent on sequential output to be parallelized.

28. References
Robert D. Blumofe and Charles E. Leiserson. Scheduling multithreaded computations by work stealing. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science, pages , Santa Fe, New Mexico, November
Matthew S. DeBergalis. A parallel file I/O API for Cilk. Master's thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, June 2000.
William Pugh. Concurrent Maintenance of Skip Lists. Departments of Computer Science, University of Maryland, CS-TR , June, 1990.

29. References
Thomas H. Cormen, Charles E. Leiserson, Donald L. Rivest and Clifford Stein. Introduction to Algorithms (2nd Edition). MIT Press. Cambridge, Massachusetts, 2001.
Supercomputing Technology Group MIT Laboratory for Computer Science. Cilk Reference Manual, November Available at pdf.
bz2 source code. Available at