1. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Fortress John Burgess and Richard Chang CS691W University of Massachusetts Amherst

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Overview Sun’s DARPA HPCS project High productivity a goal “Fortress” derived from intent to produce “secure Fortran” Not backwards compatible with Fortran Design decisions motivated by desire to make scientists/programmer’s job as easy as possible Mathematical syntax Implicit/explicit parallelism, handled by libraries Decisions still being made, no compiler currently available

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Design Goals High Performance Operations for parallel execution and distribution of large data structures Architecture-specific library implementations High Productivity Type safety Type inference used to reconstruct types Extensive Libraries Transactional memory Syntax emulates standard mathematical notation Management of APIs/Components

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Overview Fortress Syntax Objects and Traits Variables APIs and Components Parallelism in Fortress

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 “Advances” in Syntax Syntax resembling mathematical notation Dot product (Ascii version): dotProd[\T extends Field\] (left_vec:T[n],right_vec:T[n],foo_bar: T)= Dot product (Unicode version):

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Objects and Traits Traits Like Java interfaces with code, classes without fields Objects Consist of fields and methods Fortress also has top-level functions

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Traits Declare methods Abstract methods have only headers Concrete methods have definitions May extends other Traits Allow new types to be defined

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 Objects Consist of fields and methods Each object has a set of Traits Multiple inheritance of code through traits Inherits concrete methods of its Traits May override inherited methods Must include definition of all abstract methods of its traits

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 List Trait trait List{\T\] extends Object first:()->T rest:()->List[\T\] cons:T->List[\T\] append:List[\T\])->List[\T\] …(* Definitions of Object trait methods*) end

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 The Object Trait Root of trait hierarchy Every object in Fortress has trait Object Every object has implementations of these methods (directly or inherits them from a trait)

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 Empty List Object object Empty traits {List} first() = throw Error rest() = throw Error cons(x) = Cons(x, self) append(xs) = xs end Implements methods of List trait Defines an empty list object Cons is another object that implements cons method for lists

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 Variables in Fortress May be declared to be immutable or mutable Variable declarations may not have explicit type declarations Examples: myList:List[\Integer\] := Empty.cons(13) (* mutable/explicit type declaration*) myList2 := Empty.cons(10) (* mutable/inferred type*) maxValue = 99 (* immutable/inferred type *)

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Components and APIs Components Similar to modules in ML, scheme Export and import APIs APIs Define types in traits, objects, and functions implemented by components

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 Hello World component Hello import print from io export executable run(args) = print “Hello world” end api io print:String -> () end api executable run:(args:String…) -> () end

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Parallelism in Fortress Spawn expressions Transactional Memory Implicit Parallelism Generators Distributions Reductions Recursive Decomposition Matrix unpasting syntax

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Spawn Expressions Used to explicitly run sections of code in new thread executing in parallel Syntax: v = spawn do expressions end Synchronization methods v.val() returns value computed by expressions Will block until v has completed v.wait() Blocks until v completes but doesn’t return value v.ready() Returns true if v has completed, returns false otherwise

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Transactional Memory Synchronization in Fortress done using atomic blocks Code inside block appears to run atomically Multiple threads accessing shared data will have access serialized IO cannot be performed inside atomic block

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Atomic arraySum In Fortress for loops are implicitly parallel atomic modifier serializes updates to sum

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Implicit Parallelism Implicit threads created by Fortress runtime for: Tuple expressions, for loops, summations, etc Implicit barrier at the end of implicitly parallel construct

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Generators Used to express parallel iteration For loops parallel by default: Body of iterations run in separate implicit threads Example generators: 0:9 a.indices() sequential(a.indices())

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Regions Fortress programs execute within set of regions Abstractly describe structure of a machine Organized as a tree Every Fortress object resides in a region Library writers have control over how objects/threads are distributed to regions Programmers perspective: shared global address space Can specify how objects/threads are distributed using distributions

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Tree Abstraction

23. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 Distributions Attempt to separate task of data distribution from ensuring program correctness Architecture/platform specific library will provide default distributions for organizing data produced by generators and allocating and distributing arrays sequential(g) applies local distribution to generator g

24. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 Distributions Cont. Built in distributions: Default - machine/platform specific Sequential - Arrays allocated to one piece of (local) memory Par - Blocked into chucks of size 1 Blocked - Blocked into roughly equal sized chunks Plus others… Library writers can provide other distributions Programmers can specify which distribution array and generators should use: a = blocked.array(xSize, ySize)

25. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 25 Reductions Fortress provides reductions based on those of OpenMP Reductions defined to have identity value arraySum example: Variable sum is reduced Sum assigned to identify of + at beginning of each iteration At end of iterations, original value of sum is combined with final values of sum from each iteration using the reduction (+) in arbitrary order

26. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 26 Recursive Subdivision Recursively divide problems into smaller problems On-the-fly load balancing Computation independent of machine size Generators provided by library do this internally Can be used explicitly by programmers

27. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 27 Recursive Sum Recursively subdivide generator until it is small enough that runtime returns sequential generator

28. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 28 Matrix Unpasting Shorthand for breaking up matrix into parts Useful for recursive subdivision Assume M is a matrix: [topM bottomM ] = M [leftM rightM ] = M Concise, eliminates corner case errors

29. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 29 Matrix Multiplication

30. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 30 Conclusion Fortress philosophy: Use abstractions to hide details of parallel execution as much as possible Extensive libraries to handle data and thread distribution Productivity improvements Mathematical syntax Component/APIs to organize large projects Type Checking