SPMD: Single Program Multiple Data Streams Hui Ren Electrical & Computer Engineering Tufts University

Background This also increased power consumption, and had problems of heat dissipation at high clock speeds. Previously, computing performance was increased through clock speed scaling. Parallel computing allow more instructions to complete in a given time through parallel execution. Now days, parallel computing has entered main stream use, following the introduction of multi-core processors. This diagram show you how optical mammography works Four wavelength lights, (690,750,788,856nm), are modulated electronically around 70 Mhz. The modulated lights are guided and projected to breast by optical fiber bundle. The source and detector located on opposite sides of breast, and will move mechanicly and screening the breast tissue. Due to the diffusive nature of light propagation in breast tissue, the amplitude and phase of received light will change and shift. Prof. Fantini’s group can generate a image of the breast if they know the amplitude and phase data. The photon mulitiplier tube convert the optical singal to electrical signal. The amplitude and phase data, which can be measured from electrical signal . The spectral dependences of blood vessel, cancer and mastopathy are different. That’s why they use four wavelengths lights here. Here are the four images for different wavelenghs lights. The four images are combined to generate a oxygenation-index image, The arrow indicated the position of a 2.5cm invasive ductal carcinoma

What is SPMD? SPMD mode is a method of parallel computing, its processors run the same program, but execute different data. SPMD could get better computing performance through increasing the number of processors.

SPMD operation mechanism The same program code is loaded to all the processors. Data is distributed to each processor The barrier is like a control signal generated by all processors. It could synchronize the execution of processors at some point. This diagram show you how optical mammography works Four wavelength lights, (690,750,788,856nm), are modulated electronically around 70 Mhz. The modulated lights are guided and projected to breast by optical fiber bundle. The source and detector located on opposite sides of breast, and will move mechanicly and screening the breast tissue. Due to the diffusive nature of light propagation in breast tissue, the amplitude and phase of received light will change and shift. Prof. Fantini’s group can generate a image of the breast if they know the amplitude and phase data. The photon mulitiplier tube convert the optical singal to electrical signal. The amplitude and phase data, which can be measured from electrical signal . The spectral dependences of blood vessel, cancer and mastopathy are different. That’s why they use four wavelengths lights here. Here are the four images for different wavelenghs lights. The four images are combined to generate a oxygenation-index image, The arrow indicated the position of a 2.5cm invasive ductal carcinoma

The first example of SPMD: Titanium Titanium is a Java-based language for writing high performance scientific applications on large scale multiprocessors. public static void main(String[] args) { System.out.println("Hello from thread " + Ti.thisProc() ) ; Ti.barrier() ; if (Ti.thisProc() == 0) System.out.println("Done.") ; }

The first example of SPMD: Titanium Data locality: No communication between processors.

The second example of SPMD: MPI MPI is a standard interface for message passing parallel programs written in C, C++, or Fortran. begin program x = 0 z = 2 b = 7 if (rank == 0) then x = x + 1 b=x* 3 send(x) else receive(y) z=b* y (10) endif f = reduce(SUM,z) end program

The second example of SPMD: MPI we can see that the variable y will be assigned the constant value 1 due to the send of x and the corresponding receive into y. SPMD has a local view of execution.

Advantages of SPMD Locality. Data locality is essential to achieving good performance on large-scale machines, where communication across the network is very expensive. Structured Parallelism. The set of threads is fixed throughout computation. It is easier for compilers to reason about SPMD code, resulting in more efficient program analyses than in other models. Simple runtime implementation. SPMD belongs to MIMD, it has a local view of execution and parallelism is exposed directly to the user, compilers and runtime systems require less effort to implement than many other MIMD models.

Disadvantages of SPMD SPMD is a flat model, which makes it dificult to write hierarchical code, such as divide-and-conquer algorithms, as well as programs optimized for hierarchical machines. The second disadvantage may be that it seems hard to get the desired speedup using SPMD.

Expectation The advantages of SPMD are very obvious, and SPMD is still a common use on many large-scale machines. Many scientists have done researches to improve SPMD, such as the recursive SPMD, which provides hierarchical teams. So, SPMD will still be a good method for parallel computing in the future.

References 1. Kamil, A.A., Single Program, Multiple Data Programming for Hierarchical Computations. Dissertation of Philosophy Doctor, University of California, Berkeley, 2012. 2. Wierman, A., L.L.H. Andrew, and A. Tang, Power-aware speed scaling in processor sharing systems: Optimality and robustness. Perform. Eval., 2012. 69(12): p. 601-622. 3. Pais, M.S., K. Yamanaka, and E.R. Pinto, Rigorous Experimental Performance Analysis of Parallel Evolutionary Algorithms on Multicore Platforms. Latin America Transactions, IEEE (Revista IEEE America Latina), 2014. 12(4): p. 805-811. 4. Cremonesi, P. and C. Gennaro, Integrated performance models for SPMD applications and MIMD architectures. Parallel and Distributed Systems, IEEE Transactions on, 2002. 13(12): p. 1320-1332. 5. Cao, J.-J., S.-S. Fan, and X. Yang. SPMD Performance Analysis with Parallel Computing of Matlab. in Intelligent Networks and Intelligent Systems (ICINIS), 2012 Fifth International Conference on. 2012. 6. Lusk, E. MPI in 2002: has it been ten years already? in Cluster Computing, 2002. Proceedings. 2002 IEEE International Conference on. 2002. 7. Strout, M.M., B. Kreaseck, and P.D. Hovland. Data-Flow Analysis for MPI Programs. in Parallel Processing, 2006. ICPP 2006. International Conference on. 2006. 8. Numrich, R.W. and J. Reid, Co-array Fortran for parallel programming. SIGPLAN Fortran Forum, 1998. 17(2): p. 1-31.

Thank you！