1. Parallel Artificial Neural Networks Ian Wesley-Smith Frameworks Division Center for Computation and Technology Louisiana State University http://cct.lsu.edu/~iwsmith

2. Basics of ANNs Vague model of biological neural network No authoritative definition for ANNs –Group of small computational components (neurons) networked together

3. Strengths of ANNs –Inherently Non-Linear –Learn Supervised Input-Output Mapping –Simple enough to implement in hardware Good at: –Optimization problems (traveling salesman) –Pattern classification (handwriting analysis)

4. Examples of ANNs Digital Signal Processing (DSP) Optical Character Recognition (OCR) Sales Forecasting Industrial Process Control SONAR/RADAR Medical Assessment Games

5. Examples of ANNs Robot Army

6. Components of Neurons Inputs –Vector Weights –Matrix [inputs x outputs] Output (activation) Function –Threshold –Sigmoid

7. Example Neuron x1x1 x2x2 xnxn......  f(x) Weights Inputs Output Function A Single Neuron

8. Perceptrons Simplest ANN Single Layer Single Neuron –Can be more Simple pattern classifiers –Only classify linearly separable sets Learning with the delta-rule Output function is threshold

9. Sample Data

10. Perceptrons Computation x1x1 x2x2 xnxn......  f(x) Weights Inputs Output Function A Single Neuron

11. Perceptrons Delta-Learning

12. Methodology Implicit parallelism Neurons are independent of one another Calculations are relatively simple Process large datasets faster Implementation –Serial to Parallel Implementation –PETSc Portable Extensible Toolkit for Scientific Computing –Run Details AMD Dual Opteron 2 Processor Run Varying Sized Data Sets (100-10 million)

13. Error Function

14. Other Types of ANNs ● Multi-layered Perceptron ● Capable of solving more complex problems (XOR) ● Backpropagation Network ● Speech analysis ● Hopfield Network ● Pattern association ● Optimization problems ● Kohonen Feature Map ● Learning most closely related to biological learning

15. Results Parallel ANN was functional Parallel implementation performs slower than serial –This is expected Possible Reasons –Single Neuron Problem –PETSc/MPI Overhead

16. Future Work Implement more advanced (recurrent) networks Hand code MPI instead of relying on PETSc Test in larger environments –32 processors minimum

17. Acknowledgments Yaakoub Y. El-Khamra Dr. Gabrielle Allen Dr. Ed Seidel Kathy Traxler Louisiana State University –Center for Computation and Technology –Computer Science Department