1. ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University http://www.csl.cornell.edu/

2. ISTC-EC @ Cornell The Cornell Team Prof. José Martínez (PI), Prof. Rajit Manohar @ Computer Systems Lab Prof. Tsuhan Chen @ Advanced Multimedia Processing Lab MS/Ph.D. students – Yuan Tian, MS ’13 – Skand Hurkat – Xiaodong Wang

3. ISTC-EC @ Cornell The Cornell Graph

4. ISTC-EC @ Cornell The Cornell Project Provide hardware accelerators for belief propagation algorithms on embedded SoCs (retail/car/home/mobile) – High speed – Very low power – Self-optimizing – Highly programmable BP Accelerator within SoC Graph Inference Algorithm Result

5. ISTC-EC @ Cornell What is belief propagation? Belief propagation is a message passing algorithm for performing inference on graphical models, such as Bayesian networks or Markov Random Fields

6. ISTC-EC @ Cornell What is belief propagation? Labelling problem Energy as a measure of convergence Minimize energy (MAP label estimation) Exact results for trees – Converges in exactly two iterations Approximate results for graphs with loops – Yields “good” results in practice Minimum over large neighbourhoods Close to optimal solution

7. ISTC-EC @ Cornell Not all “that” alien to embedded Remember the Viterbi algorithm? Used extensively in digital communications

8. ISTC-EC @ Cornell What does this mean? Every mobile device uses Viterbi decoders – Error correction codes (eg: turbo codes) – Mitigating inter-symbol interference (ISI) Increasing number of mobile applications involve belief propagation – More general belief propagation accelerators can greatly improve user experience with mobile devices

9. ISTC-EC @ Cornell Target markets Retail/Car/Home/Mobile Image processing – De-noising – Segmentation – Object detection – Gesture recognition Handwriting recognition – Improved recognition through context identification Speech recognition – Hidden Markov models are key to speech recognition Servers Data mining tasks – Part-of-speech tagging – Information retrieval – “Knowledge graph” like applications Machine learning based tasks – Constructive machine learning – Recommendation systems Scientific computing – Protein structure inference

10. ISTC-EC @ Cornell Hardware accelerator for BP BP Accelerator within SoC Graph Inference Algorithm Result

11. ISTC-EC @ Cornell Work done so far

12. ISTC-EC @ Cornell Work done so far

13. ISTC-EC @ Cornell Work done so far

14. ISTC-EC @ Cornell Hierarchical belief propagation

15. ISTC-EC @ Cornell Results – Stereo Matching

16. ISTC-EC @ Cornell Work done so far

17. ISTC-EC @ Cornell Work done so far

18. ISTC-EC @ Cornell GraphGen synthesis of BP-M

19. ISTC-EC @ Cornell Cornell Publications (2013 only) 3x Comp. Vision & Pattern Recognition (CVPR) 3x Asynchronous VLSI (ASYNC) 2x Intl. Symp. Computer Architecture (ISCA) 1x Intl. Conf. Image Processing (ICIP) 1x ASPLOS (w/ GraphGen folks, under review)

20. ISTC-EC @ Cornell Year 3 Plans GraphGen extensions for BP applications – Multiple inference techniques Extraction of “BP ISA” – Ops on arbitrary graphs – Efficient representation Amplification work on UAV ensembles – Self-optimizing, collaborative SoCs One-day “graph” workshop with GraphGen+UIUC

21. ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University http://www.csl.cornell.edu/

22. ISTC-EC @ Cornell Where can it be used? Image processing – Stereo matching – Image segmentation – Identifying objects in context Protein structure inference Almost any algorithm that uses Markov models – Speech recognition using HMM – Handwriting recognition

23. ISTC-EC @ Cornell Graphgen generator for (BP) apps BP Generator Applications Stereo Segmentation etc Algorithms BP-M Hierarchical etc GraphGen Spec Accelerators

24. ISTC-EC @ Cornell The math

25. ISTC-EC @ Cornell The math is not so alien or Remember the Viterbi Algorithm?

26. ISTC-EC @ Cornell The math is not so alien Let or

27. ISTC-EC @ Cornell The math is not so alien The Viterbi algorithm is merely a simpler version of belief propagation!

28. ISTC-EC @ Cornell What does this mean? Servers can also benefit from BP accelerators – Data mining tasks Part-of-speech tagging Information retrieval “Knowledge graph” like applications – Machine learning based tasks Constructive machine learning Recommendation systems – Scientific computing Protein structure inference