Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University

Cornell The Cornell Team Prof. José Martínez (PI), Prof. Rajit Computer Systems Lab Prof. Tsuhan Advanced Multimedia Processing Lab MS/Ph.D. students – Yuan Tian, MS ’13 – Skand Hurkat – Xiaodong Wang

Cornell The Cornell Graph

Cornell The Cornell Project Provide hardware accelerators for belief propagation algorithms on embedded SoCs (retail/car/home/mobile) – High speed – Very low power – Self-optimizing – Highly programmable BP Accelerator within SoC Graph Inference Algorithm Result

Cornell What is belief propagation? Belief propagation is a message passing algorithm for performing inference on graphical models, such as Bayesian networks or Markov Random Fields

Cornell What is belief propagation? Labelling problem Energy as a measure of convergence Minimize energy (MAP label estimation) Exact results for trees – Converges in exactly two iterations Approximate results for graphs with loops – Yields “good” results in practice Minimum over large neighbourhoods Close to optimal solution

Cornell Not all “that” alien to embedded Remember the Viterbi algorithm? Used extensively in digital communications

Cornell What does this mean? Every mobile device uses Viterbi decoders – Error correction codes (eg: turbo codes) – Mitigating inter-symbol interference (ISI) Increasing number of mobile applications involve belief propagation – More general belief propagation accelerators can greatly improve user experience with mobile devices

Cornell Target markets Retail/Car/Home/Mobile Image processing – De-noising – Segmentation – Object detection – Gesture recognition Handwriting recognition – Improved recognition through context identification Speech recognition – Hidden Markov models are key to speech recognition Servers Data mining tasks – Part-of-speech tagging – Information retrieval – “Knowledge graph” like applications Machine learning based tasks – Constructive machine learning – Recommendation systems Scientific computing – Protein structure inference

Cornell Hardware accelerator for BP BP Accelerator within SoC Graph Inference Algorithm Result

Cornell Work done so far

Cornell Work done so far

Cornell Work done so far

Cornell Hierarchical belief propagation

Cornell Results – Stereo Matching

Cornell Work done so far

Cornell Work done so far

Cornell GraphGen synthesis of BP-M

Cornell Cornell Publications (2013 only) 3x Comp. Vision & Pattern Recognition (CVPR) 3x Asynchronous VLSI (ASYNC) 2x Intl. Symp. Computer Architecture (ISCA) 1x Intl. Conf. Image Processing (ICIP) 1x ASPLOS (w/ GraphGen folks, under review)

Cornell Year 3 Plans GraphGen extensions for BP applications – Multiple inference techniques Extraction of “BP ISA” – Ops on arbitrary graphs – Efficient representation Amplification work on UAV ensembles – Self-optimizing, collaborative SoCs One-day “graph” workshop with GraphGen+UIUC

Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University

Cornell Where can it be used? Image processing – Stereo matching – Image segmentation – Identifying objects in context Protein structure inference Almost any algorithm that uses Markov models – Speech recognition using HMM – Handwriting recognition

Cornell Graphgen generator for (BP) apps BP Generator Applications Stereo Segmentation etc Algorithms BP-M Hierarchical etc GraphGen Spec Accelerators

Cornell The math

Cornell The math is not so alien or Remember the Viterbi Algorithm?

Cornell The math is not so alien Let or

Cornell The math is not so alien The Viterbi algorithm is merely a simpler version of belief propagation!

Cornell What does this mean? Servers can also benefit from BP accelerators – Data mining tasks Part-of-speech tagging Information retrieval “Knowledge graph” like applications – Machine learning based tasks Constructive machine learning Recommendation systems – Scientific computing Protein structure inference