Artificial Intelligence – Deep Learning and its Applications 人工智能 – 深度學習及其應用 Ir Dr F C Chan Past President The Hong Kong Institution of Engineers 10 May 2018

Artificial Intelligence – Deep Learning and its applications AlphaGo Artificial Intelligence Development Deep Learning Features Extraction Applications Conclusion

Vs Lee Sedol AlphaGo Lee AlphaGo won by 4-1 9 and 15 March 2016 李世石 黃士傑 AlphaGo won by 4-1 AlphaGo A combination of neural network and algorithm 9 and 15 March 2016

Vs Ke Jie AlphaGo Master Google hosted in Wuzhen in China 23-27 May 2017 Vs Ke Jie 中國棋王柯潔 AlphaGo won by 3-0 Google has announced that the research team behind AlphaGo would now focus their energy into tackling more complex issues and develop highly advanced algorithms to tackle some of the most complex problems in the world.

AlphaGo Zero Elo ratings - a measure of the relative skill levels of players in competitive games such as Go - show how AlphaGo has become progressively stronger during its development

AlphaGo Zero Vs Deep Blue Deep Blue(IBM) beat Word Chess Champion Garry Kasparov in 1997 Won 3.5 vs 2.5 in a match of 6-game Go Vs Chess Bigger board, 19x19 vs 8x8 Simpler in rules, more move possibilities, 361 vs 28 Longer game, 150 vs 80 moves Alpha Go vs Deep Blue Deep Blue can only play chess Alpha Go is general purpose, can win 49 different arcade games AlphaGo Zero No Built-in expert knowledge Creativity and Intuitive insights Deep Learning can revolutionize everything

AI Development Artificial Intelligence (1950s) – Giving intelligence to machine Machine Learning (1980s) – realizing artificial intelligence (speech recognition, image recognition, playing go, dialogue) Deep Learning (2006) – for machine learning for higher prediction accuracy A powerful class of machine learning model Modern reincarnation of artificial neural network Collection of simple, trainable mathematical functions

AI Development 1958: Rosenblatt’s Perceptron algorithm 1969: Minksky showed Perceptron could not solve the XOR problem, connectedness, parity. 1986: Rumelhart developed Backpropagtion algorithm to train neural network Mid 90’s: Cortes and Vapnik published paper on Support Vector Machines 2006: Hinton and Salakhutdinov proposed using Restricted Blotzmann Machine for pre-train Deep Neural Network

AI Development 2007: Fei-Fei Li’s ImageNet assembling a databse of 14 million labled images (Data drives learning) 2011: Micorsoft explored Speech recognition and IBM’s Watson 2014: Google acquired DeepMind, combing deep learning and reinforcement learning 2016: DeepMind’s AlphaGo defeated world champion Lee Sedol

Deep Learning applications Google: 2011 launched deep-learning focused project. 2014 bought DeepMind with AlphaGo. Microsoft: strong in speech-recognition and translation Facebook: translate user posts in more than 40 languages Baidu: for speech recognition, translation, photo search and self-driving car project

Perceptron Human mindset: Axons send signals to other cells while Dendrites receive them. Perceptron approach

Perceptron Activation Function Network Linear Binary Sigmoid Stochastic Binary

Perceptron Logistic Regression

Handling XOR

Neutral Network Weights are the important elements in Neural Network formulation

Example: 4- layer network with 2 output units Neutral Network Example: 4- layer network with 2 output units

Additional Layers of more Abstract Deep Learning Rules Based System Hand designed Program Input Output Classic Machine Learning Hand designed Features Mapping from Features Input Output Deep Learning Additional Layers of more Abstract Features Mapping from Features Sample Features Input Output

Deep Learning

Deep Learning

Neutral Network Deep Learning The drive behind Deep Learning Nonlinear activation function Neuron could connect to every input Multi-levels with millions neurons Any function can be computed The drive behind Deep Learning Faster machines and core (CPU/GPU) Big data (with large dataset) New models and algorithms Neural Network: pattern recognition or data classification

Deep Learning

Deep Learning Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques Adaptive learning Self Organization Fault Tolerance

Deep Learning Implementation Cloud with Big Data Google, Facebook, Amazon, Baidu, Tencent PC Implementation Linux cluster with GPU servers Python TensorFlow, Theano, Keras

Deep Learning Software

Trend Prediction Recognition New Knowledge Making Sense Artificial Intelligence – Deep Learning and its applications Trend Prediction Recognition New Knowledge Making Sense Replacing Human

Artificial Intelligence – Deep Learning and its applications Information retrieval (search engines) Pattern recognition Audience targeting Sentiment analysis (based on written text) Personalization Automation Natural Language Processing Social media mining Organic search and content performance Brand and product differentiation

Artificial Intelligence – Deep Learning and its applications Language Translation Speech Recognition Generating Handwriting Face Recognition Autonomous Driving Generating Arts Imitating Famous Painters Generating Music Generating Photos

Deep Learning Applications Power Systems Peak Load Forecast (e.g. Maximum Demand) Failure Prediction (e.g. Battery) Condition Monitoring (e.g. Partial Discharge) …..

Deep Learning Applications Data Centres Increase its energy efficiency by 15% 120 different variables (sensors, temperatue gauges, fans, windows…)

Conclusion Artificial Intelligence – Deep Learning will have major impact to our living Specialized applications in various domains

Thank you Artificial Intelligence – Deep Learning and its Applications Ir Dr F C Chan Past President The Hong Kong Institution of Engineers Thank you