Download presentation
Presentation is loading. Please wait.
1
Computational Intelligence
John Sum Institute of Technology Management National Chung Hsing University Taichung, Taiwan ROC
2
Computational Intelligence
OUTLINE Historical Background Computational Intelligence Example Problems Methodology Model Structure Model Parameters Parametric Estimation Discussion Conclusion John Sum Computational Intelligence
3
Computational Intelligence
HISTORY John Sum Computational Intelligence
4
Computational Intelligence
HISTORY 1940 – First computing machine 1957 – Perceptron (First NN model) 1965 – Fuzzy Logic (Rules) 1960s – Genetic Algorithm 1970s – Evolutionary Computing John Sum Computational Intelligence
5
Computational Intelligence
HISTORY 1980s Neural Computing Swarm Intelligence 1990s (Hybrid) Fuzzy Neural Networks NFG, FGN, GNF, etc John Sum Computational Intelligence
6
Computational Intelligence
HISTORY Beyond 1990s: Research areas converge Computational Intelligence Softcomputing Intelligent Systems Covering Adaptive Systems Fuzzy Systems Neural Networks Evolutionary Computing Data Mining CI IS SC AS FS DM EC NN SA PSO GA MCMC SL John Sum Computational Intelligence
7
COMPUTATIONAL INTELLIGENCE
Heuristic algorithms (or models) such as in fuzzy systems, neural networks and evolutionary computation. Techniques that use Simulated annealing, Swarm intelligence, Fractals and Chaos Theory, Artificial immune systems, Wavelets, etc. John Sum Computational Intelligence
8
COMPUTATIONAL INTELLIGENCE
Goal: Problem Solving Financial forecast Customer segmentation (CRM) Supply chain design (SCM) Business process re-engineering System control Pattern recognition Image compression Homeland security John Sum Computational Intelligence
9
COMPUTATIONAL INTELLIGENCE
Underlying structure of the model is unknown, or the model is known but it is too complicated Example: DJI versus HIS (Time Series) Define system structure NL model (NN, ODE, etc.) Rule-based system Parametric estimation Deterministic search (Gradient descent or Newton’s method) Stochastic search (SA or MCMC) John Sum Computational Intelligence
10
COMPUTATIONAL INTELLIGENCE
Underlying model structure is known Example: Manufacturing process (SCM) Define the objective to be maximized Examples: Completion time, Cost, Profit Optimization Linear programming, ILP, NLP Deterministic search (Gradient descent or Newton’s method) Stochastic search (SA or MCMC) John Sum Computational Intelligence
11
EG1: Nonlinear Dynamic System
John Sum Computational Intelligence
12
EG2: Nonlinear Function
John Sum Computational Intelligence
13
Computational Intelligence
EG3: Car Price Predict the price of a car based on Specification of an auto in terms of various characteristics Assigned insurance risk rating Normalized losses in use as compared to other cars Number of attributes: 25 Missing values: Yes! John Sum Computational Intelligence
14
Computational Intelligence
EG3: Car Price John Sum Computational Intelligence
15
EG4: Purchasing Preference
Structural Equation Model Bayesian Network Feedforward Network Fuzzy Logic John Sum Computational Intelligence
16
EG5: Financial Time Series
John Sum Computational Intelligence
17
EG5: Financial Time Series
What would happen in the next trading day? (Time series prediction problem) Closing value Open value UP or DOWN Time series prediction + trading rules What should I do tomorrow? HOLD, SELL or BUY When should I BUY and SELL? John Sum Computational Intelligence
18
Remarks on EG1 ~ EG5 System Structure Data Types Model Dynamic System
Unknown Continuous RNN, Fuzzy NN Nonlinear Function BPN, RBF, Fuzzy NN Car Price Discrete Purchasing Preference Known (SEM) SEM Bayesian Net Financial Time Series John Sum Computational Intelligence
19
COMPUTATIONAL INTELLIGENCE Statement of Problem
Given a set of data collected (or measured) from a system (probably an unknown system), devise a model (by whatever structure, technique, method in CI) that mimics the behavior of that system as ‘good’ as possible. Making use of the devised model to (1) interpret the behavior of the system, (2) predict the future behavior of the system, (3) control the behavior of the system, (4) make money. John Sum Computational Intelligence
20
Computational Intelligence
METHODOLOGY Step 1: Data Collection Experiments or measurements Questionnaire Magazine Public data sets Step 2: Model Structure Assumption IF it is known, SKIP this step. ELSE, DEFINE a model structure John Sum Computational Intelligence
21
Computational Intelligence
METHODOLOGY Step 3: Parametric Estimation Gradient descent Newton’s method Exhaustive search Genetic algorithms (*) Evolutionary algorithms (*) Swarm intelligence Simulated annealing (*) Markov Chain Monte Carlo (*) John Sum Computational Intelligence
22
Computational Intelligence
METHODOLOGY Step 4: Model Validation (is it a reasonable good model) Hypothesis test Validation/Testing set Leave one out validation Step 5: Model Reduction (would there be a simpler model that is also reasonable good) AIC, BIC, MDL Pruning (using testing set) John Sum Computational Intelligence
23
Computational Intelligence
METHODOLOGY Beyond Model Reduction Any redundant input Any redundant sample (or outlier) Any better structure (alternative) How do we determine a ‘good’ model John Sum Computational Intelligence
24
Computational Intelligence
NN MODEL STRUCTURES Perceptron Multilayer Perceptron (MLP or BPN) Adaptive Resonance Theory Model (ART) Competitive Learning (CL) Hopfield Network, Associative Network Bidirectional Associative Model (BAM) Recurrent Neural Network (RNN) Boltzmann Machine Brain-State-In-A-Box (BSB) Radial Basis Function Network (RBF Net) Bayesian Networks Self Organizing Map (SOM or Kohonen Map) Learning Vector Quantization (LVQ) Support Vector Machine (SVM) Support Vector Regression (SVR) PCA, ICA, MCA Winner-Take-All Network (WTA) Spike neural networks Remarks Not all of them is able to learn, eg BSB, WTA Might need to combine two structures to solve a single problem Multiple definitions on the ‘neuron’ John Sum Computational Intelligence
25
Computational Intelligence
NN MODEL STRUCTURES Supply Chain Management (Optimization Problem) Hopfield Network Customer Segmentation (Clustering Problem) CL, SOM, LVQ, ART Dynamic Systems Modeling RNN, Recurrent RBF Car Price/NL Function (Function Approximation) MLP, RBF Net, Bayesian Net, SVR, +SOM/LVQ Financial TS (FA or Time Series Prediction) RNN, SVR, MLP, RBF Net, + SOM/LVQ John Sum Computational Intelligence
26
Computational Intelligence
FUZZY MODEL STRUCTURE John Sum Computational Intelligence
27
Computational Intelligence
FUZZY MODEL STRUCTURE John Sum Computational Intelligence
28
Computational Intelligence
NN MODEL PARAMETERS MLP Input Weights Output Weights Neuron model RNN Recurrent Weights John Sum Computational Intelligence
29
Computational Intelligence
NN MODEL PARAMETERS John Sum Computational Intelligence
30
Computational Intelligence
NN MODEL PARAMETERS John Sum Computational Intelligence
31
Computational Intelligence
NN MODEL PARAMETERS John Sum Computational Intelligence
32
FUZZY MODEL PARAMETERS
John Sum Computational Intelligence
33
PARAMETRIC ESTIMATION
John Sum Computational Intelligence
34
PARAMETRIC ESTIMATION Gradient Descent
John Sum Computational Intelligence
35
PARAMERTIC ESTIMATION Genetic Algorithm
John Sum Computational Intelligence
36
PARAMERTIC ESTIMATION Genetic Algorithm
John Sum Computational Intelligence
37
PARAMERTIC ESTIMATION Genetic Algorithm
John Sum Computational Intelligence
38
Computational Intelligence
DISCUSSIONS CI is not the only method (or structure) to solve a problem. Even it can solve, its performance might not be better than other methods. Should compare with other well-known or existing methods John Sum Computational Intelligence
39
Computational Intelligence
DISCUSSIONS SCM Problem LP, LIP, NLP Lagrangian Relaxation Cutting Plane CPLEX Function Approximation Polynomial Series Trigonometric Series B-Spline John Sum Computational Intelligence
40
Computational Intelligence
CONCLUSIONS IF The problem to be solved has been well formulated The structure has been selected The objective function to evaluation the goodness of a parametric vector has been defined THEN Every problem is just an optimization problem John Sum Computational Intelligence
41
JOHN SUM (pfsum@nchu.edu.tw)
Taiwan HK-Chinese, PhD (98) and MPhil (95) from CUHK, BEng (92) from PolyU HK. Taught in HK Baptist University (98-00), OUHK (00) and PolyU HK (00-04), Chung Shan Medical University (05-07) Adj. Associate Prof., Institute of Software, CAS Beijing (99-02) Short visit: CityU HK, Griffith University in Australia, FAU, Boca Raton FL US, CAS in Beijing, Ching Mai University in Thailand. Assist. Prof., IEC (07-09), Asso. Prof., ITM (09-) NCHU Taiwan 2000 Marquis Who's Who in the World. Senior Member of IEEE, CI Society, SMC Society (05-) GB Member, Asia Pacific Neural Network Assembly (09-) Associate Editor of the IJCA (05-09) Research Interests include NN, FS, SEM, EC, TM John Sum Computational Intelligence
Similar presentations
