Robust Pareto Design of GMDH-type Neural Networks for Systems with Probabilistic Uncertainties N. Nariman-zadeh, F. Kalantary, A. Jamali, F. Ebrahimi Faculty of Engineering, The University of Guilan

System identification techniques are applied in many fields in order to model and predict the behaviors of unknown and/or very complex systems based on given input-output data GMDH is a self-organizing approach by which gradually complicated models are generated based on the evaluation of their performances on a set of multi-input-single-output data In order to obtain more robust models, it is required to consider all the conflicting objectives, namely, training error (TE), prediction error (PE) in the sense of multi-objective Pareto optimization process For multi-objective optimization problems, there is a set of optimal solutions, known as Pareto optimal solutions or Pareto front Introduction

System Identification Techniques Are Applied in Many Fields in order to Model and Predict the Behaviors of Unknown and/or Very Complex Systems Based on Given Input-Output Data. Group Method of Data Handling (GMDH) Algorithm is Self- Organizing Approach by which Gradually Complicated Models are Generated Based on the Evaluation of their Performances on a set of Multi-Input-Single-Output Data Pairs (i=1, 2, …, M) X1X1 X2X2 XnXn Y1Y1.... YmYm Modelling Using GMDH-type Networks

The classical GMDH algorithm can be represented as set of neurons in which different pairs of them in each layer are connected through a quadratic polynomial and thus produce new neurons in the next layer. G1G1 G2G2 G4G4 G6G6 X1X1 X2X2 X3X3 X4X4 A Feedforward GMDH-Type Network G3G3 G5G5 Input Layer Output Layer Hidden Layer(s) Modelling Using GMDH-type Networks

A Generalized GMDH Network Structure of a Chromosome a c b d ad bc adbc a d b c b c b c Application of Genetic Algorithm in the Topology Design of GMDH-type NNs

a c b d ad bc adbc a d b c d d d d Application of Genetic Algorithm in the Topology Design of GMDH-type NNs

Crossover operation for two individuals in GS-GMDH networks

Application of Singular Value Decomposition to the Design of GMDH-type Networks SVD is the method for solving most linear least squares problems that some singularities may exist in the normal equations The SVD of a matrix,, is a factorization of the matrix into the product of three matrices, matrix, diagonal matrix with non-negative elements (Singular Values), and orthogonal matrix such that :

Genetic Algorithms and Multi-objective Pareto Optimization Genetic algorithms are iterative and stochastic optimization techniques. In the optimization of complex real-world problems, there are several objective functions to be optimized simultaneously. There is no single optimal solution as the best because objectives conflict each other. There is a set of optimal solutions, well known as Pareto optimal solutions or Pareto front.

Modelling error Prediction error Multi-objective optimization

Modelling error Prediction error Multi-objective optimization

Difference between robust optimization and traditional optimization Design Variable Objective Function Feasible Infeasible Optimal solution Robust optimal solution

Random variable PDF CDF For the discrete sampling: Stochastic Robust Analysis

Modelling and prediction of soil shear strength, Su, based on 5 input parameters, namely, SPT number (Standard Penetration Test) N′, effective overburden stress s / 0, moisture content percent W, LL liquid limit, and PL plastic limit of fine-graded clay soil The data used in this study were gathered from the National Iranian Geotechnical Database, which has been set up in the Building and Housing Research Centre (BHRC) The database has been established under a mandate from the Management and Planning Organization (MPORG), which supervises the professional activities of all of the consultancy firms in Iran

Comparison of actual values with the evolved GMDH model corresponding to optimum point C (nominal table) Training set Prediction set

PointNetwork’s structureTEPEMean of TEMean of PEVariance of TEVariance of PE A bbaebcacbcaeacee B bcaebacdbcbbadde e113.3e9 C bcaebccdbdbcaccd e102.6e6 Objective functions and structure of networks of different optimum design points

PointNetwork’s structureTEPEMean of TE Mean of PEVariance of TEVariance of PE Abbaebcacbcaeacee Bbcaebacdbcbbadde e113.3e9 Cbcaebccdbdbcaccd e102.6e6 Dabeecddd Objective functions and structure of networks of different optimum design points

Y1 Y4 Y3 Y5 Y2 Point C Point D The structure of network corresponding to point C and D Y1= N ’ σ 0 ’ N ’ σ 0 ’ N ’ σ 0 ’ Y2= w LL w LL w(LL) Y3= Y LL Y (LL) (Y2)(LL) Y4= Y PL Y PL (Y1)(PL) Y5= Y Y Y Y (Y4)(Y3)

Conclusion A multi-objective genetic algorithm was used to optimally design GMDH-type neural networks from a robustness point of view in a probabilistic approach. Multi-objective optimization of robust GMDH models led to the discovering some important trade-off among those objective functions. The framework of this work is very promising and can be generally used in the optimum design of GMDH models in real-world complex systems with probabilistic uncertainties.

Thanks for your attention…