2. Matrix-Vector Formulation of Backpropagation Learning (1) 2-Layer Case x i w (1) w (2) j = Φ [ W ( 2 ) Φ ( W ( 1 ) x )] f ( 1 ) i w j w k ij jk n f ( 1 ) f ( 2 ) n f ( 2 ) = f h h+1 n m desired actual n+1 e (1) e (2) e = - Let
Then Note :
l (2) L-layer Extension With Boundary Conditions For layer Virtual Virtual
Computational Complexity of (n, h, m) MLP per Learning Cycle in # of Operations for Logistic. Mult. Add. Forward Inner Outer h (n + 1) m (h + 1) hn hm h m Backward e e(2) = * e h ( n + 1) h (m – 1) Total h(2n + 3m + 4) + 5m h(2m + n +1) + 3m h + m For n=5, m=5, h=5, 170 M, 95A, 10 φ
What if φ is non-differentiable ? How many hidden layers are suitable for real time use ? Does the output error become more uncertain in the case of complex multilayer than simple layer ? Should we use only up to 3 layers ? How much does the bias weight affect the overall computation ? Any other algorithm than BP to train the MLP ?