1. -Artificial Neural Network- Adaptive Resonance Theory(ART) 朝陽科技大學 資訊管理系 李麗華 教授

2. 朝陽科技大學 李麗華 教授 2 Introduction ART = Adaptive Resonance Theory The ART network is proposed by Grossberg in 1976. There are two commonly used models: - ART1 ： this model takes only binary input - ART2 ： this model takes continuous or binary input

3. 朝陽科技大學 李麗華 教授 3 Introduction (cont.) The ART network features ：  It is a two layered network with the forward & backward process until the message resonate.  ART is a kind of unsupervised learning network.  The input and output layers are connected by the bottom-up weights for competitive learning and by the top-down weights for ouster pattern learning.  When unfamiliar input is fed in, the ART network will dynamically generate the output node for representing the data cluster.

4. 朝陽科技大學 李麗華 教授 4 ART Network Architecture Input layer ： Output layer ： a cluster layer The network starts from only one node and grows until all the input pattern are learned. Connections ： every input node has one bottom-up link to output node and one top-down link to input node. Y1Y1 W 11 b W 11 t W n1 b W n1 t ……… Bottom-up Top-down X 1 X 2 X n ○ ○ ……...… ○ ○ ○ ……….. ○

5. 朝陽科技大學 李麗華 教授 5 ART Process Steps (1/3) 1. Setup network, i.e. the input nodes 。 X 1 ………….. XnXn W 11 b W 1n b W 1n t （ at the very beginning, j=1 ） 6. Calculate “similar value” 5. Find the winning node j* 4. Calculate the “matching value” for every output node j. 3. Input the pattern X 2. Set initial weights, i.e.,

6. 朝陽科技大學 李麗華 教授 6 ART Process Steps (2/3) 7. Do the vigilance test for winning node Case 1 ： if V j ＜ ρ （ vigilance value ）  This means the input pattern does not similar to the connected weights and, hence, it does not belongs to this j* cluster.  Find the next winning output node to see if it can pass the vigilance test, otherwise, generate a new output node. Setup new output node k ： if j = j*, then Y k =1, else Y k =0

7. 朝陽科技大學 李麗華 教授 7 ART Process Steps (3/3) Case2 ： if V j ≧ ρ  This means the input pattern matches to the output node j*. Therefore, the j* node is the cluster for representing this pattern X. ∴ in this case, all we have to do is to update weights. 8. Repeat from step 3 to step 7 for all the input patterns. The network terminates when all the input is fed into ART network.

8. 朝陽科技大學 李麗華 教授 8 Example (1/6) Please find the cluster for the following patterns. 令圖形 X : 用數字 1 表示 O : 用數字 0 表示 X1 X2 X3 X4 X5 X6 Y1 Input Output XOX OXO OXO XOX XXX OOO XXX XXO OXX XXO XOO OXO XXX OXO OO X OXO XOO OOX OXO 1 2 3 4 5 6 7 8 9 10 11 12

9. 朝陽科技大學 李麗華 教授 9 Example (2/6) 1. Let ρ=0.5 …... net 1 = j*=1 3. calculate matching value （匹配值） 2. Input 1 st pattern X 1 = { 1,0,1,0,1,0 } 1  16 t W  ={ 1, 1, 1, 1, 1, 1} 10 t W } 7 1..... 7 1, 7 1, 7 1 { 1  b W

10. 朝陽科技大學 李麗華 教授 10 Example (3/6) ∵ V j * ＞ ρ （ =0.5 ） （通過 vigilance test, 所以引用 case 2 ） update weights 4. Calculate “similar value” 1 3 3 * *      XiXi XiXi W V t ij j

11. 朝陽科技大學 李麗華 教授 11 Example (4/6) Input 2 nd pattern: X= {0,1,0,1,0,1} match value net 1 =0 net j *=net 1 Similar value ∵ V j* ＜ ρ(=0.5) ∴沒有通過 vigilance test, 所以引用 case1 ∵ no other output node, ∴ generate Y 2 &assign new weights W 2 t ={0,1,0,1,0,1} W 2 b ={ }

12. 朝陽科技大學 李麗華 教授 12 Example(5/6) W 1 b ={ } W 1 t = { 1,0,1,0,0,0 } 引用 Case2, so we do the weight updating ＞ ρ （ = 0.5 ） V j *= net2 = match value net1 = input 3 th pattern X = { 1,1,1,0,0,0 } j*=1

13. 朝陽科技大學 李麗華 教授 13 Example(6/6) The final clustering of this example is 8 cluster for 12 input patterns. XOX OXO OXO XOX XXX OOO XXX XXO OXX XXO XOO OXO XXX OXO OO X OXO XOO OOX OXO XOX O O O X OX XO O OXO O OX OXO XO O OXO O OX X X X OXO X X X Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 1 3 2 4 5 7 6 8 9 10 11 12