Faculty of Information Engineering, Shenzhen University Liao Huilian SZU TI-DSPs LAB Aug 27, 2007 Optimizer based on particle swarm optimization and LBG (PSO-LBG) — application in vector quantization School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Outline Vector quantization (VQ) LBG LBG Particle swarm optimization (PSO) Optimizer based on PSO and LBG (PSO- LBG) PSO-LBG PSO-LBG 2-dimensional simulation 2-dimensional simulation Performance comparison Performance comparisonConclusionAcknowledgement School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Outline Vector quantization (VQ) LBG LBG Particle swarm optimization (PSO) Optimizer based on PSO and LBG (PSO- LBG) PSO-LBG PSO-LBG 2-dimensional simulation 2-dimensional simulation Performance comparison Performance comparisonConclusionAcknowledgement School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Vector quantization (VQ) School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University LBG LBG, a well-known method of VQ, was proposed by Linde, Buzo and Gray in 1980 Apply two optimality criteria iteratively: Nearest neighbour criterion during assigning training vectors Nearest neighbour criterion during assigning training vectors Centroid criterion during updating codewords (code vectors) Centroid criterion during updating codewords (code vectors)Drawbacks: Local optimization Local optimization Sensitive to the selection of initial codebook Sensitive to the selection of initial codebook School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University LBG School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University LBG School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Outline Vector quantization (VQ) LBG LBG Particle swarm optimization (PSO) Optimizer based on PSO and LBG (PSO- LBG) PSO-LBG PSO-LBG 2-dimensional simulation 2-dimensional simulation Performance comparison Performance comparisonConclusionAcknowledgement School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Particle swarm optimization PSO was proposed by Eberhart and Kennedy in 1995 Advantages: Simplicity of implementation Simplicity of implementation Few parameters Few parameters High convergence rate High convergence rate Population based optimization Remember the best location of itself (Pbest) Remember the best location of itself (Pbest) Remember the best experience in the swarm (Gbest) Remember the best experience in the swarm (Gbest) School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Particle swarm optimization School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Outline Vector quantization (VQ) LBG LBG Particle swarm optimization (PSO) Optimizer based on PSO and LBG (PSO- LBG) PSO-LBG PSO-LBG 2-dimensional simulation 2-dimensional simulation Performance comparison Performance comparisonConclusionAcknowledgement School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University PSO-LBG Based on conventional PSO and LBG algorithms PSO-LBG Structure of particle Structure of particle Particle-pair model Particle-pair model Updating process Updating process Apply in Vector Quantization (VQ) School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Structure of particle School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Updating model School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Updating process PSO-LBG performs three steps at each iteration: Step1: Basic PSO operations Step1: Basic PSO operations Step2: Classical vector quantizer, i.e. LBG algorithm Step2: Classical vector quantizer, i.e. LBG algorithm Step3: Deal with codewords “flying” over the boundary of training vector space Step3: Deal with codewords “flying” over the boundary of training vector space School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Step1 － Basic PSO operations Difference between PSO-LBG and PSO Velocity updating: (additive inertia weight ) Velocity updating: (additive inertia weight ) The parameters, and are much smaller than general PSO-based algorithms The parameters, and are much smaller than general PSO-based algorithms Apply a particle-pair instead of a large number of particles Apply a particle-pair instead of a large number of particles School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Why small parameters? One point larger parameters The solution of PSO-LBG represents N points in the training vector space School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Why just two particles? Three particles consisting of two codewords: P 1 ={ y 1, y 2 }; P 2 ={ y 2, y 1 } and P 3 ={ y 3, y 4 }. P 3 has a poorer performance During the following iterations, particle P 1 and P 2 are comparative The fly direction of particle P 3 is uncertain School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Stable convergence Unstable convergence Stable convergence Unstable convergence Why just two particles? School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Updating steps 2 & 3 Apply LBG with only 3 iterations to avoid converging early Deal with the codewords “flying” over the boundary of search space: Replace this kind of codeword with the training vector that has higher distortion School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Demonstration in 2-dimensional space Three objectives intends to achieve: Three objectives PSO-LBG intends to achieve: Disperse codewords Disperse codewords Move towards global optimum codebook Move towards global optimum codebook Codewords are settled reasonably both in high density regions and low density areas of training vectors space Codewords are settled reasonably both in high density regions and low density areas of training vectors space School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Demonstration in 2-dimensional space Initial codebook = LBG =61.26 PSO-LBG =46.85 School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Performance comparison Performance is evaluated by and PSNR : Mean square error between the training vectors and corresponding nearest codewords : Mean square error between the training vectors and corresponding nearest codewords PSNR: Peak signal to noise ratio PSNR: Peak signal to noise ratio School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Performance comparison Comparison is conducted among: LBG LBG Fuzzy k-means (FKM) Fuzzy k-means (FKM) Fuzzy reinforced learning vector quantization (FRLVQ) Fuzzy reinforced learning vector quantization (FRLVQ) FRLVQ-FVQ: Apply FRLVQ as the pre- process of FVQ FRLVQ-FVQ: Apply FRLVQ as the pre- process of FVQ PSO-LBG PSO-LBG School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Experimental images Lena Pepper Cameraman Kgirl School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University PSNR comparison on Lena School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Convergence comparison on Lena School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Computation time on Lena School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Codebook characteristic on Lena School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University PSNR comparison on pepper School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University PSNR comparison on cameraman School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University PSNR comparison on Kgirl School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Conclusion Experimental results demonstrate that PSO-LBG Outperforms existing algorithms in the field of vector quantization Future work Application in gene clustering Application in gene clustering School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Acknowledgement My supervisor: Prof. Ji Prof. Ji All of you School of Software Engineering, Shenzhen University

Faculty of Information Engineering, Shenzhen University Thank you! School of Software Engineering, Shenzhen University