Interactive E-Learning System Using Pattern Recognition and Augmented Reality Author : Sang Hwa Lee, Junyeong Choi, and Jong-Il Park Consumer Electronics, IEEE Transactions on Volume 55, Issue 2, May 2009 Page(s):883 - 890 Speaker: Po-Kai Shen Advisor: Tsai-Rong Chang Student ID: M98G0214 Date: 2010/4/13

Outline Introduction Overview of proposed e-learning system Design of interactive makers Recognition of image and object Application to public education system Conclusions

Introduction The students study their textbooks with auxiliary audiovisual contents which are played on the personal computer and specific terminals. Goal is to design a mentoring system for self- studying, which lets the students learn the audio-visual contents interactively. Design colorband and polka-dot patterns for object-based user interaction.

Overview of proposed e-learning system Fig. 1.

Design of interactive makers Polka-dot Pattern Recognition Usually, the hexagonal array patterns are robust for perspective distortion caused by different camera viewpoints Fig. 2.

Design of interactive makers Polka-dot Pattern Recognition High pass filter The candidate position is first searched in the coarse grid in the original image. When a position satisfies the conditions of polka-dot marker by (1) and (2)

Design of interactive makers Fig. 3. (a) (b)

Design of interactive makers Color-band Recognition Fig. 4. The blue color is usually best recognized and most stable in the lighting variation. The hue components in HSV color space are used for robust detection in various lighting conditions.

Design of interactive makers Color-band Recognition Fig. 5. Recognition results of color-band markers. Two markers areconsistently detected when they are moving.

Recognition of image and object Feature Extraction The distinct feature points are determined by Hessian matrix at image point x(x, y) and scale parameter σ

Recognition of image and object Feature Extraction Fig. 6. 26 neighbors to decide the feature points. The red pixel is the current point at x and σ. The neighboring scale images are considered to decide the feature points.

Recognition of image and object Feature Extraction Haar filter Fig. 7.

Recognition of image and object Feature Extraction Haar filter Hessian filter + SURF (speeded up robust Feature) The square region is divided into 16 subregions, and 25 pixels (5x5) are sampled in each subregion. And we calculate 4-D vector for every subregion 4-D vector for each subregion, the descriptor vector to describe the feature point becomes 64-D (4x16) vector. This 64-D descriptor vector is an ID number of each feature point.

Recognition of image and object Feature Matching Exploit the sign of Laplacian operation for fast feature matching =

Recognition of image and object Feature Matching Fig. 8. (a) (b)

Recognition of image and object Image and Object Recognition Fig. 9. (a) (b) (c) (d)

Recognition of image and object Image and Object Recognition

Recognition of image and object Image and Object Recognition (a) (b) Fig. 10.

Recognition of image and object Image and Object Recognition (c) (d) Fig. 10.

Application to public education system Fig. 11. (a) (b)

Application to public education system Fig. 12. (a) (b)

Conclusions Expected that the proposed e-leaning system becomes popular faster Recognition errors are reduced and authoring tools are provided to produce the educational contents and scenarios

Thank you