Assistive System Progress Report 1 Bishwo Gurung Shifali Kumar James Chou

Overview of the System Objective: To help individuals who are visually impaired and provide them independence. Object recognition system SIFT algorithm Speech recognition feature MATLAB (primary language and interactive environment)

SIFT - Scale-invariant feature transform Scale-space extrema detection Keypoint localization Orientation assignment Keypoint descriptor

The First Steps of Feature Detection Begin by constructing a scale space Replicate a natural concept of perception in digital imaging The scale space represents the image in multiple sizes to produce invariance.

Applying Gaussian Blur Intentionally eliminate unwanted detail from an image (such as textures or artifacts) Removes detail without introducing false ones by smoothing the image An important step in extracting keypoints

Combining with Octaves An octave contains a set of one specific image in the same scale but progressively blurs over each one. The next octave has the levels of blurring but is scaled half the size of the previous octave. This step is repeated over and over.

David Lowe’s ideal numbers (4 octaves, 5 blur levels)

Gaussian Blur Expression σ - scale parameter

Generate Difference of Gaussians new set of images for finding interest points.

Gaussian Filter blur_1 = imgaussfilt(I, sigma); blur_2 = imgaussfilt(blur_1, sigma); blur_3 = imgaussfilt(blur_2, sigma); blur_4 = imgaussfilt(blur_3, sigma);

Difference of Gaussians(Simplified) DoG_1 = I-blur_1; subplot(2,2,1); imshow(DoG_1) DoG_2 = blur_1-blur_2; subplot(2,2,2); imshow(DoG_1) DoG_3 = blur_2-blur_3; subplot(2,2,3); imshow(DoG_1) DoG_4 = blur_3-blur_4; subplot(2,2,4); imshow(DoG_1)

Sample Image

Next Step Key-point localization Orientation assignment Scale-space extrema detection Key-point localization Orientation assignment Keypoint descriptor Locate maxima/minima in DoG images

Speech Recognition There are 2 types of systems: isolated (having a brief pause between the words) continuous (talking fluently) Analyzing: speaker independent system (multiple people) speaker dependent system (single person)

Speech Recognition 3 stages main stages speech acquisition speech analysis user interface development We will be using the microphone built in the computer.

Speech Acquisition Matlab tool: Simulink library browser Training (part 1) This part is being done offline since we are training the system. First step, is to find a way to input in our data (word acoustic). Matlab tool: Simulink library browser Next, we need to know how the computer can differentiate between each of our inputs. A solution for this is creating our own acoustic model which will convert each sounds into numerical digits so it will be efficient for the system analyze.

Speech Acquisition Matlab tool: Data acquisition toolbox Testing (part 2) This part will be done will be live. We will need to pull the saved data and compare it with the new data while processing new speech. Matlab tool: Data acquisition toolbox Processing the new speech will be done frame by frame for isolated system.

Thank You!