Gesture Recognition Technology Knitting By: Stephanie and Sam

How it works... Gesture Recognition Technology depends on the computer being able to recognise and understand human gestures, and act upon that information There are two basic ways that visual information may be captured by computers: by the use of censors, or optically. There are three basic forms of optical data capturing: Stereoscopic, Multiple Cameras, or Single Camera

Censors! The computer only pays attention to the censors, and judges what to do on them.

Stereoscopic: is a single camera that can tell how far away something is from the camera. This enables the camera to be able to tell the difference between the human making the gestures and the background..

multiple cameras. The same effect as the stereoscopic camera, only the 3d understanding of the visual field is compiled by multiple views of the same subject.

Single camera. Uses only one camera. What the gesture making human is and what the background is identified based on programming. This is the fastest computation, but needs much more programming. It is difficult for the single computer to differentiate fingers when they are against the palm. Thus, the information gathered is usually the silhouette of the hand.

The computer can be programmed to identify people (or hands) by skin colour. An RGB ratio that describes human flesh can be programmed in, and the computer will only select these pixels to look at. Proximity and priority are used to eliminate unwanted pixels—random single pixels of the right ratio in the background, for example, will be ignored by the computer.

In all cases, the background will be edited out— usually by comparison. A picture of the background is taken by the computer, and will subsequently ignore that information.

There are two basic ways gesture can be identified There are a statistically limited number of gestures a human hand can make (because joints only bend certain ways. And so on). The computer can use a logic tree based off of these limited number of gestures to identify what the gesture is. For example, the computer will identify if the hand is open or closed. If it is open, it will ignore any closed hand gestures. Alternatively, the computer will be ‘trained’ to recognise certain hand gestures. The computer will compare the collected data with the stored memory bank of gestures. It will rank the gesture on how close it is to the known gesture.

We are using gesture tech to create a knitting training devise. One knits before a computer. The computer, having multiple cameras, can identify your hands and compare them to a sequence of hand gestures it knows to be the correct ones. If you make the wrong gesture (i.e., make a mistake) it will alert you by making an audible noise. It records what you are doing; when you make a mistake you can replay what you did, accompanied by a training video of the right sequence of movements. These images can by overlapped, or seen side by side.

Gesture control—the replay function is controlled by gesture: Bringing your hands together is ‘stop’, it will stop any video playing or function of the program(bringing the two needles with the work together is the standard ‘pause’ position in knitting—it ensures that the work doesn’t fall off the needles) ‘Go back’ is tilting your hands to the left. (Go forward is to the right) Going back into the knitting position will continue and/or restart play. When you have completed the gesture successfully, you can repeat (go back hand gesture), or advance (go forward) to another technique.

Use of gesture: Here the technology is used to support a gesture, and aid in the creation of a skill by using body memory. The technology aids in quickly achieving the correct gesture, so it may be easier memorized by the body and turned into skill memory.

In addition, the gestures used to control play back are designed to work within what gestures one can make whilst knitting, as well as what is customary to do when one wishes to stop knitting momentarily

Learning Styles There are three main types of learning styles: auditory, visual, and kinesthetic. Most people learn best through a combination of the three types of learning styles, but everybody is different.

Auditory Learners: Hear Auditory learners would rather listen to things being explained than read about them. Reciting information out loud and having music in the background may be a common study method. Other noises may become a distraction resulting in a need for a relatively quiet place.

Visual Learners: See Visual learners learn best by looking at graphics, watching a demonstration, or reading. For them, it’s easy to look at charts and graphs, but they may have difficulty focusing while listening to an explanation.

Kinesthetic Learners: Touch Kinesthetic learners process information best through a “hands-on” experience. Actually doing an activity can be the easiest way for them to learn. Sitting still while studying may be difficult, but writing things down makes it easier to understand.

With the Gesture Technology that we have come up with, it provides a clear understanding and learning experience that would help every type of learner. Auditory Learners: are able to hear and understand the instructions given, and are able to hear and understand when they make a mistake, with the how the censors react. Visual Learners: are able to see what they are doing, how they are doing it, when they make an error, where they go off track and the motions that are needed to succeed. Kinesthetic Learners: are able to hold on to the knitting needles and are able to practise the motions needed to complete the stitches, as well they are able to better understand because they are actually doing the stitch.

With the use of the Knitting Gesture Technology, the average user would be more likely to full understand the simplest stitches to the more complicated ones based on how the technology works with the users learning styles. The technology is designed so that everyone is able to get the most use out of it and learn as much as they can while using it.

The End By: Stephanie and Sam