1. Shoe-Mouse: An Integrated Intelligent Shoe ∗ Weizhong Ye, Yangsheng Xu and Ka Keung Lee Department of Automation and Computer-Aided Engineering The Chinese University of Hong Kong EB2, Shatin,CUHK,HongKong wzye@acae.cuhk.edu.hk Student ID ： M9920103 Student ： Kun-Hong Lee Adviser ： Ming-Yuan Shieh 1 PPT 製作 100%

2. Outline ABSTRACT INTRODUCTION HARDWARE DESIGN DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES EXPERIMENTAL RESULTS CONCLUSION AND FUTURE WORK REFERENCES 2

3. ABSTRACT (1/2) In this paper, we developed a sensor-integrated shoe as an information acquisition platform to sense the foot motion. The system is small, portable and wearable. The platform is mainly composed of four parts including a sensing module, a computing module, a wireless communication module, and a data visualization module. 3

4. ABSTRACT (2/2) Based on this platform, we developed a novel input device called Shoe- Mouse, which can be used by people who have difficulties in using their hands to operate computers or devices. The platform can be also used for applications such as gait recognition, human identification, and motion monitoring. 4

5. 5 INTRODUCTION (1/2) Main attributes of our system Firstly, we integrated all the sensors and circuits fully inside the shoe without adding much weight into the original shoe. It is easy to use and users will notice little difference between their normal shoes and the proposed shoes. Secondly, we built a hardware platform to collect data from the shoe. This platform is programmable, easily scalable and easy to be integrated to the other applications.

6. 6 INTRODUCTION (2/2) Thirdly, we applied the system to several successful tasks based on this platform, especially the Shoe-Mouse. By using this interface, we can operate a device with our feet. This invention can be useful for the persons who have to use computers frequently but have difficulties in using their hands.

7. 7 HARDWARE DESIGN (1/6) Fig. 2. Outline of the hardware design and sensor

8. 8 HARDWARE DESIGN (2/6) Sensing the parameters inside the shoe This component consists of a flexible instrumented insole (in Fig. 3) that is worn inside the shoe. One important parameter the insole measures is the force applied between the foot and the shoe at some key points. These key points are under the major weight-bearing points of the foot: the big toe and the heel ( which is divided into a medial and lateral portion ).

9. 9 HARDWARE DESIGN (3/6) Sensing the parameters inside the shoe Fig. 3. A flexible instrumented insole

10. 10 HARDWARE DESIGN (4/6) Sensing the parameters inside the shoe Besides, a bend sensor is installed between the toe and the heel. This bend sensor is used to measure the degree of bend between the toe and the heel. The output of the bend the sensor also contains rich information about human motion, especially loading and uploading of feet.

11. 11 HARDWARE DESIGN (5/6) Gathering information from the sensors This subsystem is mainly composed of a processor circuit board (in Fig. 4). It includes a microcomputer, an analog- to-digital converter, peripheral components, batteries, and an attached accelerometer. As can be seen from Fig. 4, the circuit board is small and it can be easily put into the heel of the shoe so that users will notice little the difference between the normal shoes and the intelligent shoes.

12. 12 HARDWARE DESIGN (6/6) Gathering information from the sensors Fig. 4. Processor circuit board

13. 13 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (1/6) Mapping motion of foot to motion of a mouse-cursor on the screen Based on the shoe-mounted data gathering platform, we use the following mapping methods to achieve the goal: (1)Use accelerometer to produce motion of mouse cursor: The accelerometer used here (ADXL202E) can output xaxis acceleration and y-axis acceleration.

14. 14 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (2/6) Mapping motion of foot to motion of a mouse-cursor on the screen Fig. 8. Rectangle motion of the shoe in a plane (left) and its corresponding output of x-axis acceleration (right)

15. 15 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (3/6) Mapping motion of foot to motion of a mouse-cursor on the screen Fig. 9. Motion of foot and its corresponding acc output

16. 16 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (4/6) Mapping motion of foot to motion of a mouse-cursor on the screen (2) Use force sensors at the toe to produce single click of the left button: Force sensor at the toe is naturally mapped into a single click. When a user has clicked the shoe using his toe, a force will be produced from the force sensor installed between the toe and the shoe.

17. 17 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (5/6) Mapping motion of foot to motion of a mouse-cursor on the screen single-click of left button of the mouse takes effect. This force will persist for about 0.5 seconds. The average force during this time is computed. If the force is bigger than the threshold, the function of single-click of left button of the mouse takes effect

18. 18 DESIGN OF SHOE-MOUSE BASED ON THE PLATFORM OF INTELLIGENT SHOES (6/6) Mapping motion of foot to motion of a mouse-cursor on the screen 3) Use two force sensors at the heel to produce single click of the right button: Here we use two force sensors to judge whether the user has pressed the right button

19. 19 EXPERIMENTAL RESULTS (1/5) The toe of the shoe contacts the ground with a force. By calculating the average value of this force, we can decide whether the user is intended to produce a left click of the mouse or not. However, not all toe motions can produce the corresponding click of the mouse. We did the following experiments to evaluate the performance of the click motion.

20. 20 EXPERIMENTAL RESULTS (2/5) A circle with the diameter 30cm was drawn manually on the screen. Each of the three volunteers wearing intelligent shoes performs a simple click of toe to produce a point in this circle. If the click is successful, it will produce a new dark point inside the circle.

21. 21 EXPERIMENTAL RESULTS (3/5) Fig. 11. Result of click test

22. 22 EXPERIMENTAL RESULTS (4/5) We drew a circle as a test path, letting the users draw several identical circles in the same place as accurately as they could. Then we computed the deviation between drawn circles and the original circle.If the deviation is bigger than the threshold, we consider this circle an unsuccessful circle. After that, we compared the performance between the ordinary mouse and our proposed Shoe-Mouse.

23. 23 EXPERIMENTAL RESULTS (5/5) Fig. 12. Performance of drawing a circle using Shoe-Mouse

24. 24 CONCLUSIONS (1/2) In this paper, we have built a shoe-based sensor integrating platform that can gather a lot of data inside the shoe. A novel input device called Shoe-Mouse is introduced by applying the shoe platform as a user wearable interface. The Shoe-Mouse is especially designed for the people who have difficulties in operating computers using hands. As proved in the experiments, the Shoe-Mouse performs satisfactorily in motion control and can partly replace the ordinary mouse.

25. 25 CONCLUSIONS (2/2) People who do a lot of work using computers can also benefit from this invention. In the future, more experiments will be conducted on potential user groups to find out more about the requirements for the systems

26. 26 REFERENCES [1] K. N. Tarchanidis and J. N. Lygouras, “Data glove with a force sensor”,IEEE Trans. on Instrumentation and Measurement, vol. 52, issue.3,pp.984-989, June 2003. [2] T. E. Starner, “Wearable computers: no longer science fiction”, IEEE Pervasive Computing, vol.1, no.1, 2002. [3] R. E. Morley, E. J. Richter, J. W. Klaesner, K. S. Maluf, and M. J.Mueller, “In-shoe multisensory data acquisition system”, IEEE Trans. on Biomedical engineering, vol.48, no.7, July 2001. [4] J. Paradiso, E. Hu, and K. Y. Hsiao, “The cybershoe: wireless multisensor interface for a dancer’s feet”, in Proceesings of Internaional Dance and Technology, Tempe, Atizona,1999. [5] I. P. Pappas, T. Keller, and M. R. Popovic, “A novel gait phase detection system”, in Proceesings of Workshop Automatisierungstechnische Verfahren fr die Medizin, Darmstadt, 1999. [6] Stacy J. Morris and J. A. Paradiso, “A compact wearable sensor package for clinical gait monitoring”, Offspring vol.1, no.1, pp.7-15, January 31, 2003. [7] A. Watts, “On exponential smoothing of discrete time series”, IEEE Trans. on Information Theory, vol.16, issue.5, pp.630-630, Sep, 1970.

27. Thanks for your attention!! 27