1. HCI / CprE / ComS 575: Computational Perception Instructor: Alexander Stoytchev http://www.ece.iastate.edu/~alexs

2. Next Time Project ideas. Brief 3-5 min presentations.

3. Administrative Stuff The Wiki is now configured. http://www.vrac.iastate.edu/575x/S2010/doku.php Please register and create your page. HW2 template is no posted.

4. Skin Color Detection HCI/ComS 575X: Computational Perception Iowa State University Copyright © Alexander Stoytchev

5. “Skin-color modeling and adaptation” Yang, Lu, and Waibel CMU-CS-97-146, May 1997 (Available on the Class Web page) [Readings Section]

6. 3D Scatter Plot for a patch of skin

7. Detecting Human Skin

10. Example [http://www.cs.ucsb.edu/~haiying/project/skin_color.htm]

11. Another Example [http://www.cs.ucsb.edu/~haiying/project/skin_color.htm]

12. A Different Color Space

13. Properties

14. Color Histogram

16. Matlab Demo of Skin Detection (see code on the class web page)

17. Motion Energy & Motion History HCI/ComS 575X: Computational Perception Iowa State University Copyright © Alexander Stoytchev

18. A.F. Bobick and J.W. Davis ``An appearance-based representation of action''. In Proceedings of IEEE International Conference on Pattern Recognition 1996, August 1996, pp. 307-312.

19. A.Davis, J. and A. Bobick ``The Representation and Recognition of Action Using Temporal Templates'', In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 1997, pp. 928-934.

20. A.F. Bobick and J.W. Davis ``An appearance-based representation of action''. In Proceedings of IEEE International Conference on Pattern Recognition 1996, August 1996, pp. 307-312.

21. A.Davis, J. and A. Bobick ``The Representation and Recognition of Action Using Temporal Templates'', In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 1997, pp. 928-934.

22. The Authors James Davis Aaron Bobick

23. What is this?

24. What action is being performed?

25. Motion Energy Image (MEI) [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

26. Average MEI for various viewing angles

27. Motion History Image (MHI) [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

28. Definitions Image Sequence Binary Images indicating regions of motion Binary Motion Energy Image

29. Motion Energy

30. Motion History The result: more recently moving pixels appear brighter

31. [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

32. MHI pyramid [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/MHI/mhi.html]

33. Motion templates for finishing LEFT-ARM-RAISE and FAN-UP-ARMS. [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

35. Aerobics Dataset

36. Results for 30 o off center camera location

37. OpenCV Demo If you want to try this at home just compile and run the motempl.c file in the../samples/c directory

38. Applications

39. A.Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, and A. Wilson (1999) ``The Kidsroom: A Perceptually- Based Interactive and Immersive Story Environment" Presence: Teleoperators and Virtual Environments, Vol. 8, No. 4, 1999, pp. 367-391.

40. The Kid’s Room [Bobick et al. 1996]

41. The Kid’s Room [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

42. The Kid’s Room [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

43. The Kid’s Room [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

44. Monsters [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

45. Scavenger Hunt [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

46. Scavenger Hunt [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

47. The Blue Monster [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

48. Spin in place [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

49. 3 Kids Spin on a Rug [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

50. The River Vorls [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

51. The Technology [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

52. Motion History Templates Making a ‘Y’ FlappingSpinning [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

53. Detecting the Bed [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

54. Man Overboard Detector [http://vismod.media.mit.edu/vismod/demos/kidsroom/kidsroom.html]

55. Movies The quality of the movies is not very good You can download them from: http://vismod.media.mit.edu/vismod/ demos/kidsroom/kidsroom.html

56. J. Davis and A. Bobick ``Virtual PAT: A Virtual Personal Aerobics Trainer'', Workshop on Perceptual User Interfaces, November 1998, pp. 13-18.

57. Interactive Virtual Aerobics Trainer [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

58. Interactive Virtual Aerobics Trainer [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

59. Interactive Virtual Aerobics Trainer [http://www.cse.ohio-state.edu/~jwdavis/CVL/Research/VirtualAerobics/aerobics.html]

60. Movies

61. The Personal Pet Project Pepe Joint work with Rawesak Tanawongsuwan

62. Tanawongsuwan, R., Stoytchev, A., and Essa, I., "Robust Tracking of People by a Mobile Robotic Agent", Technical Report GIT-GVU-99-19.

63. Project Goals build an intelligent, adaptive, user-friendly agent build an agent that has a personality make the interaction between the user and the agent as natural as possible

64. Pointing gesture Motion Profile Calculation Face and Hands Segmentation Neural Network Hidden Markov Models Gesture Recognition

65. Motion History/Energy Perform a frame-by-frame subtraction (image differencing) and accumulate the results over the history window. Motion energy is a binary version of motion history Technique from Bobick and Davis.

66. Gesture recognition

67. Gesture Recognition

68. Gesture recognition

69. Movie

70. THE END