1. Group #2 / Embedded Motion Control [5HC99] Embedded Visual Control 1 Group #5 / Embedded Visual Control Self-Balancing Robot Navigation Paul Padila Vivian Zhang Amritam Das Michail Papamichail

2. Group #2 / Embedded Motion Control Overview 1. Introduction 2. Objectives 3. Design 4. Control 5. Vision 6. Conclusions and Recommendations 2 Group #5 / Embedded Visual Control

3. Group #2 / Embedded Motion Control 1. Introduction 3 Group #5 / Embedded Visual Control ► The Self-balancing robots not so popular and do not have many applications yet. ► They are mostly used for educative purposes ► Possible reason: hard to be stabilized under certain conditions. Application of self-balancing robot Two wheels self-balance electric scooter

4. Group #2 / Embedded Motion Control 1. Introduction 4 Group #5 / Embedded Visual Control ► The color tracking method is also not very popular yet. ► Possible reasons could be that colors are hard to be tracked during intense sunshine or during the night. Application of color tracking method Color-based Object Tracking in Surveillance

5. Group #2 / Embedded Motion Control 1. Introduction 5 Group #5 / Embedded Visual Control ► The camera is mounted in a robot and not anchored in a wall. ► The robot have automated navigation and can scout areas. ► Limits the amount of cameras that are needed. ► Cameras cannot be tricked by changing clothing color in blind spots. ► There can be a network of cameras that can track the target cooperatively. Possible application in the future By Combining the previous two applications one can achieve a new improved Surveillance system with great advantages.

6. Group #2 / Embedded Motion Control 1. Introduction 6 Group #5 / Embedded Visual Control ► Gesture detection ► Shape detection Playstation eye gesture detection Other visual methods

7. Group #2 / Embedded Motion Control 1. Introduction 7 Group #5 / Embedded Visual Control ► A robot that can perform certain tasks in a hospital. –Empties the trash. –Refills supplies. Future applications in general By Combining a self-balanced robot with any of the visual method of detection. ► A robot that can identify flawed parts in constructions. –It recognizes skewed shapes. –It can work even if the construction site is closed. –It increases the safety of the construction site. –It protects the investments.

8. Group #2 / Embedded Motion Control 3. Design 8 Group #5 / Embedded Visual Control Mechanical Design ► Multi-layer. ► Rigid supports. –Electronics –Motors

9. Group #2 / Embedded Motion Control 3. Design 9 Group #5 / Embedded Visual Control Support Structure ► The selected thickness of the material is able to support the weight of the set of batteries used. ► This material is lightweight (minimizes the total weigh). This means an improvement in the energy consumption of the robot. ► MDF is easy to and inexpensive material that can be used with laser cutting machines.

10. Group #2 / Embedded Motion Control 3. Design 10 Group #5 / Embedded Visual Control Plates

11. Group #2 / Embedded Motion Control 3. Design 11 Group #5 / Embedded Visual Control Motor Base ► Motors should be perfectly aligned. ► Misalignment causes vibrations and deviations during the displacement of the Robot

12. Group #2 / Embedded Motion Control 3. Design 12 Group #5 / Embedded Visual Control Motor ► Functions: –Stabilization –Displacement of the robot ►Fast reactions ►Large torque

13. Group #2 / Embedded Motion Control 3. Design 13 Group #5 / Embedded Visual Control Batteries ► Maximum energy consumption: 12V at 5.2A. ► 18650 batteries: 3.7V(x3) at 5.3A

14. Group #2 / Embedded Motion Control 3. Design 14 Group #5 / Embedded Visual Control Arduino Arduino Shield ► Compact and easy to install. ► The interfaces between the sensors and the control are ready to use ►MPU-6050: 3-axis gyroscope and a 3- axis accelerometer in a single chip with I2C communication ►L298P: Motor driver, high voltage (50V) and high current (4A) dual channel full-bridge

15. Group #2 / Embedded Motion Control 3. Design 15 Group #5 / Embedded Visual Control Arduino UNO ► Control unit. –Sensors –Actuators

16. Group #2 / Embedded Motion Control 4. Control 16 Group #5 / Embedded Visual Control Control Problem ► Stabilization Problem ► Position Problem

17. Group #2 / Embedded Motion Control 4. Control 17 Group #5 / Embedded Visual Control Stabilization Problem ► P ► PD ► PI

18. Group #2 / Embedded Motion Control 4. Control 18 Group #5 / Embedded Visual Control Position Problem ► P ► PD ► PI

19. Group #2 / Embedded Motion Control 4. Control 19 Group #5 / Embedded Visual Control Control Design ► Different control objectives. ► Same actuator. ► Different time constants are fundamental to guarantee stability

20. Group #2 / Embedded Motion Control 4. Control 20 Group #5 / Embedded Visual Control Performance ParameterValue Settling time3 sec Position tolerance +/- 4cm Tracking tolerance +/- 4 cm

21. Group #2 / Embedded Motion Control Vision 21 Image Processing Colour Tracking Colour Tracking Open CV Integration with Control Hardware Object Following Object Following

22. Group #2 / Embedded Motion Control Choice of Hardware - Raspberry Pi 2 + pi Camera 22

23. Group #2 / Embedded Motion Control Capturing Consistent Image 23 ► To fix exposure time, set the shutter_speed attribute to a reasonable value. shutter_speed ► To fix exposure gains, let analog_gain and digital_gain settle on reasonable values, then set exposure_mode to 'off'. analog_gain digital_gain exposure_mode ► To fix white balance, set the awb_mode to 'off', then set awb_gains to a (red, blue) tuple of gains. Optionally, set iso to a fixed value. awb_mode awb_gains iso

24. Group #2 / Embedded Motion Control Capturing Consistent Image – Sample Implementation 24

25. Group #2 / Embedded Motion Control Color Tracking 25 ► Image Conversion

26. Group #2 / Embedded Motion Control Noise Elimination ► Morphological Operation ► Erosion 26 Dilation

27. Group #2 / Embedded Motion Control Thresholded Image with Morphological Operation 27

28. Group #2 / Embedded Motion Control Edge Detection + Contour Analysis 28 ► Canny Edge Detection + Gaussian Blur Filter

29. Group #2 / Embedded Motion Control Integrating Raspberry pi with Arduino 29 ► Serial Communication

30. Group #2 / Embedded Motion Control Object Tracking Algorithm 30 ► Boolean Logic

31. Group #2 / Embedded Motion Control Object Tracking Algorithm 31 ► Proportionate Controller

32. Group #2 / Embedded Motion Control Performance of Object Tracking Camera Reaction Time 32 – camera –object

33. Group #2 / Embedded Motion Control Performance of Object Tracking Change in The object Distance 33

34. Group #2 / Embedded Motion Control Performance of Object Tracking Movement of the Camera 34