1. Semi-Autonomous Control of Mobile Robotic Field Agents: The Algorithm, Interface, and Implementation Results Mr. John M. Kuperavage Mr. John Haughery Dr. John R. Wright, Jr., CSIT Department of Industry & Technology Millersville University of Pennsylvania

2. The Need for Education “Advances in the field of robotics have resulted in dependable robots that are extremely accurate and repeatable while becoming affordable. In the last decade the prices for robots have fallen nearly 40%. During the same period robots have become faster and more versatile. Analysts expect the use of robots in the electronics industry to grow at an average rate of 35% annually during the next four years” (Dunham, 2001). “It has been cited by the RIA [Robotics International Association], that only 12% of the American companies that could use a robot, are presently using one. Therefore, 88% of the market is still untapped. There are many reasons for this. But the predominant reason is the lack of a trained workforce that understands how to implement the technology. We need more engineers to develop a hands-on understanding of what constitutes a good application” (Harris, 2003). www.rixan.com

3. The Need for Education Recent advances in the development of robotics combined with significant reduction of their costs have made the implementation of such devices in today’s smaller and midsize companies increasingly necessary and feasible. According to Peter Cavallo, head of U.S. sales for Denso Robotics, “it’s well within the means of any size company to use robots. It’s no longer just the realm of large or dollar-intensive companies” (Spencer, 2003).

4. Overview of Robotic Control Industrial Robots Bang-Bang Point-to-Point Controlled Path Continuous Path Mobile Robots Hard Wired Tele-operation Semi-Autonomous Autonomous Known versus Unknown

5. Sectors of Society that Employ Mobile Robots 1) Manufacturing Automatic Guided Vehicles (AGVs) 2) Medical Hospital Food Delivery 3) Military (Urbie) Recognizance 4) Research/Exploration Space, Hazardous Environments, etc. http://robotics.jpl.nasa.gov/tasks/tmr/picts/StairPhoto.jpg

6. Progression of Semi-Autonomous Research Students began with general Basic Stamp II tutorial books which included programs to run servo motors and various sensors. What’s a Microcontroller and Robotics with the BOE Bot Once familiar with the basics, the students were able to start applying this knowledge to begin operating speed controllers and steering servos to move a mobile robotic vehicle. Independent research was performed on more advanced sensors and wireless Bluetooth technology.

7. SA-1’s Development SA-1 was first introduced using only tele-operated control but was designed to have an autonomous function integrated into it. This was done while the team members were being introduced to the technology in various classes. Independent Studies and Independent Research expanded our technical capability in the use of infrared sensors and wireless Bluetooth communications. The original SA-1, which was built for the 2004 NAIT conference employed semi-autonomous control (tele-operation and autonomous modes). SA-1 was modified for the 2005 National Robotics Challenge. The autonomous function was expanded to include a search pattern that could be activated to find an object and then grab it. This advanced our autonomous mode, leaving the tele-operation the same.

8. Motor Control Many tele-operated vehicle motors are controlled by speed controllers. Research was preformed on to how to control a hobbyist level speed controller via a microcontroller. We found that the control is similar to that of a servo motor. A pulse out range of 500 to 1000 milliseconds will drive the speed controller from full reverse to full forward (750 being neutral).

9. Infrared (IR) Sensors To detect objects, two Parallax S.S.I.R infrared sensors were used. These required no integration other then connecting them to the proper pins. These were placed inside of the gripper to detect the object and then center the robot on it. The sensors used one pin on the Microcontroller to both send and receive signals. Certain kinds of light sometimes cause erroneous signals with this type of sensor. The code had to be modified to filter out these stray signals, which allowed for more accurate detection. This code will be explained later. Sensors

10. Bluetooth Communication To allow an operator to initiate the automation and monitor the status of the robot, a wireless communication link needed to be established. AppMod Header VSS Pins Code was written to allow communications between the EmbeddedBlue transceiver and the PC adapter. The user would first open a HyperTerminal window on the PC screen where he could see a prompt which was sent from the microprocessor. Then he could input a value that could be used in the code to do things such as start an automated process. This was done using a Parallax EmbeddedBlue transceiver and a USB Bluetooth adapter for the PC. The Bluetooth adapter simply plugs into a PC USB port. The EmbeddedBlue transceiver plugs directly into the AppMod Header on the BASIC Stamp II.

11. Tele-Operated to Autonomous Control SA-1’s control was transferred from the remote control to the Basic Stamp through the wireless Bluetooth connection. The user would drive the robot to a position where the object would be within a certain range of the gripper. At this point, the user would initiate the automated function from the HyperTerminal prompt on the laptop screen. This sent a start signal to the microprocessor, which began the automated process. During this process, the microprocessor takes precedence over the remote controller, temporarily disabling it. To transfer control of the robot back to the remote controller, the microprocessor activated a control relay which temporarily shorted out the BASIC Stamp reset. This reset the program and the process.

12. Integrating the Technologies Once the students had the knowledge base and more advanced technologies had been explored, the next step was to integrate the research into one working system. Many code adaptations were necessary to allow all of the systems to function together. Our team’s task was to develop a semi-autonomous system to allow an operator drive an unknown course and engage an automatic mode that searches for an object and then picks it up. This process was automated to allow the pick up to be done quicker and more reliably then if it were done through manual control. We will now go through an overview of the logic algorithm which was formed to develop the code for the automated portion of the robots control.

13. Complete Algorithm Start Automation Open/Close Shell Object Search Grab Object

14. Starting the Automation Process SEROUT 1,84,[CR,"Press 1 to start the Automation process",CR] SERIN 0,84,[DEC1 CmdData] BRANCH CmdData,[Open_Shell] GOTO Main “SEROUT” puts a display on the computer screen prompt and “SERIN” waits for the user to give a response. A zero on the keyboard starts the automation process by directing the program to an open shell subroutine. No input re-loops the code until an input is given. START Check Bluetooth Input for Start Signal Is the Automation Procces Started? Yes No Start Automation To: Open the Shell

15. Complete Algorithm Start Automation Open/Close Shell Object Search Grab Object

16. Opening/Closing the Shell Open_Shell: IF IN4 = 0 THEN Manipulator PULSOUT Motor,870 PAUSE 20 GOTO Open_Shell This is a loop to open the shell. Once inside, the program will remain in the loop until a limit switch indicating “fully open” is depressed. That limit corresponds to “IN4 = 0” and the “THEN” goes to code to open the manipulator. If the limit switch is not depressed, the program will continue to run through the loop pulsing out to “Motor” which controls the speed controller for the hinge motor. This is the exact same process for closing the shell, except you pulse out to the speed controller in the opposite direction.

17. Complete Algorithm Start Automation Open/Close Shell Object Search Grab Object

18. Opening the Manipulator The code to open the manipulator is simply a set number of passes through a loop. (Notice, no feedback.) Each time through the loop, a pulse is sent to two servos on both sides of the manipulator to open the two doors. The program then proceeds to a search pattern. This is a left, then right zigzag pattern, checking the sensors for an object 8 times each cycle. Manipulator: FOR a = 1 TO 10 PULSOUT Servo1, Backwards PULSOUT Servo2, Fwd PAUSE 20 NEXT

19. Each time through the search, the cycle yields four pulses to the left and four to the right. This is done by always pulsing to “Motor2”, which is a drive motor speed controller. The zigzag motion is accomplished by pulsing out forward or reverse to “Pin 10”, which is the steering servo. You can also again see a “SEROUT” which gives a prompt indicating the status of the search. The “GOSUB” is a subroutine to check the sensors for object detection each after each pulse. Search: FOR x = 1 TO 4 PULSOUT 10, 1000 PULSOUT Motor2, 815 SEROUT 1,84,[CR,"The SA-1 is Searching", CR] GOSUB sensor_check NEXT PAUSE 250 FOR x = 1 TO 4 PULSOUT 10, 500 PULSOUT Motor2, 815 SEROUT 1,84,[CR,"The SA-1 is Searching", CR] GOSUB sensor_check NEXT PAUSE 250 GOTO Search The Search Pattern

20. Checking the Sensors The “LOOKUP” Command checks the sensor at different frequencies (different sensitivities). “FREQOUT” sends and receives the signal through Pin 1 or IN1. “CountUp” records how many times the something is detected. sensor_check: CountUp = 0 tally = 0 FOR freqSelect = 0 TO 6 LOOKUP freqSelect,[37500,38000,38500,39000,39500,40000,40500], irFrequency FREQOUT 1,1, irFrequency IF IN1 = 0 THEN CountUp = (Countup + 1) NEXT IF (CountUp > 1 )THEN tally = (tally + 1) IF tally = 2 THEN found ***Bolded code is for detection using two sensors If the reliability is high enough (enough instances of detection are seen), the a variable “tally” is incremented. This code is for one sensor. Setting up another sensor the same way would have the same result. To align the mobile robot on the object, all sensors must detect something reliably. In this case two sensors must set the variable “tally” to two before continuing to the object “found” section of the code.

21. Object Found! Once the robot searches and finds the object, it must straighten its wheels before proceeding to pick the object up. This is done by pulsing a centering value to a servo a set number of times to overcome the resistance of the ground. Found: FOR x = 1 TO 8 PULSOUT Servo3, 750 PAUSE 300 NEXT

22. Complete Algorithm Start Automation Open/Close Shell Object Search Grab Object

23. Grabbing the Object Within the “MOVE” loop, the code pulses to the drive motor speed controller until the object is in the gripper. This is indicated by a depressed limit switch “IN3”. Notice again that “SEROUT” is used to prompt the user of the status of the robot. Once the object is in the gripper the two servos on both sides of the gripper close the doors by receiving a set number of pulses. MOVE: PULSOUT Motor2, 815 SEROUT 1,84,[CR,"The SA-1 is now in motion", CR] IF IN3 = 0 THEN SERVO_CLOSE PAUSE 20 GOTO MOVE SERVO_CLOSE: FOR b = 1 TO 5 PULSOUT Servo2, Fwd PULSOUT Servo1, Backwards PAUSE 20 NEXT

24. Complete Algorithm Start Automation Open/Close Shell Object Search Grab Object

25. What the Process Looks Like! The algorithm in motion! The operators view **Video taken at the 2005 National Robotics Challenge, Marion Ohio

26. Seems Simple? Developing this algorithm may seem easy or difficult to some, but there is more to creating a working Semi- Autonomous system. So far, we have acquired the necessary knowledge, developed key technologies, and integrated them by forming a logic algorithm for the autonomous mode. Even though everything works as bench tested, you still need to perform electromechanical integration to make all those components function together. SA-1 had two major electromechanical systems: Opening and closing the shell and grabbing the object.

27. Opening/Closing SA-1 A hinging action was used to open/close SA-1’s shell. This also lowered manipulator to ground level to allow the object to be grabbed. This hinging was performed by means of a motor and a screw. The chassis hinged at its center and its front end, which allowed it to fold up as it opened.

28. Opened Closed

29. Hinging Motor & Screw Function Lowered & raised the manipulator. Provided a mechanical advantage for lifting a load. Motor Screw Threaded Nut

30. Hinging Motor Wiring Diagram

31. Hinging Open Limit Switch (Mechanical Interface)

32. Open Previous Slide

33. Hinging Open Limit Switch (Mechanical interface)

34. Hinging Closed Limit Switch (Mechanical interface)

35. Closed Previous Slide

36. Hinging Closed Limit Switch (Mechanical interface)

37. Limit Switch Wiring Diagram Note - The wiring diagrams for all limit switches are the same except each is wired to a different input.

38. Gripping The Object SA-1 employed an encompassing gripper to grasp the object. A limit switch was used to indicate when the object was within the gripper’s grasp. The limit switch employed an active compliance system to reduce contact shock to the object.

39. Gripper Limit Switch (Mechanical interface)

40. Conclusions We discussed the need for robotic education and an overview of mobile robotics and control. The progression of research leads students from performing simple applications to developing complex algorithms, which can be used to automate portions of the robots function. Once an algorithm is developed, code can be written to perform the tasks within each section. Finally, electromechanical integration must take place to allow the code to control the mechanical components properly.

41. Thank You! Any Questions?

42. References Dunham, P. (2001). Robots provide automated soldering solution. Electronics Manufacture and Test. Accessed online (July, 2003): http://www.emtmagazine.co.uk/ http://www.emtmagazine.co.uk/ Harris, S. (2003). Stats for grant application. Email correspondence with Stephen Harris, President of Rixan Associates, Inc. Dayton, Ohio. Lindsay, A. (2004). Robotics with the Boe-Bot (VERSION 2.2). Parallax, Inc. Spencer, R., Ed. (2003). Robotic machine tending a mature application, but still has room to grow. Robotics World, 21(5). Accessed online (July, 2003): http://www.roboticsworld.com/features.asp http://www.roboticsworld.com/features.asp What’s a Microcontroller. (VERSION 1.9). (1999). Parallax, Inc.