Omni Directional Vision Tracking Denise Fancher Kyle Hoelscher Michael Layton Eric Miller

Project Objectives Design and integrate an omni-directional vision system for mobile robot navigation and image tracking with the following requirements: 360° field of view 360° field of view Track bright objects and respond to movement Track bright objects and respond to movement Housed on mobile robot Housed on mobile robot Collision avoidance Collision avoidance

Hardware – Vision System Modified CMUcam for use with Boebot Modified CMUcam for use with Boebot SX28 micro-controller SX28 micro-controller Interfaced with OV6620 Omnivision CMOS camera Interfaced with OV6620 Omnivision CMOS camera Basic tracking information is transmitted via TTL logic Basic tracking information is transmitted via TTL logic

Features of Camera Track user defined color blobs Track user defined color blobs Find the centroid of the color blob Find the centroid of the color blob 80 x 143 Resolution 80 x 143 Resolution Arbitrary window sizing via the SW command 9600 Baud Communication

Board Additions To make an image viewable we added the following components: To make an image viewable we added the following components: MAX232 chip MAX232 chip 4 – 50 mV Capacitors 4 – 50 mV Capacitors 5 x pin header 5 x pin header Serial cable Serial cable Pin headers for jumper settings Pin headers for jumper settings

Getting an Image First add the MAX232 chip, caps, header, and break the solder on the jumper connections and add those headers, remove all jumpers housings First add the MAX232 chip, caps, header, and break the solder on the jumper connections and add those headers, remove all jumpers housings Wire the serial cable to connect to the 10 pin header Wire the serial cable to connect to the 10 pin header Note: the image in the documentation is from the back side of the female DB9 Note: the image in the documentation is from the back side of the female DB9

Camera Results: Note that the image is upside down, this must be taken into acct when coding With the hemispherical mirror the horizon is very close in the picture

Requirements for Bot Comm. Set both jumpers so baud is 9600 Set both jumpers so baud is 9600 Remove capacitors and MAX232 chip Remove capacitors and MAX232 chip Issue SERIN/SEROUT commands to pull data Issue SERIN/SEROUT commands to pull data To lock on to the side, use a SW command and then TW to lock color To lock on to the side, use a SW command and then TW to lock color

Obsacle Avoidance - HW We added sonar for the front object detection We added sonar for the front object detection Larger range Larger range More dependable, less likely to be absorbed by reflective material More dependable, less likely to be absorbed by reflective material Will not be interfered with by foreign radio frequency Will not be interfered with by foreign radio frequency Set bit object detection bit when object within 4 inches Set bit object detection bit when object within 4 inches

Obsacle Avoidance - HW We used the infrared sensors for the sides of the robot We used the infrared sensors for the sides of the robot Transmitter/Receiver Transmitter/Receiver Set object detection bit when obstacle near side Set object detection bit when obstacle near side

Software Implementation Download object coordinates and confidence level from camera to BOE-BotDownload object coordinates and confidence level from camera to BOE-Bot Perform collision detectionPerform collision detection If collision eminent, issue appropriate movement to avoid collision then return to mainIf collision eminent, issue appropriate movement to avoid collision then return to main If no sensors detect objects, calculate SpinFactor based on object coordinatesIf no sensors detect objects, calculate SpinFactor based on object coordinates Based on object coordinates: spin to face object, do nothing, or move forwardBased on object coordinates: spin to face object, do nothing, or move forward Return to mainReturn to main

Software Implementation Download object coordinates and confidence level from camera to BOE-Bot Download object coordinates and confidence level from camera to BOE-Bot serout 10, Baud, ["TC",CR] serin 9, Baud, [STR RcvData\10] Confid = RcvData(9)

Software Implementation Perform collision detection Perform collision detection PulseWidth=0 PulseWidth=0 PULSOUT SonarTrigger, 9'10 uS trigger pulse PULSOUT SonarTrigger, 9'10 uS trigger pulse PULSIN SonarEcho, 1, PulseWidth'measure distance to target PULSIN SonarEcho, 1, PulseWidth'measure distance to target Distance = PulseWidth ** 8886 'divide by Distance = PulseWidth ** 8886 'divide by IF Distance <= SOnarComfortDistance THEN BitZero IF Distance <= SOnarComfortDistance THEN BitZero IF Distance > SOnarComfortDistance THEN BitOne IF Distance > SOnarComfortDistance THEN BitOne BitOne: BitOne: 'set state.bit0 to 1 'set state.bit0 to 1 State.bit0=1 State.bit0=1 goto afterSonar goto afterSonar BitZero: BitZero: 'set state.bit0 to 0 'set state.bit0 to 0 State.bit0=0 State.bit0=0 goto afterSonar goto afterSonar

Software Implementation Perform collision detection Cont… Perform collision detection Cont…afterSonar: 'check_IR 'check_IR FREQOUT LeftIROutputPin, 1, 10212'Check left IR FREQOUT LeftIROutputPin, 1, 10212'Check left IR State.bit2 = in7 State.bit2 = in7 FREQOUT RightIROutputPin, 1, 'Check right IR FREQOUT RightIROutputPin, 1, 'Check right IR State.bit1 = in0 State.bit1 = in0 IF ( (State.bit2=1) AND (State.bit1=1) AND (State.bit0=1)) THEN CalcSpinFactor IF ( (State.bit2=1) AND (State.bit1=1) AND (State.bit0=1)) THEN CalcSpinFactor 'if previous staement not taken, then must avoid obstacle 'if previous staement not taken, then must avoid obstacle goto collisionAVOID goto collisionAVOID

Software Implementation Collision Avoidance Collision Avoidance '-----State Cases----- '-----State Cases----- '0=000=blocked left, right, front '0=000=blocked left, right, front IF ( (State.bit2=0) AND (State.bit1=0) AND (State.bit0=0)) THEN Backup IF ( (State.bit2=0) AND (State.bit1=0) AND (State.bit0=0)) THEN Backup '1=001=blocked left, right '1=001=blocked left, right IF ( (State.bit2=0) AND (State.bit1=0) AND (State.bit0=1)) THEN Backup IF ( (State.bit2=0) AND (State.bit1=0) AND (State.bit0=1)) THEN Backup '2=010=blocked left, front '2=010=blocked left, front IF ( (State.bit2=0) AND (State.bit1=1) AND (State.bit0=0)) THEN Right90 IF ( (State.bit2=0) AND (State.bit1=1) AND (State.bit0=0)) THEN Right90 '3=011=blocked left only '3=011=blocked left only IF ( (State.bit2=0) AND (State.bit1=1) AND (State.bit0=1)) THEN GradualRight IF ( (State.bit2=0) AND (State.bit1=1) AND (State.bit0=1)) THEN GradualRight '4=100=blocked right, front '4=100=blocked right, front IF ( (State.bit2=1) AND (State.bit1=0) AND (State.bit0=0)) THEN Left90 IF ( (State.bit2=1) AND (State.bit1=0) AND (State.bit0=0)) THEN Left90 '5=101=blocked right only '5=101=blocked right only IF ( (State.bit2=1) AND (State.bit1=0) AND (State.bit0=1)) THEN GradualLeft IF ( (State.bit2=1) AND (State.bit1=0) AND (State.bit0=1)) THEN GradualLeft '6=110=blocked front only '6=110=blocked front only IF ( (State.bit2=1) AND (State.bit1=1) AND (State.bit0=0)) THEN BackLeftAvoid IF ( (State.bit2=1) AND (State.bit1=1) AND (State.bit0=0)) THEN BackLeftAvoid '7=111=clear -- no change in direction '7=111=clear -- no change in direction

Software Implementation Calculate SpinFactor based on object coordinates Calculate SpinFactor based on object coordinates If ((RCVData(3) 20)) Then BigSpin If ((RCVData(3) 20)) Then BigSpin If (RCVData(2) 20)) Then CalcFactorLow If (RCVData(2) 20)) Then CalcFactorLow If (RCVData(2)>40 AND (Confid > 20)) Then CalcFactorHigh If (RCVData(2)>40 AND (Confid > 20)) Then CalcFactorHighBigSpin: SpinFactor = (76 - RCVData(3)) SpinFactor = (76 - RCVData(3)) goto IfStatements CalcFactorLow: SpinFactor = 40-RCVData(2) SpinFactor = 40-RCVData(2) goto IfStatements CalcFactorHigh: SpinFactor = RCVData(2)-40 SpinFactor = RCVData(2)-40 goto IfStatements

Software Implementation Based on object coordinates: spin to face object, do nothing, or move forward Based on object coordinates: spin to face object, do nothing, or move forward If ((RCVData(3) > 75) And (RCVData(3) 35) And (Confid > 20)) Then NoSpin If ((RCVData(2) 20)) Then RightSpin If ((RCVData(2) >= 45) And (Confid > 20)) Then LeftSpin If ((RCVData(3) > 116) And (Confid > 20)) Then Forward LeftSpin: debug "Left spin",CR for pulse_timer = 0 to SpinFactor pulsout rtwheel, 1450 pulsout lftwheel, 1456 pause 20 next goto main RightSpin: debug "right spin",CR for pulse_timer = 0 to SpinFactor pulsout rtwheel, 2250 pulsout lftwheel, 2256 pause 20 next goto main

Challenges Camera image dump to PC Camera image dump to PC Lock on issues Lock on issues Coding the basic stamp Coding the basic stamp

Gantt Chart

Questions ?