2. Robotic Hand-Eye Systems CMPUT 610 Martin Jagersand

3. What is involved in making real vision-guided motion?

4. Hand-Eye System

5. System requirements Solve many very different motion tasks – Flexible, teachable/re-programmable Real time – On special embedded computers or general workstations Different special HW Multiprocessors

6. Toolbox

7. System design Interpreted “scripting” language gives flexibility Compiled language needed for speed and HW interface. Examples Matlab Greencard Haskell C, C++, fortran PVM C, C++ Dyn linking (mex)

8. PVM Parallel Virtual Machine Message passing based distributed systems Implemented for many machine architectures Runs separate (heavyweight) unix processes. Advantage: when can’t link together binaries: – Linux libc5, libc6; IRIX lib32, libn32, lib64 High level process monitoring

9. Library structure /usr/erskine2/prof/jag/matlabdirs

10. Ideas: Data flow architecture natural. Higher order functions to go from general to specific motion primitives. Toolbox of primitive motion behaviors.

11. Usage example: Specialize robot – projandwait(zero3,’robotmovehill’,A3D,’WaitForHill’); Initialize goals and trackers – [TrackCmd3D,N] = InitTrackers([1 1],[0,1]); – PU = GetGoals([1 1],[0,1]); Servo control – J3s = LineMove(‘projandwait’,TrackCmd3D,J3i,PU,Ndi,err)

12. Fran/Frob motivation: Imperative Software Limits Integration leads to recurring implementation chores – Writing loops to step forward discretely in time – Time slicing time-varying components that operate in parallel Code reuse – Two pieces of code need to do almost the same thing, but not quite What’s correct? – The design doesn’t look at all like the code – Hard to tell if its a bug in the code, or a bug in the design Programs should describe what to do not how to do it

13. New XVision Programming Model Stream Partitioning Image Filtering Feature Tracking Image Filtering Feature Tracking Image Filtering Feature Tracking Combination Video In Data Out

14. Programming Dynamical Systems trackMouth v = bestSSD mouthIms (newsrcI v (sizeof mouthIms)) trackLEye v = bestSSD leyeIms (newsrcI v (sizeof leyeIms)) trackREye v = bestSSD reyeIms (newsrcI v (sizeof reyeIms)) trackEyes v = composite2 (split, join) (trackLEye v) (trackREye v) where split = segToOrientedPts --- some geometry join = orientedPtsToSeg --- some more geometry trackClown v = composite2 concat2 (trackEyes v) (trackMouth v)

15. Summary Most current demos solve one specific movement For solving many everyday tasks we need flexibility and reprogrammability – Interpreted scripting language – Higher order functions What is SwEng anyway?

16. Bill