1. Interactive Space – An application of OpenCV
Sam Siciliano

2. Overview Who I am and the project background. The System The Software
Components
Communication
The Software
APIs
Custom Code
User Interface
Ideas, Expansions and Surprises
Demonstration and Videos

3. Who I am
Graduated summer before last from PSU with a degree in computer engineering
Embedded Systems Engineer for Logic Product Development
OMSI Employee at the time of the project

4. The Project
Portland State University Senior Capstone Project done in conjunction with OMSI
Team of 6
Was in OMSI 3-4 months, currently lives on my laptop.

5. The System – Block Diagram

6. The System - Components
“Technobeam” intelligent light fixture
DMX-512 Lighting Protocol
180 degrees pan, 90 degrees tilt
Adjustable colors, patterns via “litho wheel” and “gobos”
Iris and adjustable focus

7. The System - Components
The DMX-512 transceiver
Shifts from RS-232 to RS-485
Increases baud rate, from 1152 to 2500

8. The System – Components
The Server
Dell, p4 2GHz Machine.
Windows XP Professional
Generic Video Capture Card

9. Software Components Computer Vision Input Video Stream via DirectX 9
and
Direct Show
Calibration
Based
Mapping
TB/PT
Serial
Control
Classes
Xvid, Yvid
XTB, YTB
Frame

10. Video Capture Via DirectX
Direct Show and Filter Graphs
Example of Simple Video Capture:
Graph Used for the Project:

11. Video Capture Via DirectX
Install DirectX 9.0 SDK (note – DirectShow is not included with latest version!!)
Use the DirectShow API to Create a Filter Graph
Insert ProxyTrans Filter that comes with OpenCV
Define a callback function and start playback

12. Computer Vision Initial Conditions Tracking “cvgoodfeaturestotrack”
Region and Trigger Selection
Tracking
Optical Flow
Point Removal Rules
Centroid Calculation

13. Throw out invalid points in P[]
Algorithim No
Object Lost
Get Trackable
Features in ROI
(points P[] – 1)
> 0 points in P[]?
Yes
Calculate Centroid of
P[]
Save Frame (Frame N – 1)
Send centroid
to remapping
functions
Get Next Frame
(Frame N)
Save Current Frame
(now N – 1)
Pass in all to
OpticalFlow
Function
To get updated
points (P[])
Save Current Points
(now P – 1)
Throw out invalid points in P[]

14. Calibration and Mapping
UI setup to gather calibration data points a set of 2 inputs, 2 outputs
Points are collected once per given setup
Accounts for “fish-eye” lens distortion

15. Calibration and Mapping - 2
Our problem context:
First Solution: get to calibration points (lower left, upper right) to obtain thresholds, linearly scale and shift:
???
Pan (from 0 to 216)
Xvid(from 0 to 320)
Yvid(from 0 to 240)
Tilt (from 0 to 216)

16. Calibration and Mapping - 3
Worked fine with initial camera we were using.
When switching though to a camera with a wide angle lens, system wasn’t adequate due to “fisheye” distortion.
New Scheme: 9 point calibration and piece-wise linear interpolation.

17. Calibration and Mapping - 4
x3,y3, pan3
x1,y1, pan1
x2,y2, pan2

18. Ideas and Surprises Feedback enabled “throwing” of the light
Hypersonics and Music
Use fuzzy logic with camera on pan/tilt unit

19. Questions?