1. MPR Intersection Experiment Beijing, 2011 Spring Jointly by PKU and HEUDIASYC

2. MPR Intersection Experiment
Motivation/Purpose

3. Scenario Design
Laser
Camera
Host V S N

4. Horizontal laser scanner Obliquely laser scanner
Host V: POSS
GPS/IMU
Two Flea2
Compose stereo Cameras
Horizontal laser scanner
Obliquely laser scanner
More?

5. Scenario Design Laser Camera Host V Haidian Gymnasium S Peking Univ. N
Laser1: SW, NPC4 S
Peking Univ. N
Laser3: NN, NPC5
Laser2: NW, NPC1
Server NPC7 & AP
An overhead bridge
ChangChun Dorm.
of Peking Univ.

6. Scenario Design Haidian Gymnasium S Peking Univ. N An overhead bridge
Host Vehicle Trajectories
Haidian Gymnasium S
Peking Univ. N
An overhead bridge
ChangChun Dorm.
of Peking Univ.

7. Intersection Experiment
Time and Place:
05/23/2011, 13:00-17:00
T word intersection, in the west of PKU
Experiment Facilities: Sensors
Intersection
3 ground lasers
1 ground mono camera
Vehicle
2 lasers: horizontal and obliquely downward at the front of car
Stereo camera: 2 mono cameras
GPS

8. Cont. Experiment Facilities: Computers Other Facilities: Intersection
3 laptops: laser data collection client
1 laptop: laser data collection server
Vehicle
1 pc: laser and GPS data collection
1 pc: video data collection
Other Facilities:
Batteries, Cables, Wireless AP, Patch Boards, Tripod, etc…

9. Experimental Procedure: Preparation
Vehicle
Sensors: Stereo Camera, Horizontal Laser and Obliquely Laser
Calibration: Online
Mono Camera + Horizontal Laser
<(u, v)、(x, y, z=1.0)>
(x, y, ?)

10. Cont. Intersection Sensors: Ground Lasers and Video Camera
Lasers must be horizontal
Time Synchronization
All computers should in one network(PC on POSS and Laptops at intersection)
Do time synchronization
Calibration
Ground Lasers: Nature object or land markers
Ground Lasers + Video Camera: Land markers

11. Ground Laser + Video Calibration4 5 6 3 2 1

12. Ground Laser + Video Calibration

13. Experiment Procedure: Measurements
Start the measurements of ground lasers and video camera.
Start the measurements of on-vehicle lasers and cameras.
Let the host vehicle run across the intersection

14. Experiment Data Intersection Ground Lasers Data Ground Video
*.lms1, *.lms2, *.lms3
Ground Video
exp _westgate_1.avi
exp _westgate_1.log : start time of video

15. Experiment Data Vehicle Lasers Stereo Camera GPS data
*.lms1: Horizontal Laser Data
*.lms2: Obliquely Laser Data
Stereo Camera
*.avi: stereo video
*.txt: start time of corresponding video
GPS data
*.pos

16. Experiment Data Primary Data Processing Time Synchronization
Laser data, GPS data
Correct the start time of video
Offline Calibration
Intersection ground laser + video
Laser and Video data fusion
Using calibration results to project laser points on video images.
Intersection
Vehicle

17. Experiment Data On Vehicle Data Fusion Incorrect calibration result
Didn’t notice that the ground is not horizontal while doing calibration

18. Experiment Data
Intersection Data Fusion
Not so exactly

19. Research Tasks What perceptual algorithm do we study?
TASK 1: Multimodal perception
Multimodal sensor fusion-based object detection, recognition and tracking
Navigable space detection (road geometry, boundaries, lanes)
Static object detection: signs, trees, facades
Moving/movable object detection and tracking: cars, cycles, pedestrians
Multimodal data constrained SLAM (GPS, GIS)
Data representation and vicinity dynamic maps
What knowledge do we need to reasoning?
TASK 2: Reasoning and scene understanding
An open comparative dataset for testing for cross-cultural robustness in traffic scene understanding
Learning for scene semantics and moving object behaviors
Traffic situation awareness with uncertainty, scene semantics and information fusion
Task 2.1: Navigable space detection (road geometry, boundaries, lanes)
Task 2.2: Static object detection: signs, trees, facades
Task 2.3: Moving/movable object detection and tracking: cars, cycles, pedestrians
Task 2.4: On-road SLAM