MPR Intersection Experiment Beijing, 2011 Spring Jointly by PKU and HEUDIASYC

MPR Intersection Experiment Motivation/Purpose

Scenario Design Laser Camera Host V S N

Horizontal laser scanner Obliquely laser scanner Host V: POSS GPS/IMU Two Flea2 Compose stereo Cameras Horizontal laser scanner Obliquely laser scanner More?

Scenario Design Laser Camera Host V Haidian Gymnasium S Peking Univ. N Laser1: SW, NPC4 S Peking Univ. N Laser3: NN, NPC5 Laser2: NW, NPC1 Server NPC7 & AP An overhead bridge ChangChun Dorm. of Peking Univ.

Scenario Design Haidian Gymnasium S Peking Univ. N An overhead bridge Host Vehicle Trajectories Haidian Gymnasium S Peking Univ. N An overhead bridge ChangChun Dorm. of Peking Univ.

Intersection Experiment Time and Place: 05/23/2011, 13:00-17:00 T word intersection, in the west of PKU Experiment Facilities: Sensors Intersection 3 ground lasers 1 ground mono camera Vehicle 2 lasers: horizontal and obliquely downward at the front of car Stereo camera: 2 mono cameras GPS

Cont. Experiment Facilities: Computers Other Facilities: Intersection 3 laptops: laser data collection client 1 laptop: laser data collection server Vehicle 1 pc: laser and GPS data collection 1 pc: video data collection Other Facilities: Batteries, Cables, Wireless AP, Patch Boards, Tripod, etc…

Experimental Procedure: Preparation Vehicle Sensors: Stereo Camera, Horizontal Laser and Obliquely Laser Calibration: Online Mono Camera + Horizontal Laser <(u, v)、(x, y, z=1.0)> (x, y, ?)

Cont. Intersection Sensors: Ground Lasers and Video Camera Lasers must be horizontal Time Synchronization All computers should in one network(PC on POSS and Laptops at intersection) Do time synchronization Calibration Ground Lasers: Nature object or land markers Ground Lasers + Video Camera: Land markers

Ground Laser + Video Calibration 4 5 6 3 2 1

Ground Laser + Video Calibration

Experiment Procedure: Measurements Start the measurements of ground lasers and video camera. Start the measurements of on-vehicle lasers and cameras. Let the host vehicle run across the intersection

Experiment Data Intersection Ground Lasers Data Ground Video *.lms1, *.lms2, *.lms3 Ground Video exp20110523_westgate_1.avi exp20110523_westgate_1.log : start time of video

Experiment Data Vehicle Lasers Stereo Camera GPS data *.lms1: Horizontal Laser Data *.lms2: Obliquely Laser Data Stereo Camera *.avi: stereo video *.txt: start time of corresponding video GPS data *.pos

Experiment Data Primary Data Processing Time Synchronization Laser data, GPS data Correct the start time of video Offline Calibration Intersection ground laser + video Laser and Video data fusion Using calibration results to project laser points on video images. Intersection Vehicle

Experiment Data On Vehicle Data Fusion Incorrect calibration result Didn’t notice that the ground is not horizontal while doing calibration

Experiment Data Intersection Data Fusion Not so exactly

Research Tasks What perceptual algorithm do we study? TASK 1: Multimodal perception Multimodal sensor fusion-based object detection, recognition and tracking Navigable space detection (road geometry, boundaries, lanes) Static object detection: signs, trees, facades Moving/movable object detection and tracking: cars, cycles, pedestrians Multimodal data constrained SLAM (GPS, GIS) Data representation and vicinity dynamic maps What knowledge do we need to reasoning? TASK 2: Reasoning and scene understanding An open comparative dataset for testing for cross-cultural robustness in traffic scene understanding Learning for scene semantics and moving object behaviors Traffic situation awareness with uncertainty, scene semantics and information fusion Task 2.1: Navigable space detection (road geometry, boundaries, lanes) Task 2.2: Static object detection: signs, trees, facades Task 2.3: Moving/movable object detection and tracking: cars, cycles, pedestrians Task 2.4: On-road SLAM