REU Project 4D Efficient Real-Time Graph Based Temporal Point Cloud Segmentation Steven Hickson

4D Segmentation Problems Noise causes grouping errors in the point clouds around the edges of data Solution: Error is somehow caused by the amount of Gaussian smoothing of the depth data. In order to fix this, the sigma value used to smooth is changed according to the number of frames included.

Volumetric Results

Cons Approximately 1 Frame/sec. Requires all of the data beforehand, ie good only for post processing. Limited length of video due to memory issues. Not as accurate as it could be 6 arbitrary values used to determine quality of segmentation

New Method New method is similar to a method Georgia Tech uses in their 2010 CVPR video segmentation paper. Do an over-segmentation of the spacio-temporal volume and then use regions and region graph- based merging for the segmentation. Instead of using volumes over time, use the new region segmentation to match the regions over time (past frame vs. current frame)

Region Trees A region tree is constructed by doing an over segmentation of the 3D volume. For each region of voxels, a branch is added onto the tree with pointers to all of the voxel data of the segmented cloud. While, the voxels are added, their LABD values are calculated (using Hunter LAB since it is faster). Each region stores the histogram of the LABD values, the 3D centroid, the 2D centroid, the area, it’s neighboring regions, and the regions beneath it or voxels beneath it. All of these values can be calculated while the tree is built and it allows the trees to become hierarchal.

Merging Region Trees Trees are merged over time using a method I call temporal correction. At fps, there is little movement between frames so the segmented regions should be similar. Therefore, the normal level of segmentation can be done and then the labels can be corrected using the region tree from the previous frame. The previous region tree and the current region tree are compared and the regions are matched based off the difference of the centroids, the difference of the size and the difference of the LABD histograms. This has been finished and results are good but can be improved.

Temporal Correction Results