HYDRO 2019 The detection and identification of drowning human using underwater 3D sonar Yung-Da Sun1, I-Fong Tsui2, Yi-Horng Lai3, Chia-Ming Tsai3, Jiun-Jiun Chen1, Yu-Jen Chung4, and Jau-Woei Perng3 1Naval Meteorological and Oceanographic Office R.O.C. 2Chung Cheng Institute of Technology, National Defense University, R.O.C 3Dept. of Mechanical and Electro-Mechanical Engineering, National Sun Yat-sen University 4Dept. of Marine Science, Naval Academy, R.O.C March 20, 2019

Outline Introduction Motivation Method Experiment and Result Conclusion Q & A

Sensing the underwater world Optical sensors Passive optical sensors Active optical sensors Acoustic sensors Passive sonar sensors Active sonar sensors Pierre, Drap., et al. 2018

Drowning Swimming pool Beach Open sea Flooding http://clipart-library.com/clipart/174091.htm

Camera-Based Drowning Detection Lei Fei, et al. 2009 Chi Zhang, et al. 2015

3D Sonar-Based Drowning Detection

Motivation Pedro, J. Navarro, et al., 2016

Motivation N. Brodu, D, Lague, 2012 The multi-scale characterizes is defined as the radius of sphere centered on each local point cloud. When the radius of the sphere vary, the geometry feature of local cloud across scales can be measured. Therefore the aforementioned disadvantage of point cloud operation (e.g. missing data etc.) can be avoided. N. Brodu, D, Lague, 2012

PCA feature extraction N. Brodu, D, Lague, 2012

PCA feature extraction N. Brodu, D, Lague, 2012

RANSAC (RANdom SAmple Consensus) 1. Sample (randomly) the number of points required to fit the model 2. Solve for model parameters using sample 3. Score by the fraction of inliers within a preset threshold of the model Repeat 1-3 until the best model is found with high confidence Line fitting example CS 4495 Computer Vision – A. Bobick

GMM model There are k components. The i’th component is called wi Component wi has an associated mean vector mi Each component generates data from a Gaussian with mean mi and covariance matrix Si The GMM model are composed of a mixture of probability distributions. Each component of model owns individual Gaussian distribution to represent the subset of data.

Experiment and result BlueView BV5000 Spherical Scan Area: 360 º Operation Frequency: 1.35 MHz Maximum Scan Range: 30 m Number of Beams: 256 Beam Width (º) 1 x 1 Vertical Spatial Resolution: 16 mm at 10 m Horizontal Spatial Resolution: 30 mm at 10 m Weight: 22 kg

The two experiments have been carried out in one of fishery harbor in Kaohsiung, Taiwan. Scenario 1 Scenario 2

Scenario 1 Collect and analyze the PCA feature of underwater object. Train and group the unsupervised GMM model into the meaningful category. The purpose of scenario 1 includes: (1) Collect and analyze the PCA feature of underwater object. (2) Train and group the unsupervised GMM model into the meaningful category. The dummy person was drop into the prefixed positon in harbor where the seabed condition was known. Then BV5000 sonar was placed near close the dummy people for obtaining a necessary dense of cloud points.

The raw point could in scenario 1. The point cloud is colored by depth. In scenario 1, the location of dump person are known and can be found clearly in the sonar scan. Besides, a lot of wasted tires distributed among on the nearby seabed surface.

Using RANSAC algorithm to delete sea floor. For detection efficiency, we only focus on the underwater object. Therefore most of point cloud about seabed surface can be filtered through RANSAC surface fitting estimation. By using iterative process, RANSAC can find the inliers of the planar surface model from the consensus set. Deleting the points of seabed surface, the outlier of point cloud just belong to the interested underwater object

The detection and identification of dummy person in 3D underwater point cloud. The ground truth of scenario 1 include dummy person, wasted tire and seabed. The components of unsupervised GMM model can be grouped into the exact final category.

Scenario 2 The main purpose of scenario 2 is to evaluate the PCA feature and the unsupervised GMM model without associated ground truth of underwater object. The dummy person was drop randomly into another dock of harbor where the seabed condition was unknown. Therefore BV5000 sonar have to scan many sites along the dock.

The raw scan in scenario 2. The point cloud is colored by depth.

Two unknown outlier point cloud datasets after seabed surface removed.

The PCA feature space in scenario 2.

The underwater visual inspection in scenario 2. The underwater visual inspection prove the point cloud of object 2 is the dummy person

Conclusion The automatic detection and identification of underwater object are studied in this article. The PCA multi-scale feature can extracted from point cloud of underwater 3D sonar. Two empirical experiments are evaluated and shown the PCA multi-scale feature is robust. The proposed algorithm can be customized and modified by the user for applying to the other detection of underwater objects

Q & A