1. TGS Salt Identification Challenge
—— Zheming Wang
Course Work of CS539 in Fall 2018

2. Introduction
Salt deposits below the surface is a big obstacle for oil and gas drillers. To detect them, expert human interpretation is still a must nowadays. The goal of this project is to tell the salt region with given seismic image as follow:

3. Data and method
The data comes from the TGS challenge on Kaggle. The picture size is 101*101, while the train and test sets have sizes of 4,000 and 18,000.
The tool kit I use is Keras with TensorFlow as backend in python.
The code I used mainly referred to some kernels on Kaggle. I keep the metric and data transforming part unchanged, revised the model structure, and wrote some code to make the training resumable and automatic on Euler.

4. Basic Summary
It is a typical image segmentation problem and there is a specific metric called Intersection over Union (IoU).
In this project, I start from a baseline U-net model with IoU of 0.675(0.700) , ranking at 81%.
After trying different methods and do lots of fine tuning, I get a 2-stage Res-Unet model with data augmentation, cross validation and voting, which obtains IoU of (0.858) , ranking at 10%.

5. Model Structure: U-net (baseline)

6. Model Structure: Runet-revised (No resize)

7. Result 1: model choosing
description
test score in leaderboard
private
public
position
U-net
baseline
0.700
0.675
80.7%
U-net - improved
use greyscale
0.753
0.722
72.3%
Res-U-net (Runet)
add ResNet; add threshold
0.818
0.797
47.9%
Runet-depth
add depth as a channel
0.809
0.782
55.5%
Runet-csum
add cum-sum as a channel
0.813
0.793
50.6%
Runet-revised
keep input size 101*101 unchanged
0.829
0.800
46.3%
General setting
Batch size: 32
Epochs: 200
Early Stop : 10
Lr schedule :
1e-2/ 8/ 0.5/ 1e-5
Monitor : val_iou
* May differ a little per model.

8. test score in leaderboard
Result 2: fine-tuning
Model
description
test score in leaderboard
private
public
position
Runet-lovasz
(baseline)
0.8323
0.8005
45.8%
Runet-lovasz - tuned
0.8457
0.8171
32.7%
cross validation 3
best-cv
0.8423
0.8180
31.9%
combination - voting
0.8505
0.8264
21.1%
cross validation 5-1
Augmentation; best-cv
0.8456
0.8236
26.6%
cross valiadation 5-2
Augmentation; best-cv
0.8539
0.8334
14.0%
0.8581
0.8352
12.0%
combination - best3
best 3 above
0.8582
0.8393
10.2%

9. Lessons Learned in Project
1. Keras has some problems when resume a saving model. After reading source code, I wrote a patch for it
2. Parallelism did not accelerate single model training. I found this fact when trying Keras mult-gpu class. Then I changed to train multiple cross-validation like models at the same time.
3. In Data Science, the model choosing is a core problem, while the data preprocessing and hyper-parameter tuning can also make a great difference but need lots of experience and trails.

10. test score in leaderboard
Further improvement
Other techniques:
Transfer learning: using other network as an encoder.
Optimizer choosing: Adam, momentum, normal SGD.
Feature engineering: tell zero-or-some salt first. ……
Model description
test score in leaderboard
private
public
position
model with X-ception as encoder, different optimizer, tricks like pseudo-labeling and Stochastic weight averaging.
0.8771
0.8547
4.2%

11. Thank you for listening.