1. Classification of highly unbalanced data using deep learning techniques
Dianjing Liu

2. Thyroid Disease Data Set
(Sick)
(Not Sick)
Negative: 3178
Positive: 250
Predicting all instances to be negative: 92.71% accuracy, 0% sensitivity.
Testing accuracy of previous classifiers:
Nearest-neighbor: 92.44%
k-Nearest Neighbor: 93.70%
Ref. [1]: %
Bayes rule: 96.1%
Nonlinear Bayes: 97.2%
Ref. [2]: %
Ref. [3]: %

3. Neural network settings
Activation: sigmoid
Training steps: 50,000
Learning rate: Decays by half every 5,000 steps.
Network size is chosen by cross validation:

4. Oversampling Original Oversampled
Table 1. The accuracy and sensitivity of neural networks trained with different positive instance ratios.

5. Oversampling
Original
Oversampled
ROC curve
PR curve

6. Other techniques Activation function Dropout Batch normalization
Adam Optimization algorithm

7. Other techniques Learning curves:
Tabel 2. Testing results of different techniques (with or without oversampling). The ReLU activation and dropout methods fail to train effective models and predict all samples to be negative.
Learning curves:
Oversampling
BN and Adam

8. Conclusion Oversampling can always improve sensitivity.
BN and Adam have very good performance: higher accuracy, sensitivity and training speed.
Best model: 98.60% accuracy % sensitivity.