1. Observations by Dance Move
Deep Dance Revolution
Collin Rooney, Rose Thomas, Kristin Meier, Eric Pawlakos | MSiA Deep Learning | Spring 2017 | Northwestern University
Problem Statement
Technical Approach
Results
Process Overview
Data processing
Noisy backgrounds, mislabeled images
Naïve baseline models
KNN & SVM - performed worse than Deep Network
Neural Networks
Convolutional Neural Network
Model Tuning
Width Shift Range, Rotation Range, and only Horizontal Flipping
Analyze output
Can ballet moves be classified using images?
Beneficial to the dance community for documenting and passing on choreography.
Current process is manual and extremely time consuming. It is challenging for a non-domain expert to classify the moves.
Future work includes generating choreography from video and real time classification.
* Potential overfitting
Stump rate: 23.5%
Best accuracy: 60%
Significantly better than non-domain expert.
Download and Clean Data
Step 1
Fit CNN Model and Analyze Output
Step 2
Test Alternative Parameters and Data
Step 3
Summarize and Present Findings
Step 4
0:40 Grand Battement
0:58 Back Attitude
1:14 Pirouette
1:33 Sous Sous
Model Specifics
We used a pre-trained VGG network in Keras.
ImageNet award winning model was chosen to save time.
This pre-trained VGG network formed the baselines for our project.
Improvements:
Setting image size to 64x64 (after trying 32x32, 128x128, 224x224)
Using the Adagrad optimizer over Adam (adaptive learning)
Shrinking the number and size of layers
Regularization:
L1 and Dropout factor of 0.2
Kept our model from overfitting on small dataset
Heatmaps:
Model looks at defining features of the pose for classification
Incorrect classifications are understandable and reasonable errors
Handles costume, background, and dancer differences well
Conclusion
Dataset
Model largely focuses on areas where the human looks
Small dataset posed challenges
Improvements/future work – Make more robust to new data. Train a model to identify various dancers (object detection) and assign each dancer independent labels.
Utilizing video resources as training data or testing data could improve and validate the model
Google images of 9 distinct dance moves
~ 1200 images total
Relatively balanced classes
Example observations
Arabesque
Grand Jete
Challenges
Augmentation
Small dataset
Manual labeling
Background editing
Felzenszwalb transformation
Canny edge detection
References and Related Work
Classification of Body Movements Based on Posturographic Data (
Convolutional Neural Net (
Felzenszwalb Segmentation (
Real-Time Classification of Dance Gestures from Skeleton Animation (vision.ucla.edu/papers/raptisKH11.pdf)
Observations by Dance Move
Grand Battement
214
Arabesque
294
Grand Jete
223
Assemble 31
Penche/Ponche
200
Back Attitude
198
Pirouette/Passe
120
Front Attitude 44
Sous Sous 98