1. Mahdi Kalayeh David Hill
Learn to Comment
Week 4
Mahdi Kalayeh
David Hill

2. LRCN Architecture
CNN

3. VGG-16 vs VGG-19 3 epochs on COCO BLEU-1 BLEU-2 BLEU-3 BLEU-4 VGG-16
50.8
32.9
20.1
12.7
VGG-19
59.6
40.7
28.2
Google NIC 63
...
27.2
Human
66.3
46.9
32.1
21.7

4. LRCN Architecture
Embedding Space

5. Embedding Space
Maps 1 of k word vectors to dense vector representations
Necessary to align feature space dimensions
Other works in NLP use similar mappings
Global Vectors for Word Representation (GloVe)

6. VGG_Emb-19 vs VGG-19 3 epochs on COCO ~100 words not found in GloVE
BLEU-1
BLEU-2
BLEU-3
BLEU-4
VGG_Emb-19
58.4
39.3
27.1
19.3
VGG-19
59.6
40.7
28.2
20.1

7. KNN on Embedding Spaces
GloVe
cat
5.054
cats
5.196
dog
5.861
pet
6.274
dogs
6.500
instance
6.612
actually
6.644
indeed
6.651
unfortunately
6.655
presumably
LRCN GloVe
cat
1.647
kitten
2.012
puppy
2.029
dog
2.101
cay
2.114
calico
2.148
tabby
2.193
tiger
bird
2.236
pug
LRCN random
cat
1.374
kitten
1.604
calico
1.708
puppy
1.757
tabby
1.778
siamese
1.781
cay
1.801
dog
1.956
tiger
2.020
tortoiseshell

8. Image Captions (-7.301) a man riding a motorcycle down a street
a man with a red helmet on a small moped on a dirt road
man riding a motor bike on a dirt road on the countryside
a man riding on the back of a motorcycle
a dirt path with a young person on a motor bike rests to the foreground of a verdant area with a bridge and a background of cloud wreathed mountains
a man in a red shirt and a red hat is on a motorcycle on a hill side

9. Image Captions (-7.957) a pizza sitting on top of a wooden table
a table with plates and forks and knives close by
a display of a very long food is seen layer across a counter
a large meal on a dining room table
a large french bread pizza spanning the length of a table
the entree on the long plate is in the middle of the wooden table

10. Image Captions (-8.631) a group of people sitting around a table
man is holding up a photograph and smiling
a man from a clock shop is holding up a photo of an enormous ribbon clock
a man in a shop holding a picture of two men
a man holding a photograph up in front of him with both hands
an older gentleman is holding up an antique at a clock store

11. Next Week 3+ epochs with GoogLeNet
Continue optimizing model hyperparameters
Work on further integrating Cuda