1. Deep Learning for Natural Language Processing Tambet Matiisen 31.08.2015

2. We will learn about word vectors and distributed semantics, neural networks and backpropagation, recurrent neural networks for language modeling. Can be also considered as a general course in artificial neural networks, especially recurrent ones.

3. Where to get info Seminar: – on Mondays 12:15 @ Liivi 2-512. Homepage: – https://courses.cs.ut.ee/2015/dmseminar/fall/Main/DeepLearning https://courses.cs.ut.ee/2015/dmseminar/fall/Main/DeepLearning List: deeplearning@lists.ut.eedeeplearning@lists.ut.ee – Send “SUBSCRIBE deeplearning ” to sympa@lists.ut.ee.sympa@lists.ut.ee Other issues: – Send e-mail to tambet@ut.ee.tambet@ut.ee

4. Prerequisites Experience with Python and Numpy. –...or willingness to learn. Basic calculus and linear algebra. – taking derivatives, multiplying matrices. Basic probability and statistics. – conditional probability, log-likelihood. Machine learning basics. – loss function, gradient descent.

5. Organization At home: – Watch lectures (preferably with your study group) – Review lecture notes for test – Work through home assignments In seminar: – A test based on lecture notes (4x) – Homework presentation (7x) – Project presentations (1-2x)

6. Tests Created by one of the students (or study group?). – Must be presented to me 3 days before the class for review. Can use Google Forms, paper, etc. – Google Forms might be easier to process later. Based on lecture notes/video/slides. – Should measure undestanding, i.e. mini-tasks are welcome. In seminar: – Discussion of material in forthcoming test (~30 min). – Test itself, open-book (~30 min). – Discussion of right answers (~30 min). Graded by the creator. – Results are scaled accordig to max points. Can be taken only in class. – Must collect at least 60% of points from tests you didn’t create.

7. Homeworks Mostly in Python. – Version 2.7. – In Windows use Anaconda distribution. Some derivations as well. Will be presented in seminar. Everybody must do all homeworks. – But you are welcome to work in groups. Must be sent to me 3 days after presentation.

8. Projects In addition you can do a project based on things you learned in this course. This can be application of the method to your dataset, replication of some results etc. Must register to “MTAT.03.275 Special Assignment in Data Mining” (later in semester).

9. How to pass To get 3 points: – Create one test or present one homework. – Collect at least 60% of points from tests. – Submit all homeworks. To get additional 3 points: – Produce report of your project. – Present your project to others.

10. For the next time Form study groups Watch intro video Go through Python tutorial – There will be a test test (non-graded). Review math prerequisites