Presentation and project

Presenter Rule of all: Be informative Be organize Anticipate audience can learn something “new” Should be easy to digest but not too trivial Avoid significant overlap with prior presentations Be organize From high-level to detail Simple demo will be great

Audience You are asked to rate each presenter I will put the “task” up on canvas It will be “graded” and a part of the 70% Be critical. Treat yourself as a judge for some context. Need to comment on strength and weaknesses of a presentation Grading metric for review (2 points max) Below average (1 point) Average (1.5 point) Above average (2 points) Your rating should be submitted before mid-night of the day after presentation You have ~30 hours Late submission won’t be accepted Your input will be used to grade the presenter Presenter should submit review also. It is treated as a self-appraisal and won’t be used to grade your presentation

Final Project Most important (40%) and lasting result Project types: Start early and clearly define your task and dataset Project types: Apply existing neural network model to a new problem Implement a complex architecture for old problem Come up with a new neural network model Theory

Procedures for Type 1-3 Define Task Define Dataset Define your metric Example: summarization of text Define Dataset Search for public dataset first Sometimes conference/competition will provide dataset E.g., Document Understanding Conference (DUC) Define your own (harder, need more new baselines) Dataset from your own research Clever use of public data E.g., For text summarization, can use Wikipedia: intro paragraph and the rest of the article Be creative, can try to look into blogs and news also for natural language processing Define your metric Search online for well established metrics on your task Don’t invent your own arbitrarily. E.g., for text summarization, Rogue (Recall-oriented understudy for gisting evaluation) defines n-gram overlap to human summaries

Implement existing neural net model Split your dataset Training/validation/testing Academic dataset often come pre-split Don’t try to peek into your test set until the very last moment Establish a baseline Implement the simplest model first (for classification, logistic regression on some simple features?) Compute metrics on training and validation sets Analyze errors If metrics are amazing and no errors: done, problem was too easy, restart :) Implement existing neural net model Compute metric on training and validation sets Analyze output and errors Minimum bar for this class (guarantee half of the project score)

Try out different model variants CNN/RNN/Hybrid? Depth/width variation? ReLU/tanh/leaky ReLU/etc.? Regularization tricks? Gradient flow improvement? … Some tips and suggestions: Always be close to your data Try to visualize the dataset collect summary statistics Inspect the error events Analyze how different hyperparameters affect performance

Project proposal (Due on 3/14) Define problem Tentative approach(s) Names of group members How tasks will be split (if group project) Work on the project now if you can. Don’t wait until after submitting your proposal. If you are not sure whether your project idea is good, just drop me an email

Grading metric (40%) Presentation: (10 out of 40) clarity, structure, references Informative: e.g., background literature survey good insights and discussions of methodology, analysis, results, etc. Technical: (15 out of 40) Correctness, innovation, depth (ad hoc/theoretical-based) Evaluation and results: (15 out of 40) sound evaluation metric thoroughness in analysis and experimentation results and performance Labor cost adjustment No of members 1 2 3 4 5 6 7> Deduction (in %) 10 20 40 80

Submission (due on 5/9) 20 - 40 minutes screencast/video “presentation” explaining what you have done Longer video is not always better Should include Problem description and motivation What did you try to solve Can be shorter if it is a common problem (e.g., segmentation) Your approach(s) Evaluation Discussion: why it works or doesn’t work Written report: not mandatory. But maximum 20% extra credit 5 times late penalty (25% per day)

Expectation Quality Grade Just submitting 1/4 Department-level competition 2/4 University-level competition 3/4 National-level competition 4/4

Final grade A: ~ 85% or above B: 70%-85% C: 55%-70% D: 40%-55% F: Below 40%