Mixed reality week (2nd Nov) CSCI 2951-I 19th October 2016

Week 9: Mixed Reality Week Class splits into subgroups (5 groups, 4x3 + 1x2) Each subgroup takes one facet of these devices: Hardware + basic input image generation (e.g., depth) Tracking + mapping Gesturing + interface Content creation + rendering Eye + optics 25 minute slots – 15-18 minute presentation, 7-10 discussion A great literature review

Week 9: Mixed Reality Week Concentrate on two devices: HoloLens and Magic Leap Research challenge: how do they work? No generic marketing speak; no ‘GPU of the brain’. Given the known/hypothesized constraints on the platforms, which algorithms are they most likely to use? Not specifically published, but clues everywhere… Who makes them? What have those companies/labs published? Who did they hire recently, and what did they publish that might be causal?

Week 9: Mixed Reality Week Other possible questions: What are the alternative approaches? Given the constraints on the platform, why didn’t they go with those alternatives? What might be possible in the future, given modest predictions? In 2 years In 5 years In 10 years? Serious punt here; v. difficult. Fill in the form on the course webpage (by Weds 9am next week). I will let you know the groups; after that, self organize.

Project proposals (2nd Nov)

Projects (40% of grade) Week 7-9: Form groups (max 4 person). Week 9: Propose a project to me that is appropriate for the size of the group and the time allowed (2 pages). Both the student group and the instructor will agree on the project scope. Weeks 10-14: Work on the project in a self-directed manner. Week 12: Check in with instructor via a project report stating the progress and current challenges (2 pages). Week 14: Demo the project in the last week of class, discuss their outcomes, and submitting a write-up of the project in the format of a conference paper.

Project (40% of grade) Open-ended If you need equipment, I will try and get it (within budget). Implement existing state of the art work: OK! Everything moderated by scope, so proposal is important: Basic implementation of PatchMatch -> ~ 4 hours work for one person; insufficient. Fast implementation of PatchMatch, on GPU, with demo app -> Serious.

Project (40% of grade) Core algorithmic work + evaluation is fine. Trying to implement and improve a technique. Comparing against benchmark dataset. Systems work + evaluation is fine. Taking existing implemented components. Put together to solve an applied problem. Justifying why they should be combined; evaluate against alternative choices.

Project ideas HyperDepth implementation + evaluation (Marley) Learning-based stereo from image pairs (Lucas) Style transfer (Prisma) (for AR?) (Sam) Generative adversarial network for synthesis (Vivek) Satellite-based laser scanning (Aaron) Body-mounted camera array image synthesis + gesture control How to ‘edit’ existing objects in augmented reality? (deformation lamps?) 360 video combination/interface/synthesis (class footage, Muse concert)

Discussion board for forming groups piazza.com/brown/fall2016/csci2951i/home URL on course web page. Proposal due 2nd Nov. (two weeks)