1. Mixed reality week (2nd Nov)
CSCI 2951-I
19th October 2016

2. Week 9: Mixed Reality Week
Class splits into subgroups (5 groups, 4x3 + 1x2)
Each subgroup takes one facet of these devices:
Hardware + basic input image generation (e.g., depth)
Tracking + mapping
Gesturing + interface
Content creation + rendering
Eye + optics
25 minute slots – minute presentation, 7-10 discussion
A great literature review

3. Week 9: Mixed Reality Week
Concentrate on two devices: HoloLens and Magic Leap
Research challenge: how do they work?
No generic marketing speak; no ‘GPU of the brain’.
Given the known/hypothesized constraints on the platforms, which algorithms are they most likely to use?
Not specifically published, but clues everywhere…
Who makes them?
What have those companies/labs published?
Who did they hire recently, and what did they publish that might be causal?

4. Week 9: Mixed Reality Week
Other possible questions:
What are the alternative approaches? Given the constraints on the platform, why didn’t they go with those alternatives?
What might be possible in the future, given modest predictions?
In 2 years
In 5 years
In 10 years? Serious punt here; v. difficult.
Fill in the form on the course webpage (by Weds 9am next week).
I will let you know the groups; after that, self organize.

5. Project proposals (2nd Nov)

6. Projects (40% of grade) Week 7-9: Form groups (max 4 person).
Week 9: Propose a project to me that is appropriate for the size of the group and the time allowed (2 pages). Both the student group and the instructor will agree on the project scope.
Weeks 10-14: Work on the project in a self-directed manner.
Week 12: Check in with instructor via a project report stating the progress and current challenges (2 pages).
Week 14: Demo the project in the last week of class, discuss their outcomes, and submitting a write-up of the project in the format of a conference paper.

7. Project (40% of grade) Open-ended
If you need equipment, I will try and get it (within budget).
Implement existing state of the art work: OK!
Everything moderated by scope, so proposal is important:
Basic implementation of PatchMatch -> ~ 4 hours work for one person; insufficient.
Fast implementation of PatchMatch, on GPU, with demo app -> Serious.

8. Project (40% of grade) Core algorithmic work + evaluation is fine.
Trying to implement and improve a technique.
Comparing against benchmark dataset.
Systems work + evaluation is fine.
Taking existing implemented components.
Put together to solve an applied problem.
Justifying why they should be combined; evaluate against alternative choices.

9. Project ideas HyperDepth implementation + evaluation (Marley)
Learning-based stereo from image pairs (Lucas)
Style transfer (Prisma) (for AR?) (Sam)
Generative adversarial network for synthesis (Vivek)
Satellite-based laser scanning (Aaron)
Body-mounted camera array image synthesis + gesture control
How to ‘edit’ existing objects in augmented reality? (deformation lamps?)
360 video combination/interface/synthesis (class footage, Muse concert)

10. Discussion board for forming groups
piazza.com/brown/fall2016/csci2951i/home
URL on course web page.
Proposal due 2nd Nov. (two weeks)