Hololens: first impressions Bart Trzynadlowski http://trzy.org NYC HoloLens Developers Meetup November 28, 2016 New York, NY

overview Who Am I? What Have I Done? What Have I Learned? How I am approaching HoloLens What Have I Done? What Have I Learned? Unity HoloLens Game design challenges in AR Technical challenges Game Concepts

Who am i? I’m Bart! http://trzy.org Twitter: @BartronPolygon Electrical engineer turned software developer Perhaps best known for my open source Sega Model 3 arcade platform emulator: http://supermodel3.com

Video of Sega Model 3 emulator removed. Visit http://www. youtube Video of Sega Model 3 emulator removed. Visit http://www.youtube.com/BartTrzynadlowski for sample footage.

WHO AM I? How I AM approaching hololens My strengths Deep programming background (C/C++, assembly language) Knowledge of 3D graphics Experienced in high-performance systems (low-latency trading) My weaknesses Zero artistic ability Never developed a complete game from start to finish, let alone in a modern development environment

What have I done? In the past ~3 months: Learned Unity Learned the very basics of 3DS Max to create simple inanimate objects and rig others’ 3D models (from http://www.turbosquid.com) HoloLens: Spatial perception Occlusion/3D hole-in-wall effects

Video of Desert Strike for Sega Genesis removed Video of Desert Strike for Sega Genesis removed. The Strike series served as inspiration for my helicopter demo.

HoloLens demo footage removed. View it at: https://youtu

WHAT HAVE I LEARNED? UNity

WHAT HAVE I LEARNED? UNity Impressions: Very conducive to rapid prototyping Powerful engine – I feel redundant as a programmer! Easy to begin learning (start w/ Roll-a-Ball tutorial: https://unity3d.com/learn/tutorials) Extremely extensive support community – even my ultra-specific, ultra-obscure Google “How do I …?” queries yield useful forum answers! Overall: very impressed!

WHAT HAVE I LEARNED? UNity Some cons: Lots of “magic” Extensive use of C# getters/setters that do heavy/non-obvious work (e.g., Mesh.vertices[]) Occasional violation of the “principle of least surprise” Best practices not obvious You’ll see some weird, hacky stuff on the forums

WHAT HAVE I LEARNED? HoloLens Dev advice: Familiarize yourself with Microsoft’s HoloToolkit package! Learn to save your own room mesh with HoloLens web GUI and use in Unity Editor with HoloToolkit’s ObjectSurfaceObserver #if UNITY_EDITOR  Very useful! Room mapping best done ahead of time (e.g., Young Conker and RoboRaid) Too slow in real-time Analyzing the environment will be expensive and ought to be done once

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR No control over camera! Hard to deliver a cinematic experience User might not care to look where you want them to Also a challenge in VR! Environment totally unpredictable Object placement, level design Spatial meshes can be noisy Object scale

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR HoloLens specific: First-time users befuddled by gestures and often have no idea what to do Instinctively use their hands to grasp and reach for things Pain in the neck! Gameplay on floors and tables is awesome but the device is front-heavy and can cause neck pain quickly Consider having gameplay take place above eye level! (More comfy to look upwards.)

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Smarter object placement

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Smarter object placement

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Real-time spatial mesh deformation

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Real-time spatial mesh deformation

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Real-time spatial mesh deformation

WHAT HAVE I LEARNED? GAME DESIGN CHALLENGES IN AR Work-in-progress: Real-time spatial mesh deformation

WHERE TO FROM HERE? Game concepts Simulation / world-building

WHERE TO FROM HERE? Game concepts Simulation / world-building

WHERE TO FROM HERE? Game concepts Observation (e.g., Nintendo’s Fruit from Game & Wario collection)

WHERE TO FROM HERE? Game concepts Real-time strategy (I mean, duh…) EtherWars 

GAME CONCEPTS Physics puzzlers

THAT’S ALL, FOLKS! (Any questions?)