Camera-less Smart Laser Projector Author: Alvaro Cassinelli∗, Alexis Zerroug, and Masatoshi Ishikawa Ishikawa-Komuro Laboratory, University of Tokyo Jussi Angesleva The Berlin University of the Arts --------------- SIGGRAPH 2010 Emerging Technology TNUA M9893010 Virtual Reality 陳育羣

Outline Abstract Introduction Previous work 'Sticky Light' New prototype and apps Technical details Hardware and setup

Abstract Smart laser projector 'Augmenting' surfaces Prototype : 1. Sticky Light : Contours of drawing or Bouncing on flat figure 2. simulation of the refraction of light beams on 2d surface Snell Law Smart laser projector 可以投射在任何表面的雷射投影機，並同時將雷射當作掃描光束來擷取像是表面形狀、材 質、反射，相對運動等資訊 'Augmenting' surfaces 在任何材質上，將人眼看不見的資訊標示出來，基於實際或美學目的。 Prototype : 1. Sticky Light : Contours of drawing or Bouncing on flat figure 可生動地在3D物件邊緣描繪輪廓或平面彈跳。 simulation of the refraction of light beams on 2d surface 模擬光束在2d表面的反射。採用snell-law計算灰階區域的光束偏差來繪製。 Snell Law

3d座標可能不需要，在利用測量顏色、反射值後，用來調整可視光。 如果不用stereo camera立體掃描將會是掃描機的物理限制 1. Introduction Raster scanning mode or Vector mode sensing and scanning the surface Advantages : [1] Calibration is not needed at all Stereo Scan :Vein Viewer [3], infrared camera+DLP Extremly large depth of focus Variable resolution Simple and Compact optical system Projection at very long distance Raster scanning mode or Vector modesensing and scanning the surface 光柵掃描模式和向量模式 Advantages : [1] Calibration is not needed at all 不需校準，3D紋理的座標，可採用laser-based rangefinding tech.得知。 代表著不需要 立體攝影機或標記物。 3d座標可能不需要，在利用測量顏色、反射值後，用來調整可視光。 Q: 3D並有顏色，測量值不會被干擾？ Stereo Scan :Vein Viewer [3], infrared camera+DLP 一開始使用紅外線鏡頭去發現皮下紋理，再用DLP(一般投影機)標示出來 要顯示出真的位置，皮膚與鏡頭的相對位置和相對方向的得知是很重要的。 如果不用stereo camera立體掃描將會是掃描機的物理限制 Extremly large depth of focus Variable resolution 可對特定範圍精細掃描& Vector- graphic mode Simple and Compact optical system Projection at very long distance Vector-graphics mode 可以投在上百公尺以外的表面很適合戶外應用

1. Introduction cont. Applications of SLP: Translation of barcode information into directly readable text Augmentation of paintings, mural Ad. and logos using laser light Automatic contrast compensation Medical imaging, biosensing and biometrics. Advantage Architectural Spaces Spatial cueing Exo-displays Entertainment Applications of SLP: Translation of barcode information into directly readable text 掃描並解碼及投影成可讀資訊。 Augmentation of paintings, mural Ad. and logos using laser light 不用先校準，動態廣告看板 Automatic contrast compensation 自動對比補償：改進光源糟時的可讀性，不需外部光也可辨識，完全黑暗也可。 Medical imaging, biosensing and biometrics. 醫學影像、生物計量識別、生物傳感︰結合近紅外線鐳射光後，可看到偵測和註明皮下組織；可用不同波長偵測皮下的瘤，也可以用UV波長偵測牙齒的表面狀況，在牙齒病變區上投射出指示的光；還有可能直接用鐳射光做光線療法。 Advantage Architectural Spaces 作者們對於空間中的邊線及角落的即時偵測很有興趣，並用鐳射光對這些角落、邊線做擴增應用，ex: 未事先校準的投影 Spatial cueing 從視覺觀點來看可以做透視的增強。Ex:Mapping 對視線不清或有行動障礙的環境偵測並依此做指示的註明強化。 樓梯的箭頭 路標等。 很有實踐的可能性，因為可穿戴性，可以戴著邊走邊偵測及投影！ Exo-displays 外部顯示投影︰因為可戴性，所以可將資訊投在穿戴者周圍的任何表面。Ex: 有趣的領域是投在汽車上，這系統可以在路上行進時直接顯示和強調訊號，需依靠透視的演算法。 Entertainment 擴增實境的娛樂應用，可能可以引起興趣的是應用範圍的擴大(投影距離很遠)，所以此系統可以用在很大的場域，ex攀岩牆、足球場、滑雪坡。下一個例子裡，滑雪者的動作可以被偵測並繪製出來，然後可以顯示在巨大的滑雪坡上。雪面是很妙的投影表面，提供了高對比的顯示。

1. Introduction cont. Problems : Real – time extraction of relevant 3D features, radiometric properties, texture, speed and even temperature of the illuminated surface. This research is build on 'smart laser scanner' [7] and experiments on real-time scanning and projection of deformable surface using active lighting and a custom vision chip [8] 麻煩問題：輸入資料的即時解析，ex: 3D特徵、輻射特性、材質、速度和表面溫度 作者們對於可穿、迷你的感測投影機特別感興趣，可以外掛在衣服及車上。最近的實驗是關於：使用客製化的影像晶片和動態照明在可變形表面的即時掃描和投影。

2.Previous work 'Sticky Light' Air-hockey, Pin ball game It actually augmented the drawing content by scanning the lines and bouncing on the colors.

3.New Prototype and Apps Bouncing on lines and edges Ray tracing The scanned material cab be virtually anything. The needed is that scanned object presen enought contrast for the scanning beam, Ray tracing Note the scanning 'saccade' at the intterface between gray areas, and the deviation of the 'beam' following a simulated Snell Law.

4.Technical Details Baced on 3D tracking technology. Previeous Developed is 'smart laser scanner' [7] In the near future, it will use an RGB laser projector for completing the color palette.(RG) Hardware: Arduino Mega Custom made Lock-In amplifier A laser head projector of two laser diodes(r and g) Couple of galvano mirrors.(could be steering ) A sigle non-imaging photodetector. 沒有鏡頭跟投影機(像素式感測)，追蹤就像動作一樣非常的滑順和流暢很多雷射點可以被scanning head製造和控制 Uses a microcontroller both for steering the mirrors and for processing the photodiode signal. Arduino mega 產生100kHz 的方波，這訊號用來調整紅色雷射光束和被lock- in amplifier當作參考訊號 lock-in amplifier是超高Q-factor的頻帶過濾器,通常使用'一對參考訊號'來計算訊號的相位和振幅 為了簡化系統，和因為可調AM訊號的對應波長範圍在數百公尺之間, 我們估計相位是重要的基本常數。因此我們一直校準和設定混和訊號的相位,來獲得大的回應。

4.Technical Details cont. Processing: The microcontroller reads the lock-in amplifier output signal to discriminate between dark and bright zones (and a discrete number of gray levels). It also controls the position of the laser spot by generating analog signals driving the X/Y galvano-mirror pair. In such case, it generates a circular scanning saccade in order to compute the normal vector to the interface between the two different regions. From this information, it is straightforward to compute either the bouncing or the refracting direction.

5. Hardware and Setup The overall size is about 30x30x40cm

6. Conclusion We have demonstrated a portable prototype of a ‘smart laser pro-jector’ having endless applications from entertainment, to human computer interfaces to medical imaging. In the near future, we plan to demonstrate medical imaging applications (in the field of derma-tology and phlebo therapy), as well as research on ways to rendering the whole system compatible with a hand-held instrument using MEMS technology.

Thanks for your attention.