STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM – Mobicom 2014

STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM – Mobicom 2014 A Tightly-Packed Coding Scheme for Camera-based Visible Light Communication

4 STYROFOAM – Mobicom 2014 Screen  Camera Communication

STYROFOAM – Mobicom Screen  Camera Communication

STYROFOAM – Mobicom Benefits Pairless infrastructure Multi-cast transmission Targeted interaction Use Cases Device-Specific Broadcast Wearable Transmissions Easy Access Media

Camera-based VLC Research Challenges Efficient Marker Tracking (COBRA) Blur-aware Ordering (COBRA) Unsync’ed Camera/Screen rates (LightSync) Increasing block bit-depth (Styrofoam) 7 STYROFOAM – Mobicom 2014

OFF ON 8 STYROFOAM – Mobicom 2014 Increasing block bit-depth (Styrofoam)

STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation

STYROFOAM – Mobicom sDsD sCsC sBsB sAsA

sAsA sBsB sCsC sDsD 11 STYROFOAM – Mobicom 2014 TX:

sAsA sBsB sCsC RX: sAsA sBsB sCsC sDsD 12 STYROFOAM – Mobicom 2014

sAsA sBsB Inter-symbol interference sAsA sBsB sCsC sDsD 13 STYROFOAM – Mobicom 2014 t TX: RX:

TX: RX: sDsD sAsA sBsB r j = α 0,j s A + α 1,j s B sAsA sBsB sCsC TX_ α 0 : TX_ α 1 : RX_ α 0 : RX_ α 1 : α 0,j α 1,j 14 STYROFOAM – Mobicom 2014 t

STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 15 STYROFOAM – Mobicom 2014

TX: t RX: sDsD sAsA sBsB r j = α 0,j s A + α 1,j s B sAsA sBsB sCsC 16 STYROFOAM – Mobicom 2014 Styrofoam Coding Scheme Guaranteeing resolvability

Constraint #1 Symbol Visibility TX: RX: sDsD sAsA sBsB sCsC Capture Period 17 STYROFOAM – Mobicom 2014

Constraint #1 Symbol Visibility 18 STYROFOAM – Mobicom 2014 TX: RX: sDsD sAsA sBsB sCsC Capture Period Capture Period < Symbol Period

Constraint #1 Symbol Visibility Capture Period < Symbol Period This guarantees each symbol is RXed with α > STYROFOAM – Mobicom 2014 TX: RX: sDsD sAsA sBsB sCsC Capture Period

Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels. TX: RX: sDsD sAsA sBsB sAsA sBsB sCsC Color-cross-talk 20 STYROFOAM – Mobicom 2014

Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels. TX: RX: sDsD sAsA sBsB sAsA sBsB sCsC Color-cross-talk 21 STYROFOAM – Mobicom 2014

Constraint #3 Symbol Resolvability TX: RX: sDsD sAsA sBsB r j = α 0,j s B + α 1,j s C sAsA sBsB sCsC s 1B sCsC r i = α 0,i s A + α 1,i s B sCsC sDsD r k = α 0,k s 1 + α 1,k s 2 22 STYROFOAM – Mobicom 2014 Mixed Capture: 1 eqn, 2 var

sAsA sBsB sBsB sCsC sCsC Constraint #3 Symbol Resolvability TX: RX: sDsD sAsA sBsB sCsC 23 STYROFOAM – Mobicom 2014 Mixed Capture: 1 eqn, 2 var Pure Capture: 1 eqn, 1 var r j = α 0,j s B + α 1,j s C r i = α 0,i s A + α 1,i s B r k = α 1,k s C Ensure pure captures with Styrofoam Blocks

Constraint #3 Symbol Resolvability sAsA sBsB sCsC ~~ αCαC αBαB αAαA A mixed capture target is always followed by the mixed symbol’s capture. Spacing => α A < α B < α C 24 STYROFOAM – Mobicom 2014 Ensure pure captures with Styrofoam Blocks Constraint #1: Visibility Spacing Spacing => Every symbol appears with α>0.5

Styrofoam Block Size Styrofoam blocks are only useful when symbol α>0.5 Remainder of block is < 0.5. Styrofoam blocks need only be half as wide as symbol blocks sAsA sBsB sCsC ~~ 25 STYROFOAM – Mobicom 2014 ~~

Display each symbol for two frames, Styrofoam Block for one frame Display Frame Rate < 2 x Camera Frame Rate e.g. (Display : 30 FPS, Camera : 15 FPS)` Styrofoam Frame Timing 26 STYROFOAM – Mobicom 2014 sDsD sEsE sFsF ~~ sDsD sEsE sFsF ~ sDsD sEsE sFsF ~ sDsD sEsE sFsF ~ sDsD sEsE sFsF sGsG sHsH sIsI ~ sGsG sHsH sIsI ~ sAsA sBsB sCsC ~ sAsA sBsB sCsC s1s2 s3s3 ~ s1s2 s3s3 s1s2 s3s3 ~ s1s2 s3s3 ~ s1s2 s3s3 ~ s1s2 s3s3

Styrofoam Coding sAsA …sNsN ~ Data Rate sAsA …sNsN Bit depth/Block/Display Frame ~ Decode Latency 27 STYROFOAM – Mobicom 2014 Frame Latency

Per-block, Per-Display Frame Rate Datarate Comparisons Traditional (1-bit, blank alternate frames) : 0.5 bits COBRA (5-values, half-frame rate) : 1.16 bit LightSync ( “Synchronization” interference blocks ): 1 bit Styrofoam: 1.7 bits 1.7 bits * 720 blocks * 60 fps = 73.4 kbps STYROFOAM – Mobicom

STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 29 STYROFOAM – Mobicom 2014

Implementation In paper: –Transmitter: Blackberry Z30, 60 FPS –Receiver: Nokia Lumia, 30 FPS Hardware ISO & Exposure control –Decoded offline w/ MATLAB Demo: –Nexus 5 + Nexus 5 Android L hardware controls 30 STYROFOAM – Mobicom 2014

Styrofoam Barcode (Nexus 5) STYROFOAM – Mobicom

Styrofoam Barcode (Simplified Version) 32 STYROFOAM – Mobicom 2014 α max α1α1 α0α0

Sample TX STYROFOAM – Mobicom EvenOdd

34 STYROFOAM – Mobicom 2014 s1s2 ~~

Decoded Pairs 35 STYROFOAM – Mobicom 2014

Additional Styrofoam Research Challenges Steganography –Can we hide the Styrofoam blocks? Power Efficiency –Of the display? Of the receiver? Leveraging YUV color channels –Mixing colors safely? Geometric problems –Screen -> Camera angles? 36 STYROFOAM – Mobicom 2014

David Ramirez Robert LiKamWa Jason Holloway STYROFOAM Characterize and constrain inter-symbol interference with Styrofoam spacing blocks 37 STYROFOAM – Mobicom 2014 Behnaam Aazhang Lin Zhong Ashok Veeraraghavan Advisors: Authors:

Constraints Display Frame > Symbol Frame –Symbol Visibility for at least half capture Styrofoam block spacing –To guarantee pure captures –Block at least half width of symbol Color Matching –To deter color filter issues STYROFOAM – Mobicom