1. STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM – VLCS @ Mobicom 2014

3. STYROFOAM Robert LiKamWa, David Ramirez, Jason Holloway Advisors: Lin Zhong, Behnaam Aazhang, Ashok Veeraraghavan Rice University STYROFOAM – VLCS @ Mobicom 2014 A Tightly-Packed Coding Scheme for Camera-based Visible Light Communication

4. 4 STYROFOAM – VLCS @ Mobicom 2014 Screen  Camera Communication

5. STYROFOAM – VLCS @ Mobicom 2014 5 Screen  Camera Communication

6. STYROFOAM – VLCS @ Mobicom 2014 6 Benefits Pairless infrastructure Multi-cast transmission Targeted interaction Use Cases Device-Specific Broadcast Wearable Transmissions Easy Access Media

7. Camera-based VLC Research Challenges Efficient Marker Tracking (COBRA) Blur-aware Ordering (COBRA) Unsync’ed Camera/Screen rates (LightSync) Increasing block bit-depth (Styrofoam) 7 STYROFOAM – VLCS @ Mobicom 2014

8. OFF ON 8 STYROFOAM – VLCS @ Mobicom 2014 Increasing block bit-depth (Styrofoam)

9. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation

10. STYROFOAM – VLCS @ Mobicom 2014 10 sDsD sCsC sBsB sAsA

11. sAsA sBsB sCsC sDsD 11 STYROFOAM – VLCS @ Mobicom 2014 TX:

12. sAsA sBsB sCsC RX: sAsA sBsB sCsC sDsD 12 STYROFOAM – VLCS @ Mobicom 2014

13. sAsA sBsB Inter-symbol interference sAsA sBsB sCsC sDsD 13 STYROFOAM – VLCS @ Mobicom 2014 t TX: RX:

14. TX: RX: sDsD sAsA sBsB r j = α 0,j s A + α 1,j s B sAsA sBsB sCsC TX_ α 0 : TX_ α 1 : RX_ α 0 : RX_ α 1 : α 0,j α 1,j 14 STYROFOAM – VLCS @ Mobicom 2014 t

15. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 15 STYROFOAM – VLCS @ Mobicom 2014

16. TX: t RX: sDsD sAsA sBsB r j = α 0,j s A + α 1,j s B sAsA sBsB sCsC 16 STYROFOAM – VLCS @ Mobicom 2014 Styrofoam Coding Scheme Guaranteeing resolvability

17. Constraint #1 Symbol Visibility TX: RX: sDsD sAsA sBsB sCsC Capture Period 17 STYROFOAM – VLCS @ Mobicom 2014

18. Constraint #1 Symbol Visibility 18 STYROFOAM – VLCS @ Mobicom 2014 TX: RX: sDsD sAsA sBsB sCsC Capture Period Capture Period < Symbol Period

19. Constraint #1 Symbol Visibility Capture Period < Symbol Period This guarantees each symbol is RXed with α > 0.5 19 STYROFOAM – VLCS @ Mobicom 2014 TX: RX: sDsD sAsA sBsB sCsC Capture Period

20. Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels. TX: RX: sDsD sAsA sBsB sAsA sBsB sCsC Color-cross-talk 20 STYROFOAM – VLCS @ Mobicom 2014

21. Constraint #2 Symbol Color Fidelity Adjacent symbols must match color channels. TX: RX: sDsD sAsA sBsB sAsA sBsB sCsC Color-cross-talk 21 STYROFOAM – VLCS @ Mobicom 2014

22. Constraint #3 Symbol Resolvability TX: RX: sDsD sAsA sBsB r j = α 0,j s B + α 1,j s C sAsA sBsB sCsC s 1B sCsC r i = α 0,i s A + α 1,i s B sCsC sDsD r k = α 0,k s 1 + α 1,k s 2 22 STYROFOAM – VLCS @ Mobicom 2014 Mixed Capture: 1 eqn, 2 var

23. sAsA sBsB sBsB sCsC sCsC Constraint #3 Symbol Resolvability TX: RX: sDsD sAsA sBsB sCsC 23 STYROFOAM – VLCS @ Mobicom 2014 Mixed Capture: 1 eqn, 2 var Pure Capture: 1 eqn, 1 var r j = α 0,j s B + α 1,j s C r i = α 0,i s A + α 1,i s B r k = α 1,k s C Ensure pure captures with Styrofoam Blocks

24. Constraint #3 Symbol Resolvability sAsA sBsB sCsC ~~ αCαC αBαB αAαA A mixed capture target is always followed by the mixed symbol’s capture. Spacing => α A < α B < α C 24 STYROFOAM – VLCS @ Mobicom 2014 Ensure pure captures with Styrofoam Blocks Constraint #1: Visibility Spacing Spacing => Every symbol appears with α>0.5

25. Styrofoam Block Size Styrofoam blocks are only useful when symbol α>0.5 Remainder of block is < 0.5. Styrofoam blocks need only be half as wide as symbol blocks sAsA sBsB sCsC ~~ 25 STYROFOAM – VLCS @ Mobicom 2014 ~~

26. Display each symbol for two frames, Styrofoam Block for one frame Display Frame Rate < 2 x Camera Frame Rate e.g. (Display : 30 FPS, Camera : 15 FPS)` Styrofoam Frame Timing 26 STYROFOAM – VLCS @ Mobicom 2014 sDsD sEsE sFsF ~~ sDsD sEsE sFsF ~ sDsD sEsE sFsF ~ sDsD sEsE sFsF ~ sDsD sEsE sFsF sGsG sHsH sIsI ~ sGsG sHsH sIsI ~ sAsA sBsB sCsC ~ sAsA sBsB sCsC s1s2 s3s3 ~ s1s2 s3s3 s1s2 s3s3 ~ s1s2 s3s3 ~ s1s2 s3s3 ~ s1s2 s3s3

27. Styrofoam Coding sAsA …sNsN ~ Data Rate sAsA …sNsN Bit depth/Block/Display Frame ~ Decode Latency 27 STYROFOAM – VLCS @ Mobicom 2014 Frame Latency

28. Per-block, Per-Display Frame Rate Datarate Comparisons Traditional (1-bit, blank alternate frames) : 0.5 bits COBRA (5-values, half-frame rate) : 1.16 bit LightSync ( “Synchronization” interference blocks ): 1 bit Styrofoam: 1.7 bits 1.7 bits * 720 blocks * 60 fps = 73.4 kbps STYROFOAM – VLCS @ Mobicom 2014 28

29. STYROFOAM Screen -> Camera Communication Understanding the channel Styrofoam Coding Scheme Implementation 29 STYROFOAM – VLCS @ Mobicom 2014

30. Implementation In paper: –Transmitter: Blackberry Z30, 60 FPS –Receiver: Nokia Lumia, 30 FPS Hardware ISO & Exposure control –Decoded offline w/ MATLAB Demo: –Nexus 5 + Nexus 5 Android L hardware controls 30 STYROFOAM – VLCS @ Mobicom 2014

31. Styrofoam Barcode (Nexus 5) STYROFOAM – VLCS @ Mobicom 2014 31

32. Styrofoam Barcode (Simplified Version) 32 STYROFOAM – VLCS @ Mobicom 2014 α max α1α1 α0α0

33. Sample TX STYROFOAM – VLCS @ Mobicom 2014 33 EvenOdd

34. 34 STYROFOAM – VLCS @ Mobicom 2014 s1s2 ~~

35. Decoded Pairs 35 STYROFOAM – VLCS @ Mobicom 2014

36. Additional Styrofoam Research Challenges Steganography –Can we hide the Styrofoam blocks? Power Efficiency –Of the display? Of the receiver? Leveraging YUV color channels –Mixing colors safely? Geometric problems –Screen -> Camera angles? 36 STYROFOAM – VLCS @ Mobicom 2014

37. David Ramirez Robert LiKamWa Jason Holloway STYROFOAM Characterize and constrain inter-symbol interference with Styrofoam spacing blocks 37 STYROFOAM – VLCS @ Mobicom 2014 Behnaam Aazhang Lin Zhong Ashok Veeraraghavan Advisors: Authors:

38. Constraints Display Frame > Symbol Frame –Symbol Visibility for at least half capture Styrofoam block spacing –To guarantee pure captures –Block at least half width of symbol Color Matching –To deter color filter issues STYROFOAM – VLCS @ Mobicom 2014 38