1. Discovery project Find swarmlets for devices around you
Victor Nouvellet
Terraswarm - April 2017
Visiting Student at University of California Berkeley
Berkeley 04/05/2017

2. [Dad] Which app should I use to turn on the light?
[Mum] Can you explain me how to close the garage from my phone?
[Sister] I forgot that we had a connected padlock!
[Brother] I don’t have time to setup all this things!
[Grandma] Way too complicated for me…
[Me] ➡ Explaining. Configuring. Repeat.

3. Who am I? Victor Nouvellet
Visiting Student Researcher, University of California Berkeley
&
Senior MSc Student,
Computer Science & Information Technology
Department at INSA Lyon

4. Outline The Discovery Problem Description & challenges
A solution with accessors
Work in progress
I will mainly talk about vision, design and work in progress. Do not wait awesome results… yet!

5. THE DISCOVERY PROBLEM

6. But much less people know how to use them...
28.1 billion
# of IoT devices in 2017
But much less people know how to use them...
Source: Statistica

7. IoT: Number of connected devices worldwide (2012 to 2020)

8. A simple example : the Kubi
Implementation time: 1 week
Number of implemented accessors: 2 accessors
Number of implemented mobile app: 1 iOS app
Number of reused accessors: 1 accessor
Skills needed: Mobile, Bluetooth & JS development skills
What if we could improve that?

9. DESCRIPTION & CHALLENGES

10. What are we talking about?
Analogy: (almost) a piece of cake
Recipe
Swarmlet
Ingredients
Accessors / Actors
Chef
Software Engineer
Not everyone is a chef but…
...many want to use well-designed recipes to make the best cake

11. Where to find them?
Contextually (in your environment) mixing location systems
Location systems:
Global Positioning System (GPS) - outdoor
Networks signal strength: Wifi, Bluetooth, Zigbee - indoor & outdoor
Beacons - indoor & outdoor
Sounds - indoor & outdoor
Light - indoor
Absolute and relative location systems

12. When to find them? Devices have to be up to date for user usability
On user needs
On fixed periods
Using deterministic temporal semantics to synchronize devices actions
Absolute and relative location systems

13. Why is this needed?
Cyber-Physical Systems have to find a way to reach people needs.
It has to be done
Easily
Without code
With less configuration
When we think about it we obviously think about a sandboxed marketplace inside user’s devices

14. Challenges The global solution to tackle the problem should be:
Decentralized
Scalable
Adjustable
Modular
Simple (and it is hard)
Adjustable : levels for different kind of users

15. A SOLUTION WITH ACCESSORS

16. Missing parts & connections
Actors & accessors are the main components
A lot of Web standard components can already be used
Only a few components are missing

17. Discover reachable devices
Gather devices metadata you can reach on multiple kind of networks
Type
Range
Applications
Standards
Personal area network (PAN)
Within reach of a person
Cable replacement for peripherals
Bluetooth, ZigBee, NFC
Local area network (LAN)
Within a building or campus
Wireless extension of wired network
IEEE (WiFi)
Metropolitan area network (MAN)
Within a city
Wireless inter-network connectivity
IEEE (WiMAX)
Wide area network (WAN)
Worldwide
Wireless network access
Cellular (UMTS, LTE, etc.)
Each of us has at least one device with ubiquitous connectivity and few always-connected devices around him.

18. Discover reachable devices
Gather devices metadata you can reach on multiple kind of networks
Considering PAN & LAN only, discovery accessors for:
Wifi: arp protocol and management frames
Bluetooth/BLE: basic advertising information and eddystone
Zigbee: manufacturer-defined application objects analysis
NFC: tags protocols (libnfc)
Other: infrared, audio, etc...

19. Monitor connectivity and events
Devices can be in or out of reach, connected, disconnected, in idle mode...
Also, this information need to be constantly available along with other information like battery or signal strength.

20. Discovery project in a nutshell
Top-down approach:
Discovery Accessors: Discover devices around
Raw Context Accessor: Monitor connectivity and events
Context Aggregator Accessor: Find and store accessors
Contextual Broker: Classify accessors in user context
Mutable Accessors: Give general definitions of classes of accessors

21. WORK IN PROGRESS
(Bottom-up approach)

22. Classify and filter available accessors
Contextual Broker (100% done)
Use of mutable accessors to test for reification
Classify available accessors
Provide the list of available classes
Make a filter available to only detect some classes
Trigger a new discovery to get more information about a device and find a better top level accessor

23. Find accessors for device metadata
Accessor4Device (10% done)
One webservice :
[Soon] Multiple supported devices
[Soon] Multiple accessors
Context Aware (100% done)
This accessor generalizes the capability of REST.js by providing a simple interface for a user to select a concrete REST service and provides the context (set of inputs needed) to invoke that concrete REST service and set of outputs needed from the service.

24. Keep raw device data & monitor connectivity
Raw Context Accessor (not started yet)
Store devices data, metadata & states
Trigger lost & found devices
Trigger on connect & disconnect
Communicate with Discovery Accessors to start/end scans

25. Discovery Accessors
Should use capabilities of the host (Wifi, Bluetooth, Zigbee, NFC, ...).
The Discovery Accessor (100% done, made by Elizabeth Latronico)
Use Nmap or ARP to get MAC Address from IP
Other Discovery Accessors (not started yet):
A Bluetooth Discovery Accessor (BDA)
A Wifi Discovery Accessor (WDA)

26. Discovery Accessors Bluetooth Discovery Accessor (BDA)
Steps that BDA should follow:
Scan for devices
Connect to devices
Read services and characteristics
Detect Eddystone protocol

27. Discovery Accessors Wifi Discovery Accessor (WDA)
Use Nmap with -A and/or -sV options to detect OS and standard services.
A trade off decision between detection time and services information to be gathered has to be made.

28. Swarmlets & iOS devices relationship
iOS app as swarmlet host (100% done) - demo
JXCore host let iOS apps instantiate accessors and run swarmlets.
iOS app as accessor enabled device (80% done)
AccessorsKit is an iOS framework to drop in your app project and use in conjunction with AccessorModule iOS frameworks.
For now, two AccessorModules are being implemented:
KubiAccessorModule
CameraAccessorModule

29. Summary
Democratize cyber physical systems by enabling users to reuse existing swarmlets in their own context.
Big picture

30. What is next? Create a marketplace for accessors and swarmlets
Use machine learning to find the appropriate swarmlet in the context
Let Smart Assistants (Siri, Cortana, Alexa, Google Assistant) handle the swarmlet selection and configuration process with user
Locally (or not) store configurations for already known accessors
Allow bots to create swarmlets themselves based on existing swarmlets, mutable accessors and I/O descriptions

31. Thank you! Any questions?