1. Pi In The Sky (Storing Big Data on Cloud) Jenish Koirala Claflin University Mentors: Dr. Raghu Raj, Dr. Richard Loft SIParCS at Mesa Lab, NCAR Boulder, Colorado

2. Objective To provide easy access to observational weather data through a low cost platform To structure and synchronize data for querying and displaying purposes 2

3. Technologies Used Raspberry Pi Weather Sensors Cloud Storage Web Interface ~$200 3

4. What is a raspberry pi? GPIO pins usb ports Ethernet port HDMI port Raspberry Pi 2 Model B (Why?) 900MHz quad-core ARM Cortex A7-Processor 1GB RAM Raspberry Pi 2 Model B ~6 times faster Its just $35 4

5. System Overview Collects Data Displays Information Uploads Files to cloud User asks information Transmits Data Queries Receives Reads data from file Populates database Weather Sensors Sensor Processor Web Server Cloud Server Web InterfaceDatabase OwnCloud 5

6. Cloud Storage Why Cloud Storage? Data can be distributed and can be accessed anywhere from the world Cloud is cool! Without providing a detailed knowledge on the infrastructure used behind it. An internet based storage system What is Cloud Storage? 6

7. OwnCloud File hosting software (like Dropbox) Version used: 7.0.1 Why OwnCloud? Open Source Already been used and tested on RPi Free Free Free!! 7

8. OwnCloud System Architecture How this was achieved? disk1 disk2 disk3 Sensors Sensor Processor 8

9. Modules and Libraries Pushing files using Pyocclient Library disk2 Making a copy of file on disk1 or disk3 use Secure Copy disk2 disk3 Sensor Processor 9 put_file(‘path/to/owncloudDir’, ‘path/to/localfile’) os.system(”rsync –rvz /path/to/localdatabase pi@ipaddress: /path/to/remotedatabase”) Syncing Database on all disks using Remote SYNC os.system(”scp FILE USER@SERVER:PATH”)

10. Data and Storage Specification Data: ~8 Kilo Bytes per hour (1 sensor Module) ~70.08 Mega Bytes per year Storage: Toshiba 1 Tera Bytes Canvio Interface Transfer Rate: Up to 5 Giga Bytes per second (USB 3.0) Up to 480 Mega Bytes per second (USB 2.0) Capacity Check: 1000 more sensor modules ~8000 Kilo Bytes per hour 8000 * 24 * 365 = ~70.08 Giga Bytes Capacity to store ~14 years data 10

11. Populated Database Connect to MySQL Server Populate the Database Fully Populated Database Composite Primary Key 11 Imported ‘MySQLdb’ module in python script to connect to MySQL database

12. Snap Shots 12 Pressure(KiloPascals) Time(HH:MM)

13. Future Work Compare and implement (if necessary) commercially available cloud platform like Google cloud platform and Amazon EC2 Implement a parallel distributed file system for example, Lustre on Raspberry Pi. 13

14. Image Credits https://upload.wikimedia.org/wikipedia/en/thumb/c/cb/Raspberry_P i_Logo.svg/810px-Raspberry_Pi_Logo.svg.png https://upload.wikimedia.org/wikipedia/en/thumb/c/cb/Raspberry_P i_Logo.svg/810px-Raspberry_Pi_Logo.svg.png http://i0.wp.com/www.switchdoc.com/wp- content/uploads/2014/10/IMG_0833.jpg http://i0.wp.com/www.switchdoc.com/wp- content/uploads/2014/10/IMG_0833.jpg http://www.businesskorea.co.kr/sites/default/files/field/image/cloud %20computing_0.jpg http://www.businesskorea.co.kr/sites/default/files/field/image/cloud %20computing_0.jpg http://png- 2.findicons.com/files/icons/977/rrze/720/database_mysql.png http://png- 2.findicons.com/files/icons/977/rrze/720/database_mysql.png 14

15. Acknowledgements Dr. Richard Loft Dr. Raghu Raj Kumar Rashmi Oak Harish Ramchandaran Priyanka Sanghavi Amogh Simha 15

16. Questions? 16