Pi In The Sky (Storing Big Data on Cloud) Jenish Koirala Claflin University Mentors: Dr. Raghu Raj, Dr. Richard Loft SIParCS at Mesa Lab, NCAR Boulder,

Slides:

Advertisements

Similar presentations

Andrew Stratton Linux based cut down mobile phone New Technology Old performance SD Card boot and disk Debian image standard Python.

Advertisements

Distributed Data Processing

Microsoft ® Official Course First Look Clinic Overview of Windows 8 By Ragowo Riantory, S.Kom, MCP.

1. Topics Is Cloud Computing the way to go? ARC ABM Review Configuration Basics Setting up the ARC Cloud-Based ABM Hardware Configuration Software Configuration.

MCITP: Microsoft Windows Vista Desktop Support - Enterprise Section 1: Prepare to Deploy.

IoT Hydroponics Project

CLOUD COMPUTING.  It is a collection of integrated and networked hardware, software and Internet infrastructure (called a platform).  One can use.

Addition to Networking.  There is no unique and standard definition out there  Cloud Computing is a general term used to describe a new class of network.

Cloud computing Tahani aljehani.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

A global, public network of computer networks. The largest computer network in the world. Computer Network A collection of computing devices connected.

RaspberryPi Ira Goldstein Siena College. What is a Raspberry Pi? University of Cambridge’s Computer Laboratory – Decline in skill level – Designed for.

IBIS System: Requirements and Components Lois M. Haggard Office of Public Health Assessment.

The Glasgow Raspberry Pi Cloud: A Scale Model for Cloud Computing Infrastructures By: Mugasa Hatwib.

Google Chromebook A New Way of Low Cost, Low Maintenance Computing.

PHOTOSWAP Albert Park & Brandon Ochs. What is PhotoSwap?  Social networking platform for iOS  Users share images with each other  Extract sensor data.

Juan Miguel Valverde Martínez.  What is a Home Media Center?  Raspberry Pi  Idea  Interface  Potential.

Computer Literacy for IC 3 Unit 1: Computing Fundamentals © 2010 Pearson Education, Inc. | Publishing as Prentice Hall.1 Chapter 1: Identifying Types of.

Computer Applications Unit D Remote Desktop, Cloud Storage, Dropbox.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

Pi In The Sky (Web Interface) Gaston Seneza Philander Smith College, Little Rock, AR SIParCS Intern Mentors: Dr. Richard Loft & Dr. Raghu Raj Kumar 1.

Computing Essentials 2014 Secondary Storage © 2014 by McGraw-Hill Education. This proprietary material solely for authorized instructor use. Not authorized.

Amazon Web Services BY, RAJESH KANDEPU. Introduction  Amazon Web Services is a collection of remote computing services that together make up a cloud.

Cloud Computing & Amazon Web Services – EC2 Arpita Patel Software Engineer.

LOGO 2 nd Project Design for Library Programs Supervised By Dr: Mohammed Mikii.

Presented by: Mostafa Magdi. Contents Introduction. Cloud Computing Definition. Cloud Computing Characteristics. Cloud Computing Key features. Cost Virtualization.

Chapter 3 Installing and Learning Software. 2Practical PC 5 th Edition Chapter 3 Getting Started In this Chapter, you will learn: − What is in an application.

Home Intrusion Detection System Andre, Irena, Priyanka, Balta.

Eduroam Monitoring Probes Scott Armitage. eduroam Monitor Probe Development Prototype Raspberry Pi ( £26 ) Dlink DWA160 ( £20 ) 4GB SD Card ( £7 )

CLASS Information Management Presented at NOAATECH Conference 2006 Presented by Pat Schafer (CLASS-WV Development Lead)

{ Cloud computing. Exciting and relatively new technologies allow computing to be a part of our everyday lives. Cloud computing allows users to save their.

Enterprise Cloud Computing

Data-Intensive Cloud Control for GENI GEC 10 Orca control framework March 15 th, 2011 Michael Zink, Prashant Shenoy, Jim Kurose, David Irwin and Emmanuel.

HOME SECURITY SYSTEM Senior Design 1 Fall 2015 [1]

Under The Guidance of Smt. Ch.Ratna Kumari Asst.Professor Submitted by M Ravi Kumar Roll No:10021F0006 M.C.A.

CLOUD COMPUTING. What is cloud computing ? History Virtualization Cloud Computing hardware Cloud Computing services Cloud Architecture Advantages & Disadvantages.

Introduction :  In the beginning, the calculator is the basic idea of computers out, then that arrived at the large computers.  in1980 or late seventies.

FILE MANAGEMENT Computer Basics 1.3. FILE EXTENSIONS.txt.pdf.jpg.bmp.png.zip.wav.mp3.doc.docx.xls.xlsx.ppt.pptx.accdb.

Information Systems in Organizations 5.2 Cloud Computing.

STORAGE LOCAL OR ONLINE. DATA STORAGE: DATA YOU STORE ONLINE FILES SUCH AS IMAGES, SPREADSHEETS, VIDEO OR MUSIC. ONLINE DATA STORAGE: WHEN FILES ARE STORES.

Cloud Computing By Reedy McGeady. What is Cloud Computing? Cloud Computing is using another organisations computer, which are known as hosts.

SQL Server 2012 Session: 1 Session: 4 SQL Azure Data Management Using Microsoft SQL Server.

Rasberry pi 2 model B. Selve computeren i rasberry pi’en.

Cloud Computing from a Developer’s Perspective Shlomo Swidler CTO & Founder mydrifts.com 25 January 2009.

Introducing the Raspberry Pi Nauru ICT Department April 2016.

Farhin Al Masud What is Raspberry PI? o Low cost, credit card sized computer o SOC (System on a chip) o Founded by Raspberry PI foundation.

An introduction to the Raspberry Pi. What is a Raspberry Pi?  University of Cambridge’s Computer Laboratory  Decline in skill level  Designed for education.

IPlant Collaborative Tools and Services Workshop iPlant Collaborative Tools and Services Workshop Overview of Atmosphere.

CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Operating Systems.

Get Started with Raspberry Pi- Single Board Computer.

Unit 3 Virtualization.

Building Raspberry Pi Controllers with Python

Jenny Pange University of Ioannina

Cloud Computing Cloud computing: (the Internet represents the Cloud).

Tools and Services Workshop Overview of Atmosphere

Andrew McCombs March 10th, 2011

Cloud based Open Source Backup/Restore Tool

INTRODUCTION TO THE RASPBERRY PI Darrell Davis KT4WX

The Internet of Things (IoT)

What we learn during Program

AT91RM9200 Boot strategies This training module describes the boot strategies on the AT91RM9200 including the internal Boot ROM and the U-Boot program.

Cloud Computing BY: Udit Jain.

Cloud computing mechanisms

Oracle HFM Implementation Boot Camp

BusinessObjects IN Cloud ……InfoSol’s story

RecTech - Associated Recreation Council

Introduction to Single Board Computer

Amazon Web Services.

Revision PowerPoint By Nicole Davidson.

Presentation transcript:

Pi In The Sky (Storing Big Data on Cloud) Jenish Koirala Claflin University Mentors: Dr. Raghu Raj, Dr. Richard Loft SIParCS at Mesa Lab, NCAR Boulder, Colorado

Objective To provide easy access to observational weather data through a low cost platform To structure and synchronize data for querying and displaying purposes 2

Technologies Used Raspberry Pi Weather Sensors Cloud Storage Web Interface ~$200 3

What is a raspberry pi? GPIO pins usb ports Ethernet port HDMI port Raspberry Pi 2 Model B (Why?) 900MHz quad-core ARM Cortex A7-Processor 1GB RAM Raspberry Pi 2 Model B ~6 times faster Its just $35 4

System Overview Collects Data Displays Information Uploads Files to cloud User asks information Transmits Data Queries Receives Reads data from file Populates database Weather Sensors Sensor Processor Web Server Cloud Server Web InterfaceDatabase OwnCloud 5

Cloud Storage Why Cloud Storage? Data can be distributed and can be accessed anywhere from the world Cloud is cool! Without providing a detailed knowledge on the infrastructure used behind it. An internet based storage system What is Cloud Storage? 6

OwnCloud File hosting software (like Dropbox) Version used: Why OwnCloud? Open Source Already been used and tested on RPi Free Free Free!! 7

OwnCloud System Architecture How this was achieved? disk1 disk2 disk3 Sensors Sensor Processor 8

Modules and Libraries Pushing files using Pyocclient Library disk2 Making a copy of file on disk1 or disk3 use Secure Copy disk2 disk3 Sensor Processor 9 put_file(‘path/to/owncloudDir’, ‘path/to/localfile’) os.system(”rsync –rvz /path/to/localdatabase /path/to/remotedatabase”) Syncing Database on all disks using Remote SYNC os.system(”scp FILE

Data and Storage Specification Data: ~8 Kilo Bytes per hour (1 sensor Module) ~70.08 Mega Bytes per year Storage: Toshiba 1 Tera Bytes Canvio Interface Transfer Rate: Up to 5 Giga Bytes per second (USB 3.0) Up to 480 Mega Bytes per second (USB 2.0) Capacity Check: 1000 more sensor modules ~8000 Kilo Bytes per hour 8000 * 24 * 365 = ~70.08 Giga Bytes Capacity to store ~14 years data 10

Populated Database Connect to MySQL Server Populate the Database Fully Populated Database Composite Primary Key 11 Imported ‘MySQLdb’ module in python script to connect to MySQL database

Snap Shots 12 Pressure(KiloPascals) Time(HH:MM)

Future Work Compare and implement (if necessary) commercially available cloud platform like Google cloud platform and Amazon EC2 Implement a parallel distributed file system for example, Lustre on Raspberry Pi. 13

Image Credits i_Logo.svg/810px-Raspberry_Pi_Logo.svg.png i_Logo.svg/810px-Raspberry_Pi_Logo.svg.png content/uploads/2014/10/IMG_0833.jpg content/uploads/2014/10/IMG_0833.jpg %20computing_0.jpg %20computing_0.jpg 2.findicons.com/files/icons/977/rrze/720/database_mysql.png 2.findicons.com/files/icons/977/rrze/720/database_mysql.png 14

Acknowledgements Dr. Richard Loft Dr. Raghu Raj Kumar Rashmi Oak Harish Ramchandaran Priyanka Sanghavi Amogh Simha 15

Questions? 16