Progress Report 2010/1/04. Compared to the last report… Due to a miscommunication/misunderstanding about the last report, it covered up to Dec. 15. That.

Slides:



Advertisements
Similar presentations
AP Computer Science Anthony Keen. Computer 101 What happens when you turn a computer on? –BIOS tries to start a system loader –A system loader tries to.
Advertisements

The map and reduce functions in MapReduce are easy to test in isolation, which is a consequence of their functional style. For known inputs, they produce.
12 August 2004 Strategic Alignment By Maria Rojas.
Roles of Variables with Examples in Scratch
Characteristic Functions. Want: YearCodeQ1AmtQ2AmtQ3AmtQ4Amt 2001e (from fin_data table in Sybase Sample Database) Have: Yearquartercodeamount.
MapReduce.
Computer Science 2212a/b - UWO1 Structural Testing Motivation The gcov Tool An example using gcov How does gcov do it gcov subtleties Further structural.
EHarmony in Cloud Subtitle Brian Ko. eHarmony Online subscription-based matchmaking service Available in United States, Canada, Australia and United Kingdom.
Developing a MapReduce Application – packet dissection.
1 Appendix A: Writing and Submitting SAS ® Programs A.1 Writing and Submitting SAS Programs.
M-grid Using Ubiquitous Web Technologies to create a Computational Grid R J Walters and S Crouch 21 January 2009.
ASTR1002 Mystery Object Tute. Patterns in the Night Sky Imagine, once again, that you live in Mog, where the clouds recently cleared (for the first time.
DEFINITION OF LIMIT We are going to learn the precise definition of what is meant by the statements and. We must make precise our intuitive notion that.
Progress Report 2010/3/30. 1.Successful client side behavior Since the last report, we have solved the issues involving client-side local file system.
Ruo-Hua ’ s Research log 2004/7 ~ 2004/12. Half-Year Plan(2004/7~2004/12) Due Job 8/28/168/309/139/2710/1 1 10/2 5 11/811/2 2 12/612/2 0 1/51/20 Wall.
Verbs Definition and Use.
Hadoop: The Definitive Guide Chap. 8 MapReduce Features
Making Apache Hadoop Secure Devaraj Das Yahoo’s Hadoop Team.
Hola Hadoop. 0. Clean-Up The Hard-disks Delete tmp/ folder from workspace/mdp-lab3 Delete unneeded downloads.
Hybrid Cloud Security: Replication and Direction of Sensitive Data Blocks Glenn Michael Koch Eric Drew Advisor: Dr. XiaoFeng Wang Mentor: Kehuan Zhang.
CSE 486/586 CSE 486/586 Distributed Systems PA Best Practices Steve Ko Computer Sciences and Engineering University at Buffalo.
Cloud MapReduce : a MapReduce Implementation on top of a Cloud Operating System Speaker : 童耀民 MA1G Authors: Huan Liu, Dan Orban Accenture.
Session 5 Select the best solution. Step #4: Select the best solution Once a variety of possible solutions have been identified the best solution should.
Spreadsheets in Finance and Forecasting Presentation 8: Problem Solving.
Financial Services Developer Conference Excel Solutions with CCS Antonio Zurlo Technology Specialist HPC Microsoft Corporation.
Nutch in a Nutshell (part I) Presented by Liew Guo Min Zhao Jin.
1. Goal setting is one of the most powerful techniques that you can use to help you accomplish what you really want in your life. The problem is that.
f ACT s  Data intensive applications with Petabytes of data  Web pages billion web pages x 20KB = 400+ terabytes  One computer can read
CSC 107 – Programming For Science. History of C  Dennis Ritchie developed C from 1969 – 1973  Based upon B (& other) earlier languages  Since its creation,
PHY 107 – Programming For Science. History of C  Dennis Ritchie developed C from 1969 – 1973  Based upon B (& other) earlier languages  Since its creation,
Investigation into CAD designed Products Find some examples of 3D Computer Aided Design from the internet and paste the images onto this page. Label and.
CSC 107 – Programming For Science. Announcements  Locations of Macs to use outside of lab time  Can find on Library ground floor (6) & main floor (6)
FOR OUR BBES 3 RD GRADERS MULTIPLICATION PROPERTIES.
Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA
Grade Book Database Presentation Jeanne Winstead CINS 137.
Math 104 Calculus I Part 6 INFINITE SERIES. Series of Constants We’ve looked at limits and sequences. Now, we look at a specific kind of sequential limit,
Geography Flip Books 6 th Grade Social Studies Shiloh Middle School Mr. Christman.
Unix Commands PowerPoint Presentation developed for LS 560 Information Technology online class - University of Alabama by Debey Sklenar TENacious Cohort.
Apache Hadoop Daniel Lust, Anthony Taliercio. What is Apache Hadoop? Allows applications to utilize thousands of nodes while exchanging thousands of terabytes.
Agenda Executive Summary Key Performance Indicators
Fall 2015 ECEn 490 Lecture #8 1 Effective Presentations How to communicate effectively with your audience.
JFK-103B1W17 &. Welcome ! Welcome to (company name)! We have new ideas that are coming to you! We are going to start a training program to help build.
A scalable and flexible platform to run various types of resource intensive applications on clouds ISWG June 2015 Budapest, Hungary Tamas Kiss,
COPYRIGHT 2010: Dr. David Scanlan, CSUS OBJECTIVES: How to write classes How to create an object from a class using the "New" keyword. How to communicate.
Set up environment for mapreduce developing on Hadoop.
Progress Report 2009/12/15. Add pipe in hadoop For now on hadoop can only do one thing, in one command like bin/hadoop fs –ls Pipes have the potential.
PI2134 Software Engineering IT Telkom.  Software definition  Characteristic of software  Software myths  Software Engineering definition  Generic.
1 Project 2: Using Variables and Expressions. 222 Project 2 Overview For this project you will work with three programs Circle Paint Ideal_Weight What.
Lecture 02 File and File system. Topics Describe the layout of a Linux file system Display and set paths Describe the most important files, including.
Sessions and cookies (part 2) MIS 3501, Fall 2015 Brad N Greenwood, PhD Department of MIS Fox School of Business Temple University 11/19/2015.
MCast Errors and HV Adjustments Multicast Errors (seen on the DATA ERIS connection) have caused a disruption of a HV Adjustment due to a timeout (since.
From Al Scarfone. COLLABORATION A practice that is always performed by two or more people working together to receive a common goal, or achieve by agreeing.
Data-Centric Systems Lab. A Virtual Cloud Computing Provider for Mobile Devices Gonzalo Huerta-Canepa presenter 김영진.
Click Here to Start Central Fire Command Welcome. I am glad that you have accepted the challenge of becoming a fire rescue professional. We are going.
Instructor: Chris Trenkov Hands-on Course Python for Absolute Beginners (Spring 2015) Class #003 (February 14, 2015)
L131 Assignment Operators Topics Increment and Decrement Operators Assignment Operators Debugging Tips rand( ) math library functions Reading Sections.
인공지능연구실 이남기 ( ) 유비쿼터스 응용시스템: 실습 가이드 인공지능연구실 이남기 ( )
Set up environment for mapreduce developing on Hadoop
The Three Constraints.
2008/10/27: Lecture 13 CMSC 104, Section 0101 John Y. Park
Lecture Notes – Week 3 Lecture-2
Assignment Operators Topics Increment and Decrement Operators
Assignment Operators Topics Increment and Decrement Operators
2008/10/27: Lecture 13 CMSC 104, Section 0101 John Y. Park
Assignment Operators Topics Increment and Decrement Operators
Assignment Operators Topics Increment and Decrement Operators
Assignment Operators Topics Increment and Decrement Operators
Assignment Operators Topics Increment and Decrement Operators
Assignment Operators Topics Increment and Decrement Operators
Presentation transcript:

Progress Report 2010/1/04

Compared to the last report… Due to a miscommunication/misunderstanding about the last report, it covered up to Dec. 15. That was only 3 weeks ago. So there is not a great amount of new progress. The students are still trying to resolve the same issues. So I, the professor, have kept their slides and just added some new explanations in italics. I have made some progress on collaborations. This will be discussed at the end.

Add pipe in hadoop For now on hadoop can only do one thing, in one command like bin/hadoop fs –ls Pipes have potential of reducing communication & increasing the distributedness of computation We are still work on fully understanding hadoop’s local –cloud-node mechanisms – Students were offered a bonus if they could produce a specific outcome: “A file is created, not using Hadoop’s file creation feature, on one of the cloud nodes that is not the server, when some hdfs command is run.” So far, no student has won the bonus.

About MapReduce To add the pipe feature in hadoop, we need to create local temp file in each client The challenge about that is how to tell JobTracker to create the file The short-term reason that we are trying to create a file is as a test-of-understanding. For a new file to appear on a client is undeniable evidence that that code which we added is, in fact, running on the client. (Print statements won’t work, because the client doesn’t have terminal.)

WordCount example(1) We traced the wordCount example and tried to figure out how it works: The above code definitely seems to be the place where the word count program really runs. We inserted code here to create a file. The file only creates when the computer with the modified code is the server.

WordCount example(2) We used wireshark to find how the server told client to do the job

But we’re still working From the last slide, we see that the job is executed locally as a java code. But what exact java code is that? The answer is not as obvious as you would think. We hope to understand it soon. We still hope so…

ISPAN The professor and several students recently attended ISPAN (it was in Kaohsiung) There were several sections ostensibly about clouds and grid computing, but in reality they were all about grids. But I had fruitful discussions with Dr. Anthony of ANU and Dr. Wang of HKU, plus his student, King Tin. – Dr. Antony had good industry knowledge that suggests that our work is desirable. – Dr. Wang works with JESICA2 (now 3), which I have been looking at for possible use. I will visit HKU this month to discuss it further.