1. 인공지능연구실 이남기 ( beohemian@gmail.com )
유비쿼터스 응용시스템: 실습 가이드
인공지능연구실
이남기
( )

2. Environment Cloudera QuickStart VM with 5.4.2 Guide for Download
%BF%BC%ED%84%B0%EC%8A%A4+%EC%9D%91%EC%9A%A9%EC%8B%9C% EC%8A%A4%ED%85%9C&uid=660

3. Contents Using HDFS Running MapReduce Job : WordCount
How To Use
How To Upload File
How To View and Manipulate File
Exercise
Running MapReduce Job : WordCount
Goal
Remind MapReduce
Code Review
Run WordCount Program
Importing Data With Sqoop
Review MySQL

4. Using HDFS
With Exercise

5. Using HDFS How to use HDFS How to Upload File
How to View and Manipulate File

6. Using HDFS – How To Use (1)
You see a help message describing all the commands associated with HDFS $ hadoop fs

7. Using HDFS – How To Use (2)
You see the contents of directory in HDFS: $ hadoop fs –ls / $ hadoop fs –ls /user $ hadoop fs –ls /user/cloudera

8. Exercise
How To Use

9. Using HDFS – How To Upload File (1)
Unzip ‘shakespeare.tar.gz’: $ cd ~/training_materials/developer/data $ tar zxvf shakespeare.tar.gz

10. Using HDFS – How To Upload File (2)
Insert ‘shakespeare’ directory into HDFS: $ hadoop fs -put shakespeare /user/cloudera/shakespeare

11. Exercise
How To Upload

12. Using HDFS – How To View and Manipulate Files (1)
Remove directory $ hadoop fs –ls shakespeare $ hadoop fs –rm shakespeare/glossary

13. Using HDFS – How To View and Manipulate Files (2)
Print the last 50 lines of Herny IV $ hadoop fs –cat shakespeare/histories \ | tail –n 50

14. Using HDFS – How To View and Manipulate Files (3)
Download file and manipulate $ hadoop fs –get shakespeare/poems \ ~/shakepoems.txt $ less ~/shakepoems.txt
If you want to know other command: $ hadoop fs

15. Exercise
How To View and Manipulate Files

16. Importing Data With Sqoop
Review MySQL and Exercise

17. Importing Data With Sqoop
Log on to MySQL: $ mysql --user=root \ --password=cloudera
Select Database > use retail_db;
Show Databases: > show databases;

18. Importing Data With Sqoop – Review MySQL (1)
Log on to MySQL: $ mysql --user=root \ --password=cloudera
Show Databases: > show databases;
Select Databases: > use retail_db;
Show Tables: > show tables;

19. Importing Data With Sqoop – Review MySQL (2)
Review ‘customers’ table schema: > DESCRIBE customers;

20. Importing Data With Sqoop – Review MySQL (3)
Review ‘customers’ table: > DESCRIBE customers; … > SELECT * FROM customers LIMIT 5;

21. Importing Data With Sqoop – How To Use (1)
List the databases (schemas) in your database server: $ sqoop list-databases \ --connect jdbc:mysql://localhost \ --username root --password cloudera
List the tables in the ‘retail_db’ database: $ sqoop list-tables \ --connect jdbc:mysql://localhost/movielens \ --username root --password cloudera

22. Importing Data With Sqoop – How To Use (2)
Import the ‘customers’ table into HDFS $ sqoop import \ --connect jdbc:mysql://localhost/retail_db \ --table customers --fields-terminated-by '\t' \ --username root --password cloudera
Verify that the command has worked $ hadoop fs –ls customers $ hadoop fs –tail movie/part-m-00000