1. WordCount
빅데이터 분산컴퓨팅
박영택

2. Copy Local to VM Local
장치  드래그 앤 드롭  “양방향”

3. Copy Local to VM Local 장치  드래그 앤 드롭  “양방향”
링크에서 training_materials.zip 다운로드
홈페이지 첨부파일에서 wc.jar 다운로드
~/Desktop/training/developer/data/shakespeare
Unzip the training_materials.zip and rename!
Drag & Drop!

4. Running a MapReduce Job – Goal
Works of Shakespeare
Final Result
ALL'S WELL THAT ENDS WELL
DRAMATIS PERSONAE
KING OF FRANCE (KING:)
DUKE OF FLORENCE (DUKE:)
BERTRAM Count of Rousillon.
LAFEU an old lord.
PAROLLES a follower of Bertram.
Steward |
| servants to the Countess of Rousillon.Clown |
A Page. (Page:)
COUNTESS
OFROUSILLON mother to Bertram. (COUNTESS:)
HELENA a gentlewoman protected by the Countess. …
Key
Value A
2027
ADAM 16
AARON 72
ABATE 1
ABIDE
ABOUT 18
ACHIEVE
ACKNOWN …
Run WordCount

5. Upload Files into HDFS Upload Local to HDFS Make a directory in HDFS
$ hadoop fs –mkdir shakespeare
Upload the shakespeare file into HDFS
$ hadoop fs –put ~/Desktop/training/developer/data/shakespeare/comedies shakespeare

6. Running a MapReduce Job – Run WordCount in HDFS
Submit a MapReduce job to Hadoop using your JAR file to count the occurre nces of each word in Shakespeare: $ hadoop jar ~/Desktop/wc.jar WordCount shakespeare wordcounts
wc.jar – jar file
WordCount – Class Name containing Main method(Driver Class)
shakespeare – Input directory in HDFS
wordcounts – Output directory in HDFS

7. Processing of MapReduce Job

8. Check result
In terminal