WordCount 빅데이터 분산컴퓨팅 2016. 3. 22 박영택

Copy Local to VM Local 장치  드래그 앤 드롭  “양방향”

Copy Local to VM Local 장치  드래그 앤 드롭  “양방향” http://ailab.synology.me:5000/fbsharing/KantrsNS 링크에서 training_materials.zip 다운로드 홈페이지 첨부파일에서 wc.jar 다운로드 ~/Desktop/training/developer/data/shakespeare Unzip the training_materials.zip and rename! Drag & Drop!

Running a MapReduce Job – Goal Works of Shakespeare Final Result ALL'S WELL THAT ENDS WELL DRAMATIS PERSONAE KING OF FRANCE (KING:) DUKE OF FLORENCE (DUKE:) BERTRAM Count of Rousillon. LAFEU an old lord. PAROLLES a follower of Bertram. Steward | | servants to the Countess of Rousillon.Clown | A Page. (Page:) COUNTESS OFROUSILLON mother to Bertram. (COUNTESS:) HELENA a gentlewoman protected by the Countess. … Key Value A 2027 ADAM 16 AARON 72 ABATE 1 ABIDE ABOUT 18 ACHIEVE ACKNOWN … Run WordCount

Upload Files into HDFS Upload Local to HDFS Make a directory in HDFS $ hadoop fs –mkdir shakespeare Upload the shakespeare file into HDFS $ hadoop fs –put ~/Desktop/training/developer/data/shakespeare/comedies shakespeare

Running a MapReduce Job – Run WordCount in HDFS Submit a MapReduce job to Hadoop using your JAR file to count the occurre nces of each word in Shakespeare: $ hadoop jar ~/Desktop/wc.jar WordCount shakespeare wordcounts wc.jar – jar file WordCount – Class Name containing Main method(Driver Class) shakespeare – Input directory in HDFS wordcounts – Output directory in HDFS

Processing of MapReduce Job

Check result In terminal