Labs 2: Palabras

Palabras Archiecture Slave1Slave2Slave3 SlaveM Directory Master1Master2Master3MasterN … … Jobs1 Jobs2 Jobs3 JobsM Slave1 Slave2 Slave3 … SlaveN

Step 1: Get Started Login: – Username: nombre\cc5212 – Password on board – C:/Program Files (x86)/eclipse/ (in Spanish ) – File > Import > …

Step 2: Run Locally ~ abstracts ~ non-unique words ~320 MB uncompressed org.mdp.cli.RunWordCountLocally Right Click > Run As > Run Configurations > Arguments -i /abstracts-es.txt.gz -igz –k 500 How long will it take? Will it even run? -Xmx256M

Step 3: Start the Directory I start the directory! – vm116.dcc.uchile.cl ( ) – Port 1985 Remind me to set heap-space

Step 4: Prepare Slave org.mdp.cli.StartWordCountSlave 1.Implement openDirectoryStub() 2.Add the slave’s name to the directory 3.Review the other code

Step 5: Run Slave Build the.jar using build.xml (dist) Open cmd and go to directory java –jar –Xmx256M mdp-2.jar StartWordCountSlave –dn vm116.dcc.uchile.cl –dp 1985 –sn

Step 6: Prepare Master org.mdp.cli.StartWordCountMaster 1.Connect to the directory 2.Get the list of slaves from the directory 3.Clear words from the slave for you 4.Choose a slave for each word 5.Send the add-words job to each slave

Step 7: Run Master For small dataset! org.mdp.cli.StartWordCountMaster Right Click > Run As > Run Configurations > Arguments -i \es-abstracts-10k.txt.gz -igz -dp dn vm116.dcc.uchile.cl -mn - k 500

Step 8: Run Big Master For big dataset! org.mdp.cli.StartWordCountMaster Right Click > Run As > Run Configurations > Arguments -i \es-abstracts.txt.gz -igz -dp dn vm116.dcc.uchile.cl -mn -k 500

Step 9: Run Distribution Locally 1.Start a directory server – Build and use the jar – java -jar mdp-2.jar StartRegistryAndServer -n localhost -p r -s 1 -sp 2.Start 4 slaves (give different names) in four different CMD windows – Use the jar – java -jar mdp-2.jar StartSlave -dn localhost - dp 1985 –wn 3.Start a master – Can use Eclipse or jar (as preferred) – Point it to local directory – Use small file (large file if successful) -Xmx256M

Final Step: Teach Me Spanish Ask me words in the top 500!