MWDriver: An Object-Oriented Library for Master-Worker Applications Mike Yoder, Jeff Linderoth, Jean-Pierre Goux June 3rd, 1999

Talk Outline MWDriver review New features –Logical checkpointing of the master –Task list management –Miscellaneous Future work

Introduction to MW Provide an object-oriented framework to develop master/worker applications Use Condor-PVM to handle acquiring / releasing nodes, message passing MW is fault tolerant –Handles workers arriving / leaving –Handles workers getting suspended and resumed Assigns tasks to workers

More MW User must implement three classes –MWDriver Setup; packing initial data; acting on completed tasks –MWTask Definition of one unit of work Holds the work to be done and the results –MWWorker How to execute some work, given a task

Master Wid 1 Wid 2Wid 3 Wid 4 W1 W2W3 W4 Workers T2 T3 T4 T5 Running To Do T6 T7 T8... Global Data

Checkpointing - Goals Save progress in case of failure Ease of use Hide details from the user Automatic restart from a checkpoint file

Checkpointing the Master Virtual functions to implement: –In the MWDriver: write_master_state() read_master_state() –In the MWTask: write_ckpt_info() read_ckpt_info() Use of set_checkpoint_frequency()

Checkpointing - Internals Write periodic ckpt. Upon restart: –Check for ckpt. file –If it exists, restart from it. –All tasks on to-do list at start. Checkpoint file: MWDriver internal info write_master_state() write_ckpt_info() of all tasks on the running list write_ckpt_info() of all tasks on the to-do list

Task List Management - Goals Allow users to specify how to access the task list Hide implementation details Not efficiency! –Implemented as singly linked list –Sorting done in O(n 2 ) time

Task List Management - Implementation Each task is assigned a “MWKey” through a user specified function. –MWKey is simply a double –Lower is better –The function takes an MWTask and returns an MWkey The user can change the key at any time –set_task_key_function( MWKey(*)( MWTask * ) );

Using MWKeys Adding tasks –ADD_AT_BEGIN, ADD_AT_END, ADD_BY_KEY Retrieving tasks –GET_FROM_BEGIN, GET_BY_KEY Sort the task list: –int sort_task_list() The task list can be “fathomed” –int delete_tasks_worse_than( MWKey );

Example 1 Implementing “Best Bound” MWKey taskkey1( MWTask *t ) { BBTask *bt = dynamic_cast ( t ); return (MWKey) bt->lower_bound; } // In get_userinfo() set_task_add_mode( ADD_BY_KEY ); set_task_retrieval_mode( GET_FROM_BEGIN ); set_task_key_function( &taskKey1 );

Example 2 A branch and bound strategy that goes depth-first, but occasionally chooses to evaluate the node with the best lower bound value. // In get_userinfo() … set_task_add_mode( ADD_TO_BEGIN ); set_task_retrieval_mode ( GET_FROM_BEGIN ); set_task_key_function ( &taskKey1 ); // In act_on_completed_task() if ( best_bound_move ) set_task_retrieval_mode( GET_BY_KEY ); else set_task_retrieval_mode( GET_FROM_BEGIN );

Other MW Goodies rematch_task_to_workers() {un}pack_driver_task_data() New statistics class –Stats now collected across checkpoints MWprintf() –Just like printf(), but has “debug levels” Code skeleton

Future Work Heterogeneous worker support Gathering workers faster Ranking workers by abilities –Processor speed, memory, etc –Communication bandwidth/latency More…. suggestions?