1. Design and Implementation of PARK (PARallel Kernel for data fitting) Paul KIENZLE, Wenwu CHEN and Ziwen FU Reflectometry Group

2. Distributed Computing Environment Service Server Master Node User Cluster Working Nodes User/Client ServiceServer Management WorkingServer User

3. Service Server WorkingServer WorkingClient Service User JobQueueSys MessageQueueSy s ServiceServer

4. Prototype Demo Download Source code: –svn co svn://svn@danse.us/park Edit cluster config file: –park/config/hosts Start service server –park/servers/mapServer.py Start client –park/client/AppJob.py Provide services –park/services

5. Further Improvement (working server) 3 running modes –Command: simple commands, short time ls, top, kill, … –Thread: python functions, short time Multithreading, not recommend –Process: normal services, long time any executable programs Task and task queue management: –Restrict the number of running commands, threads, and processes, and their running times –Kill the task by its task ID –Query the task queue status

6. Further Improvement (Service server) –More stable and efficient for the service server –More functions for service server Submit job Register as a listener Query job status and services Kill job by its ID Run job on specified node (s) –Better job queue and message queue management –Pluggable reduce functions –Restart the working node when it crashes

7. User Interface Service Server Scripting User Interface GUI Files Email Web External Apps data

8. Software Infrastructure of PARK for data fitting Service Server Service Working Nodes User Interface Scientist View DeveloperReduce Service Developer Data reduction Model Developer Data simulation Data presentation Data View

9. Procedure for data fitting dataset Reduction / filter metadata Theory parameter Experimental data Simulation data uniplexor multiplexor optimizer Variables Dependency Constraints

10. XML tree for data fitting DataFitting Optimizers* Multiplexor DataSet Variables* DataFilterFunctions* Dependency Constraints Uniplexors* Theory* Parameters* Uniplexors DataSource Variable

11. Base Classes for XML/ Object tree For the developer, XML tree is actual the object tree. It is the common task to transfer between the XML tree and object tree

12. Classes from XML tree DataFitting Optimizer Multiplexor DataSetVariables DataFilterFunction Dependency Constraints Uniplexor Theory Parameter DataSource Variable XmlObject XmlAttributeObject Model DeveloperReduction Developer

13. Basic functions for classes Datasource: –read(), write() :read/write to local file system –fetch(), send() : read/write to remote file system –getdata(hints:object), getmetadata(hints:object): Get the data and meta data DatafilterFunction: –filter (input:object, hints: object)

14. Basic functions for classes Theory: –calculate(): do the calculation –getSimulationData(): get the simulation data Parameter: –hasDerivative(order: int, hints:object) Query whether the its derivatives are available

15. Example: sans optimization

16. XML tree for request Request JobQuery JobSubmission* JobGroup* DataFitting JobRegister JobKill ReduceFunction Jobs* Task TaskQuery Command User Inputs* TaskKill Reduction function Developer External application Developer, UI Developer Service Server Working Server

17. XML tree for UI UI Request Viewer Viewer Developer, UI developer WebViewerFileViewer GUI Viewer Viewer

18. GUI (MVC pattern) OptimizerViewer ParameterViewer ModelViewer DataSetViewer Variables Dependency Constraints ReplyViewerRequestViewer GUIViewer StatusViewer

19. GUI Demo Prototype GUI MVC GUI