Review of PARK Reflectometry Group 10/31/2007

Outline Goal Hardware target Software infrastructure PARK organization Use cases Park Components. GUI / UI client for fitting Servers Fitting service Todo list

Goal  Provide simple and easy used distributed computing environment for scientific computing  Provide a framework for data fitting service, including GUI and scripting.  Support cross-platform and pluggable components.

Target Distributed Computing Environment Service Server Master Node User Cluster Working Nodes User/Client ServiceServer Management WorkingServer User

Software Infrastructure of PARK Service Server Service Working Nodes User Interface Scientist GUI/UI DeveloperReduce Service Developer Data reduction Model Developer Data simulation Data presentation Data View

PARK Organization SVN root directory for Source code: Will move to Release files: (*.zip:Windows exe file, *.bz2, *.gz: Python source code ) Directory of PARK: park/xmlUtilBasic data structure for job request park/fitBasic data structure for fitting service park/theoryHelp functions and classes for theory park/optimizerImplementation of optimization algorithms park/parkAuiGUI client of PARK based on wx.Aui park/scriptHelp functions and classes for scrip-based client park/serversImplementation of service and worker servers park/examplesExamples park/testerTesting park/docDocuments park/servicesSome basic services and help tools to develop services park/configExamples of configuration files

Use case for PARK PARK Application (GUI, script) PARK Fitting Frame (GUI) Scientists GUI/UI Developers Reduction DevelopersModel DevelopersOptimizer Developers PARK script PARK Reduction PARK optimizer PARK theory PARK model implement define implement use develop

Server Working Nodes Message Queue Job Scheduler Pending Job Queue Job Priority Queue Running Job Queue Scheduler Controller Servic e Client User Interface (UI) PARK Components

Fitting GUI Framework FittingConstrain FittingModelBuilderFittingDatasetViewer FittingDatasetEditor FittingModelPage Dataset event Model event Model events: 1. The whole model is updated 2. The parameter value is changed FittingViewer Network event

Fitting GUI Framework

Script Client

Servers

Server

Worker Server

Worker (Fitting Service)

Fitting doFitting() –Do the fitting, and return the object representing the fitting results getOptimizer() –Return a real optimizer object getXmlOptimizer() –return the object that is the xml representation of the optimizer setXmlOptimizer(optimizer) –set the object that is the xml representation of the optimizer getMultiplexor() –get the multiplexor object setXmlMultiplexor(xor) –set the multiplexor object

Multiplexor getVariables() –Return a list of variable definitions –Variable attributes: Name: read only, model_name.parameter_name.attribute_name Flag: ‘optimized’ | ‘fixed’ | ‘constrains’ Value: initial value Range: [value0, value1] getConstrains() –Return a list of variable constrains –Constrain attributes: Target: model_name.parameter_name.attribute_name Constrain expression: string representation of constrain evaluate(): evaluate and set the parameter’s value getModels() –Return a list of models addModel (model) –Add a model

Model getDataSet() –Return the data set object representing the experimental data and meta data getWeight() / setWeight(weight) –Get/set the weight getTheory() / setTheoryName(string name) –Return /set the theory object to calculate the theoretical data. getParameters() / addParameter(pm) –Return the parameters representing the model updateTheoryData() –Update the theoretical data associated with the model getTheoryData()/setTheoryData(data –Get/set the theory data associated with the model

Dataset addData(data) –Add a data removeData(data) –Remove a data getData() –Return a list of data getReductionData() –Return the reduction data MetaData / Parameter metadata.para_name = para_value parameter.attr_name = attr_value

Data getDataSource() / setDataSource(src) –Get/set the destination of data source getReductionData() / getRawData() –Return the reduction/raw data getMetaData() –Return the meta data associated with this data _fetchData() –Fetch the data from data source _sendData () –Send the data to the data source _readData () –Read the raw data from the data source _writeData () –Write the data to the data source

Todo List Write the document and tutorial for PARK Improve the stability and performance of the server, especially in the single node environment. Redesign and implement of the server, including: –Support database to handle the various managements, such as user/project, job, message, service, … –Data pre-fetching and security –Support user-code and auto-searching of components (services)

Fully Distributed Services ? Service Register Cluster Management Service Management Job Queue Message Queue Data Fetching Archive Logging Task Management User Client Services Shared Files

Multi-tier of PARK Service Server Working Server Message Server Data Server Client Server Explicit direct connection Implicit direct connection Possible connection All are working as both the server and the client

Security: authentication and authorization Working Server Job Server Security Server MessageServer

Data Transfer 1.Provide the center data server for the cluster, which will retrieve data from remote data server, and store the data for the accessing by the local working nodes. Necessary for diskless nodes in the cluster. 2.Provide the reference to the remote data (similar to url), and each working node will access the data individually.