1. Physics data processing with SOA
4/10/2019
V. Gyurjyan. CLAS12 software meeting

2. V. Gyurjyan. CLAS12 software meeting
Architecture
Formal description
Purpose, functions, external visible properties, interfaces
Internal components and their relationships
Scalability and maintenance
Evolution
4/10/2019
V. Gyurjyan. CLAS12 software meeting

3. V. Gyurjyan. CLAS12 software meeting
4/10/2019
V. Gyurjyan. CLAS12 software meeting

4. V. Gyurjyan. CLAS12 software meeting
What is SOA?
Style of building reliable systems that deliver functionality as services
Loose coupling between interacting services
4/10/2019
V. Gyurjyan. CLAS12 software meeting

5. V. Gyurjyan. CLAS12 software meeting
Service
Atomic unit of an SOA.
Encapsulates logic, data, or process.
Autonomous
Location Transparency
It is defined by the messages it can accept and the responses it can give. Messages can be one-way, synchronous or asynchronous.
They can be integrated to provide higher-level services
It can be completely self-contained, or it may depend on the availability of other services or other resources.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

6. V. Gyurjyan. CLAS12 software meeting
Service interaction
Service
Registry
Discover
Advertise
Service
Consumer
Service
Provider
Interact
4/10/2019
V. Gyurjyan. CLAS12 software meeting

7. V. Gyurjyan. CLAS12 software meeting
Loose Coupling
Flexibility
Scalability
Replaceability
Fault tolerance
4/10/2019
V. Gyurjyan. CLAS12 software meeting

8. V. Gyurjyan. CLAS12 software meeting
Statelessness
Service
Registry
Discover
Discover
Advertise
Service
Provider
Service
Consumer 1
Data storage
Service
Provider 2 3
Service
Provider
4/10/2019
V. Gyurjyan. CLAS12 software meeting

9. SOA is for distributed computing
SOA design can be used for a single system - individual processes as services - communication through well defined interfaces, using local channels or high-speed interconnect.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

10. V. Gyurjyan. CLAS12 software meeting
SO vs. OO
OO advocates tight coupling, i.e. users have static knowledge of the object.
Service has an internal thread of control.
SOA promotes the design of the system in which it is not necessary that requests and responses are handled by the same set of communicating entities.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

11. Do we need distributed physics data processing?
HEP and NEP go GRID
GRID – distributed computing in which resources often spread across different physical locations and administrative domains.
Two classifications for GRID: - compute - data
4/10/2019
V. Gyurjyan. CLAS12 software meeting

12. V. Gyurjyan. CLAS12 software meeting
SO GRID
Grid Standards
OGSI
Extension to WSDL
WS-Resource
WS-ResourceLifetime
WS-ResourceProperties
WS-RenewableReferences
WS-ServiceGroup
WS-BaseFaults
SOA Standards
WSDL
UDDI
BPEL
WS-Profile
WS-Security
WS-Choreography
4/10/2019
V. Gyurjyan. CLAS12 software meeting

13. CLAS12 mini-GRID infrastructure
CNU
Grid infrastructure
ODU
JLAB
UVA
4/10/2019
V. Gyurjyan. CLAS12 software meeting

14. CLAs Reconstruction and Analyses framework
CLAS12 software framework and middleware
CLAS12 physics data processing services
4/10/2019
V. Gyurjyan. CLAS12 software meeting

15. Framework design choices
Separation between: a) data and algorithm b) transient and persistent data
4/10/2019
V. Gyurjyan. CLAS12 software meeting

16. V. Gyurjyan. CLAS12 software meeting
Algorithm services
Algorithm services will process one type of data and generate data objects of a different type.
Algorithm
Service
Data
Data
Data
4/10/2019
V. Gyurjyan. CLAS12 software meeting

17. Persistent and Transient Data
Algorithm service should not use directly data objects in the persistent storage.
Transient data storage as a means of communication between physics algorithm services.
Data service translates data from persistent storage into transient storage.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

18. V. Gyurjyan. CLAS12 software meeting
Why not one data object?
Two different optimization criteria for applications using persistent and transient data.
Being independent from the persistent storage technology.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

19. V. Gyurjyan. CLAS12 software meeting
Data service
Persistent
Data
Transient
Data
Data
Data
Service
Adapter
4/10/2019
V. Gyurjyan. CLAS12 software meeting

20. Data Object categories
Event data
Detector data
Statistical data
4/10/2019
V. Gyurjyan. CLAS12 software meeting

21. V. Gyurjyan. CLAS12 software meeting
Framework Services
One Supervisor service for each application.
Algorithm services (Tracking, Calibration, SegmentFinder, VertexReconstruction, TruthAssociation,etc.)
Data services (Geometry, PersistentEvtData, PersistentDetectorData, TransientEvtData, etc.)
JobOptions
Messaging
etc.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

22. Hypothetical tracking
Supervisor
start
retrieve
SpacePointFormation
EvtData
TransientEvtData
record
start
CoarseTrackFinder
DetectorData
retrieve
Raw Data
SeadMaker
StatData
Space Points
VertexFinder
record
ClusterAnalyzer
Track candidates
TransientDetData
retrieve
start
AmbiguitySolver
TransientStatData
record
Resolved Tracks
start
retrieve
TrackFitter
Final Tracks
TrackScoring
Transient Storage
Tracking State machine
4/10/2019
V. Gyurjyan. CLAS12 software meeting

23. ClaRA implementation details
Platform
4/10/2019
V. Gyurjyan. CLAS12 software meeting

24. ClaRA Service Categories
Normative services administrative service
Crate and manage all other services
Environment administration
Supervisor service - Contains state machine of the particular data processing
Data processing service NS SS S
4/10/2019
V. Gyurjyan. CLAS12 software meeting

25. Communication between services
All service communications are through message transfer. Message format is ACL (Agent Communication Language) defined by FIPA (Foundation for Intelligent Physical Agents).
Each message is one of several predefined communication types.
Message structure: - Sender - Receiver - Content (Data Object or Ontology Object) - Language - Ontology - Protocol
Not understood
Inform-content
Refuse-reason
Types
Agree
Failure-reason
4/10/2019
V. Gyurjyan. CLAS12 software meeting

26. Service Creation NC NR NA ClaRA platform Data processing description
Administrative domain and container
ClaRA platform NC NR
Data processing description NA
Service
Service container
Service: Java or c/c++ through JNI
4/10/2019
V. Gyurjyan. CLAS12 software meeting

27. Legacy software/hardware virtualization
Service abstracts software/hardware component
Service invoke actions which monitor or change the component state. S S S
ClaRA
Platform
IPC
IPC
IPC
Fortran Program
Farm Node
Storage
4/10/2019
V. Gyurjyan. CLAS12 software meeting

28. V. Gyurjyan. CLAS12 software meeting
Summary
SOA is widely seen as the basis for developing flexible, highly scalable software systems, that can span management and ownership domains, regardless of the hardware and software platforms deployed in each.
SOA encourages modularity and encapsulation, and promotes system hierarchical structure.
Normative services of the framework are programmed and tested.
Legacy software/hardware virtualization as a service has bin developed and tested.
System distribution and scalability has bin tested.
4/10/2019
V. Gyurjyan. CLAS12 software meeting

29. V. Gyurjyan. CLAS12 software meeting
Future plans
Develop PersistentEvtDataService, capable of handling EVIO data format (based on JEVIO).
Define persistent detector data IO, and develop PersistentDetectorDataService.
Define persistent statistics data format, and develop PersistentStatisticsDataService.
Define transient data format and interface, and develop TransientDataServices.
Develop ClaRA specific ontology concepts, used in service description and communications.
4/10/2019
V. Gyurjyan. CLAS12 software meeting