1 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango SPYC project : Command Line Interface and.

Slides:



Advertisements
Similar presentations
TSpaces Services Suite: Automating the Development and Management of Web Services Presenter: Kevin McCurley IBM Almaden Research Center Contact: Marcus.
Advertisements

Tango at Petra III Teresa Núñez Hasylab-DESY Tango Meeting ESRF,
The Experience Factory May 2004 Leonardo Vaccaro.
1 An introduction to design patterns Based on material produced by John Vlissides and Douglas C. Schmidt.
Data Structures and Programming.  John Edgar2.
The Project AH Computing. Functional Requirements  What the product must do!  Examples attractive welcome screen all options available as clickable.
1 AMMW2013 workshop 13 d – 15 th of November, 2013 Asset management tools: use and improvement in SOLEIL – P. Betinelli Asset management tools: use and.
Software Development Concepts ITEC Software Development Software Development refers to all that is involved between the conception of the desired.
Introduction to Information System Development.
Introduction to the Enterprise Library. Sounds familiar? Writing a component to encapsulate data access Building a component that allows you to log errors.
1 Shawlands Academy Higher Computing Software Development Unit.
The Sardana device pool for SPEC lovers - BLISS Seminar - January 15, 2007 The Sardana device pool for SPEC lovers BLISS Seminar January 15, 2007 Tiago.
Dessy, 17 september 2007 Tango Meeting Development of Tango Client Applications in Python Tiago Coutinho and Josep Ribas.
 To explain the importance of software configuration management (CM)  To describe key CM activities namely CM planning, change management, version management.
Identify steps for understanding and solving the
Configuration Management (CM)
EGEE is a project funded by the European Union under contract IST Testing processes Leanne Guy Testing activity manager JRA1 All hands meeting,
1 Chapter 5 Software Engineering Practice. 2 What is “Practice”? Practice is a broad array of concepts, principles, methods, and tools that you must consider.
A DΙgital Library Infrastructure on Grid EΝabled Technology ETICS Usage in DILIGENT Pedro Andrade
Coming up: Software Engineering: A Practitioner’s Approach, 6/e Chapter 5 Practice: A Generic View copyright © 1996, 2001, 2005 R.S. Pressman & Associates,
Adaptive Hypermedia Tutorial System Based on AHA Jing Zhai Dublin City University.
SE: CHAPTER 7 Writing The Program
Tool Integration with Data and Computation Grid GWE - “Grid Wizard Enterprise”
LHCb Software Week November 2003 Gennady Kuznetsov Production Manager Tools (New Architecture)
Budapest, September 5th, 2002 DataGrid Accounting System DGAS Current status & plans Stefano Barale INFN Budapest, September.
Towards a Global Service Registry for the World-Wide LHC Computing Grid Maria ALANDES, Laurence FIELD, Alessandro DI GIROLAMO CERN IT Department CHEP 2013.
CAC01 – April 2010B11 – Data Format and Data Reduction Synchrotron SOLEIL Alain BUTEAU : Head of Controls and Data Acquisition software group) The Data.
Petra III Status Teresa Núñez Hasylab-DESY Tango Meeting DESY,
The Software Development Process
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
Réunion Contrôle Expérience 28/03/ Experiments Controls Vision, ideas, tasks to begin with … Alain Buteau Andy Götz.
11 Software Design CSCU 411 Software Engineering.
Tool Integration with Data and Computation Grid “Grid Wizard 2”
Requirement engineering & Requirement tasks/Management. 1Prepared By:Jay A.Dave.
1 The Software Development Process ► Systems analysis ► Systems design ► Implementation ► Testing ► Documentation ► Evaluation ► Maintenance.
XML 2002 Annotation Management in an XML CMS A Case Study.
“This improved a lot since I started using Tango (three years ago) from scratch so I'm happy to see the efforts from the developers. Still there is room.
LIU Configuration Management EDMS Documentation, Layout and ECRs This presentation follows the LIU-PSB specific presentation done on 9 th October 2014.
7/8/2016 OAF Jean-Jacques Gras Stephen Jackson Blazej Kolad 1.
Tango Collaboration Meeting May 13 th 2009Sardana Status Report Sardana Status Report Tango Collaboration Meeting May 13 th, 2009 Tiago Coutinho - ALBA.
JDemetra+ as an innovative tool for seasonal adjustment
ITIL: Service Transition
22/04/2018 SOLEIL PROGRESS IN PROVIDING REMOTE DATA ANALYSIS SERVICES Majid OUNSY: Data Analysis Software Project Leader.
Software Configuration Management
The software side of the moon
Archiving System News Tango Meeting (Elettra, April 2008)
Software Tango Meeting - May 2011 N. Leclercq on behalf of the SOLEIL Computing Team.
PLM, Document and Workflow Management
The Development Process of Web Applications
Status report SOLEIL May 2009
Overview – SOE PatchTT November 2015.
Software testing
Chapter 11 Object-Oriented Design
Unified Modeling Language
Project Context Different BeamLine Users
Tau: PyQT GUI for Tango TANGO Collaboration Meeting April 18, 2008
Testing for patch certification
Renovation of the Accelerators Controls Infrastructure and its Assets Management Asset and Maintenance Management Workshop November 14th, 2013 Cl.Dehavay.
Chapter 2: System Structures
PANIC/PyAlarm at SOLARIS – S2Innovation in action
SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION
Module 01 ETICS Overview ETICS Online Tutorials
Tango in a Nutshell 31/12/2018.
INTEGRATING THE SKA-MPI DISH INTO MEERKAT
Metadata The metadata contains
Rekayasa Perangkat Lunak
Towards SOLEIL unique portal
Architectural Mismatch: Why reuse is so hard?
COMPONENT – BASED SOFTWARE ENGINEERING MODULE 2 – SECOND SEMESTER MKHIZE, BSC HONS COMPUTER SCIENCES, DIP IT, ICDL.
From Use Cases to Implementation
Presentation transcript:

1 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango SPYC project : Command Line Interface and Scripting solution on top of Tango S. PIERRE-JOSEPH ZEPHIR (Computing Division - ICA – Synchrotron SOLEIL)

2 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango SPYC project : Command Line Interface and Scripting solution on top of Tango S. PIERRE-JOSEPH ZEPHIR (Computing Division - ICA – Synchrotron SOLEIL)  History  Working tools  Main Guidelines  Architecture  Status  Human Resources  Conclusion

3 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango HISTORY

4 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Historic  Before 2011 : for various reason, there is one CLI per beamline who need it  End 2011 : A Scripting Working Group created by J. Daillant (Soleil General Director)  Goal of the working group and of the Scripting project :  Provide a Soleil CLI fully integrated to our control system  With well known and common commands for all beamlines  These commands should be scripted  7 beamlines implemented their own Python Command Line

5 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango  Existing Solutions studied  SARDANA at ALBA  SPEC at ESRF Has inspired the required set of commands  Both have their own integrated Control System  Soleil Control System is built with Tango devices The command line has to manage these devices  Then the SoleilPYthonCommand or Spyc is born Historic

6 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Historic  SPYC is based on Python/IPython/C++ development languages  Begin 2013 :  11 beamlines had their own Python Command Line  For 3 new beamlines the CLI usage was not so clear  November 2013 : first release deployed on a Beamline

7 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango WORKING TOOLS

8 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Working Tools : 1 - Working group  It is composed of :  General Direction : Soleil General director  Experiment Division :  5 beamline scientists  1 Transversal Engineer  Computing Division  Service Information Infrastructure – for calculation cluster and its python installation  3 persons of Computing Control and Acquisition team (or ICA)  Meetings  At the beginning, each 1,5 month working group meeting : decides of the project orientation. Now more spaced.  Regularly depending on the needs technical meetings with Scientists and Spyc developers : o To prioritize the requirements o To define the content the commands  Importance of constructive exchange between the customer and the provider

9 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Working Tools : 2 - Documentation  “Scripting Requirements”  Written by the working group scientists  Help to start the discussion on the Scripting subject  Describes what is required by this project  No more updated : users demands arrived through our Requests Tracker – Jira.  “Detailed Specification”  Written by ICA  Detailed explanation about the commands content (algorithm, syntax, errors …)  Allowed to discuss with beamline people at migration step  Updated whenever a new command is added in Spyc  “Technical Description”  Redacted by ICA  Explained technical points : o how Spyc works, o which are the interactions with other control system tools  Updated often : at each new Spyc release

10 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango MAIN GUIDELINES 1 - HOMOGENEOUS CONTROL SYSTEM 2 - IMPROVE CURRENT SUPPLY

11 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango  SPYC has to be fully integrated to the existing control system to keep it Homogeneous Existing notions are used : i.e : scan with Sensors/Actuators/Timebase notions already used by scanning tools, Salsa GUI and ScanServer device Tango Database is used for some configuration aspects – Spyc config (start with a set of equipments), Tango attribute alias usage Spyc requirements can impact the existing software applications: for instance Salsa/ScanServer or Tango devices, attribute alias impact Main Guidelines: Homogeneous Control System

12 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango  Tango attribute alias usage impacts our Control System Main Guidelines: Homogeneous Control System Device TANGO Software bus = Beamline Control System Salsa I06-C-C07/EX/FZP-X/position = full attribute name Spyc fastshx = attribute alias Attribute alias usage Now: Attribute alias usage too Attribute alias is also used by « Experimental Data file management » Service – also called NeXus At the beggining : Full attribute name usage DataBrowser

13 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Main Guidelines : improve current supply  Take advantage of working process description to improve the current supply During migration step we discover some control systems issues which were solved through abnormal workarounds in existing Beamline Python code o We tried to solve them asap o Or at least described them in the bugs tracker for future solutions  These two guidelines help all beamlines ( even those not involved in Spyc !)

14 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango ARCHITECTURE

15 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Architecture  Description  PyTango : allow access to Tango Devices access  PyNexus : Experimental data files access (H5py, pytables)  PyScientific : Python scientific modules (Numpy, scipy, matplotlib..)  Responsibilities are shared between ICA and Beamlines PyTangoRoot Beamline code = PyBeamLineRoot PyCommandRoot SPYC PyAbstractDevice PyTANGO PyNeXusRoot h5py tables PyScientificRoot matplotlib scipy numpy Legend : C++ Python Java DeviceRoot ScanServer ScanUtils (Plugins) SimpleScan (Salsa API) SimpleScan (Salsa API) ShutterControl DataFitter shopen … calcafterscan amove, ct, limit … ascan, dmesh… IPython Python Beamline responsability ICA responsability

16 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Architecture  Spyc  Main commands are provided  Specifics Beamline commands can be easily added  Some behaviors can be customized – add code before/after scan operations  PyBeamline : toward a beamline standardization development  Contains beamline specific code  Gives a certain autonomy – flexible system  Same code organization is recommended for all the beamlines  Beamline code started to be organized into a configuration source code tool (true for 2 beamlines)  To help beamline persons who are not familiar with this kind of tool, a procedure description is still a work in progress

17 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango STATUS

18 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango  Solution implementation rules:  “Inspired” from the existing Beamline code To facilitate the merge with it  Reuse existing tools of our Control System  Keep a technical consistency with the current ICA supplies by putting all the intelligence in C++ libraries Can be reused by other Tango devices Consistent with the ICA team people competences Easily integrated in our continuous build mechanism (Maven/Jenkins)  Last implementations :  Better management of the exception inspired of PyTango with python_error and tango_error functions  Specification redaction is on going for commands dedicated to new services on our control system Status: PYCommand Implementation

19 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Contents  Available commands

20 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Deployment  Deployment Status on Beamlines (can be useful for at least 14 beamlines)  Spyc is used in production : 7 beamlines  Migration starting point is not defined by time missing : 2 beamlines  Not interested for the moment : 5 beamlines

21 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Migration process and difficulties  Migration of a beamline consists in transferring the existing Python code to Spyc solution  Migration is done in collaboration between the beamline contact (one of the main requirements is to have designated a unique entry point) and ICA  When the Spyc philosophy is fully understood by the beamline contact, then ICA is no more needed  But when the command line implementation of the beamline is very complex, then it is needed that ICA help deeper the migration process  Accommodate “testing time” is not always easy.

22 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango HUMAN RESOURCES

23 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Human Resources  Direct ICA human resources  A temporary contractor, who works full time on the subject  Myself, not full time  Indirect human resources  Punctually, all ICA people, if some requirements lead to improvements in other existing software components  More Indirect human resources  ICA and Beamlines developers work together to merge the solutions and test them.

24 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango CONCLUSION

25 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Conclusion  Migration step will require a lot of time  A beautiful project based on good exchanges between working group people and Spyc contact on beamlines

26 30 th Tango collaboration meeting, June 2016 SPYC Project : CLI and scripting solution on top of Tango Questions ?