1. 1 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango SPYC project : Command Line Interface and Scripting solution on top of Tango S. PIERRE-JOSEPH ZEPHIR (Computing Division - ICA – Synchrotron SOLEIL)

2. 2 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango SPYC project : Command Line Interface and Scripting solution on top of Tango S. PIERRE-JOSEPH ZEPHIR (Computing Division - ICA – Synchrotron SOLEIL)  History  Working tools  Main Guidelines  Architecture  Status  Human Resources  Conclusion

3. 3 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango HISTORY

4. 4 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Historic  Before 2011 : for various reason, there is one CLI per beamline who need it  End 2011 : A Scripting Working Group created by J. Daillant (Soleil General Director)  Goal of the working group and of the Scripting project :  Provide a Soleil CLI fully integrated to our control system  With well known and common commands for all beamlines  These commands should be scripted  7 beamlines implemented their own Python Command Line

5. 5 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango  Existing Solutions studied  SARDANA at ALBA  SPEC at ESRF Has inspired the required set of commands  Both have their own integrated Control System  Soleil Control System is built with Tango devices The command line has to manage these devices  Then the SoleilPYthonCommand or Spyc is born Historic

6. 6 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Historic  SPYC is based on Python/IPython/C++ development languages  Begin 2013 :  11 beamlines had their own Python Command Line  For 3 new beamlines the CLI usage was not so clear  November 2013 : first release deployed on a Beamline

7. 7 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango WORKING TOOLS

8. 8 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Working Tools : 1 - Working group  It is composed of :  General Direction : Soleil General director  Experiment Division :  5 beamline scientists  1 Transversal Engineer  Computing Division  Service Information Infrastructure – for calculation cluster and its python installation  3 persons of Computing Control and Acquisition team (or ICA)  Meetings  At the beginning, each 1,5 month working group meeting : decides of the project orientation. Now more spaced.  Regularly depending on the needs technical meetings with Scientists and Spyc developers : o To prioritize the requirements o To define the content the commands  Importance of constructive exchange between the customer and the provider

9. 9 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Working Tools : 2 - Documentation  “Scripting Requirements”  Written by the working group scientists  Help to start the discussion on the Scripting subject  Describes what is required by this project  No more updated : users demands arrived through our Requests Tracker – Jira.  “Detailed Specification”  Written by ICA  Detailed explanation about the commands content (algorithm, syntax, errors …)  Allowed to discuss with beamline people at migration step  Updated whenever a new command is added in Spyc  “Technical Description”  Redacted by ICA  Explained technical points : o how Spyc works, o which are the interactions with other control system tools  Updated often : at each new Spyc release

10. 10 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango MAIN GUIDELINES 1 - HOMOGENEOUS CONTROL SYSTEM 2 - IMPROVE CURRENT SUPPLY

11. 11 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango  SPYC has to be fully integrated to the existing control system to keep it Homogeneous Existing notions are used : i.e : scan with Sensors/Actuators/Timebase notions already used by scanning tools, Salsa GUI and ScanServer device Tango Database is used for some configuration aspects – Spyc config (start with a set of equipments), Tango attribute alias usage Spyc requirements can impact the existing software applications: for instance Salsa/ScanServer or Tango devices, attribute alias impact Main Guidelines: Homogeneous Control System

12. 12 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango  Tango attribute alias usage impacts our Control System Main Guidelines: Homogeneous Control System Device TANGO Software bus = Beamline Control System Salsa I06-C-C07/EX/FZP-X/position = full attribute name Spyc fastshx = attribute alias Attribute alias usage Now: Attribute alias usage too Attribute alias is also used by « Experimental Data file management » Service – also called NeXus At the beggining : Full attribute name usage DataBrowser

13. 13 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Main Guidelines : improve current supply  Take advantage of working process description to improve the current supply During migration step we discover some control systems issues which were solved through abnormal workarounds in existing Beamline Python code o We tried to solve them asap o Or at least described them in the bugs tracker for future solutions  These two guidelines help all beamlines ( even those not involved in Spyc !)

14. 14 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango ARCHITECTURE

15. 15 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Architecture  Description  PyTango : allow access to Tango Devices access  PyNexus : Experimental data files access (H5py, pytables)  PyScientific : Python scientific modules (Numpy, scipy, matplotlib..)  Responsibilities are shared between ICA and Beamlines PyTangoRoot Beamline code = PyBeamLineRoot PyCommandRoot SPYC PyAbstractDevice PyTANGO PyNeXusRoot h5py tables PyScientificRoot matplotlib scipy numpy Legend : C++ Python Java DeviceRoot ScanServer ScanUtils (Plugins) SimpleScan (Salsa API) SimpleScan (Salsa API) ShutterControl DataFitter shopen … calcafterscan amove, ct, limit … ascan, dmesh… IPython Python Beamline responsability ICA responsability

16. 16 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Architecture  Spyc  Main commands are provided  Specifics Beamline commands can be easily added  Some behaviors can be customized – add code before/after scan operations  PyBeamline : toward a beamline standardization development  Contains beamline specific code  Gives a certain autonomy – flexible system  Same code organization is recommended for all the beamlines  Beamline code started to be organized into a configuration source code tool (true for 2 beamlines)  To help beamline persons who are not familiar with this kind of tool, a procedure description is still a work in progress

17. 17 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango STATUS

18. 18 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango  Solution implementation rules:  “Inspired” from the existing Beamline code To facilitate the merge with it  Reuse existing tools of our Control System  Keep a technical consistency with the current ICA supplies by putting all the intelligence in C++ libraries Can be reused by other Tango devices Consistent with the ICA team people competences Easily integrated in our continuous build mechanism (Maven/Jenkins)  Last implementations :  Better management of the exception inspired of PyTango with python_error and tango_error functions  Specification redaction is on going for commands dedicated to new services on our control system Status: PYCommand Implementation

19. 19 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Contents  Available commands

20. 20 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Deployment  Deployment Status on Beamlines (can be useful for at least 14 beamlines)  Spyc is used in production : 7 beamlines  Migration starting point is not defined by time missing : 2 beamlines  Not interested for the moment : 5 beamlines

21. 21 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Spyc Status : Migration process and difficulties  Migration of a beamline consists in transferring the existing Python code to Spyc solution  Migration is done in collaboration between the beamline contact (one of the main requirements is to have designated a unique entry point) and ICA  When the Spyc philosophy is fully understood by the beamline contact, then ICA is no more needed  But when the command line implementation of the beamline is very complex, then it is needed that ICA help deeper the migration process  Accommodate “testing time” is not always easy.

22. 22 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango HUMAN RESOURCES

23. 23 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Human Resources  Direct ICA human resources  A temporary contractor, who works full time on the subject  Myself, not full time  Indirect human resources  Punctually, all ICA people, if some requirements lead to improvements in other existing software components  More Indirect human resources  ICA and Beamlines developers work together to merge the solutions and test them.

24. 24 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango CONCLUSION

25. 25 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Conclusion  Migration step will require a lot of time  A beautiful project based on good exchanges between working group people and Spyc contact on beamlines

26. 26 30 th Tango collaboration meeting, 21-23 June 2016 SPYC Project : CLI and scripting solution on top of Tango Questions ?