1. IRMIS Components Overview Gabriele Carcassi 18 April 2011

2. IRMIS3 Modules  IRMIS has been conceptually divided into independent modules Components (installation/cabling/lightweight inventory), Lattice, eLog, PV, …  Motivation: Easier to have independent development User can pick and choose  This talk is about the Components module

3. IRMIS3 DATA MODEL

4. IRMIS3 Data Model - Components Where is it? How is it powered? How is it controlled? How is it connected?

5. IRMIS3 Data Model - Components  Distinguishes between Component instance - physical component with inventory information associated with it Component installation - set of instructions of how a particular component is installed (housed, powered, controlled, cabled, …)  Independent concepts (can track an instance in the inventory without having it installed; can track an installation without caring which instance)

6. IRMIS3 Data Model - Components  Each component is of a particular type The type defines attributes: manufacturer, ports, which parents/children can have, generic key/value pairs (e.g. pointers to documentation, maximum drawn current, local contact, …)  Each component installation Lives in multiple hierarchies: control, housing and power Can be cabled Can be associated with a Field Name (i.e. ioc name, site naming convention) and a set key/value pairs (i.e. MAC address, geolocation)

7. IRMIS3 Data Model - Components  Instances of each type can be tracked in 3 ways: NONE: components of that type are not tracked COUNT: only the number of components is tracked INSTANCES: each instance is tracked individually - is identified by a serial number which has to be unique within the same type (“N/A” is only non-unique allowed): use barcode if available - can have multiple key/value pairs (i.e. last inspection date, purchase order number, …)

8. IRMIS3 Data Model - Components  Cable type defines what type of cable (copper, fiber, diameter, …) and how is it used (high power, BPM, …)  Each cable connects two ports of two component instances  Support for cable bundle/pull Each bundle/pull includes a set of cables Each bundle/pull can be associated with a path of different components (i.e. this bundle goes from this horizontal tray, to this vertical tray, …)

9. IRMIS3 SERVICE ARCHITECTURE

10. Project infrastructure Production server since January 2009 “alpha” - AuthZ and AuthN only designed Build and test server Automated tests IE or firefox on Windows/Linux/Mac supported SourceForge project (mailing list, bug tracking, wiki). Monthly release cycle.

11. IRMIS3 Architecture

12. Software Stack XML protocol (REST style WS) Few database utilities: backup, consistency check, etc… Client Server Integration with external tools (i.e. physcs) 3 rd party Perl/Pyton scripts 3 rd party Java apps Jython scripts Web applications JavaScript bridge Applets and Widgets Java Client API Data Service layer Database layer

13. Getting data... XML returned REST style (data service) Id references De-normalized data

14. Writing data <transaction xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://xml.bnl.gov/schema/irmis' xsi:schemaLocation='http://xml.bnl.gov/schema/irmis irmis.xsd'> XML describing a transaction sent through a POST allows remote/disconnected work no lock “FOR UPDATE” idempotent actions (error or same result) different independent “commands”

15. Implementation MySQL JPA JAXB SAX Business logic Business logic Servlet Read Write Stream out XML as soon as possible to reduce delay JAXB to convert transaction request to Java objects JPA vs JDBC: simpler queries to maintain 6% performance loss

16. Performance  Dell Precision M4300 laptop, Intel Core Duo T7500, 2.20 GHz - APS dataset Entire component data set streamed in about 5 seconds (31338 components, 63911 relationships) Most other queries under 100 ms (manufacturers 88ms, interfaces 95ms)  Roughly half of the time is spent in DB access and half in XML generation Streaming starts right after DB access  Only tuning done is on the database query (was “good enough”) Other areas: hardware/os tuning, xml generation improvements, profiling, caching, …

17. IRMIS3 CLIENT API

18. Java API: design principles  The Java API implement the XML protocol and provides objects that represent the data  All reads are cached: no need to keep caches in your code.  All reads are synchronized you can have reads on multiple threads on the same data and will not trigger multiple reads on the server (extremely useful for GUIs)

19. Java API design principles  The read access is object oriented Data is exposed through interfaces - Component, ComponentType, Manufacturer Some classes function as “finders” for the classes containing the data - Components.getById(123), FormFactors.getByName(“Rack”) You can follow references from one object to the other - component.getComponentType().getManufacturer()

20. Java API design principles  The write access is protocol oriented You create a transaction You call methods to add elements in the transaction - Manufacturers.createManufacturer(transaction, “ACME”) You save the transaction, which may fail The read access is refreshed only after the data is saved

21. Importing components from Excel (using Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful"

22. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Load excel file

23. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Connect to the service

24. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Loop over the rows

25. Importing from Excel (Jython) from gov.bnl.irmis.api import * from java.net import URI from java.io import File from jxl import * workbook = Workbook.getWorkbook(File(“components.xls")) sheet = workbook.getSheet(0) nRows = sheet.getRows() nColumns = sheet.getColumns() irmisDB = IrmisDB.getInstance(); irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data")) transaction = irmisDB.createTransaction(); for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber) transaction.save(); print "Operation successful" Get data out Create components

26. Fluent API  Navigating through hierarchies can be tedious  On top of the “beans” API there is a fluent API (internal domain specific language) Search API Builders Change Set API

27. IRMIS3 USER INTERFACE

28. UI design  Web interface: uses applets and Web Start (both greatly improved with Java 6u10)  Uses Swing standard components plus open source (SwingX, Flamingo, JIDE, …)  UI elements can optionally be used in custom pages and integrated with javascript

29. UI elements Component hierarchy tree widget (filtering, sorting, search, …) Component path widget

30. Component applications

31. Component inventory

32. Cabling applications

33. Status  Component and Component types applications is considered production level  Cabling UI is considered beta while the API and service support is considered production