New perfSonar Dashboard Andy Lake, Tom Wlodek

What is the dashboard? I assume that everybody is familiar with the “old dashboard”: The “new dashboard” is supposed (at very minimum) to reproduce the functionalities of the old one, but with better code organization If possible we would like to add new features: interface to central configuration system, gather gridftp information. Other ideas?

What has changed since Madison? Matrix services added It is possible to create, modify, delete throughput matrices. Latency and traceroute matrices also added but not fully tested. Uploading matrix data from collector still needs some debugging: probably we have some data format inconsistency between collector and data store. Should be resolved soon.

Old/New dashboard - overview Data Store Collector API Collector PS Host database user User API

New dashboard Minimum objective: New dashboard should keep all the functionalities of the old one, but have code that is better organized, documented, extensible and scalable. Beyond minimim objective: Dashboard should interact with the centralized configuration management. Other ideas, suggestions as what should be included in „beyond minimal objective”?

Proposed structure of new dashboard framework Data Store Data Access API Data Persistence Layer Database Display GUIObject config GUIAlarmsAuthenticationCollectorOther?User mgmt

Status of the components We wrote a formal description of the data objects, Andy Lake maintains a design document, available on request. I will describe status of each component and what is expected from it

Central data store The central store maintains list of known hosts and services, It also maintains lists of sites, matrices and clouds. So far it has no database back end So far it does not keep history of services

Central data store I plan to add back end database as soon as we are done with debugging matrix services. If you are interested in working on this you are welcome to join, especially if you know hibernate. Service history – We need to design and code the service history. Andy had some ideas how to define service record, so it should not take much time to put this on paper and code. Preferably we would like to keep as much as possible of history info in memory and not in db, but I am not sure whether this will be feasible.

Data Store Status I keep adding new features to data store almost daily. To see current status I refer you to my “data store work progress” document.

Interaction between modules Modules talk to the data store by exchanging JSON objects using POST and GET methods Data access API is a servlet, responding to GET and POST requests from clients returns JSON objects Client programs obtain data from JSON objects. Details follow in the second powerpoint presentation.

Collector Collector is a program which runs on remote site Periodically it connects to data store and asks „do you have jobs to run?” Data store returns list of JSON objects representing the jobs. Collector executes the jobs and Uploads results back to data store.

Collector The code has been written by Andy and he maintains it. Collector has been tested to work on primitive services. Matrix services need some debugging, but should be ready soon.

What next? GUI Persistence/Db History Matrix Users Config Gui Alarms and filters Authentication Interface to centralized config management? Gridftp usage data?

GUI We need a client to display service information It has been suggested that the GUI can be integrated in OSG display. I am fine with that, but we will some sort of display for the debugging while the system is developed anyway. We can have several customized GUI’s.

Status GUI The status of matrix services – maybe we could adapt Andy’s code for that. The status of primitive services – we do not have candidate code, project can be taken by anyone who wants it. Cloud display – integrates site and matrix displays. Volunteers? History display: We need to finish history data first. Once ready – volunteers are needed.

Config GUI (needs: back end database and persistence) Define, modify and delete objects and services. STATUS: developer needed.

Display and Config GUI The actual technology to be used does not really matter We can have many different GUIs Unless you have a very good idea on how to do it I think it would be prudent to follow the layout of the old dashboard, as users know it and they seem to like it.

User Management Define, modify and delete user information Nice, self contained project good for a summer student or undergrad intern. Must know basic java and preferably have internet programming class. It has been suggested that we use existing OSG user management system.

Authentication OperationURLServletFilter Get services{url}servicesGet services servlet Update Service{url}/updateServiceUpdate service servlet DN filter Build latency host{url}/addLatencySer vicesToHost?id={id} Add latency servletDN filter Upload probe results{url}/resultsUpload results servlet Host filter... Etc, etc,...

We need two types of filters: DN and host DN filter: javax.security.cert.X509Certificate Compare to list of DN’s of known users Host filter: Client host IP is obtained from request context. Compared to list of known hosts

Alarms (requires: config GUI, Users) Define what criteria have to be met to raise an alarm and who should be alerted Current dashboard has alarms for primitive services. It is hard to define them for matrix services Project is moderately hard for primitive services and hard for matrix services.

Status Filters (not to be confused with servlet filters, discussed earlier!) Users should be able to define (via GUI) which lower level inputs (like service statuses) should be combined to obtain higher level status (like: site status). Filter definition should be configurable via GUI (requires skill in UI design) Filters should be then assigned to objects (sites or clouds) Not easy, but probably rather interesting project. I was told in Madison that status filters already exist in OSG. If yes, we should think how to integrate them with dashboard.

Interface to centralized configuration Aaron Brown is about to release the centralized config for testing We will have to interface the data store to it. As of today we know next to nothing about how the interface should look like Very likely it will be a rather big and interesting project, with lots of design work.

Gridftp monitoring In Madison it has been suggested that we add gridftp monitoring You have to find a way to tell gridftp servers to upload usage data to the data store. This is probably rather hard. Once this is solved then we have to agree on format of the gridftp usage data and then define service type “gridftp” in the data store. However from the data store point of view it is not hard: a modest amount of fairly routine coding.

Documentation Andy’s design document – ask Andy for access DataStore API and work progress document – ask me for access DataStore code - will be in BNL public subversion soon, you will need doe cert to access it DataStore javadocs will be available Mailing list SNAPSHOT/dump

That’s All Questions, comments, suggestions?