Visualization Ideas for Management Dashboards John Shade CERN IT/GD it is basically a way for business users to get an at-a-glance understanding of metrics of importance to them. In addition to acting as a summarization device, the dashboard also serves to highlight specific data and allows the user to drill down and inspect specific items. […] From the perspective of information technology staff, the dashboard often represents a consolidation of data from disparate data sources. The effort is usually considerable when it comes to the necessary extraction, transformation and presentation of the data. -from http://dashboardspy.wordpress.com/ CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it

J.Shade: Visualization Ideas for Management Dashboards How will we spend 20’? Quick overview of current displays/tools Examples of 3rd party dashboards What do we want from a Management Dashboard? Some observations Q & A Note the loss of branding identity. I work in GD for EGEE SA1 operations, but this is a WLCG workshop – so I’ve adopted an organisation-neutral slide template (which also allows more real-estate for screen shots) 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards CIC Dashboard Interfaced with GGUS – good dashboard for operations which has “come on a lot” since the first version. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards MonALISA Performance – should be non-invasive From CalTech 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards SAM “dashboard” Not very user-friendly 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards GStat Displays what’s in the BDII + SAM – too much info http://goc.grid.sinica.edu.tw/gstat/ 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards GridIce VO View, Site View – tabs for different views Also has charts of job status etc. http://gridice2.cnaf.infn.it:50080/gridice/site/site.php 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards NAGIOS Mainly a fabric monitor, but grid-specific sensors have been developed by the monitoring group. https://pps-monitoring.cern.ch/nagios/cgi-bin/status.cgi?host=all 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards WiatG http://bdii.phy.bg.ac.yu/WiatG/pl/WiatG.pl (SEE-GRID) Display BDII contents 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards GridView 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards GridMap http://gridmap.cern.ch/gm/ Max’s s/w – note the popup! SAM visualization tool. Size of squares etc.is calculated on the server side. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

GridMap (site reliability) 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

Experiment Dashboards ALICE ATLAS CMS LHCb Site efficiency reports Common architecture, Python 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

ALICE (site efficiency) http://dashb-alice.cern.ch/dashboard/request.py/Home Note! Help Use of portlets and widgets (can add, but not remove?) 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards ATLAS data transfers http://dashb-atlas-data-tier0.cern.ch/dashboard/request.py/site?statsInterval=720 Note links to User Guide & feedback button 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

CMS I/O rate monitoring http://lxarda16.cern.ch/dashboard/request.py/cmsio Note that slide is built once you’ve selected parameters 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards GridPP reports Real-Time Monitor pages from Imperial College Job-related information – interesting information buried in reports http://gridportal.hep.ph.ic.ac.uk/rtm/reports.xhtml 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

Examples of 3rd party dashboards 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards VitalStream.com VitalStream bought by Internap. The dashboard features reports on web traffic by geography, Keynote Streaming Perspective StreamQ ratings, concurrent viewers and more. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards DashboardSpy.com Airport arrival & departure information. Simple & effective. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards HP OpenView BMC, CA – Business Process Views Framework that allows dashboard to be built by user, according to needs. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Obvient.com Dials, fancy bar graphs, colour – shows cost-compliance So… the displays are not very different to what we have in terms of display – however, most vendors provide frameworks that allow users to customize their own displays. We should probably investigate the use of toolboxes of display widgets rather than writing our own. 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

What do we want from a Management Dashboard? “show the state and spot problems” per-VO, up-to-the-minute, view of the infrastructure how much of available resources are being used? how many jobs are running, are queued, why are they queued? (i.e. any issues) need to include jobs not submitted via the RB what resources are available to [e.g. ATLAS] per site, per region? include data transfers (view of problems) FTS, GridFTP and others 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

What do we want from a Management Dashboard? (contd.) Need a summary for each of the 5-6 largest VOs Need operations view to spot "black holes“ Consolidation of data from disparate data sources 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Some Observations “the information is there, but needs to be summarized” Observations: GridView navigation is tedious. The graphs are useful GridMaps are a good way of conveying a lot of information at a glance Explorer is good for navigating hierarchies Ops Dashboard will probably be some form of GridView, with a different navigation paradigm 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

EGEE Operations Management Dashboard

1st Pass at a Management View Kiviat Graph from ACDC dashboard 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

EGEE Operations Management Dashboard VO selection (or All) Site availability/reliability table Explorer navigation linked to GridMap GridMap controls (CPUs, storage, jobs, data transfers etc.) Idea: navigation can be done using Explorer-like interface on the left (click on a site & it will be highlighted on GridMap), or directly by clicking on GridMap view. Idea: passing mouse over the GridMap will popup minimal info as today, but *hovering* will retrieve more info, eg. Availability stats from GridView. How to identify black holes? Ratio of queued jobs to running jobs? (simplistic)

Possible site-specific pop-up AEGIS01 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

Displaying historical data Possible ways for GridView to display graphs – note the horizontal scroll-bar that allows scrolling back in time. Taken from BMC Patrol console (display data from an agent, built on the fly) 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Topology View show how key components are interconnected e.g (ui-> wms-> bdii), (sam-->wsm->bdii) show the inter-dependencies between services. E.g. what dies if I kill service X? 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Navigation & Status Hierarchical View , with colour-coded percolation of status Allows quick navigation to problem element Check tools that provide Business Process Views (eg. BMC Patrol, Unicenter TNG) 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Key Points Custom views are desirable Consider using existing toolkits Make navigation easy Data from existing tools should be exportable (XML) Remember the Goal: At-a-glance understanding of important metrics 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards

J.Shade: Visualization Ideas for Management Dashboards Q & A 4-Dec-2007 J.Shade: Visualization Ideas for Management Dashboards