1. www.see-grid-sci.eu SEE-GRID-SCI Branko Marovic University of Belgrade branko@rcub.bg.ac.rs JRA1 progress PSC07 Meeting, Žabljak, 22-23 January 2010 The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

2. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 2 Deliverables Three JRA1 deliverables produced  DJRA1.1 - Grid application requirements and commonalities assessment  M06, 7 PM, UoB-RCUB, Editor: B.M.  DJRA1.2 - Application-specific monitoring and control tools design  M08, 12 PM, IPP, Editor: E.A.  DJRA1.3 - Application services design report  M12, 10 PM, UoB-RCUB, Editor: B.M.  DJRA1.4 - Application services implementation and deployment report  M22, 30 PM, UoB-RCUB, Editor: B.M.  DJRA1.5 - Assessment of operational tools developed  M23, JRA1 28,5 PM, ICI, Editor: A.S.

3. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 3 Milestones MJRA1.1 - Baseline analysis of operational tools  M03, IPP: wiki, informal report MJRA1.2 - Application commonalities assessed and areas of development identified  M06, UoB-RCUB: DJRA1.1 MJRA1.3 - Operational tools implemented and running  M16, IPP: wiki, informal report  5 in production, Logwatch-G and AMS in beta MJRA1.4 - Selected application services implemented and deployed  M24  Demonstration at review

4. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 4 Ongoing Developments and Metrics (PSC06 -> PSC07) 11 AS (7 major, 3 pre-existing as prototypes, 1 small effort with some initial interest)  Described in DJRA1.1, DJRA1.3 and DJRA1.4 7 OT  Described in DJRA1.2 and DJRA1.5 Black: As in DoW Green: Achieved M0M12M16M19M22M24 MTJRA1.1 AS developed 0/3 preexisting 2/4 production + 4 alpha/beta 6+57+48+3 ->1 α 6 MTJRA1.2 OT developed 0/02/4 production + 2 alpha 5+26+17+04

5. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 5 Application Services (PSC06 -> PSC07) NameStatusAvailability date Partner s SEE-GRID File Management Java APIProduction/stableAvailableUOB Data Management Web PortalProduction/stableAvailableUOB Event LoggerProduction/stableOctober 08UOB Work Binder Production/live -> Production/stable October 08UOB SDS (Seismic Data Server) Applications Service Beta -> Production/live January 09 ULAKBI M TUBITA K User Level Monitoring ToolBeta Since beginning of 09 Production version in second half of 09 UKIM Common Workflow Repository Extension of P-GRADE Portal Production/live -> Production/stable May 09SZTAKI Environment oriented Satellite Data Processing Platform Production/live 1. October 09 (Phase 1) 30. March 10 (Final) UTCN Mathematical Expressions Web ServiceProduction/liveJune 09ETFBG Rendering Application Service M-PSC06 Alpha -> Beta July 09FGA Advanced-Workflow Development Tool and Orchestration Service AlphaSeptember 09UKIM

6. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 6 Operational Tools (PSC06 -> PSC07) NameStatusAvailability datePartners JTS, Job Track Service M-PSC06, M-PSC07 Production/liveFebruary 09IPP-BAS/UKIM BBmSAM Extensions M-PSC06 Production/liveMarch 09UoBL Grid Site Software Vulnerability Analyzer Production/live -> Stable March 09SZTAKI No Mercy Trouble Ticketing System M-PSC06 Production/liveMay 09ICI User/application specific grid infrastructure monitoring extension Production/live -> Stable Operational Release in July 09 SZTAKI Logwatch Plug-ins for Grid Services M-PSC06 Beta -> Prod/stable September 09RBI Alert Messaging Service Beta -> Production/live September 09 UoB, IPP/UoBL/ICI

7. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 7 Status (OT + AS)

8. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 8 Internal JRA1 Metrics Measure and compare the progress of AS & OT Current status (only one of offered values, maximum is 8 points):  1 - Planning - Conceptual description, possibly with initial functional specification, or architectural description  3 - Development - in progress, no working version available  5 - Alpha - working version exists, available only to developers  6 - Beta - working version exists, available to users  7 - Production/live - released, open to adding new features and bug-fixes  8 - Production/stable - released, developers still work on bug-fixes but it is closed for new features  7 - Production/closed - final version released, development closed  NaN - Abandoned/Inactive - project abandoned before being released to production Usage (additive for each named usage, up to 6 points):  2 – per each application currently using an application service (AS only)  1 - per each site having tool deployed (if applicable, OT only) or  1 - per each user (e.g. grid site administrator) using it in last 3 months (for OT only) Presentation (only one of offered values, maximum is 6 points):  0 - No presentation or outdated presentation  2 - Up-to-date brief presentation (based on JRA1 presentation template) available (not more than 3 months old)  4 - Up-to-date detailed presentation available (not more than 3 months old)  6 - Detailed presentation available with usage or performance analysis and live or recorded demo Publications (additive 0.5 for each, up to 6 points) Visibility towards wider community (additive for each fulfilled indicator, but up to maximally 8 points)  1 - Software license defined and declared  1 - Introductory web site or Wiki with basic data and functional overview  2 - Architectural description available, online or within the corresponding JRA1-related project deliverable  2 – User documentation available (preferably online) fully covering typical usage, including deployment, if applicable  1 - Source code available publicly or to all eligible users (in accordance with license), e.g. via download.  1 - Source code repository with version control available publicly or to eligible users

9. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 9 Visibility towards wider community

10. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 10 JRA Metrics Reporting Did not respond by the start of PSC06  BBmSAMeX (Mihajlo)  JTS (Emanouil/Dimitar)  Logwatch-G (Valentin)  NMTT (Alex)  RAS (Davor/Mirsad) AWT added in an AP Expected to provide an overview report at PSC07! JRA1 Wiki has been greatly improved – also visible in metrics  51 edit since PSC06 JTS was late in reporting for PSC07 (again) 50 conference and publication references reported In 2 cases there is a slight regression due to outdated presentations

11. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 11 General ranking (OT + AS) Presentations and status change helped to some Many presentations become outdated Some were inconsistently weighted on two occasions DevelopmentPartnerM16M19 Comment Work BinderUOB2526.5 DM-WebUOB2225 ESIP PlatformICI / UTCN2024 Event LoggerUOB2323.5 FM-J-APIUOB22 AMSUOB1321 ULMONUKIM1220.5Visibility 3 Presentation 2 Usage 2 SDSASULAKBIM2319Old presentation, visibility -4 GSSVAMTA SZTAKI2318.5Inconsistent reporting USGIMEMTA SZTAKI2318.5Inconsistent reporting AWTUKIM918Status 3 Visibility 5 Pubs 1. But WIKI!? MEWSRS-ETFBG16 CWREMTA SZTAKI2010.5Status -3 Presentation -2 Visibility -5 JTSIPP-BAS24- NMTTICI16- BBmSAMeXUoBL12- RASRBI10- Logwatch-GRBI9-

12. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 12 Access to JRA1 Developments http://wiki.egee-see.org/index.php/JRA1_Commonalities Significant progress since PSC06: 51 edits Missing summary  BBmSAMeX  JTS  NMTT  ESIP-P  MEWS Issues persist  Missing summaries  Resource links  Missing publications  Collaborations (applications etc.)

13. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 13 Application – AS Alignment

14. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 14 Per-application conclussions In good standing  AMS, DM-Web, Event Logger, FM-J-API, Work Binder, CWRE, USGIME, GSSVA, SDSAS, BBmSAMeX, ESIP Platform, Logwatch-G, JTS Need to improve metrics  RAS, MEWS, NMTT, AWT, ULMON Need to improve usage  AMS, USGIME Need to improve presentation (PPT and Web/Wiki)  Event Logger, FM-J-API, Work Binder, SDSAS, ESIP Platform, Logwatch-G OK, but should improve visibility towards wider community  GSSVA, JTS

15. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 15 Application – AS Alignment Compared without taking SEE-GRID(-2) apps, but with GreenLand RAS desperately needs WRG-ARW, ROMS or whatever CWRE should measure access frequency or something MEWS remains being a solution looking for a problem ULMON was initially very popular, but now only MDSSP-WA uses it Usage of OTs not thoroughly reported…

16. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 16 Issues and News NMTT – AMS, one way “communication” on integration WRF-ARW - DM-Web: “vantage” IRB User Interface, UI access policy... RAS – not a single image yet!  Meteorologists are happy with their 2D tools  Target the general public?  Or try with ROMS?  Possible usage of Work Binder of secondary importance. Announced CCIAQ – JTS (JTS is an OT!?) ROMS – Work Binder? ESIP Platform also used by GreenLand RESEWA uses Event Logger Service authentication – checked with Work Binder, may also be needed elsewhere –CA, VO policies

17. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 17 Dissemination and Plans Achieved  Further publishing of details and documentation of JRA1 developments  1 at EGEE’09 Conference in Barcelona  6 at SEE-GRID-SCI UF  Reported 30 (23 in JRA1 wiki) published papers  EGEE UF5, call extended until 14 December 2009  Are we exhausting the pool of globally relevant developments?  JRA1 umbrella presentation at EGEE UF5? Focus on (unchanged bullets!)  Improvement of existing documentation, installation and user documentation  Making both code and packaged (installable) software available  Start promotion of developments when they reach adequate level of maturity  Possible external verification: should advertise our software and collaborate with other related projects and user communities

18. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 18 Dissemination and Plans EGEE UF5  JRA1 umbrella presentation at EGEE UF5 in Uppsala, Sweden, April 12-15 by Anastas Mišev Possible external verification: should advertise our software and collaborate with other related projects and user communities  Information exchange (and possible collaboration) with Grid Observatory – more details at EGEE UF5  Work Binder and ESIP-P made submissions to RESPECT program Focus on (unchanged bullets!)  Improvement of existing documentation, installation and user documentation  Making both code and packaged (installable) software available  Start promotion of developments when they reach adequate level of maturity

19. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 19 Possible OT Collaboration with Grid Observatory The initial contacts with scientific and technical leaders. Delayed due to insufficient OT documentation Difficult to communicate the info without representative and up-to-date links Proposed  Exchange of ideas and information related to JTS (Job Track Service) and ULMON (User-Level Monitoring Tool).  Sharing the (job) traces of gLite-based SEE-GRID infrastructure. Opportunity to compare the behavioral patterns of EGEE and SEE-GRID. Still waiting for a pointing response by GO

20. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 20 AS dissemination through RESPECT Candidates must go through RESPECT evaluation The case must be built by  Clear description of purpose  Existence of supporting documentation  Installation procedure well worked out  Assistance in running of a test  All this is anyway related with DNA4.3 and DJRA1.4 Work Binder – same area as DIANE (Python/Ganga based framework), while we have a multi-application service ESIP-P and SDAAS in good shape, but too domain-specific ULMON could be OK, halfway towards OTs All OTs are out

21. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 21 SEE-GRID-SCI User Forum 2009 Good opportunity, also used by applications – indirect dissemination of ASs JRA1 featured participations  Experiments on ESIP - Environment oriented Satellite Data Processing Platform - Dorian Gorgan, Victor Bacu, Denisa Rodila, Florin Pop, Dana Petcu  Seismic Data Server Application Service and Web Interface - Can Özturan, Bilal Bektas, Mehmet Yımazer  Work Binder - a service for quick access and communication with jobs - Branko Marovic, Milan Potocnik  Job Track Service: Architecture and Features - Emanouil Atanassov, Todor Gurov, Dimitar Dimitrov  DM-Web - Web access and management of application-specific files in LFC catalog - Branko Marovic, Milan Potocnik, Dragan Okiljevic, Jovan Oklobdzija  Event Logger – A solution for tracking and analysis of application-generated events Branko Marovic, Milan Potocnik Plus  Operational Grid tools developed at SCL - Vladimir Slavnic, Branimir Ackovic, Dusan Vudragovic, Antun Balaz, Aleksandar Belic  Grid Site Monitoring tools developed and used at SC  Vladimir Slavnic, Branimir Ackovic, Dusan Vudragovic, Antun Balaz, Aleksandar Belic

22. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 22 EGEE-SEE/SEE-GRID-SCI SVN-Trac Repository usage https://trac.egee-see.org/trac/XXX, XXX: JRA1  ams.see-grid-sci.eu - Alert Messaging Service  bbmsam.see-grid-sci.eu - BBmSAM Portal (Restricted)  event-logger.see-grid-sci.eu - Event Logger Application  work-binder.see-grid-sci.eu - Work Binder Application Service NA4  refs.meteo.see-grid-sci.eu - Regional scale multi-model, multi- analysis Ensemble Forecasting System (Restricted)  wrf-arw.see-grid-sci.eu - Advanced Research WRF Can be seen as a infrastructural service provided by HellasGrid NGI to SEE-GRID-SCI NA4 and JRA1 Public access  SVN requires a certificate even with Trac opened to anonymous  Just contact support@grnet.gr for public R/O access

23. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 23 Source code availability and repository usage Source and download 11 Source only 2 Download 2 In metrics, 9 confirmed usage of a repository No source or download links for  BBmSAM, JTS, RAS Should provide at least source code, if OK with licensing EGEE-SEE/SEE-GRID-SCI SVN-Trac Repository is available, anonymous R/O access also works Download of binaries is a nice commodity

24. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 24 Ongoing issues Final push for usage by applications  AMS - NMTT?  BBmSAM – applications?  Alignment with application commitments Usage of sites/admins od OTs  E.g.Logwatch-G looks rather handy, Valentin is a bit shy in promotion Missing user documentation  RAS, GSSVA, AWT, JTS AWT still alpha! Address reviewer OT recommendations in DJRA1.5 (there is also an analogy for AS in DJRA1.4)

25. SEE-GRID-SCI PSC07 Meeting, Žabljak, 22-23 January 2010 25 Actions to be Taken One slide each AS/OT for EGEE UF Contributions to DJRA1.5 Wiki, wiki wiki  DJRA1.4 helped AS  DJRA1.5 for OT  User documentation (4 missing)  At least basic installation instructions, as done for RESPECT Make at least source code available Final metrics report at M24