LLNL-PRES-XXXXXX This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA Lawrence Livermore National Security, LLC ESGF Node Manager, Tracking and Feedback ESGF F2F December 9, 2015 Sasha Ames (LLNL/DOE) Prashanth Dwarakanath (NSC/IS-ENES)

LLNL-PRES-xxxxxx 2 Introduction  Development effort to replace Java Node Manager Track node status Manage node membership in federation, projects: eg. CMIP, CORDEX, ACME, Manage “shards” esgf_*.xml files Info feed for desktop - registration.xml  How else can we use NM to coordinate node configuration for additional modules?  Tracking and Feedback: user notification of new or updated data sets  Who: Node Manager – Sasha, Prashanth, Luca (JPL/ESRL), Katharina (DKRZ), Ben, Claire (ANU), Nicolas (IPSL), Sandro (CMCC)  T & F: looking for volunteers

LLNL-PRES-xxxxxx 3 How can Node Manager support ESGF Modules? Compone nt Node Manger Role IdPWhitelist updates ORPKnown providers PublisherEsg.ini / config deploy securityats, azs files (redundancy) desktopRegsistration.xml esg-searchShards lists CoGFederated cog site status ComputeResource status Esg Search Automating these processes will improve user experience and reduce node administrator time in manually managing site configuration, fewer node restarts, etc.

LLNL-PRES-xxxxxx 4 CY 2015 status (Node Manager)  Major aspects of design completed Two-tier principle: Super vs member nodes Backup roles included Asynchronous communication Principal modules  Functioning v0.1 Use Django framework; testing via “runserver” Incoming messages from endpoints placed on queue for serial processing Python daemon connects to other nodes; handles actions from requests SuperNodes check on each others status using round-robin Member nodes join, can be reassigned Registration.xml generated with simple metrics: users, download counts

LLNL-PRES-xxxxxx 5 Next steps for Node Manager .xml / config file deployment implementation  Integrate with Mod_wsgi / apache ESGF installer Deployment mirrors (for basic config)  Cleanup module configuration (Too many env. variables)  Standby Node failover  AccessLoggingFilter replacement  Trust and security

LLNL-PRES-xxxxxx 6 Roadmaps  Node Manager Implementation Test federation deployment Testing in virtual, local, federated environment Deploy first version to ESGF- PROD  Support minimal features at first New features? o Eg, Compute resources management o Requirements gathering, Design, Implement, Test, Deploy  Need to design for incremental adoption  Tracking and Feedback Team telco for requirements CMIP6 requirements a start Design task force to propose solution, design Meet to review Implement Deploy in test federation Test