Resource Provisioning EGI_DS WP3 consolidation workshop, CERN Fotis Karayannis, GRNET

EGI_DS WP3 consolidation workshopCERN January2 Contents Why this function is needed in EGI Activities in the field of this function –Input and outputs of related actors Actors involved for a federated act –Who and how many people –Who triggers the technical actions Goal and advantages Funding sources Reporting scenarios

EGI_DS WP3 consolidation workshopCERN January3 Why this function is needed In many cases new VOs do not have enough resources (computing, storage) to run their applications nor personnel to support them with the required services This problem has been highlighted in the EGEE project No easy solution to solve the problem

EGI_DS WP3 consolidation workshopCERN January4 Activities in the field of this functions (I) Provisioning through NGI resources –Regarding computing resources the NGI coordinator signs service level agreements with the resource owners (for the resources not owned by the NGI) A resource allocation policy per discipline is defined, according to the NGI agreement (or MoU) A percentage can be retained for new VOs –In case no new VOs the computing resources are used for supported ones (technically feasible) –Similar approach for the human resources, NGI contributes human resources to the supported VOs A percentage of human resources is retained for new VOs –In case no new VOs the human resources are used for supported ones Third parties (JRU members) can be used for such purpose –ROC can play the role of a buffer for both computing and human resources

EGI_DS WP3 consolidation workshopCERN January5 Activities in the field of this functions (II) Provisioning through a central EGI pool of resources (proposed by CERN and adopted in EGEE-III) –The central pool of resources mostly refers to the human resources Enabling the running of the VO required core services upon request –Simulating the EC “trans-national access” funding scheme of researchers travelling to work in different research centres for some time –Regarding computing resources, Either owned by the EGI –With EC or NGIs funding? Or contributed by NGIs –Possibly with remote management »Some resources per region/federation could be enough

EGI_DS WP3 consolidation workshopCERN January6 Actors inputs/outputs Input requests coming from: –New application communities and VOs –NGIs which cannot serve a local app community or VO –Application projects or other infrastructure projects with related regional VOs –Other stakeholders such as Supercomputing for resilience purposes? –End users who cannot be served by their NGI? Output: –Provide a slice of NGIs and/or EGI central pool of resources –Provide support through NGI and EGI personnel –NGIs or EGI can guide the new community to submit proposals for funding or to link with other efforts

EGI_DS WP3 consolidation workshopCERN January7 Actors involved for federated act If NGI resources used: –a specific plan (incl. SLAs) needs to be developed agreed by the NGIs and supported by EGI The actors will be the NGI operators and support groups, assisted by the EGI related operations and applications teams If an EGI pool of resources is developed: –then this needs to be run by the EGI team members (application and operations leaders and members) with the executive decision power and support by the NGIs* *It is not clear at present if EGI needs to hire and employ separate teams, or these can be provided by NGIs in the form of Task Forces

EGI_DS WP3 consolidation workshopCERN January8 Who and how many people In the NGI case, it is estimated to 2 unfunded FTEs in total –it could be a team of 5-7 part time funded NGI operators – support teams –In addition, slices of NGI resources need to be able to accommodate such requests. In the EGI case, this is probably also around 2 FTEs, plus the related central computing resources.

EGI_DS WP3 consolidation workshopCERN January9 Who triggers the technical actions VO coordinators and/or application community leaders should trigger such requests NGIs should first try to accommodate the requests locally (or maybe through regional catch-all VOs, services & resources In case they cannot they can pass the request to the EGI, which according to the selected method (a or b), they route it appropriately In some cases (small team or very small community) end-users might trigger the requests, but this should be avoided for scalability reasons. –It would be better that the NGIs receive such requests in the first place.

EGI_DS WP3 consolidation workshopCERN January10 Goal and advantages The goal is to better support European Grid application communities (and new VOs if cannot be served by existing ones) which do not have enough human and computing resources to serve deploy their application Advantage for the NGIs is the better exploitation of resources and for the app community to be served efficiently

EGI_DS WP3 consolidation workshopCERN January11 Funding sources and sharing The usual suspects: –NGIs and EGI (if EC or other sources besides NGIs can be the case) EC can support the EGI project as well as separate application projects –Other sources (e.g. industry, or third party resource providers) could be investigated In the first scenario (NGI based) mostly funded by NGIs (computing and human resources) plus some EGI coordination In the second scenario (central pool funded by EGI), this can be either paid by NGIs and/or EC or in the form of a task force (with current NGI personnel)

EGI_DS WP3 consolidation workshopCERN January12 Reporting scenarios The reporting should be done either by the NGIs and their interfaces to EGI in cooperation with the EGI teams (first case) Or through the EGI related teams (operations/app support) with the support of an NGI decision making body, task force or policy committee (second case).

EGI_DS WP3 consolidation workshopCERN January13 Summary of comments received Polish NGI proposed a third scenario: –Provisioning through direct contracts between Resource Centers and VOs This was discussed and it is felt that the NGI should play central role in the country organisation, thus such a scenario is not favourably seen –Other comments are valuable e.g. “that the EGI/NGIs supervise and regulate the resources market” DFN does not support the second scenario (central pool of EGI resources) because of sustainability reasons –See separate ppt

EGI_DS WP3 consolidation workshopCERN January14 Conclusions Resource provisioning in the current Grid Infrastructure environment is a challenge (and a risk for its existence) –The first scenario might require more tweaking and policy agreements among NGIs –The second scenario is more straightforward, but funding is a complex / crucial issue Resource provisioning is strongly related to the new EGI/NGIs resources market and we need to agree on specific assumptions (currently not agreed)

EGI_DS WP3 consolidation workshopCERN January15 Other (challenging?) proposals Allow only national VOs –Prioritization in each NGI VO is given through VO sub-groups (or other technical way) –The cross-border collaboration of VO researchers is done through other means (network of excellence projects, task forces, mailing lists, etc.)  This requires the change of the Grid definition!  In this scenario each NGI is aware of what it supports/pays for  Specify policies for using other NGI VOs (e.g. according to the amount of resources contributed to the EGI infrastructure) Use “cloud computing” cycles/storage for new small communities and/or VOs? –Pay per usage principle (convenient for the central pool of resources) –Tender for industrial services

EGI_DS WP3 consolidation workshopCERN January16 Fotis