Computing Division Helpdesk Activity Report Rick Thies October 10, 2006

DOE Certificate RA’s Helpdesk staff are Grid Registration Authorities (RA’s) and approve DOE Grid Certificate requests for FNAL and OSG Support Centers at Fermilab. Helpdesk staff are Grid Registration Authorities (RA’s) and approve DOE Grid Certificate requests for FNAL and OSG Support Centers at Fermilab. Approval process for each request is usually ~5 minutes varying on how many databases need to be searched. Approval process for each request is usually ~5 minutes varying on how many databases need to be searched. Some requests for experiments (i.e. CDF and DZero are sent to a local mail list may take a few days for a responses. Some requests for experiments (i.e. CDF and DZero are sent to a local mail list may take a few days for a responses. The Grid affiliation PPDG began phasing out in August, replaced by OSG. The Grid affiliation PPDG began phasing out in August, replaced by OSG. OSG metrics show granular detail of Helpdesk approving certificates. (see next slide) OSG metrics show granular detail of Helpdesk approving certificates. (see next slide)

DOE Grid Certificate Approvals 2006

OSG Support Center On the OSG front CD Helpdesk is listed as contact for several support centers. On the OSG front CD Helpdesk is listed as contact for several support centers. FermiGrid and USCMS support centers have been active for sometime. FermiGrid and USCMS support centers have been active for sometime. SDSS, CDF and DZero support centers are just getting started. SDSS, CDF and DZero support centers are just getting started. The support centers have other groups that triage and fix problems. The support centers have other groups that triage and fix problems. Through Remedy automation Rick has setup the assignment of problems to the various Grid Support Centers. Through Remedy automation Rick has setup the assignment of problems to the various Grid Support Centers. Other than Grid certificate approvals, Helpdesk staff to date are quite immune to Grid issues. Other than Grid certificate approvals, Helpdesk staff to date are quite immune to Grid issues.

OSG Support Center There has been activity creating Remedy accounts and training for people at FNAL and ~9 universities. There has been activity creating Remedy accounts and training for people at FNAL and ~9 universities. These people are working with tickets relating to Grid issues. These people are working with tickets relating to Grid issues. Most recently trained people are Shaowen Wang and company at the University of Iowa, working with USCMS. Most recently trained people are Shaowen Wang and company at the University of Iowa, working with USCMS. Others that have been trained range from Caltech to MIT and University of Wisconsin to University of Florida. Others that have been trained range from Caltech to MIT and University of Wisconsin to University of Florida.

OSG Support Center In 2005 there were 89 Remedy Grid tickets. In 2005 there were 89 Remedy Grid tickets. This calendar year there has been 207 tickets to date. This calendar year there has been 207 tickets to date. Remedy (FNAL) and FootPrints (OSG) activity. Remedy (FNAL) and FootPrints (OSG) activity. Last year the first phase of pushing our Remedy tickets to the OSG FootPrints tracking software was completed. Last year the first phase of pushing our Remedy tickets to the OSG FootPrints tracking software was completed. May/June 2006 work was completed where tickets from either software are pushed to create, update and set to resolve tickets in both directions. May/June 2006 work was completed where tickets from either software are pushed to create, update and set to resolve tickets in both directions.

Remedy/Footprints Metrics

Tel*Assist Answering Service Tel*Assist has been serving after hour’s Helpdesk calls for 2 years 3 months. Tel*Assist has been serving after hour’s Helpdesk calls for 2 years 3 months. Service keeps up to date pager schedules and contact numbers as backup Service keeps up to date pager schedules and contact numbers as backup 362 Remedy tickets have been created since starting for after hour’s calls. 362 Remedy tickets have been created since starting for after hour’s calls. Though many of these calls have gone well, those that haven’t rise up and cause flurries of extra work. Though many of these calls have gone well, those that haven’t rise up and cause flurries of extra work. Two major problems with off hour calls are; Two major problems with off hour calls are; 1. Clients difficulty reporting what is wrong 2. Time for service to take action.

Tel*Assist Answering Service In September the after hour process changed In September the after hour process changed Phone tree/messages were simplified Phone tree/messages were simplified Calls involve answering service passing info to Helpdesk on-call, where the on-call calls the client, determines action and takes action (normally creating ticket to start notification process). Calls involve answering service passing info to Helpdesk on-call, where the on-call calls the client, determines action and takes action (normally creating ticket to start notification process). This is an interim plan while configuring a new process that will work well This is an interim plan while configuring a new process that will work well Early results indicate ”happy clients” as Helpdesk staff calls the user within a few minutes of clients initial call Early results indicate ”happy clients” as Helpdesk staff calls the user within a few minutes of clients initial call Helpdesk staff involvement takes less time and improves routing of issues Helpdesk staff involvement takes less time and improves routing of issues

Remedy and Remedy Metrics Added phone functionality to Remedy/TelAlert for notifications Added phone functionality to Remedy/TelAlert for notifications Work at the Helpdesk (to date 10/06) Work at the Helpdesk (to date 10/06) 4.3k tickets new accounts, PW changes, principles, accounts, etc. 4.3k tickets new accounts, PW changes, principles, accounts, etc..7k Account requests from Off-site visitors.7k Account requests from Off-site visitors 1.9k Hardware service requests 1.9k Hardware service requests.6k Walk-ins.6k Walk-ins Windows Desktop issues Windows Desktop issues Patches, Spyware Patches, Spyware Remedy tickets from automation 1.4k (NGOP, Network, MSS, BSS tape mounts, etc.) Remedy tickets from automation 1.4k (NGOP, Network, MSS, BSS tape mounts, etc.)

Remedy Metrics (Source) % more tickets % more tickets Automated tickets +13% Automated tickets +13% Manual tickets –17% Manual tickets –17% Number of phone calls doubled Number of phone calls doubled Walk-ins +64% Walk-ins +64% This year 14,153 tickets (2.5 months to go) This year 14,153 tickets (2.5 months to go)

Remedy Metrics 2005

Remedy/Misjob Metrics 2005

Tasks to work on..... BCM400 Helpdesk phone switch BCM400 Helpdesk phone switch Setup web server for reports Setup web server for reports Get Unified Messaging to run again Get Unified Messaging to run again Remedy Remedy Automate process for pager schedule entries Automate process for pager schedule entries Automate metrics to monthly statistics Automate metrics to monthly statistics Off hour support Off hour support Process for system administrators to enter/update systems and services Process for system administrators to enter/update systems and services Hardware Service call automation Hardware Service call automation

Helpdesk Time Consuming Chores Remedy and other manual tasks consume work cycles Remedy and other manual tasks consume work cycles Staff spends a lot of time cutting & pasting data Staff spends a lot of time cutting & pasting data Maintaining Remedy forms for systems, pager schedules and staff changes. Maintaining Remedy forms for systems, pager schedules and staff changes. Client and CD staff Client and CD staff Accounts (Kerberos, mail, windows, crypto cards, etc.) Accounts (Kerberos, mail, windows, crypto cards, etc.)  Validating individuals  Verifying and entering Database information and forms to complete new accounts  Databases & forms accessed & updated for accounts CNAS, UserDB, Crypto Card Excel spreadsheet, CMS special principle, etc.

Helpdesk Time Consuming Chores Remote Account Request Form Remote Account Request Form This process is tightly tied to Users Office due with ID #’s. This process is tightly tied to Users Office due with ID #’s. This added cycles to helpdesk work where there was previously no involvement. This added cycles to helpdesk work where there was previously no involvement. Hardware Service Hardware Service Additional time used when Misjob data is not up to date Additional time used when Misjob data is not up to date Waiting on task numbers, location, serial numbers, etc. Waiting on task numbers, location, serial numbers, etc. Checking system warranties Checking system warranties Keeping NGOP/Remedy in sync Keeping NGOP/Remedy in sync

Remedy Remedy Leverage all CD staff to use forms Leverage all CD staff to use forms Have client replies directly update tickets Have client replies directly update tickets Revamp Remote account request form Revamp Remote account request form Process to automate and have system administrators maintain systems, pager schedules and staff changes Process to automate and have system administrators maintain systems, pager schedules and staff changes Accounts (Kerberos, mail, windows, crypto cards, etc.) Accounts (Kerberos, mail, windows, crypto cards, etc.) Develop plan to automate process Develop plan to automate process Pre-certify clients have accounts available when clients start Pre-certify clients have accounts available when clients start Helpdesk Time Consuming Chores and possible Remedies