1. Classic Storage Element
Luca Vaccarossa
EGEE tutorial for administrators – 7 June 2007
INFN-CNAF Bologna

2. Classic SE Main components:
A service that make disk resources to be accessed/moved from the GRID (GridFTP)
An information service that publishes information about the SE (LDAP Server)
The Grid Security stack for authentication.
EGEE Tutorial for administrators

3. Classic SE
GridFTP:
Essentially an FTP server extended/optimized for large data transfers:
Parallel streams for speed.
Allows checkpoints during file transfers, for later resuming.
Authentication through GSI certificates instead of user name + password
GridFTP server supports secure data transfers
EGEE Tutorial for administrators

4. Classic SE
LDAP SERVER
Information organised according to the GlueSchema: specifically by the GlueSEUniqueID entity.
Information describing the SE such as its name and listening port of service.
Information specific to each VO that the SE is serving such as the local path to the file holding directory, available space, etc.
Part of the information is updated dynamically, especially that concerning the disk space available and disk space occupied.
It is done through LDAP Providers found in /opt/lcg/libexec.
The providers run periodically scripts which update the dynamic information.
Finally the rest of the grid information system periodically polls the information made available by the SE present there.
EGEE Tutorial for administrators

5. Classic SE The Grid Security Stack
Grid Security Infrastructure (GSI) ,GLOBUS libraries: used for secure data transfers
VOMS is the user management service that handles user certificates and allows Grid access as appropriate..
LCAS - LCMAPS The LCAS service can authorize local users and grant local resources. LCMAPS keeps track of credentials and their mapping between Grid and the local site.
EGEE Tutorial for administrators

6. Classic SE Set of scripts run by cron jobs to manage pool accounts:
/opt/edg/sbin/edg-mkgridmap creates a gridmap file by reading a local configuration file that specifies sources of allowed credentials, from VOMS/LDAP servers or a specific file.
/opt/edg/sbin/lcg-expiregridmapdir used to remove the mapping to local credentials when a grid user no longer is working on that machine.
/opt/edg/sbin/edg-fetch-crl used to retrieve revocation lists of invalid certificates.
EGEE Tutorial for administrators

7. Limitations
Basically Classic SE consists of a GridFTP server in front of a physical single disk or disk array
It is FTP! A user can do what an FTP client allows to be done!
There is no separation of what can be done from the grid, and the actual transport protocol.
There is no explicit and separate list of file manipulation operations that can be done from the grid!
There is no uniform view of the possible file manipulations: they are linked to the underlying transport protocol!
Depending on the protocol you may not have the same functionality
For the same functionality the specific protocol must be used: it may not be possible to access seamlessly all SEs!
EGEE Tutorial for administrators

8. Limitations
FTP allows file manipulation compatible with underlying Unix filesystem permissions.
If grid control on files is needed, it is the underlying filesystem that must be carefully managed!
Map users to specific local accounts: not pool accounts. Each grid user can be controlled individually once it gets into the machine.
Partition local accounts into especially created groups: reflects data access patterns.
Carefully crafted directory tree guides data access.
So a grid user with no access rights to a file is stopped because the GridFTP server gets stopped on its track by the local filesystem
EGEE Tutorial for administrators

9. Quota & Disk Management
Quota per VO is through physical partitioning of the disk
a single VO might fill up the entire SE
it is user reponsability to monitor disk usage in the case of a Classic SE.
the classic SE does NOT support the SRM interface
EGEE Tutorial for administrators

10. Other Services Hosted on ClassicSE
Classic SE does NOT support SRM interface (and never will!)
Not used in present Data Management Services
Q. What to do of Classic SE ?
A. Hosting other Grid Services 
FMON Server (GridICE Site Collector)
R-GMA Server
EGEE Tutorial for administrators

11. Typical SE related problems
Replica Manager is unable to find your SE: Check if globus-mds service is running both on your CE and SE. Simple “service globus-mds status” command is not sufficient in many cases. Check the running processes to see if “slapd” process is there. If not, restart the service.
Access problems for members of one or more VOs: Check if information provided by your information system reflects the reality. Look at the section that describes your SE in the report generated by printInfo Are all the supported VOs properly configured on your SE? (grid-mapfile, pooled accounts, storage directories, permissions...)
Replica Manager is unable to copy and register or replicate file to your SE (default SE): Check if globus-gridftp service is running on your SE. Check if the storage directory (eg. /flatfiles/SE00/ops) exists and has correct permissions.
EGEE Tutorial for administrators

12. Typical SE related problems
Site BDII works, but SE information is missing:
First, check if slapd demon is running (service globus-mds status command is not sufficient)
You can check infoproviders manually:
/opt/lcg/libexec/lcg-info-wrapper
Use ldapsearch command line interface:
ldapsearch -x -h <SE> -p b mds-vo-name=local,o=grid
Use graphical LDAP browser (GQ...)
Interesting entries:
GlueSEUniqueID, GlueSEName, GlueCESEBindSEUniqueID, GlueSAStateAvailableSpace, GlueSAStateUsedSpace...
EGEE Tutorial for administrators

13. Log Files Gridftp server Gridftp Lcas-Lcmaps (authorization)
/var/log/globus-gridftp.log
Gridftp Lcas-Lcmaps (authorization)
/var/log/gridftp-lcas_lcmaps.log
Globus-mds (GRIS)
/var/tmp/edginfo-globus-mds.log
Authentication & Mapping
/var/log/messages
EGEE Tutorial for administrators