CS Communication & Systèmes Applied Grid For SMEs

Workshop on Grid Utilization - IAS – 20/10/ Long-term partner for our customers in design, integration and operations of critical applications, systems and infrastructures Focus on applications at the heart of our customers' business : expertise, commitment, innovation, sense of service and ability to manage large and complex projects +3,300 employees (70% engineers) Headquarters and Regional offices in France 6 subsidiaries in Europe Subsidiaries in Canada, USA, Brazil and Chile 2004 Consolidated Sales : € 348 Millions

Workshop on Grid Utilization - IAS – 20/10/ Intelligent Transport Systems Cross-Disciplinary Offers Banking and Finance Energy, Utilities and Telecom Public Administrations Defence and Military Aviation Aeronautics and Automotive Space and Civil Aviation Critical Systems Integration Outsourcing & Transitioning IS Network Services Industrial & Critical Applications Critical Real Time software Mission and Control Centres Simulation and Scientific Computing Technical Information Systems (PLM, EDMS, EAI, Web-services, Virtual Reality) Security & Business Continuity Subsidiaries SCOT Germany Canada Romania Air Traffic Control Centre Simulator Communication Systems Intelligence C4I Logistic Information Systems Toll Systems & Contact less toll collection Traffic Management Subsidiaries United States Chile Croatia Distributed Systems Networks End-to-End Production Data Migration Systems Conversion Application Maintenance Architecture and Design Integration and Deployment Maintenance and Supervision Security Digital Video Surveillance Application Management / Security / Linux & Open-source Software / Grid Computing Infrastructure ServicesSystems and Applications Business skills adapted to our clients needs

Applied Grids for SMEs

Workshop on Grid Utilization - IAS – 20/10/ Computing and storage facilities Security and confidentiality Capacity to use a set of dedicated applications closed to their industrial needs Costs reduction by mutualisation and computing on demand Externalization of IT tasks ‘User friendly’ interfaces to HPC infrastructures Business model matching to their financial capacities Industrial Applications driven Grids RUGBI : To SMEs of Biotechnology Industry Openplast : To SMEs of Polymer Processing Industry SMEs Needs and Challenges

Workshop on Grid Utilization - IAS – 20/10/ Application driven Grid Challenges Needs Non computing specialist users. Light Client with grid frontier (portal). Information System for user request, management and exploitation. Technologies Web compliant. Quality Of Services (QoS) Transparency. Security / Confidentiality. Performance / Permanence. Easy business oriented increase and scalability. Interoperability, Evolution compliant, Portability. Administration / Exploitation / Maintenance. Distributed Architecture. Open source and Open file format solutions. Initial state of the art DataGrid, E-Toile, GRASP, …

Workshop on Grid Utilization - IAS – 20/10/ Two Grid projects gathering: OpenplastRUGBI Funded by the French Ministry of Research IT Services company Software Editor Business oriented interface Academic institution Infrastructure provider

Workshop on Grid Utilization - IAS – 20/10/ Academics & IT services company : Computing cluster MPI Academics & IT services company : Storage bay IT services company: Controller server IT services company: Exploitation server Actors driven Architecture Client SMEs : End - User Technological park & Editors & Academics : Web Portal Business oriented

Workshop on Grid Utilization - IAS – 20/10/ High speed network ECP [Paris] PEP [Oyonnax] CS SI [Grenoble] CC-IN2P3 [Lyon] Client INSA [Oyonnax] LPC [Clermont Ferrand] Grids Sites Infrastructures OpenplastRUGBI Controller server Web Portal server Computing Cluster Storage Bay Exploitation Server CC-IN2P3 [Lyon] CS SI [Grenoble] High speed networks ECP [Paris] Client IBCP [Lyon] Client Academic institutions BioPôle [Clermont limagne]

Workshop on Grid Utilization - IAS – 20/10/ The grid portal must provide a user friendly interface for grid management, exploitation and easy use. User management Resources management Sites management Workflow management Files management Exploitation services Grid management Jobs management Grid Portal

Workshop on Grid Utilization - IAS – 20/10/ Resources management services : Applications (commercial or not): Creation. Deployment. Configuration. Update. Databases (commercial or not): Creation. Deployment. Configuration. Update. Storage resources with specific roles: Transit (all uploaded data) on portal. Repository (all permanent data) on storage element. Working (for job submission) on computing element.

Workshop on Grid Utilization - IAS – 20/10/ CS GRID Controller : multi-layer architecture Operating System Batch Manager Globus Grid Services Controller Information System Data DBMS XML-DB XML Files Soft & DB deployment User ResourcesRights Execution/ Workflow Location/ Allocation Files management Exploitation Web Portal Business oriented applications Generic layersSpecific layers Controllers provide services to manage users, resources and users rights on resources. Controllers provide allocation services to allocate jobs execution. Controllers provide exploitation services : accounting, monitoring, …

Workshop on Grid Utilization - IAS – 20/10/ Globus 2.4, 3.2, and now 4.0 Java (servlet, CoG) Web Services SOAP OpenSource schedulers (PBS, Condor, …) Exchange format : XML Information System hosted on DBMS Oracle / Mysql / Xindice wrappers Technological choices

Workshop on Grid Utilization - IAS – 20/10/ Computing Storage Controller Exploitation Portal Client Client Web Client SOAP Client CoG Globus GridFtp, MyProxy Client Mail Web Server Client WS-SOAP GridFtp Server WS-SOAP Server CoG Globus Gram, GridFtp, MyProxy Clients SGDB-XML Server WS-SOAP Server MyProxy Server NTP Server CA Server Mail Client GridFtp Server GRAM Server SOAP Server GridFtp Server FIREWALL SOAP [8080] MyProxy [7521] FIREWALL GRAM-SOAP [8080] GridFtp [2811] FIREWALL GRAM-SOAP [8080] GridFtp [2811] DMZ FIREWALL SOAP [8080] FIREWALL SOAP [8080] FIREWALL GridFtp [2811] HTTPS [8443] Grid internal Communication Client external Communication Portal-Controller Communication Grid Network Architecture

Workshop on Grid Utilization - IAS – 20/10/  Internet provider  Storage and Computing Network  Intranet CS Network  DMZ CS Node Network Architecture compliant with internal and external grids

Workshop on Grid Utilization - IAS – 20/10/ Computing cluster MPI Storage bay Computing cluster MPI Storage bay Controller server Exploitation server Site A Site B User portal Connection Authorization Authentication proxy creation certificate creation Grid basic job submission principles Web Portal server

Workshop on Grid Utilization - IAS – 20/10/ Computing cluster MPI Storage bay Site B User portal File Computing cluster MPI Storage bay Controller server Exploitation server Site A File Job submission Workflow settings File upload Web Portal server Workflow generation Workflow execution File transfer File Job execution Grid basic job submission principles

Workshop on Grid Utilization - IAS – 20/10/ Computing cluster MPI Storage bay Site B User portal Computing cluster MPI Storage bay Controller server Exploitation server Site A Web Portal server Workflow execution File Job execution result Files transfer to user space Worflow engine informs user Updates portal / send mail Grid basic job submission principles

Screen Shots of generic grid functions

Workshop on Grid Utilization - IAS – 20/10/ Examples :  I nstallation packages  Controller configuration Intuitive Grid Portal/Controller installation & configuration

Workshop on Grid Utilization - IAS – 20/10/ Examples :  Grid-client Files Management  User management User friendly interfaces

Workshop on Grid Utilization - IAS – 20/10/ Exploiting and Monitoring Subscription forms Statistical use diagrams

Specific tools

Workshop on Grid Utilization - IAS – 20/10/ RUGBI Tools and databases Automatically Updated Databases Protein knowledgebase Swissprot Trembl 3D protein structure data PDB Encyclopedia of genes and genomes KEGG nucleotide sequence database EMBL compounds diversity set NCI Tools Sequence alignment Blast Fasta Multiple sequence alignment Clustalw Protein 2D structure prediction Predator Sympa96 GorIV … Docking of compounds to proteins Autodock Metabolic Pathway Analysis

Workshop on Grid Utilization - IAS – 20/10/ RUGBI alignment tool: the BLAST example

Workshop on Grid Utilization - IAS – 20/10/ Platform for polymer injection molding simulation Project management : case-studies oriented Allows to specify all the stages of polymers injection modeling Qualifying and Validating each stage of the process CAD Project Mesh Subproject Data settings Data settings Study Molding Case Analysis Defects Study view Project view Data enrichment Mesh Soft. Solver Post- treatment CAD + Mesh Plast3D Plast2D PlastTherm3D PlastPart2D PamView PamRTM Geomesh CAD + Mesh + DS [LC, IC, FC] PlastDefects XML Data settings Analysis PlastPre CADCAD + Mesh + DS + Raw results CAD + Mesh + MED + Raw res. + Analysed res. CAD + Mesh + DS + Raw res. + Analysed res. + Post-treated res. CATIA DS : Data settings XML

Workshop on Grid Utilization - IAS – 20/10/ Study Framework Prototype Openplast client modules Post-treatment framework

Workshop on Grid Utilization - IAS – 20/10/ Applied grids for SMEs : Security based on Globus authentication by proxy and certificate. Confidentiality managed by a strict use of user directories (temporary, working, storage,…) based on user rights. Exploitation and Accounting facilities for different user type. Easy of use for non grid specialists. Basic set of applications (client and grid side). Expected Grid End-Users RUGBI IBCP portal gets daily 2000 connections that will be transferred on RUGBI nodes. Different SMEs from Clermont Limagne BioPole and others. Openplast Polymer processing industry from which 70% are SMEs. European Polymer Institute. Conclusion

Workshop on Grid Utilization - IAS – 20/10/ Starting point: the SMEs real needs Business Interface partners Openplast : European Polymer Institute RUGBI : BioPôle Clermont Limagne. In progress: RUGBI: study market with Lyon Biopôle, Arteb, Théogone … Openplast: partnership with PEP. IT service company (IN2P3, CS). Academic institutions for their up-to-date research knowledge. Software editors: Openplast: ESI, Fluent, … & RUGBI: Accelrys, Tripos, … Infrastructure: extend from a regional level to national RUGBI: Theogone and different biopoles and interoperability with future national french bioinformatics grid. Openplast: HPC pole - Iles de France (IOLS). European: EGEE2 (functional test OK with GT2.4, and interoperability with Glite in progress). For a good industrial accomplishment

Contacts Project manager :

Questions and Answers ?

More Technical slides

Workshop on Grid Utilization - IAS – 20/10/ Manage users, groups and rights on resources. Manage logical resources : softwares and databases. Manage physical resources : Computing / Storage elements, Portals … Information System 1/3

Workshop on Grid Utilization - IAS – 20/10/ Database updating RUGBI service allows : Releases updates Versions updates Versions management Versions deployment FTP server Available files list XML FILE Comparison Metadata (names, date) Available files list XML FILE Missing files Download - Waiting end of jobs on old database - Deletion of the old database - Replication - Registration Update

Workshop on Grid Utilization - IAS – 20/10/ Computing cluster MPI Storage bay Master Controller server Exploitation server Site B Web Portal server Computing cluster MPI Storage bay Slave Controller server Site A Web Portal server data modification Computing cluster MPI Storage bay Slave Controller server Site C Web Portal server User portal data Modification notification data Modification notification data updates Grid information system update mechanisms

Workshop on Grid Utilization - IAS – 20/10/ Computing cluster MPI Storage bay Master Controller server Exploitation server Site B Web Portal server Computing cluster MPI Storage bay Slave Controller server Site A Web Portal server Computing cluster MPI Storage bay Slave Controller server Site C Web Portal server User portal data update notification data updates data updates Grid information system update mechanisms

Workshop on Grid Utilization - IAS – 20/10/ Controller Interface <<DataRequestor>> MySQL XIndice Wrapper MySQLRequestor Wrapper XIndiceRequestor DataRequestorFactory Dump.xml RequestsBackup Restore Wrappers : MySQL, Oracle, PostGres, XIndice compliant. Backup / Restore throw XML files driven by the master controller. Information System 3/3

Workshop on Grid Utilization - IAS – 20/10/ GRID Workflow Engine Application developer Submit jobs, transfer files, invoke grid services, receive notification events, etc. While: - Monitoring execution - Handling errors - Recovering workflow failures - Freeing allocated resources - etc. End-user Choose applications, input files, arguments, etc. Web Portals + Control ler job1 job2 jobs execution environment GRID : End-user interface / Application developer interface

Workshop on Grid Utilization - IAS – 20/10/ <![CDATA[/grid/software/BLAST/2.4/bin/blastall -p blastn -i $INPUT -d /grid/db/ecoli/ecoli.aa -o $OUTPUT -v 500 > $OUTPUT]]> <wf:input wf:VAR="INPUT" wf:source="file:///grid/transit/user/1/projects/8433/input.in" wf:destinationdir="$HOME/tmp/8433" > <wf:output wf:VAR="OUTPUT" wf:destination="file:///grid/transit/user/1/projects/8433/output.out" wf:sourcedir="$HOME/tmp/8433"/> <wf:mail wf:NAME="mail-8433" wf:contact="mail.grid.fr" wf:depends="job-8433" wf:subject="your job 8433 is finished." Workflow used spaces Portal transient space (input and output) /grid/transit/user/1/projects/8433 Temporary CE spacegsiftp://sirius.si.c-s.fr:2811/grid/vo/CS-VO/cs-user1/projects/ Temporary Working node space $HOME/tmp/8433 CE software location /grid/software/BLAST/2.4/bin/blastall : CE database location /grid/db/ecoli/ecoli.aa Execution sequence executed by the workflow engine Execution sequence example