Download presentation
Presentation is loading. Please wait.
Published byBethanie Alexander Modified over 8 years ago
1
1 IDGF International Desktop Grid Federation NEW TRENDS IN DESKTOP GRID COMPUTING Mikhail Posypkin Centre for Distributed Computing IITP RAS
2
2 DEGISCO WP4 24/06/2016 2 PRESENTATION PLAN Intro to Desktop Grid Computing Tools and technologies in DG computing Russian Chapter of IDGF
3
3 WHAT “DESKTOP GRIDS” ARE Use free CPU cycles Volunteer computing Use master-slave model of distributed computation Major technologies: BOINC XTREMWEB-HEP OURGRID CONDOR X-COM SARD
4
4 DESKTOP GRIDS BENEFITS Low-cost: DGs are leaders in terms of Flops/$ Ease of setup and maintenance Reuse of existing resources DGs are “Green” Restricted area of application: coarse grain master-slave Complex technology Low w.r.t. supercomputers efficieincy STRONG POINTSWEAK POINTS NOT a REPLACEMENT for SUPECOMPUTERS/SERVICE GRIDS/ CLOUDS
5
5 DEGISCO WP4 24/06/2016 5 BOINC: Berkeley Open Infrastructure for Network Computing BOINC is an open source middleware system for deploying a desktop “grid” based on volunteer computing. Originally created from the SETI@home project by David Anderson from University of California, Berkeley. Now has over 22 full scale production projects and dozens of alpha projects exist Projects are based on different scientific disciplines BOINC has over 538,354 active computers (hosts) worldwide processing on average 7,777.280 TeraFLOPS TFLOPS (June 28, 2014)
6
6 BOINC
7
7 BOINC CREDIT SYSTEM The project's server keeps track of how much work your computer has done. This is called credit. To ensure that credit is granted fairly, most BOINC projects work as follows: Each task may be sent to two computers. When a computer reports a result, it claims a certain amount of credit, based on how much CPU time was used. When at least two results have been returned, the server compares them. If the results agree, then users are granted the smaller of the claimed credits.
8
8 BOINC Client
9
9 BOINC Server
10
10 BOINC App Specific Daemons work generator (creates stream or batches of jobs) assimilator (handles correct result) validator (compares replicas, selects “correct” result) BOINC
11
11 DEGISCO WP4 24/06/2016 11 BOINC DEVELOPMENT TOOLS C++ API – BOINC native API – DC-API WRAPPERS – BOINC Wrapper – GenWrapper VIRTUALIZED WRAPPERS – BOINC VBox WRAPPER – GBAC
12
12 BOINC Native API (BOINC team) WU Generator: - work unit generation, submission - work unit generation, submission - input and output file handling - input and output file handlingValidator: - validator frameworks (simple, low-level) - validator frameworks (simple, low-level)Assimilator: - handler function linked with a special program - handler function linked with a special program Initialisation Input and output file identification I/O wrappers Checkpointing Progress, credit reporting Critical sections Atomic file update Time handlers Graphics API (screen savers, icons) Other APIs (status, diagnostics, etc.) Grid System Client Application BOINC API Client Application BOINC API WU GeneratorValidatorAssimilator BOINC API
13
13 DC-API (SZTAKI team, Hungary) Grid System Client Application DC-API Client Application DC-API Initialisation Input and output file identification Event processing (e.g. checkpointing) Progress reporting Configuration, logging, Common features: Master Application DC-API Initialisation Work unit generation Input and output file handling Work unit submission Event processing (e.g. results) Result assimilation Work unit deletion
14
14 BOINC Wrapper (BOINC team) BOINC core client BOINC application compiled with the BOINC libraries Wrapper Legacy Application Without the WrapperWith the Wrapper Executes the legacy application(s) in a specific order
15
GenWrapper technology (SZTAKI team) BOINC core client ApplicationApplication Work unit A ZIP file (optional) Launcher GenWrapper binaries Legacy application Profile script Application script Input files 3. Extracts 5. Executes 1. Downloads 2. Downloads Acts as the BOINC client application 4. Launches GitBox Can do almost anything that a shell script can (e.g. can start the legacy application) Output files 7. Produces GBAC
16
16 GenWrapper technology (SZTAKI team) BOINC core client ApplicationApplication Work unit A ZIP file (optional) Launcher GenWrapper binaries Legacy application Profile script Application script Input files 3. Extracts 5. Executes 1. Downloads 2. Downloads Acts as the BOINC client application 4. Launches GitBox Can do almost anything that a shell script can (e.g. can start the legacy application) Output files 7. Produces GBAC BINARY COMPATIBILITY ISSUE
17
17 DEGISCO WP4 24/06/2016 17 VBoxWrapper (BOINC team) Developers work in environment of choice App consists of – compressed minimal VM image (same across multiple apps) – executable – BOINC “Vbox wrapper” Volunteers need to install VirtualBox
18
18 VBoxWrapper BOINC client Vbox wrapper VM instance shared directory: executable input, output files
19
19 GBAC : Generic BOINC Application Client Developed by SZTAKI ( Hungary )
20
20 DEGISCO WP4 24/06/2016 20 VBoxWrapper advantages Advantages – Portability issues solved (no “mininal” linux is needed) – Safety is almost absolute – No special checkpointing is needed Drawbacks – Virtual Box should be installed by volunteers – Virtual appliances load network traffic
21
BOINC on mobile devices
22
22 AndroBOINC: Remote control tool for BOINC clients This light-weighted manager for BOINC clients: - can connect to BOINC core client via WiFi or mobile network - displays status of client: projects, tasks, transfers and messages -provides basic controls for client Pavol Michalec https://code.google.com/p/androboinc/
23
23 NativeBOINC for Android NativeBOINC is Android version of the BOINC client. This software moves BOINC to mobile device's world. Main application is fully-fledged BOINC manager (based on AndroBOINC), which can controls BOINC client on the Android OS. This version provides manager, BOINC client and especially optimized applications for ARM processors. Mateusz Szpakowski (Poland) http://nativeboinc.org/site/uncat/starthttp://nativeboinc.org/site/uncat/start
24
24 BOINC on Android (by Berkeley team) BOINC harnesses the unused computing power of your Android device and runs jobs for scientific research projects. BOINC computes only when your device is plugged in and charged, so it won't run down your battery. It transfers data over WiFi, so it won't use up your cell phone plan's data limit.
25
25 DEGISCO WP4 24/06/2016 25 Interoperability among Desktop Grids and other HPC resources System-level interoperability – 3G-Bridge – CluboRUN Application-level interoperability – gUSE/WS-PGRADE
26
26 3G-BRIDGE Service grid or Cloud BOINC
27
27 DEGISCO WP4 24/06/2016 27 CluboRUN: use idle supercomputer cycles Supercomputers also have ‘idle cycles’ – free rooms in a queue Periodic script queries the queue and submits BOINC jobs to cluster if there is a room for it MPI wrapper for BOINC jobs is used to run multiple clients on the cluster IDSTU RAS, IITP RAS, volunteers
28
WS-PGRADE ARCHITECTURE (SZTAKI) Graphical User Interface: WS-PGRADE Workflow Engine Workflow storage File storage Application repository Logging gUSE information system Submitters Local resources, service grid VOs, Desktop Grid resources, Web services, Databases gUSE Meta-broker Submitters File storage Submitters
29
Desktop Grid infrastructure on Demand
30
30 The activity of Russian Chapter of International Desktop Grid Federation
31
31 DEGISCO WP4 24/06/2016 31 What is IDGF? http://desktopgridfederation.org
32
32 What is Russian Chapter of IDGF? http://desktopgrid.ru
33
33 RUSSIAN CHAPTER’S BOARD
34
34 DEGISCO WP4 24/06/2016 34 GOALS Communication, Dissemination, Community management Joint Research activity Collaboration with volunteers’ community International collaboration
35
35 DEGISCO WP4 24/06/2016 35 Communication, dissemination, and Community management Tutorials: GRID’2010, IT-EDU’2010,DICR’2010,PAVT’2011, GRID’2012, GRID’2014 Special tracks: GRID’2010, GRID’2012, GRID’2014, NSCF’2013 Scientific publications and presentations
36
36 DEGISCO WP4 24/06/2016 36 PRESS RELEASES Андреев А.Л., Манзюк М.М., Ватутин Э.И. Весь мир как суперкомпьютер // Троицкий вариант наука. № 16 (110) от 14 августа 2012. Ивашко Е.Е. Выскопроизводительные вычисления в DesktopGrid // Суперкомпьютеры 2013 Посыпкин М.А. Как стать компьютерным донором? // Научная Россия 2013
37
37 BOINC-FAST in Karelia 2013 BOINC-FAST in Karelia 2015 is coming!
38
38 DEGISCO WP4 24/06/2016 38 Joint Research IITP RAS provides support for SAT@home, for new team from Crimea CluboRUN (joint project of IDSTU RAS, IITP RAS and volunteers) IITP RAS donates unused computing power for volunteer computing
39
39 DEGISCO WP4 24/06/2016 39 Collaboration with Volunteers’ Community Promoting and getting resources for scientific projects Testing scientific projects Feedback Joint events Joint publications
40
40 COMMUNITY STUDY Details: Dr. Ilya Kurochkin, Prof. Vladimir Yakimets RESEARCH OF PREFERENCES OF PARTICIPANTS OF THE VOLUNTARY DISTRIBUTED COMPUTING, today, 18.00 at Desktop Grid section
42
42 DEGISCO WP4 24/06/2016 42 International Collaboration Invited talks to BOINC-FAST (Robert Lovasz, David Anderson) Invited talks to GRID’2010, GRID’2012, GRID’2014 by Robert Lovasz, Gabor Terstzyanski, Attila Csaba Marosi Joint projects – FP-7 DEGISCO project (completed) – RSCF project under supervision of Miklos Kozlovzsky submitted
43
43 DEGISCO WP4 24/06/2016 43 DEGISCO FP7 PROJECT Main objective: Expand European DCIs into ICPC countries by supporting the creation of new Desktop grids for e-Science in those countries and in Europe and by connecting them using the EDGeS bridge technology. Support applications on this expanded infrastructure, disseminate, promote and provide training about this expanded infrastructure and its usage.
44
44 DEGISCO WP4 24/06/2016 44 RSCF Project Combined distributed computing infrastructure, mathematical models, methods and tools for fundamental and applied resource-consuming research problems Partners: SZTAKI (Hungary), IITP RAS, KAR RC RAS, IDSTU RAS
45
45 DEGISCO WP4 24/06/2016 45 Welcome to our section! Today, 14:30, room 407
46
46 http:// desktopgridfederation.eu http://desktopgrid.ru Globe adapted from http://upload.wikimedia.org/ wikipedia/commons/f/fa/ Globe.svg IDGF International Desktop Grid Federation
47
47 DESKTOP GRID ACTIVITY IN RUSSIA ISA RAS: OPTIMA@home, DG technology, tools for performance monitoring, dissemination and user support ISDCT SB RAS: SAT@home KRC RAS: Useful tools (BOINC portal), DG based data mining, dissemination BOINC.RU – Russian team of BOINC Volunteers Other teams not mentioned here…
48
48 DEGISCO WP4 24/06/2016 48 ISA RAS Desktop Grid Infrastructure BOINC Educational Infrastructure boinc-edu.isa.ru BOINC Test Infrastructure boinc-test.isa.ru BOINC Production Infrastructure boinc.isa.ru, sat.isa.ru Based on SZTAKI Desktop Grid Package
49
49 DEGISCO WP4 24/06/2016 49 The Typical Pass NetmaxOPTIMA@home SAT@home Lots of (student) projects
50
50 DEGISCO WP4 24/06/2016 50 OPTIMA@home Project OPTIMA@home - a research project that uses Internet-connected computers to solve challenging large-scale optimization problems http://boinc.isa.ru/dcsdg/ http://boinc.isa.ru/dcsdg/ Institute for Systems Analysis of RAS, M. Posypkin, N. Khrapov
51
51 More than 800 active hosts connected
52
52 DEGISCO WP4 24/06/2016 52 Exhibitions – Softool’2010 – Softool’2011 – SIMO Network(Spain)’2011 Talks 1 st year: IT-EDU’2010,DICR’2010,PAVT’2011 2 nd year: – IT-EDU’2011 (regular talk) – PAVT’2012 (regular talk) – GRID’2012 (regular and plenary + Desktop Grid Track) Communication, dissemination, and Community management
53
53 DEGISCO WP4 24/06/2016 53 Communication, dissemination, and Community management Courses – Moscow State University – Moscow Institute for Physics and Technology Localized material – Posters – Flyers – Russian Chapter of IDGF site: desktopgrid.ru
54
54 DEGISCO WP4 24/06/2016 54 SAT@home: Most Successful BOINC Project in Russia Project SAT@home – research project aimed at solving hard combinatorial problems reduced to SAT problems http://sat.isa.ru/pdsat/ http://sat.isa.ru/pdsat/ Institute for System Dynamics and Control Theory of Siberian Branch of Russian Academy of Sciences (ISDCT SB RAS) A. Semenov, O. Zaikin
55
55 SAT@home
56
56 SAT@home: More than 5500 hosts connected
57
57 SAT@home: Finds Useful Solutions
58
58 KRC RAS: BOINC BASED DATA MINING Based on Partition algorithm, the special BOINC- based software has been developed in High- performance Data Center of Karelian Research Centre, RAS (HpDC) for discovering associative rules in large data sets. The software performs three stages of data processing with two points of synchronization. The software includes a client's software and the following developed BOINC's services: work generator, scheduler and assimilator. Also it uses the single MySQL database to store workunits.
60
OPTIMA@home
61
61 DEGISCO WP4 24/06/2016 61 DEGISCO FP7 PROJECT Main objective: Expand European DCIs into ICPC countries by supporting the creation of new Desktop grids for e-Science in those countries and in Europe and by connecting them using the EDGeS bridge technology. Support applications on this expanded infrastructure, disseminate, promote and provide training about this expanded infrastructure and its usage.
62
62 DEGISCO WP4 24/06/2016 62 International Desktop Grid Federation The International Desktop Grid Federation (IDGF) brings together organisations and persons interested in Desktop grid computing and related technologies. Focuses on application developers and Desktop grid operators. IDGF is a member organisation that provides services for its members (legally: participants). IDGF brings them together, so they can get better information, documentation and training, a platform to exchange experiences. IDGF serves as a marketing channel for its members
63
63 DEGISCO WP4 24/06/2016 63 The Goals of Russian Chapter Help Russian research and industry teams to develop, deploy and maintain their desktop-grid distributed applications Maintain DCI infrastructures for our projects and for other organizations Provide courses and tutorials on desktop grids and volunteer computing for Russian academic and industrial community Promote Russian BOINC projects Explain the mission of IDGF and benefits of the membership
64
64 DEGISCO WP4 24/06/2016 64 The Future of Russian Chapter Maintain and improve Web-site desktopgrid.ru Dissemination activity: Tracks Tutorials Courses Papers Create “Umbrella” project for Russian BOINC applications International collaboration
65
65 DEGISCO WP4 24/06/2016 65 The Future of Russian Chapter Joint Activity (development) BOINC Web-portal (easy application deployment and job submission) Scientific Gateways Supporting interoperability: Bridges Messaging service for BOINC Alternative DG systems ….
66
66
67
67 WP2: Service grid infrastructure EGI Grid site: RU-ISA-CGTDC http://grid.isa.ruhttp://grid.isa.ru
68
68 DEGISCO WP4 24/06/2016 68 WP3: Application and User Support ISA RAS support local developers from – Institute for System Dynamics and Control Theory of Siberian Branch of Russian Academy of Sciences – Moscow Institute for Electronic Technology (MIET) – Joint Institute for Nuclear Research
69
69 WP2: 3G-Bridge Infrastructure 3gbridge.isa.ru
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.