ABS Parameter Optimizer

Slides:



Advertisements
Similar presentations
D ARMSTADT, G ERMANY - 11/07/2013 A Framework for Effective Exploitation of Partial Reconfiguration in Dataflow Computing Riccardo Cattaneo ∗, Xinyu Niu†,
Advertisements

SkewReduce YongChul Kwon Magdalena Balazinska, Bill Howe, Jerome Rolia* University of Washington, *HP Labs Skew-Resistant Parallel Processing of Feature-Extracting.
Cloud Computing Resource provisioning Keke Chen. Outline  For Web applications statistical Learning and automatic control for datacenters  For data.
June 22-23, 2005 Technology Infusion Team Committee1 High Performance Parallel Lucene search (for an OAI federation) K. Maly, and M. Zubair Department.
CIS 101: Computer Programming and Problem Solving Lecture 8 Usman Roshan Department of Computer Science NJIT.
Bin Fu Eugene Fink, Julio López, Garth Gibson Carnegie Mellon University Astronomy application of Map-Reduce: Friends-of-Friends algorithm A distributed.
4/27/2006 ELEC7250: White 1 ELEC7250 VLSI Testing: Final Project Andrew White.
4/26/05Han: ELEC72501 Department of Electrical and Computer Engineering Auburn University, AL K.Han Development of Parallel Distributed Computing System.
CS 267 Spring 2008 Horst Simon UC Berkeley May 15, 2008 Code Generation Framework for Process Network Models onto Parallel Platforms Man-Kit Leung, Isaac.
Automatic Generation of Parallel OpenGL Programs Robert Hero CMPS 203 December 2, 2004.
Undergraduate Poster Presentation Match 31, 2015 Department of CSE, BUET, Dhaka, Bangladesh Wireless Sensor Network Integretion With Cloud Computing H.M.A.
Bandwidth Measurements for VMs in Cloud Amit Gupta and Rohit Ranchal Ref. Cloud Monitoring Framework by H. Khandelwal, R. Kompella and R. Ramasubramanian.
To run the program: To run the program: You need the OS: You need the OS:
Abstract Load balancing in the cloud computing environment has an important impact on the performance. Good load balancing makes cloud computing more.
CS492: Special Topics on Distributed Algorithms and Systems Fall 2008 Lab 3: Final Term Project.
Elastic Applications in the Cloud Dinesh Rajan University of Notre Dame CCL Workshop, June 2012.
XSEDE14 Reproducibility Workshop: Reproducibility in Large Scale Computing – Where do we stand Mark R. Fahey, NICS Robert McLay, TACC XSEDE14 - Reproducibility.
Introduction to Apache Hadoop Zibo Wang. Introduction  What is Apache Hadoop?  Apache Hadoop is a software framework which provides open source libraries.
Building a Parallel File System Simulator E Molina-Estolano, C Maltzahn, etc. UCSC Lab, UC Santa Cruz. Published in Journal of Physics, 2009.
Frank Casilio Computer Engineering May 15, 1997 Multithreaded Processors.
LOGO PROOF system for parallel MPD event processing Gertsenberger K. V. Joint Institute for Nuclear Research, Dubna.
LOGO A Convolution Accelerator for OR1200 Dawei Fan.
too.
Autonomic scheduling of tasks from data parallel patterns to CPU/GPU core mixes Published in: High Performance Computing and Simulation (HPCS), 2013 International.
A Novel Local Patch Framework for Fixing Supervised Learning Models Yilei Wang 1, Bingzheng Wei 2, Jun Yan 2, Yang Hu 2, Zhi-Hong Deng 1, Zheng Chen 2.
1 CMPE 511 HIGH PERFORMANCE COMPUTING CLUSTERS Dilek Demirel İşçi.
Presentation by Tom Hummel OverSoC: A Framework for the Exploration of RTOS for RSoC Platforms.
Computing Simulation in Orders Based Transparent Parallelizing Pavlenko Vitaliy Danilovich, Odessa National Polytechnic University Burdeinyi Viktor Viktorovych,
Program Design. The design process How do you go about writing a program? –It’s like many other things in life Understand the problem to be solved Develop.
+ Clusters Alternative to SMP as an approach to providing high performance and high availability Particularly attractive for server applications Defined.
Grid Appliance The World of Virtual Resource Sharing Group # 14 Dhairya Gala Priyank Shah.
Rounding scheme if r * j  1 then r j := 1  When the number of processors assigned in the continuous solution is between 0 and 1 for each task, the speed.
HPC HPC-5 Systems Integration High Performance Computing 1 Application Resilience: Making Progress in Spite of Failure Nathan A. DeBardeleben and John.
Xi He Golisano College of Computing and Information Sciences Rochester Institute of Technology Rochester, NY THERMAL-AWARE RESOURCE.
Evolving RBF Networks via GP for Estimating Fitness Values using Surrogate Models Ahmed Kattan Edgar Galvan.
Self-Hosted Placement for Massively Parallel Processor Arrays (MPPAs) Graeme Smecher, Steve Wilton, Guy Lemieux Thursday, December 10, 2009 FPT 2009.
Load Rebalancing for Distributed File Systems in Clouds.
Computing Facilities CERN IT Department CH-1211 Geneva 23 Switzerland t CF Cluman: Advanced Cluster Management for Large-scale Infrastructures.
Developing resource consolidation frameworks for moldable virtual machines in clouds Author: Liang He, Deqing Zou, Zhang Zhang, etc Presenter: Weida Zhong.
Cofax Scalability Document Version Scaling Cofax in General The scalability of Cofax is directly related to the system software, hardware and network.
Copyright © Univa Corporation, All Rights Reserved Using Containers for HPC Workloads HEPiX – Apr 21, 2016 Fritz Ferstl – CTO, Univa.
Server Consolidation in Clouds through Gossiping Moreno MarzollaOzalp Babaoglu Fabio Panzieri Università di Bologna, Dip. di Scienze dell'Informazione.
Welcome to Indiana University Clusters
Service Fabrik Manage Enterprise Grade Services
Cognitive Link Layer for Wireless Local Area Networks
基于多核加速计算平台的深度神经网络 分割与重训练技术
Welcome: Intel Multicore Research Conference
Parallel Density-based Hybrid Clustering
SUNNY-CP: A Multicore Tool for Constraint Solving Jacopo Mauro University of Oslo NordConsNet Uppsala, May 2017.
Speeding up R with Parallel Programming in the Cloud
Verilog to Routing CAD Tool Optimization
PACE: Power-Aware Computing Engines
دانشگاه شهیدرجایی تهران
Vitali Pavlov - HashCoins.com
تعهدات مشتری در کنوانسیون بیع بین المللی
Welcome to the Automatic Salary Review Please answer the following question. Your salary adjustment will depend on your answer!!! Next.
Numerical Algorithms Quiz questions
Md. Abul Kashem, Chowdhury Sharif Hasan, and Anupam Bhattacharjee
Bin Ren, Gagan Agrawal, Brad Chamberlain, Steve Deitz
OS Sim (Operating System Concepts Simulator)
CS639: Data Management for Data Science
New cluster capabilities through Checkpoint/ Restart
Parallel Speedup.
CSE 1020:Software Development
A Virtual Machine Monitor for Utilizing Non-dedicated Clusters
FREERIDE: A Framework for Rapid Implementation of Datamining Engines
Research: Past, Present and Future
Java Coding Environment Installation & Execution guide
FREERIDE: A Framework for Rapid Implementation of Datamining Engines
Presentation transcript:

ABS Parameter Optimizer Jacopo Mauro University of Oslo ABS Workshop, Jun 2017

Motivation: Optimize Cloud Scaling 2

The Cloud Framework

Idea Model the system in ABS Scalability settings exposed as parameters def Int X() = 0; Use automatic reconfigurator to find best settings SMAC: Sequential Model-based Algorithm Configuration

How to Define Parameters: an example initial_instances_1 integer [1,1] [1] scale_in_threshold_1 integer [4,300] [4] scale_out_threshold_1 integer [3,300] [3] cooling_off_time_1 ordinal { 240, 300, 360, 420, 480 } [300] ... { scale_in_threshold_4 <= scale_out_threshold_4 || scale_in_threshold_1 <= scale_out_threshold_1 }

Preliminary (Parallel) Tests Run ABS Optimizer on: Docker container (4 processors) 1..4 runs in parallel Simulation took ~10 minutes Numascale cluster (104 Nodes) 32,64,104 runs in parallel Simulation took ~1h (compilation > 10 min!) Stop after a day of computation

Preliminary (Parallel) Tests Output as expected Some errors (i.e., no output for X seconds): Models sometimes hang (possible Erlang backend bug) Too much resources used (OS scheduler having problems) → Less error with less simulation in parallel

Conclusions Open Source https://github.com/HyVar/abs_optimizer Deployable by using Docker (one button click installation) Need to better understand limits of Erlang simulations (how many in parallel?) Apply to other examples → Welcome to try!

Thank you for your attention! Questions?