MaxSAT Evaluations SURVEY & RECENT RESULTS. Outline  Max-SAT overview  Sub problems of Max-SAT  Applications of Max-SAT  Annual Evaluations, 2015.

Slides:



Advertisements
Similar presentations
Interactive Configuration
Advertisements

UIUC CS 497: Section EA Lecture #2 Reasoning in Artificial Intelligence Professor: Eyal Amir Spring Semester 2004.
Proofs from SAT Solvers Yeting Ge ACSys NYU Nov
“Using Weighted MAX-SAT Engines to Solve MPE” -- by James D. Park Shuo (Olivia) Yang.
Methods of Proof Chapter 7, second half.. Proof methods Proof methods divide into (roughly) two kinds: Application of inference rules: Legitimate (sound)
1 NP-Complete Problems. 2 We discuss some hard problems:  how hard? (computational complexity)  what makes them hard?  any solutions? Definitions 
Properties of SLUR Formulae Ondřej Čepek, Petr Kučera, Václav Vlček Charles University in Prague SOFSEM 2012 January 23, 2012.
Ryan Kinworthy 2/26/20031 Chapter 7- Local Search part 1 Ryan Kinworthy CSCE Advanced Constraint Processing.
NP-Complete Problems Reading Material: Chapter 10 Sections 1, 2, 3, and 4 only.
The Theory of NP-Completeness
Efficient SAT Solving for Non- clausal Formulas using DPLL, Graphs, and Watched-cuts Himanshu Jain Edmund M. Clarke.
NP-Complete Problems Problems in Computer Science are classified into
1 Satisfiability Checking of Non-clausal Formulas using General Matings Himanshu Jain Constantinos Bartzis Edmund Clarke Carnegie Mellon University.
State-of-the-art in SAT solvers
1 Discrete Structures CS 280 Example application of probability: MAX 3-SAT.
Computability and Complexity 24-1 Computability and Complexity Andrei Bulatov Approximation.
Integer Programming Difference from linear programming –Variables x i must take on integral values, not real values Lots of interesting problems can be.
1 CS 4700: Foundations of Artificial Intelligence Carla P. Gomes Module: Satisfiability (Reading R&N: Chapter 7)
Knowledge Representation II (Inference in Propositional Logic) CSE 473 Continued…
1 Message Passing and Local Heuristics as Decimation Strategies for Satisfiability Lukas Kroc, Ashish Sabharwal, Bart Selman (presented by Sebastian Brand)
Cardinality & Sorting Networks. Cardinality constraint Appears in many practical problems: scheduling, timetabling etc’. Also takes place in the Max-Sat.
Relaxed DPLL Search for MaxSAT (short paper) Lukas Kroc, Ashish Sabharwal, Bart Selman Cornell University SAT-09 Conference Swansea, U.K. July 3, 2009.
SAT Solver Math Foundations of Computer Science. 2 Boolean Expressions  A Boolean expression is a Boolean function  Any Boolean function can be written.
Optimization and Relaxation in SAT Search Sharad Malik Princeton University Symposium on Satisfiability Solvers and Program Verification (SSPV) Seattle.
Boolean Satisfiability and SAT Solvers
Large-scale Hybrid Parallel SAT Solving Nishant Totla, Aditya Devarakonda, Sanjit Seshia.
Introduction to Job Shop Scheduling Problem Qianjun Xu Oct. 30, 2001.
10/9/2015COSC , Lecture 51 Real-Time Systems, COSC , Lecture 5 Stefan Andrei.
MBSat Satisfiability Program and Heuristics Brief Overview VLSI Testing B Marc Boulé April 2001 McGill University Electrical and Computer Engineering.
Tonga Institute of Higher Education Design and Analysis of Algorithms IT 254 Lecture 8: Complexity Theory.
CHAPTERS 7, 8 Oliver Schulte Logical Inference: Through Proof to Truth.
INTRODUCTION TO ARTIFICIAL INTELLIGENCE COS302 MICHAEL L. LITTMAN FALL 2001 Satisfiability.
Explorations in Artificial Intelligence Prof. Carla P. Gomes Module 3 Logic Representations (Part 2)
Analysis of algorithms Analysis of algorithms is the branch of computer science that studies the performance of algorithms, especially their run time.
Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard Despite a numerous attempts we do not know any efficient.
Combination of Exact and Approximate Methods for SAT and MAX-SAT Problems Frédéric Lardeux, Frédéric Saubion and Jin-Kao Hao Metaheuristics and Combinatorial.
1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst case.
Explorations in Artificial Intelligence Prof. Carla P. Gomes Module Logic Representations.
On the Relation between SAT and BDDs for Equivalence Checking Sherief Reda Rolf Drechsler Alex Orailoglu Computer Science & Engineering Dept. University.
On Finding All Minimally Unsatisfiable Subformulas Mark Liffiton and Karem Sakallah University of Michigan {liffiton, June 21, 2005.
Boolean Satisfiability Present and Future
SAT 2009 Ashish Sabharwal Backdoors in the Context of Learning (short paper) Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal Cornell University SAT-09.
CSE 589 Part V One of the symptoms of an approaching nervous breakdown is the belief that one’s work is terribly important. Bertrand Russell.
Finding Models for Blocked 3-SAT Problems in Linear Time by Systematical Refinement of a Sub- Model Gábor Kusper Eszterházy Károly.
Saleem Sabbagh & Najeeb Darawshy Supervisors: Mony Orbach, Technion & Ilia Averbouch, IBM Started at: Spring 2012 Duration: Semester.
CS6045: Advanced Algorithms NP Completeness. NP-Completeness Some problems are intractable: as they grow large, we are unable to solve them in reasonable.
Satisfiability and SAT Solvers CS 270 Math Foundations of CS Jeremy Johnson.
Custom Computing Machines for the Set Covering Problem Paper Written By: Christian Plessl and Marco Platzner Swiss Federal Institute of Technology, 2002.
Young CS 331 D&A of Algo. NP-Completeness1 NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and.
SAT Solving As implemented in - DPLL solvers: GRASP, Chaff and
Onlinedeeneislam.blogspot.com1 Design and Analysis of Algorithms Slide # 1 Download From
Inference in Propositional Logic (and Intro to SAT) CSE 473.
TU/e Algorithms (2IL15) – Lecture 12 1 Linear Programming.
REU 2007-ParSat: A Parallel SAT Solver Christopher Earl, Mentor: Dr. Hao Zheng Department of Computer Science & Engineering Introduction Results and Conclusions.
Solving Weakened Cryptanalysis Problems for the Bivium Keystream Generator in the Volunteer Computing Project Oleg Zaikin, Alexander Semenov,
Extracting Minimum Unsatisfiable Cores with a Greedy Genetic Algorithm Jianmin Zhang, Sikun Li, and Shengyu Shen School of Computer Science, National University.
Hard Problems Some problems are hard to solve.  No polynomial time algorithm is known.  E.g., NP-hard problems such as machine scheduling, bin packing,
Chapter 10 NP-Complete Problems.
Inference in Propositional Logic (and Intro to SAT)
Inference and search for the propositional satisfiability problem
Richard Anderson Lecture 26 NP-Completeness
Oleg Zaikin, Alexander Semenov, Mikhail Posypkin
EA C461 – Artificial Intelligence Logical Agent
Hard Problems Introduction to NP
Objective of This Course
Richard Anderson Lecture 25 NP-Completeness
ECE 667 Synthesis and Verification of Digital Circuits
CSE 6408 Advanced Algorithms.
Dichotomies in CSP Karl Lieberherr inspired by the paper:
Faster Extraction of High-Level Minimal Unsatisfiable Cores
Presentation transcript:

MaxSAT Evaluations SURVEY & RECENT RESULTS

Outline  Max-SAT overview  Sub problems of Max-SAT  Applications of Max-SAT  Annual Evaluations, 2015 evaluation  Example of a solver  evaluations  Summary

V

Reminder What is MaxSAT? The problem of determining the maximum number of clauses, of a given Boolean formula in conjunctive normal form (CNF), that can be made true by an assignment of truth values to the variables of the formula (Wikipedia).

Reminder NP-HARD Optimization of SAT Difficult to find an approximate solution of the problem Large-size MAX-SAT instances cannot be solved exactly, and one must resort to approximation algorithms and heuristics.

Reminder The weighted maximum satisfiability problem (Weighted MAX-SAT) asks for the maximum weight which can be satisfied by any assignment, given a set of weighted clauses. The partial maximum satisfiability problem (PMAX-SAT) asks for the maximum number of clauses which can be satisfied by any assignment of a given subset of clauses (Hard clauses). The rest of the clauses is like in Max-SAT.

Applications of MaxSAT Cook Levin theorem Many real-world applications can be encoded to MaxSAT: Eclipse platform uses MaxSAT for managing the plugins dependencies Error localization in C code Debugging of hardware designs Reasoning over Biological Networks (Genes) Course timetabling...

Applications of Max-SAT in Automotive (Re)Configuration Applications of MaxSAT in Automotive Configuration paper Written by: Rouven Walter and Christoph Zengler and Wolfgang Kuchlin The after-sales business asks for extensions, replacements, or removal of components of a valid configuration with minimal effort. Automotive configuration can be represented as a constraint satisfaction problem and also as a CNF formula in propositional logic, where each satisfying assignment is called a valid configuration of a car.

Applications of Max-SAT in Automotive (Re)Configuration SAT example With the SAT based configuration, two main problems arise. First, if the configuration is not valid, it is not possible to know which components cause the conflict. Second, even if we know the components causing the conflict, we do not know, which components to omit to get a valid configuration with a maximal number of components we wanted originally.

Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT  Annual Evaluations, 2015 evaluation  Example of a solver  evaluations  Summary

Introduction The International Conferences on Theory and Applications of Satisfiability Testing are the primary annual meetings for researchers studying the propositional satisfiability problem (SAT), a prominent problem in both theoretical and applied computer science. SAT lies at the heart of the most important open problem in complexity theory (P vs NP) and underlies many applications in, among other examples, artificial intelligence, operations research, and electronic design engineering.

Introduction The primary objective of the conferences is to bring together researchers from various areas and communities, including theoretical and experimental computer science as well as the many relevant application areas, to promote collaboration and the communication of new theoretical and practical results in SAT-related research and its industrial applications. Location of conferences SAT 2015 conference website Journal

Introduction There have been 10 evaluations so far. The objective of the evaluation is assessing the state of the art in the field of Max-SAT solvers, as well as creating a collection of publicly available Max-SAT benchmark instances. The evaluation allows the submission of incomplete solvers in a Special Track, with the same three categories than for complete solvers, but with a reduced number of instances.

Dates Opening – April 25 th, 2015 Submission deadline – June 30 th, 2015 Results of the evaluation -September 24 th -27 th, 2015 After a few days of the submission, a representative sample of benchmark instances were provided to the authors of solvers with the aim of allowing them to correct possible bugs and tune their solvers. We will focus on last evaluation in 2015

Rules The most attention is on complete solvers, but there is also a special track for incomplete solvers. We will discuss the difference and definitions of complete and incomplete solvers soon, but first lets see how we register to this event. There are three categories: Max-SAT (Unweighted) Partial Max-SAT Weighted Partial Max-SAT We will focus on the Max-SAT category

What is a solver? Solver

Benchmarks and solver requirements Input The input file must be read from the file given in parameter, for example:./mysolver The input file format for Max-SAT will be in DIMACS format, for example: c comments p cnf

Benchmarks and Solver Requirements DIMACS The file can start with comments, that is lines beginning with the character 'c'. Right after the comments, there is the line "p cnf nbvar nbclauses" indicating that the instance is in CNF format; nbvar is the number of a variables appearing in the file; nbclauses is the exact number of clauses contained in the file. Then the clauses follow. Each clause is a sequence of distinct non-null numbers between –nbvar and nbvar ending with 0 on the same line. Positive numbers denote the corresponding variables. Negative numbers denote the negations of the corresponding variables.

Benchmarks and Solver Requirements Output The solvers must output messages on the standard output that will be used to check the results. The output format is inspired by the DIMACS output specification of the SAT competition and may be used to manually check some results. The solver cannot write to any files except standard output and standard error (only standard output will be parsed for results, but both output and error will be memorized during the whole evaluation process, for all executions).

Benchmarks and Solver Requirements Output messages Lets go over it in the website: MaxSAT 2015

Benchmarks and Solver Requirements Benchmarks categories industrial - come from a real application random - are randomly generated crafted - all others - they are usually designed to give a hard time to solvers

Machine Specifications Only two versions of the same solver will be accepted for each category. In earlier competitions one version was allowed per solver. For each instance and solver there will be a time limit of 30 minutes. Assessment of solvers will be based on the number of successfully solved instances and the time needed to solve them. Solvers will run on machines with the following specification: Operating System: CentOS release x86_64 GNU/Linux Processor: Intel(R) Xeon(R) CPU E GHz Memory: 3.5 GB Cache: KB Compilers: GCC 4.4.6, javac J2RE 2.3

Complete vs incomplete Complete - identify provenly optimal solutions in finite time. Meaning if the solver is finished then an optimal solution is given, if exists. Incomplete solvers cannot determine wither the solution given is an optimal one. In general, incomplete solvers are faster.

Complete vs incomplete Per instance a solver gets - Complete : 1800 seconds (30 minutes) - Incomplete : 300 seconds (5 minutes)

Incomplete track The input format is the same than for complete solvers. The output format will be the following: The solver must print the optimal solution lines ("o" line) as soon as possible. For each "o" the environment will record the user time at which the line has been printed. In order to verify the optimum, a values line ("v" line) will be required. Only the last "v" line printed will be considered, in which all the solution values must be. The best solver, for a given instance, will be the one giving the best solution. Ties will be broken by user time of the best optimum The benchmarks in each category of this Special Track will be a selection of instances of each sub-category

Results - Incomplete Results random

Presentation Max-SAT 2015 Presentation given in 2015 after evaluations

Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation  Example of a solver  evaluations  Summary

Solvers Branch and Bound (B&B) based: Clone, MaxSatz, IncMaxSatz, IUT_MaxSatz, GIDSHSat. Satisfiability based: SAT4J, QMaxSat. Unsatisfiability based: WBO, msuncore, WPM1, PM2.

Example of a solver Open-WBO Authors: Ruben martins, Vasco Manquinho, Ines Lynce Contributors: Saurabh Joshi An open source version of the MaxSAT solver WBO Won several awards in MaxSAT evaluation 2014 Implements unsatisfiability-based algorithm and linear search algorithm Can solve all MaxSAT variants, namely, unweighted MaxSAT, partial MaxSAT, and weighted partial MaxSAT Result in 2015

Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT

Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT

Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT

Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT

Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT

Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT

Open-WBO Unsatisfiability-based algorithms Consists of identifying unsatisfiable subformulas, relaxing each clause in each unsatisfiable subformula, and adding a new constraint requiring exactly one relaxation variable to be relaxed in each unsatisfiable subformula. x xn ≤ 1 Ladder encoding Open-WBO

Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver  evaluations  Summary

The First and Second Max-SAT Evaluations Written by: Josep Argelich Chu-Min Li Felip Many`a Jordi Planes

The First and Second Max-SAT Evaluations Reasons for establish and organize the evaluations: Growing interest of the SAT community in developing fast exact Max-SAT solvers Lack of a good collection of benchmarks Good experiences gathered from the SAT competitions Competitions Competitions

The First and Second Max-SAT Evaluations Main objectives of the evaluations: Assessing the advancements in the field of Max-SAT solvers through a comparison of their performances Identifying successful solving techniques Encouraging researchers to develop new ones Creating a publicly available collection of challenging Max- SAT benchmarks

2006 Max-SAT 6 Solvers Unweighted Max-SAT and Weighted Max-SAT

2007 Max-SAT 12 Solvers Two additional categories: Partial Max-SAT and Weighted Partial Max-SAT

Focus on complete solvers Most of the benchmarks used in the first two evaluations were contributed The organizers added random Max-2-SAT and Max-3- SAT instances Changing the clause–to–variable ratio The First and Second Max-SAT Evaluations

zChaff A solver participated in 2006 Max-SAT. Based on Chaff algorithm, which is an improvement of DPLL algorithm. The First and Second Max-SAT Evaluations

DPLL algorithm Davis–Putnam–Logemann–Loveland 50 Years ago Basic algorithm for SAT Many of the algorithms and heuristics are expansions and improvement of this algorithm The First and Second Max-SAT Evaluations

DPLL algorithm Davis–Putnam–Logemann–Loveland Unit Propagation Pure Variable Assignment Backtracking The First and Second Max-SAT Evaluations

Results unweighted Max-SAT The First and Second Max-SAT Evaluations

What the first evaluations have contributed? Provided a quite accurate snapshot of the current state-of-the-art of complete Max-SAT solvers Contributed to increase the interest and activity of the research community on Max-SAT Allowed to identify a number of good performing solving technique The First and Second Max-SAT Evaluations

What the first evaluations have contributed? Promoted the creation of a publicly available collection of challenging Max-SAT benchmarks Most authors of solvers had the opportunity of correcting a significant number of bugs in their code From benchmarks, in each category, are divided into 3 types: random, crafted and industrial The First and Second Max-SAT Evaluations

Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations  Summary Thank You!