Comparative Succinctness of KR Formalisms Paolo Liberatore.

Slides:



Advertisements
Similar presentations
Computational Complexity
Advertisements

NP-Hard Nattee Niparnan.
Lecture 24 MAS 714 Hartmut Klauck
UIUC CS 497: Section EA Lecture #2 Reasoning in Artificial Intelligence Professor: Eyal Amir Spring Semester 2004.
Department of Computer Science & Engineering
The Theory of NP-Completeness
Great Theoretical Ideas in Computer Science for Some.
Umans Complexity Theory Lectures Lecture 2a: Reductions & Completeness.
CSC5160 Topics in Algorithms Tutorial 2 Introduction to NP-Complete Problems Feb Jerry Le
Complexity 15-1 Complexity Andrei Bulatov Hierarchy Theorem.
1 Introduction to Computability Theory Lecture12: Decidable Languages Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture15: Reductions Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture12: Reductions Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture13: Mapping Reductions Prof. Amos Israeli.
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Read sections 7.1 – 7.3 of the book for next time.
1 Polynomial Church-Turing thesis A decision problem can be solved in polynomial time by using a reasonable sequential model of computation if and only.
1 Perfect Matchings in Bipartite Graphs An undirected graph G=(U  V,E) is bipartite if U  V=  and E  U  V. A 1-1 and onto function f:U  V is a perfect.
Analysis of Algorithms CS 477/677
Automata & Formal Languages, Feodor F. Dragan, Kent State University 1 CHAPTER 5 Reducibility Contents Undecidable Problems from Language Theory.
CSE 421 Algorithms Richard Anderson Lecture 27 NP Completeness.
CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Lecture NP-Completeness Jan Maluszynski, IDA, 2007
Chapter 11: Limitations of Algorithmic Power
Toward NP-Completeness: Introduction Almost all the algorithms we studies so far were bounded by some polynomial in the size of the input, so we call them.
Halting Problem. Background - Halting Problem Common error: Program goes into an infinite loop. Wouldn’t it be nice to have a tool that would warn us.
1 L is in NP means: There is a language L’ in P and a polynomial p so that L 1 ≤ L 2 means: For some polynomial time computable map r : x: x L 1 iff r(x)
1 The Theory of NP-Completeness 2012/11/6 P: the class of problems which can be solved by a deterministic polynomial algorithm. NP : the class of decision.
Nattee Niparnan. Easy & Hard Problem What is “difficulty” of problem? Difficult for computer scientist to derive algorithm for the problem? Difficult.
February 18, 2015CS21 Lecture 181 CS21 Decidability and Tractability Lecture 18 February 18, 2015.
Computational Complexity Theory Lecture 2: Reductions, NP-completeness, Cook-Levin theorem Indian Institute of Science.
Theory of Computing Lecture 17 MAS 714 Hartmut Klauck.
NP Complexity By Mussie Araya. What is NP Complexity? Formal Definition: NP is the set of decision problems solvable in polynomial time by a non- deterministic.
Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard Despite a numerous attempts we do not know any efficient.
CSCI 3160 Design and Analysis of Algorithms Tutorial 10 Chengyu Lin.
Theory of Computing Lecture 21 MAS 714 Hartmut Klauck.
1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst case.
1 P P := the class of decision problems (languages) decided by a Turing machine so that for some polynomial p and all x, the machine terminates after at.
Umans Complexity Theory Lectures Lecture 1a: Problems and Languages.
NP-COMPLETE PROBLEMS. Admin  Two more assignments…  No office hours on tomorrow.
NP-Complete problems.
NP-Completness Turing Machine. Hard problems There are many many important problems for which no polynomial algorithms is known. We show that a polynomial-time.
Young CS 331 D&A of Algo. NP-Completeness1 NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and.
CSCI 2670 Introduction to Theory of Computing December 2, 2004.
CSCI 2670 Introduction to Theory of Computing December 7, 2005.
TU/e Algorithms (2IL15) – Lecture 9 1 NP-Completeness NOT AND OR AND NOT AND.
Knowledge Repn. & Reasoning Lecture #9: Propositional Logic UIUC CS 498: Section EA Professor: Eyal Amir Fall Semester 2005.
1 The Theory of NP-Completeness 2 Review: Finding lower bound by problem transformation Problem X reduces to problem Y (X  Y ) iff X can be solved by.
Computability Examples. Reducibility. NP completeness. Homework: Find other examples of NP complete problems.
The NP class. NP-completeness Lecture2. The NP-class The NP class is a class that contains all the problems that can be decided by a Non-Deterministic.
Lecture 9: Query Complexity Tuesday, January 30, 2001.
NP-Completeness A problem is NP-complete if: It is in NP
The NP class. NP-completeness
P & NP.
L is in NP means: There is a language L’ in P and a polynomial p so that L1 ≤ L2 means: For some polynomial time computable map r :  x: x  L1 iff.
NP-Completeness (2) NP-Completeness Graphs 7/23/ :02 PM x x x x
Perfect Matchings in Bipartite Graphs
CS21 Decidability and Tractability
Intro to Theory of Computation
Intro to Theory of Computation
Intractable Problems Time-Bounded Turing Machines Classes P and NP
Richard Anderson Lecture 25 NP-Completeness
Alternating tree Automata and Parity games
Chapter 34: NP-Completeness
Halting Problem.
Chapter 11 Limitations of Algorithm Power
CSE 6408 Advanced Algorithms.
NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979.
Umans Complexity Theory Lectures
CSE 589 Applied Algorithms Spring 1999
Instructor: Aaron Roth
Lecture 23 NP-Hard Problems
Presentation transcript:

Comparative Succinctness of KR Formalisms Paolo Liberatore

Outline The problem; Direct proofs; Compilability proofs; Applications of succinctness.

Representation: Explicit/Succinct Explicit: a set of propositional models (tuples of binary values); Implicit: a propositional formula. Explicit: an ordering of models; Implicit: a formula in a language for preference representation.

Stupid Example x 1 =Italian x 2 =French, x 3 =German Ciccio is either Italian or French or German: –Explicit: x 1 x 2 x 3, x 1 -x 2 x 3, x 1 x 2 -x 3, … –Succinct: x 1  x 2  x 3 Explicit: all possible cases; Succinct: can be even more intuitive.

Running example Knowledge: a set of modes; KB: something representing a set of models Language: method for associating a KB to a set of models (and vice versa). Example of languages: set of models, 3CNFs, set of terms, formulae, 3CNFs+new variables, default logic, etc.

Expressivity Given: two languages LA and LB; Question: does every set of models that can be expressed in LA be expressed in LB? Not in this talk!

Succinctness Given: two languages LA and LB; Question: do every set of models that can be expressed in LA be expressed in LB in polynomial space? This talk is about this.

Reformulation The question is the same as: Can every knowledge base K 1 in LA be translated into a K 2 in LB such that: K 1 and K 2 express the same set of modes; K 2 is at most polynomially larger than K 1

Notation Model: I 1, I 2, I 3,… Set of models: S Knowledge base: K 1, K 2, … Languages: LA, LB

Results on Succinctness: 2 Kinds 1.Possibilty of polysize translations: ad-hoc proofs (not in this talk); 2.Impossibility: –Direct proofs; –Proofs based on complexity classes.

Direct Proofs: 2 (Sub-)kinds 1.Based only on combinatorial arguments; 2.Based on circuit complexity theory. Not a theoretical difference.

A Trivial Direct Proof Two languages: LA: a KB is a set of complete terms LB: a KB is a 3CNF Terms: {x 1 x 2 x 3, -x 1 x 2 x 3, x 1 -x 2 x 3, …} 3CNF: x 1  x 2  x 3 LB (3CNFs) is “obviously” more succinct.

Considerations Most of the languages allow more than one KB to represent the same set of models; A language can be short in representing one set of models but longer on another one; Size is relevant only for large KB’s.

Equivalent KB’s Term: x 1  x 2  x 3 3CNF: {x 1  x 2  x 3, -x 1  x 2  x 3, x 1  -x 2  x 3, …} Sets of terms are more succinct than 3CNFs? Equivalent 3CNF: {x 1, x 2, x 3 }; Always consider the most succinct KB’s!

Specific Sets Incomparable languages: –LA: S is short but R is large; –LB: S is large and R is short. Comparable: every S that is short in LA is short in LB as well.

Asymptotic Behavior Reduction from LA to LB is possible if: –For every S That can be represented in LA in size n –It can also be represented in LB in p(n) Impossibility: –Exists S 1, S 2, …, S n, … such that: –S i can be represented in LA in size n –S i cannot be represented in LB in p(n)

Example The proof for terms vs. 3CNFs: {{x 1,x 2,x 3 }} is a specific set of models {{{x 1  x 2  …  x n }} | n>0} is a set of sets 3CNFs can be more succinct than sets of terms: proved by the second, not the first.

Circuit Complexity Classes within P; Non-conditioned results. A useful result: PARITY is not in AC 0 Meaning: no polynomial-size CNF formula represents the set of all models with an even number of 1’s.

A Language Language of 3CNFs with new variables KB=(F,X,Y) where: –F: a 3CNF formula on variables X  Y (disjoint) Represents sets of models on variables X I (a model on variables X) is in the set represented by KB=(F,X,Y) if there exists a model J on variables Y such that I  J is a model of F

Application of PARITY LA=language of 3CNFs; LB=language of 3CNFs with new variables. We can use PARITY to prove that LB is more succinct than LA

PARITY in Action S n =all models of n variables with an even number of 1’s In LA: not in polynomial space; In LB: since parity can be checked in polynomial time, there exists a circuit (a specific kind of formulae with new variables) that represents S n in polynomial space.

Proofs Using Complexity Classes Largest part of the talk Idea: given a problem on S that –is hard if S is expressed in LA –is simple if S is expressed in LB translating from LA to LB must be difficult! (otherwise, solve by first translating!)

More Notations… I  S means that I is a model of S I  KB, where KB is a knowledge base, means that KB represents a set of models that contains S Checking I  KB is a decision problem. Can be represented by a set: A={(K,I)|I  K}

Easy Result I  S is a polynomial-time problem; I  KB can NP-hard: –It is if KB is in the language of 3CNFs+new variables. Have we proved that the language of 3CNFs+new variables is more succinct than the explicit representation? NO!

Hardness and Size I Hardness: how long does it take; Succinctness: how much space is needed. Referred to a language: Hard: takes a long time to translate; Succinct: translating produce large result.

Hardness and Size II Languages for representing a single bit: –LA: explicit representation (0 or 1); –LB: a bit is represented by a Turing machine: the machines that always terminate represent 1; the others represent 0. Translating from LB to LA is undecidable. Is LB more succinct?

Hardness  Size Fact: –Translating from LB to LA is hard (undecidable in this case!); –Translation result is polynomially-sized. Consequence: –Hardness cannot be used to compare succinctness. (btw: both 0 and 1 have short TM representation: LA and LB are succinctly equivalent)

Compilability Digression (>10 slides!); How hard is a problem if part of its data can be preprocessed? Example: in diagnosis, we have: –the description of the system to diagnose; –the specific faults. They do not have the same status.

Assumptions on Preprocessing Solving is done in two steps: –First preprocess one part of the input only; –Then, solve the problem. The first phase (the preprocessing step): –Can take arbitrarily long time; –Must produce a polynomially-sized result.

Preprocessing, Pictorially Preprocessing Step On-line processing In-part 1 In-part 2 out

Classes of Compilability Complexity of the on-line part; The complexity of the preprocessing step is not counted. Complexity: P and NP. Compilability: ~>P and ~>NP.

Classes: Formal Definition A problem is a set of pairs of strings; –E.g, A={(x,y)} Solving=telling whether (x,y)  A for a given pair of strings (x,y) Idea: x is the part we can preprocess; Usual formalization of decision problems.

Formal definition II Class ~>P: is a set of problems A={(x,y)} A  ~>P if there exists: –Problem B  P –Function f from strings to strings (see below!) Such that: –(x,y)  A if and only if (f(x),y)  B

The function f Is the in/out function of the preprocessing step Its computation is not bounded on time; Its result must be of polynomial size w.r.t. the size of its argument. Formally: f is polysize if there exists a polynomial p such that, for every string x, it holds |f(x)|<p(|x|)

Must f be computable? Depending on what we try to prove: That a problem is in ~>P; reasonable to assume that f is computable; That a problem is not in ~>P: stronger results if f is not bounded.

Back to Succinctness… The question was: given K 1 in LA, is there any equivalent K 2 in LB that is (at most) polynomially larger? Equivalence means: I  K 1 iff I  K 2 ; Question, reformulated: solve the problem I  K 1 by preprocessing K 1 into K 2.

Complexity and Compilability Problem A is I  K 1 ; Problem B is I  K 2 ; Complexity of B: polynomial; If every K 1 in LA has an equivalent K 2 in LB of polynomial size, then: A  ~>P (f=the function that gives K 2 given K 1 )

Why? Facts: –I  K 1 is equivalent to I  K 2 ; –K 1 can be translated into K 2 (not in P!) –I  K 2 is in P –f defined as f(K 1 )=K 2 is a polysize function –I  K 1 iff I  K 2 Consequence: –Solving I  K 1 is in ~>P

So What? The other way around: Prove that I  K 1 is not in ~>P Conclude that K 1 cannot be translated into a polynomially-sized K 2 This is a method for obtaining negative results (impossibility of polysize translations).

How to prove non-membership? Membership to ~>P: no general method; Non-membership: proofs based on hardness Seen: definition of ~>P is based on P; Now: definition of ~>NP based on NP; Generalization to an arbitrary class of problems C.

Compilability Classes Replace P with another class C everywhere: –A  ~>C if there exists B and f such that: –B  C –(x,y)  A iff (f(x),y)  B Function f is polysize: –Result is at most polynomially larger than argument.

Compilability-Hardness Based on polynomial reductions; Direct definition of hardness not useful; Classes ||~>C: the preprocessing step can use the first part of data and the size of the second part; The corresponding hardness is useful.

Monotonic Reductions Proving ||~> hardness is… hard; Sufficient conditions: –Monotonic reductions; –Representative equivalence. Only sufficient; Usually work.

Monotonic Reductions: the Base Problem A={(x,y)} is NP-hard; –Complexity, not compiability; Means: –there exists two polynomial functions r,h; –F is sat iff (r(F)),h(F))  A How can A be proved ||~>NP-hard?

Monotonic Reductions r, h: polynomial reduction from 3sat to A For every two 3CNF formulae F and G that: –Have the same variables; –F  G (i.e., G has some clauses more than F) If: (r(F),h(F))  A iff (r(G),h(F))  A Then: problem A is ||~>NP-hard. [there is no typo in this slide]

Operatively… Usually, A is already known NP-hard; Polynomial-time reduction from 3sat to A known; Often, does not satisfy the condition of representative equivalence. In such cases: find a new reduction.

Reduction: Guideline I A is the problem of checking whether a model I satisfied a knowledge base K; A={(K,I)|I is a model of K} Reduction from 3sat to A: F is safisfiable iff I is a model of K If K depends only on the number of variables of F the reduction is monotonic.

Reduction: Guideline II F=variables+structure (clauses) Variables of F  K Whole formula F  I How can this be done? F is a 3CNF of n variables Given n variables, there are only O(n 3 ) possible clauses of three variables.

Reduction: Guideline III F  G={(v i  c i )|c i  C n } v i are new variables C n =set of all 3-clauses on the same variables of F F is “almost” equivalent to G  {v i |c i  F} Reduce: –G  K –{v i |c i  F}  I Easier to reduce a set of variables to a model.

Reduction: Example Language of 3CNF with new variables; Is NP-hard; by reduction from 3sat: –3CNF formula F on variables X is sat if and only if the empty model is a model of (F, ,X) This reduction is not monotonic.

A Monotonic Reduction F  G={(v i  c i )|c i  C n } where: –C n =all clauses of three variables over the same variables of F F is sat iff G  {v i |c i  F} is sat; Consequence: F is sat iff {v i |c i  F} is a model of G; Is a monotonic reduction.

Does it always work? Sufficient condition; G to K and {v i |c i  F} to I is hard sometimes. Intuitive meaning, based on structures.

Generalization Often, we have: A collection of objects (e.g., propositional variables); These objects form structures (e.g., clauses, defaults, etc.) K is a collection of these structures. Idea: use subcase with few possible structures.

Application I Object: nodes; Structures: edges; Knowledge base: graph. n nodes: at most n 2 possible edges.

Application 2 Object: variables; Structures: formulae and defaults; Knowledge base: default theory. Limit to the case of defaults containing only a fixed number of variables.

Intuition What these reductions prove? F contains two pieces of information: –The number of variables; –The clauses. We reduce the clauses to I and the number of variables to K; The complexity is in I, not in K; Preprocessing K is useless.

Preprocessing and Succinctness A=checking whether a model is a model of a knowledge base in language LA B=the same for LB If A is ||~>NP-hard and B is in ~>P; There exists knowledge bases in LA that cannot be polynomially expressed in LB.

Time/Space Tradeoff LA is compilability-hard  it is succinct LB is compilability-simple  is not succinct Compilability hardness prove succinctness. Note: a language that is hard but not compilability hard is both hard and not succinct.

Knowledge Bases Structures that represent knowledge; –So far: knowledge=set of models; Could also be: –Knowledge=set of propositional formulae; –Knowledge=ordering of models; –???

References Cadoli et al. Preprocessing of intractable problems, I&C 176(2), Liberatore, Monotonic reductions, representative equivalence, and compilation of intractable problems, JACM 48(6), Cadoli et al. Space efficiency of propositional knowledge representation formalisms, JAIR 2000.