1. 1/44 A simple Test For the Consecutive Ones Property

2. 2/44 P-Q Trees Two types of internal nodes: P-nodes & Q-nodes Children of a P-node can be “permuted arbitrarily” Children of a Q-node can only be “reversed” Q P 12 34 L(T) = { all permutations generated by T } In the example, L(T) = { 1234,1243,4321,3421 }

3. 3/44 Intermediate On-Line Operations …… … … …… … … …… PQ-tree and PC-tree implementations for the COP are “on-line”, i.e. you can see the rows only one at a time

4. 4/44 Off-Line vs On-Line PC-tree algorithm is an on-line algorithm We design a simple off-line algorithm –If one arranges the rows in a good order, then even a kid knows how to test C1P In general we can apply graph decomposition to obtain a good ordering

5. 5/44 An intelligent off-line test Assume all rows of the matrix are given at the beginning Can we find a “good” order of the rows to be processed for the COP test so that the test becomes easier? –No more PQ-trees nor PC-trees. –Even a child can perform the test for you!!

6. 6/44 Strictly Overlapping (S.O.) Relationships Two rows are said to overlap strictly if they overlap but none is contained in the other. Such a pair of rows implies the following column partition: 1 - - - - - - 1 1 - - - 1 1 - - - - 1 u v Ｖ\ｕＶ\ｕ Ｖ∩ｕＶ∩ｕｕ\Ｖｕ\Ｖ

7. 7/44 Ideal Situation If there is a row ordering v 1, v 2,…,v m such that each v i strictly overlaps with some v j, j ＜ i, then it is trivial to test the consecutive ones property Partition Before 1- - - - - - 1 1 - - - 1 1 - - - - 1 After 1 - - - 1 1 - - 1 1 - - 1 1 - - - 1 1 - - - - 1

8. 8/44 The strictly overlapping graph Define the graph G on the set of rows whose edge set consists of those strictly overlapping pairs of columns. Each connected component of G satisfies the above “ideal situation”. –Why? Consider a spanning tree and a breadth-first order –The corresponding submatrices are called prime –Can show that the matrix satisfies the COP iff each of its prime submatrices does

9. 9/44 Strictly Overlapping Graph G 2 7 1 5 9 8 4 6 3 10 11

10. 10/44 Decomposition of Consecutive Ones Matrix 1 2 3 4 5 6 7 8 9 10 11

11. 11/44 Decomposition – A Divide and Conquer Strategy Divide and conquer 1.To reduce complexity 2.To make the problem easier to solve To simplify a graph algorithm, we can decompose the graph until the problem becomes easier to solve on the final components –If the decomposition operation is also very efficient, then we will get an efficient algorithm at the end

12. 12/44 Another example of the Graph G 1 2 3 4 5 6 7 8 9 10 1 6 4 3 7 9 8 5 2

13. 13/44 A spanning subgraph G’ of G However, we cannot afford to compute all the edges in G, which could take O( r 2 ) time. We shall compute a subset of edges that contain a spanning tree of each connected component. Note that the process of obtaining the component actually decompose the matrix into prime submatrices

14. 14/44 COP Testing with Good Row Ordering

15. 15/44 Exact Algorithm for Consecutive Ones Testing 1. Construct a subgraph G’ which contains a spanning tree of G ( the S. O. graph ). Each connected component corresponds to a prime submatrix. ( matrix decomposition ) 2. Decide a good ordering of a prime matrix based on BFS. 3.For each prime matrix determine the ordering of columns, using the set partition strategy as described in the following slides. Process the rows from small to large

16. 16/44 An Efficiency Note Assume every row in A strictly overlaps every one in B. The # of strictly adjacent pairs is |A| |B|. Let a, b be the least indexed rows in A,B, respectively. To connect A,B in the graph, it suffices to make a adjacent to all rows in B and b adjacent to all rows in A. A B a b

17. 17/44 An Efficiency Note The # of strictly adjacent pairs is |A| |B|. Let a, b be the least indexed rows in A,B, respectively. To connect A,B, it suffices to make a adjacent to all rows in B and b adjacent to all rows in A. A B a b

18. 18/44 1 1 1 1 1 1 1 1 1 Representative Rows v A and v B v v 1/2 Let v be adjacent to both A and B. But, v A and v B are forbidden to be made adjacent to A, B, respectively (to avoid incorrect s. o. edge formed) vAvA vBvB vAvA vBvB vBvB vAvA

19. 19/44 Classifying the neighbors of a row u u B D C A 1.Append A(u),B(u) and D(u) to PT(u) (the set of candidates that are potentially s. o. with u). 2.Append u D to PT(w) for all w in C(u) whose index is smaller than Ind(u D ) 3.Delete the row u and use an artificial column [u] to replace the region covered by columns of u 4.Add edges from u to nodes of PT(u)-FB(u) (the set of rows forbidden to be s. o. with u)

20. 20/44 Partition the columns within row u This is relatively unique u At the end of the iteration, no longer have to worry about the columns within u

21. 21/44 After a row u is processed All of its columns are shrunk to one artificial column [u] (the main reduction) All the 1’s for rows in A, B in columns of u are eliminated except to save ½ for v A and v B (to discover future strictly overlapping relations) Save a 1 for rows in C and eliminate all rows in D.

22. 22/44 Which S. O. relations have been Changed? B F C A u E vAvA vBvB All those rows which have s. o. relation changed are connected to u i.e. they belong to the same component as u does

23. 23/44 Afterwards B F C A u E vAvA vBvB The connected components do not change after the reduction of u Rows that were not s. o. cannot become s. o.

24. 24/44 6 1 4 5 3 2 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 0 1 1 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 16 1 2 3 5 6 4 0 0 0 0 0 0 0 1 0 1 1 0 0 0 1 1.5 0 1 1 1 1 0 0 4 5 2 63 1632 2 3 5 6 4

25. 25/44 6 4 5 3 16 5 32 1.5 0 1 0 0 0 0 0 0 1 1 0 0 1.5 1 1 3 5 6 4 0.5 0 6 4 5 16 45 32 1 0 1 1 0 0.5 5 6 4 6 5 16 45 32.5 1 0 0.5 5 6

26. 26/44 Lemma 1 If u j  FB(u i ) ∪ PT(u i ), i<j, u i and u j are connected in G’

27. 27/44 Lemma 2 If one of the u i and u j (i<j) is contained in the other and the containment is changed before iteraion i, u i and u j are connected in G’. 0.5 uiui uiui ujuj ujuj [u k ] ukuk 0

28. 28/44 The sub-graph G’ generated by the algorithm G’ is a spanning sub-graph of G' with the same components. Claim 1. G’ is a subgraph of G. If(u i,u j ) G, (u i,u j ) G’ Claim 2. if(u i, u j ) G, then u i and u j belong to the same component of G’

29. 29/44 Claim 1 G’ is a subgraph of G vBvB vAvA u [u][u] 0.5 In this case, u i is in FB(u j ) and u j is in FB(u i ) 1. u i and u j are independent originally. The only case they could be s.o. is when they become v A and v B for some u. 2. u i is contained in u j originally. (apply Lemma 2)

30. 30/44 Claim 2 If(u i, u j )  E(G), then u i and u j belong to the same component of G’. Suppose not. Let u i,u j be the minimal bad pair. (for all other bad pair (u p,u q ) either i<p or j<q) Consider the changing of intersection relationship –“intersect” to “contain” (case 1) –“intersect” to “independent” (case 2)

31. 31/44 Case 1. “intersect” changed to “contain” u i and u j intersect originally. Let one of the u i and u j be contained in the other after iteration k. Consider the following two subcases: Case 1.1: Both u i and u j overlap u k. Case 1.2: Only one of the u i and u j (say, z) overlaps u k ( The other is named e A )

32. 32/44 Case 1.1 Both u i and u j overlap u k ukuk ukuk u i is connected to u j through u k uiui uiui ujuj ujuj

33. 33/44 Case 1.2 one of u i and u j (say, z) overlaps u k z eAeA z eAeA ukuk z eAeA ukAukA u k is connected to z and u k A. We shall verify if u k A is connected to e A. ukuk

34. 34/44 Case 1.2 Only one of the u i and u j (said) z overlaps u k Case (i) u k a is contained in e A originally By lemma 2, u k a is connected to e A. Case (ii) u k a contains e A originally z eAeA ukAukA ukuk π -1 (e A ) < π -1 (u k A ) < π -1 (z) If z is deleted at iteration t (t< π -1 (e A ) ) z eAeA ukAukA ukuk t π-1(e A ) < π-1(z) < π-1(u t D ) e A connects u t D. u t D connects t. t connects z.

35. 35/44 Case 1.2 Case (iii) u k A is indepenet e A originally Let u k A overlaps e A after iteration t. u k A is connected to e A via u t Case (iv) u k A intersects e A originally (u k A, e A ) becomes the minimal bad pair. (a contradiction) It concludes that u k A is connected to e A in G" such that e A and z is connected in G".

36. 36/44 Case 2. ““intersect” changed to “independent” u i and u j intersect originally. Let one of the u i and u j become indepedent after iteration k. consider the following two subcases: Case 2.1: Both u i and u j overlap u k. Case 2.2: Only one of the u i and u j (said) z intersects u k (The other is named e A )

37. 37/44 Case 2.1 Both u i and u j overlap u k ukuk ukuk u i is connected to u j through u k in G’ uiui uiui ujuj ujuj

38. 38/44 Case 2.2 Only one of the u i and u j (say, z) intersects u k z eAeA z eAeA z eAeA ukAukA u k is connected to z and u k A. We shall verify if u k A is connected to e A. ukuk

39. 39/44 Case 2.2 Only one of the u i and u j (said) z intersects u k (i) u k A is independent to e A or one is contained in the other originally. Check Claim 1 (ii) u k A intersects e A originally. If u k A is not connected to e A, (u k A,e A ) becomes the minimal bad pair. (a contradiction)