Solving the Multiple-Instance Problem with Axis-Parallel Rectangles By Thomas G. Dietterich, Richard H. Lathrop, and Tomás Lozano-Pérez Appeared in Artificial Intelligence, Volume 89, Issue1-2, (January 1997), Pages: 31 – 71. Presented by Shuiwang Ji Part of this slides are from the original paper and from Dr. Yixin Chen.

The key-lock problem Problem statements: 1.There is a keyed lock on the door to the supply room; 2.Each staff member has a key chain containing several keys; 3.One key on each key chain can open the supply room door; 4.For some staff members, their supply key may open one or more other doors; Positive bag Negative bag

The key-lock problem Question: You are a lock smith and you are attempting to infer the most general required shape that a key must have in order to open the supply room door.

Drug activity prediction Problem statements: A good drug molecule will bind very tightly to the desired binding site; The input object is a molecule, and the observed result is “bind” or “not bind”; Conformation determines bind or not; Each drug molecule has many conformations;

Drug activity prediction Problem statements: A good drug molecule will bind very tightly to the desired binding site; The input object is a molecule, and the observed result is “bind” or “not bind”; Conformation determines bind or not; Each drug molecule has many conformations; Positive bag Negative bag

Drug activity prediction Question: The goal is to infer the most general shape required for binding to the binding site.

Supervised learning

Multiple-instance learning Key chain Molecule Key Conformations Open? Bind? bag

Formal problem definition Supervised learning: Multiple-instance learning:

Ray-based representation A set of 162 rays emanating from the origin; Each feature value is the distance from the origin to the molecular surface.

Three classes of algorithms

Standard APR algorithm x1x1 x2x2 2 For each negative instance, count the number of instances that must be excluded from the APR in order to exclude the negative instance. Greedy algorithm: Eliminate the negative instance that requires eliminating the fewest positive instances

Standard APR algorithm Only the NUMBER of positive instances is considered when eliminating negative instances; The resulting APR may not cover at least one instance from all positive bags; The cost of eliminating each positive instance should be different.

Outside-in APR algorithm x1x1 x2x2 2 Consider excluding positive instances that are expendable in the sense that every positive molecule still has at least one positive instance Greedy algorithm: define a cost function for the elimination 4

Outside-in APR algorithm x1x1 x2x2 2 Consider excluding positive instances that are expendable in the sense that every positive molecule still has at least one positive instance Greedy algorithm: define a cost function for the elimination 4

Outside-in APR algorithm The cost of excluding a positive instance of molecule depends on the other not yet excluded positive instances of the same molecule; A cost function based on the density estimation of the positive instances was proposed; The employed density estimation is expensive to compute.

Inside-out APR algorithm x1x1 x2x2 2 Choose an initial seed positive instance Grow the APR by identifying the positive instance that when added to the APR would least increase its size.

Grow: An algorithm for growing an APR with tight bounds along a specified set of features Inside-out APR algorithm Discrimination: An algorithm for choosing a set of discriminating features Expand: An algorithm for expanding the bounds of an APR to improve its generalization ability

Inside-out APR algorithm (grow) The size of an APR is the sum of the widths of all of its bounds. Greedy: Identify the positive instance of a not yet covered positive molecule that would least increase its size; 1,2,…,d-1,d, d+1 Backfitting: After making the d-th decision, all previous d- 1 decisions are revisited until no changes. 1,2,…,d-1, d, d+1

Inside-out APR algorithm (discrimination) A feature strongly discriminates against a negative instance if that instance is far outside of the bound of this feature; In each step, choose a feature that strongly discriminates against the largest number of negative instances; Repeat until all negative instances are excluded.

Inside-out APR algorithm (expand) APR resulting from the first two steps is too tight since it is designed to cover at least one positive instance of each positive bag; Apply kernel density estimation to estimate the probability that a positive instance will satisfy the bounds on that feature; Expand the bounds so that with high probability, new positive instance will fall inside the APR.

Performance of iterated discrimination on artificial data set

Performance of iterated discrimination on musk data set 1

Performance of iterated discrimination on musk data set 2

Thank you!