1. More about the Learning Algorithm More about the Class being learned
Define the model
Four examples
More about the Model
More about the Learning Algorithm
More about the Class being learned
More about the Teacher
More about the Learning Complexity
Back to the examples
11/30/2018
Content

2. Exact Learning Problem from Membership Queries
Interpolation
Exact Learning MQ
Inferring from Q
Active Learning
Guessing Game
Combinatorial Search
Hitting
Testing
Verification
Black Box PIT
White Box PIT
Find 𝑓 exactly
Test whether 𝑓≡𝑔, 𝑓≡0 etc. 𝑓
𝑓(𝑥)
𝑓∈𝐶 𝑥
𝑓:𝐷→𝑅
Goal: Find 𝑓 exactly
11/30/2018
Exact learning from MQ

3. אוניית מלחמה - צוללות (משחק)
Battleship game
Type of ship
Size
aircraft carrier 5
battleship 4
submarine 3
destroyer
patrol boat 2
A B C D E F G H I J
Each player secretly arranges their ships אֳנִיָּה on their primary grid
The game proceeds in a series of rounds. In each round, each player takes a turn to announce a target square in the opponent's grid which is to be shot יָרָה at.
The opponent announces whether or not the square is occupied by a ship, and if it is a "hit" they mark this on their own primary grid.
If all of a player's ships have been sunk שָׁקַע, the game is over and their opponent wins.
11/30/2018
Battleship game

4. 𝒇: 𝟏,𝟐,⋯,𝟏𝟎 × 𝑨,𝑩,⋯,𝑱 →{𝟎,𝟏} 𝒇 𝟒,𝑫 =𝟏 a "hit" 𝒇 𝟕,𝑬 =𝟎 a “miss" 𝑪={𝒇}1 2 3 4 5 6 7 8 9 10 A B C D E F G H I J
𝒇: 𝟏,𝟐,⋯,𝟏𝟎 × 𝑨,𝑩,⋯,𝑱 →{𝟎,𝟏}
𝒇 𝟒,𝑫 =𝟏
a "hit"
𝒇 𝟕,𝑬 =𝟎
a “miss"
𝑪={𝒇}
11/30/2018
Battleship game

5. 1 2 3 4 5 6 7 8 9 10 A B C D E F G H I J Find 𝑓 exactly
Test whether 𝑓≡0.
Hitting set
11/30/2018
Battleship game

6. Group Testing
Robert Dorfman's paper in 1943 introduced the field of (Combinatorial) Group Testing. The motivation arose during the Second World War when the United States Public Health Service and the Selective service embarked upon a large scale project. The objective was to weed out all syphilitic נדְבִּק men called up for induction. However, syphilis testing back then was expensive and testing every soldier individually would have been very cost heavy and inefficient.
We can combine עִרְבּוּב blood samples and test a combined sample
together to check if at least one soldier has syphilis.
11/30/2018
Group Testing

7. Group Testing & Learning
𝑥 1 , 𝑥 2 , 𝑥 3 , 𝑥 4 ,
𝑥 5 , 𝑥 6 , 𝑥 7 , 𝑥 8 ,
𝑥 9 , 𝑥 10 , 𝑥 11 , 𝑥 12
𝑥 13 , 𝑥 14 , 𝑥 15 , 𝑥 16
YES NO
YES
( )
( )
( )
𝑓= 𝑥 3 ∨ 𝑥 10
11/30/2018
Group Testing & Learning

8. Decision Tree Group Testing
Decision Tree of Depth 𝑑 1
𝑥 2 1 1 1 1
𝑥 4
𝑥 3
2 𝑑 1 1
𝑥 3 1
𝑥 1 1
Group Testing 𝑑 1 1
𝑥 3
𝑓: 0,1 𝑛 →{0,1}
𝑓= 𝑥 3 ∨ 𝑥 10
𝑓 1011 =1 1
𝑥 10 1
𝑓 1000 =0
11/30/2018
Decision Tree

9. Battleship as Decision Tree
𝑥 1 1 2 3 4 5 6 7 8 9
𝑥 2 1 1
𝑥 2 =5 1
𝑥 1 <3 1
𝑥 2 >4 1
𝑥 2 >1 1 1
𝑥 1 =3
11/30/2018
Battleship as Decision Tree

10. The structure of part of a DNA
𝑦𝑒𝑠 1
𝑥 2
𝑦𝑒𝑠 1 1
𝑥 4
𝑥 3 𝑛𝑜
𝑦𝑒𝑠
𝑥 3 1 𝑛𝑜 1
𝑥 1
The structure of part of a DNA
𝑦𝑒𝑠
11/30/2018
DNA

11. Resistorנַגָּד Combinations
Change the
resistance
הִתְנַגְּדוּת
𝑅= 𝑅 𝑅 4 + 𝑅 𝑅 𝑅 2 + 𝑅 3
11/30/2018
Resistor Combination

12. More about the Learning Algorithm More about the Class being learned
Define the model
Four examples
More about the Model
More about the Learning Algorithm
More about the Class being learned
More about the Teacher
More about the Learning Complexity
Back to the examples
11/30/2018
Content

13. 𝑓 𝑓∈𝐶 𝑥 𝑓(𝑥) 𝑓:𝐷→𝑅 𝐴 𝐶 (𝑀 𝑄 𝑓 ,𝐼,𝑟) Learning Algorithm
Goal: Find 𝑓 exactly
Learning Algorithm
𝐴 𝐶 (𝑀 𝑄 𝑓 ,𝐼,𝑟)
11/30/2018
Algorithm

14. Learning Algorithm Sequential Parallel Deterministic Randomized
Monte Carlo Alg.
Pr 𝑟 𝐴 𝐶 𝑀 𝑄 𝑓 ,𝐼,𝑟 =𝑓 ≥1−𝛿
Deterministic
Randomized
Las Vegas Alg.
Pr 𝑟 𝐴 𝐶 𝑀 𝑄 𝑓 ,𝐼,𝑟 =𝑓 =1
Adaptive
Non-adaptive
Rounds
11/30/2018
Learning Algorithm

15. Input Taget 𝒇 𝑪 𝑪 𝒕 Boolean 𝒇:𝑫→{𝟎,𝟏} Arithmetic 𝒇: 𝑹 𝒏 →𝑹 Discrete
𝒇:𝑫→{𝟎,𝟏,⋯,𝒎}
11/30/2018
Target

16. Output Interpolation Exact Learning MQ Inferring from Q
Active Learning
Guessing Game
Combinatorial Search
Hitting
Testing
Verification
Black Box PIT
Test if 𝑓≡𝑔, 𝑓≡0
𝒇∈𝑪
Find 𝑓 exactly
NonProper
computable 𝒉
Proper
𝒉∈𝑪
PAC, PAC+MQ
Find 𝑓 approximately
11/30/2018
Summary

17. Teacher, Black Box Opponent player
Honest
הוֹגֵן
Liar שַׁקְרָן
Incomplete MQ
Angluin Slonim 94
I DON’T KNOW wp 𝒑
(Limited) Malicious MQ
Angluin Krikis Sloan Turan 95
m Incorrect
Answers ? with probability p
+persistent
Malicious MQ
Valiant 85- Kearns Li 88
Incorrect wp 𝒑
Limited MQ
Angluin Krikis Sloan Turan 95
m I DON’T KNOW
Answers incorrect to m
+ persistent
Incorrect answers with probability p
+ persistent
Answers ? with to m
+ persistent
11/30/2018
Teacher

18. Query Complexity=𝑂𝑃𝑇(𝐶)𝑓 𝐶
𝑓(𝑥)∈𝐶
𝑥 =𝑛
𝑠𝑖𝑧𝑒 𝑓 ≤𝑠
Query Complexity=𝑂𝑃𝑇(𝐶)
Query Complexity
Time Complexity
Learnable
𝑝𝑜𝑙𝑦(𝑂𝑃𝑇(𝐶),𝑛,𝑠)
Efficient Learnable
𝑝𝑜𝑙𝑦(𝑂𝑃𝑇(𝐶))
𝑝𝑜𝑙𝑦(𝑂𝑃𝑇(𝐶),𝑛,𝑠)
Optimally Learnable
𝑂𝑃𝑇(𝐶) 1+𝑜 1
𝑝𝑜𝑙𝑦(𝑂𝑃𝑇(𝐶),𝑛,𝑠)
Optimally Learn. in 𝑛
𝑂𝑃𝑇(𝐶) 1+𝑜 1 , 𝑠=𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
Randomized
Deterministic
Teacher
Adaptive
Non-adaptive
Honest
Liar
11/30/2018
Complexity

19. More about the Learning Algorithm More about the Class being learned
Define the model
Four examples
More about the Model
More about the Learning Algorithm
More about the Class being learned
More about the Teacher
More about the Learning Complexity
Back to the examples
11/30/2018
Content