Consider the machine learning problem in the figure below (only four examples of each class are shown for simplicity, imagine that we have 1,000 examples to train our machine learning algorithms). Which of the following algorithms could potentially achieve close to 100% classification accuracy on this problem? Hint, plot out a bunch of examples in a 2D plot A) Nearest Neighbor and Decision tree B) Simple Linear Classifier C) Decision tree onlyD) No algorithm can do well on this problem. A learning problem. (The rule used to determine class is this, if the sum of the two heights is less than or equal to ten, it is class A, otherwise it is class B).

Consider the machine learning problem below. Suppose I learn a Nearest Neighbor for this problem. The resulting classifier would have an accuracy rate of about… A) ) 99 to 100%B) 20 to 25% C) 70 to 90%.D) Nearest Neighbor is not defined for this problem Now suppose I learn a simple linear classifier for this problem. The resulting classifier would have an accuracy rate of about… A) ) 99 to 100%B) 45 to 55% C) 70 to 90%.D) 0%

A) Do nothingB) Make the decision tree more accurate. C) Make both algorithms more accurateD) Make nearest neighbor more accurate. A) Abandoning the projectB) Does not effect my recommendation C) Decision treeD) Nearest neighbor A) Does not effect my recommendationB) Decision tree C) Abandoning the projectD) Nearest neighbor I am working for the government on a secret project. The send me a dataset with ninety-two million instances and 17 continuous features. I have no idea what the features are. I quickly test both the nearest neighbor and decision tree classifiers, they both have about 87% accuracy on this problem, so I have no reason to choose one over the other… Given the above, suppose that the government tells me that it now knows that features 2 and 3 are useless for the problem at hand. Using this information I can Given the above, suppose that the government tells me that it the problem is to classify whether a image on a radar screen is an civilian airplane or incoming missile. In the latter case the government has 0.03 seconds to try to shoot the missile down. This information would make me lean towards recommending Given the above, suppose that the government tells me that one of the goals of the project is to gain some understanding of the data, not just to build a accurate classifier. This information would make me lean towards recommending

Look at the classification problem to the left. How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a nearest neighbor would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch)

Look at the classification problem to the left. How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a nearest neighbor would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch)

Look at the classification problem to the left. How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a nearest neighbor would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch)

Look at the classification problem to the left. How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a nearest neighbor would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch) Note: This is a tricky one. There is a good decision tree for this problem, but it is not clear that the algorithm we discussed would find it. For now, just draw the best decision tree for this problem

Look at the classification problem to the left. In this example we know both the x ans y value for all our examples: However, suppose that we only had the X value How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch) Now suppose that we only had the Y value How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch) X Y

Look at the classification problem to the left. How well to you think a simple linear classifier would work? Draw the decision boundary for it (just a rough sketch) How well to you think a nearest neighbor would work? Draw the decision boundary for it (just a rough sketch) How well to you think a decision tree would work? Draw the decision tree, and draw the decision boundary (just a rough sketch) Note: This is a tricky one. There is a good decision tree for this problem, but it is not clear that the algorithm we discussed would find it. For now, just draw the best decision tree for this problem

(Western Pipistrelle (Parastrellus hesperus) Photo by Michael Durham

Characteristic frequency Call duration Western pipistrelle calls We can easily measure two features of bat calls. Their characteristic frequency and their call duration