Silvino Barreiros. Cross Validation Bootstrapping Testing Prepared for final submission REU Poster.

Slides:

Advertisements

Similar presentations

Human Detection Phanindra Varma. Detection -- Overview  Human detection in static images is based on the HOG (Histogram of Oriented Gradients) encoding.

Advertisements

Classification Classification Examples

My name is Dustin Boswell and I will be presenting: Ensemble Methods in Machine Learning by Thomas G. Dietterich Oregon State University, Corvallis, Oregon.

Learning Algorithm Evaluation

Data Mining Methodology 1. Why have a Methodology  Don’t want to learn things that aren’t true May not represent any underlying reality ○ Spurious correlation.

Ragib Hasan University of Alabama at Birmingham CS 491/691/791 Fall 2011 Lecture 10 09/15/2011 Security and Privacy in Cloud Computing.

A Classification-based Glioma Diffusion Model Using MRI Data Marianne Morris 1,2 Russ Greiner 1,2, Jörg Sander 2, Albert Murtha 3, Mark Schmidt 1,2 1 Alberta.

Supervised classification performance (prediction) assessment Dr. Huiru Zheng Dr. Franscisco Azuaje School of Computing and Mathematics Faculty of Engineering.

Machine Learning IV Ensembles CSE 473. © Daniel S. Weld 2 Machine Learning Outline Machine learning: Supervised learning Overfitting Ensembles of classifiers.

Object Class Recognition Using Discriminative Local Features Gyuri Dorko and Cordelia Schmid.

Pedestrian Recognition Machine Perception and Modeling of Human Behavior Manfred Lau.

Fitting. Choose a parametric object/some objects to represent a set of tokens Most interesting case is when criterion is not local –can’t tell whether.

Examples of Ensemble Methods

TTI's Gender Prediction System using Bootstrapping and Identical-Hierarchy Mohammad Golam Sohrab Computational Intelligence Laboratory Toyota.

COS 429 PS3: Stitching a Panorama Due November 4 th.

Learning and Recognizing Activities in Streams of Video Dinesh Govindaraju.

Exercise Session 10 – Image Categorization

Evaluating Classifiers

Toshihide IBARAKI Mikio KUBO Tomoyasu MASUDA Takeaki UNO Mutsunori YAGIURA Effective Local Search Algorithms for the Vehicle Routing Problem with General.

Anomaly detection Problem motivation Machine Learning.

2015 AprilUNIVERSITY OF HAIFA, DEPARTMENT OF STATISTICS, SEMINAR FOR M.A 1 Hastie, Tibshirani and Friedman.The Elements of Statistical Learning (2nd edition,

Learning Visual Bits with Direct Feature Selection Joel Jurik 1 and Rahul Sukthankar 2,3 1 University of Central Florida 2 Intel Research Pittsburgh 3.

Creating With Code.

EMIS-R Data Collector Uncovered Teresa Williams NWOCA/SSDT OAEP

Categorical data. Decision Tree Classification Which feature to split on? Try to classify as many as possible with each split (This is a good split)

Object Recognition in Images Slides originally created by Bernd Heisele.

Ensembles. Ensemble Methods l Construct a set of classifiers from training data l Predict class label of previously unseen records by aggregating predictions.

Manu Chandran. Outline Background and motivation Over view of techniques Cross validation Bootstrap method Setting up the problem Comparing AIC,BIC,Crossvalidation,Bootstrap.

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

S. Salzberg CMSC 828N 1 Three classic HMM problems 2.Decoding: given a model and an output sequence, what is the most likely state sequence through the.

Parallel lines are like train tracks. They never cross.

Bilinear Logistic Regression for Factored Diagnosis Problems Sumit Basu 1, John Dunagan 1,2, Kevin Duh 1,3, and Kiran-Kumar Munuswamy-Reddy 1,4 1 Microsoft.

BOĞAZİÇİ UNIVERSITY DEPARTMENT OF MANAGEMENT INFORMATION SYSTEMS MATLAB AS A DATA MINING ENVIRONMENT.

Data Mining Practical Machine Learning Tools and Techniques By I. H. Witten, E. Frank and M. A. Hall Chapter 5: Credibility: Evaluating What’s Been Learned.

A Multiresolution Symbolic Representation of Time Series Vasileios Megalooikonomou Qiang Wang Guo Li Christos Faloutsos Presented by Rui Li.

Improved Video Categorization from Text Metadata and User Comments ACM SIGIR 2011:Research and development in Information Retrieval - Katja Filippova -

WEEK4 RESEARCH Amari Lewis Aidean Sharghi. PREPARING THE DATASET  Cars – 83 samples  3 images for each sample when x=0  7 images for each sample when.

Weekly Presentation Anderson G Moura 07/13/2015. Goals Wavelets – Set up an experiment to add a new feature to the htk files generated by the exp0022.

Validation methods.

Chapter 5: Credibility. Introduction Performance on the training set is not a good indicator of performance on an independent set. We need to predict.

CISC Machine Learning for Solving Systems Problems Presented by: Eunjung Park Dept of Computer & Information Sciences University of Delaware Solutions.

Project Overview CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.

Conductor, insulator and ground. Force between two point charges:

Silvino Barreiros. Team all on the same page Initial System Complete Bootstrapping – Research and Coding Started.

Cold-Start KBP Something from Nothing Sean Monahan, Dean Carpenter Language Computer.

COS 429 PS3: Stitching a Panorama Due November 10 th.

Week 4: 6/6 – 6/10 Jeffrey Loppert. This week.. Coded a Histogram of Oriented Gradients (HOG) Feature Extractor Extracted features from positive and negative.

Mustafa Gokce Baydogan, George Runger and Eugene Tuv INFORMS Annual Meeting 2011, Charlotte A Bag-of-Features Framework for Time Series Classification.

1 Machine Learning Lecture 8: Ensemble Methods Moshe Koppel Slides adapted from Raymond J. Mooney and others.

Week 3 Emily Hand UNR. Online Multiple Instance Learning The goal of MIL is to classify unseen bags, instances, by using the labeled bags as training.

Course Project Lists for ITCS6157 Jianping Fan. Project Implementation Lists Automatic Image Clustering You can download 1,000,000 images from You can.

Data Science Credibility: Evaluating What’s Been Learned

REU Report Meetings – Week 7

CSE P573 Applications of Artificial Intelligence Decision Trees

Interval Estimation.

PROBLEM 1 Training Examples: Class 1 Training Examples: Class 2

Construct a Convolutional Neural Network with Python

Three classic HMM problems

CSE 573 Introduction to Artificial Intelligence Decision Trees

Learning Algorithm Evaluation

30 A 30 B 30 C 30 D 30 E 77 TOTALS ORIGINAL COUNT CURRENT COUNT

Approaching an ML Problem

Herbert G. Mayer, PSU CS Status 8/2/2013

Replace this image. Problem Solver- TITLE.

Chair Professor Chin-Chen Chang Feng Chia University

Incremental Programming

What are the real world implications of this project?

REU Report Meetings – Week 9

Presentation transcript:

Silvino Barreiros

Cross Validation Bootstrapping Testing Prepared for final submission REU Poster

Train on the hardest images Test images with high PFA moved to train set Images in training replaced Keeps the training set from growing to large

Steps Extract all positives and and equal number of negatives for training sets Take 100 negatives that we didn't train on as the testing set Compute the codebook once using the training images Use codebook to generate word counts and the classifier

Steps continued Test the 100 images in the negative testing set Move the worst false alarms from the negative testing set into the negative training set Add a corresponding number of duplicate positives into the positive training set Replenish the negative testing set with more negatives in order to bring it back to 100 images Iterate at step 4, but don't recompute the codebook! Use the original codebook, but generate new wordcounts using the additional training images.

Features Tested Airplane_flying, Bus, Dog, Doorway and Person-eating Results Each iteration produces varying results Reasons Not enough positive images used Images moved from the training set are better then ones moved in

IterationPDPFATrain SetTest Set %28% %22% %12% %10% %6% %6% %3% %8% AVG69.64%11.875% Airplane_flying

IterationPDPFATrain SetTest Set 160%64% %19% %20% %14% %5% %9% %11% %8% AVG57.5%18.75% Chair

IterationPDPFATrain SetTest Set 170%34% %20% %19% %12% %9% %7% %14% %8% AVG61.25%15.375% Infant

Started testing on harder features Airplane Flying, Doorway… etc After testing complete change parameters

Method to test positive images Group positive images Cycle one group to test and train on others

Crunch time!! Final Run Complete submission code