Visual Inspection Planning with Sensor Constraint Graph

Slides:



Advertisements
Similar presentations
Approximation algorithms for geometric intersection graphs.
Advertisements

CS 206 Introduction to Computer Science II 04 / 01 / 2009 Instructor: Michael Eckmann.
Motion Planning for Point Robots CS 659 Kris Hauser.
 Distance Problems: › Post Office Problem › Nearest Neighbors and Closest Pair › Largest Empty and Smallest Enclosing Circle  Sub graphs of Delaunay.
Label Placement and graph drawing Imo Lieberwerth.
Graphs Chapter 12. Chapter Objectives  To become familiar with graph terminology and the different types of graphs  To study a Graph ADT and different.
University of Pennsylvania 1 GRASP CIS 580 Machine Perception Fall 2004 Jianbo Shi Object recognition.
Models and Security Requirements for IDS. Overview The system and attack model Security requirements for IDS –Sensitivity –Detection Analysis methodology.
Data Transmission and Base Station Placement for Optimizing Network Lifetime. E. Arkin, V. Polishchuk, A. Efrat, S. Ramasubramanian,V. PolishchukA. EfratS.
Announcements Final Exam May 16 th, 8 am (not my idea). Practice quiz handout 5/8. Review session: think about good times. PS5: For challenge problems,
Totally Unimodular Matrices Lecture 11: Feb 23 Simplex Algorithm Elliposid Algorithm.
- 1 - Intentional Mobility in Wireless Sensor Networks Deployment, Dispatch, and Applications Dr. You-Chiun Wang ( 王友群 ) Department of Computer Science,
Contents Description of the big picture Theoretical background on this work The Algorithm Examples.
Graphs Chapter 12. Chapter 12: Graphs2 Chapter Objectives To become familiar with graph terminology and the different types of graphs To study a Graph.
Spring 2010CS 2251 Graphs Chapter 10. Spring 2010CS 2252 Chapter Objectives To become familiar with graph terminology and the different types of graphs.
Randomized Planning for Short Inspection Paths Tim Danner Lydia E. Kavraki Department of Computer Science Rice University.
Job Scheduling Lecture 19: March 19. Job Scheduling: Unrelated Multiple Machines There are n jobs, each job has: a processing time p(i,j) (the time to.
Fall 2007CS 2251 Graphs Chapter 12. Fall 2007CS 2252 Chapter Objectives To become familiar with graph terminology and the different types of graphs To.
Approximation Algorithms Motivation and Definitions TSP Vertex Cover Scheduling.
Visualization and graphics research group CIPIC January 21, 2003Multiresolution (ECS 289L) - Winter Surface Simplification Using Quadric Error Metrics.
כמה מהתעשייה? מבנה הקורס השתנה Computer vision.
1 Shortest Path Calculations in Graphs Prof. S. M. Lee Department of Computer Science.
Efficient Gathering of Correlated Data in Sensor Networks
Systems of Inequalities in Two Variables Sec. 7.5a.
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
1  The Problem: Consider a two class task with ω 1, ω 2   LINEAR CLASSIFIERS.
Graphs Chapter 12. Chapter 12: Graphs2 Chapter Objectives To become familiar with graph terminology and the different types of graphs To study a Graph.
Single-view geometry Odilon Redon, Cyclops, 1914.
Linear Programming: A Geometric Approach3 Graphing Systems of Linear Inequalities in Two Variables Linear Programming Problems Graphical Solution of Linear.
Approximation Algorithms based on linear programming.
Spanning Trees Dijkstra (Unit 10) SOL: DM.2 Classwork worksheet Homework (day 70) Worksheet Quiz next block.
Research directions on Visual Inspection Planning By Alexis H Rivera.
Network Topology Single-level Diversity Coding System (DCS) An information source is encoded by a number of encoders. There are a number of decoders, each.
Math 3121 Abstract Algebra I Lecture 6 Midterm back over+Section 7.
Presented By Ravindra Babu, Pentyala.  Real World Problem  What is Coloring  Planar Graphs  Vertex Coloring  Edge Coloring  NP Hard Problem  Problem.
Prof. Yu-Chee Tseng Department of Computer Science
The NP class. NP-completeness
Linear Programming for Solving the DSS Problems
Uniformed Search (cont.) Computer Science cpsc322, Lecture 6
Computational Geometry
Applied Discrete Mathematics Week 14: Trees
Lap Chi Lau we will only use slides 4 to 19
BackTracking CS255.
Topics in Algorithms Lap Chi Lau.
Graph Searching.
Mean Shift Segmentation
CS 3343: Analysis of Algorithms
Lecture 12 Algorithm Analysis
Exact Inference Continued
Uniformed Search (cont.) Computer Science cpsc322, Lecture 6
Based on slides by Y. Peng University of Maryland
Presented by: Cindy Yan EE6358 Computer Vision
Craig Schroeder October 26, 2004
Enumerating Distances Using Spanners of Bounded Degree
Quantum One.
Minimum Spanning Tree.
Randomized Algorithms CS648
Graphs Chapter 13.
CSE 373 Data Structures and Algorithms
3.5 Minimum Cuts in Undirected Graphs
Lecture 12 Algorithm Analysis
Minimum Spanning Tree.
Licensed Electrical & Mechanical Engineer
Graphs.
The Greedy Approach Young CS 530 Adv. Algo. Greedy.
Lecture 12 Algorithm Analysis
Major Design Strategies
Applied Discrete Mathematics Week 13: Graphs
Concepts of Computation
Locality In Distributed Graph Algorithms
Presentation transcript:

Visual Inspection Planning with Sensor Constraint Graph By Alexis H Rivera

Overview Definition the problem Nonlinear optimization problem Visual Inspection Visual Inspection Planning Nonlinear optimization problem Constraint Sensor Graph Visual Inspection Plan

Problem Visual Inspection Given an object, determine if it satisfies the design specification Uses camera as measurement tool

Problem (cont.) Visual Inspection Planning Where should the camera be located to optimally inspect the desired geometric entities? location: camera pose or sensor setting geometric entities: vertices, edges need to define optimality

Problem (cont.) Optimality criterion minimize the inherent measurement errors such that desired entities are: resolvable in focus within field of view visible satisfy dimensional tolerances minimize number of camera sensors needed

Measurement Errors Two inherent errors involved in camera placement Displacement Errors Quantization Errors

Displacement Errors Ideal pose: Actual pose: Consequence: (tx, ty, tz, Φ, θ, Ψ) Actual pose: (tx, ty, tz, Φ, θ, Ψ) + (dtx, dty, dtz, dΦ, dθ, dΨ) Consequence: a vertex ideally mapped to image point (u, v) now is mapped to (u’, v’) causing measurement error

Quantization Error Actual Length: L = lrx + u + v Quantized Length

Error Model Goal Denote the total error as the R.V. ε = εd + εq Minimize Mean Square Error E[ε2] = σε2 + ηε2 E[ε2] = E[εd2] + E[εq2]

Resolution For each entity j, there is a constraint g1j() Map smallest line entity to l pixels Example: l = 2

Focus Two constraints, g2a(), g2b() Require closest and furthest entity vertices from the camera position to be within the far and near limits of the depth of field camera rf rc Far limit Near limit

Field Of View One constraint: g3() Bounding cone must be contained within the viewing cone Bounding cone Viewing cone

Visibility Many equations: g4i() for i=1 to m Plane equations that bound the visibility of the desired entities x y e1 e2 e3 e4 Example: To see entities e1, e2, e3, e4, the camera must satisfy equation y < 0

Visibility Equations are determined from the Entity Aspect Graph Each node contains visibility equations and entities that are visible on that aspect

Visibility (optional) Equations are determined from the Entity Aspect Graph Works by finding visual events Type C Type A Type B x y e1 e2 e3 e4

Nonlinear optimization program For a given set of entities, S minimize E[ε2] = f (tx, ty, tz, Φ, θ, Ψ, S) subject to: g1j <= 0 (resolution), for j=1 to k g2a <= 0 (focus) g2b <= 0 g3 <= (field of view) g4i <= 0 (visibility) for i=1 to m

Dimensional Tolerance From optimal pose, the observed dimensions of entities are as a close as possible to the actual real world dimension of the entity For each line entity want probability of deviation <= threshold

Definitions Passing entity Failing entity Passing optimization entity whose dimensional error satisfies tolerances Failing entity entity whose dimensional error violates tolerances Passing optimization All geometric entities are passing entities Failing optimization At least one failing entity

Inspection Planning (so far) Determine optimal camera pose for current entity Determine optimal camera pose for current entity More entities? yes Passing optimization? Regroup entities Save camera pose no yes How are entity sets determined and regrouped?

Sensor Constraint Graph SCG node: 4 tuple (E,O,G,I) E: set of desired geometric entities to be observed O: objective function (MSE) G = { V, V’} V visibility constraints V’ focus, resolution, and field of view constraints I: initial camera pose

Sensor Constraint Graph SCG Arcs solid arcs adjacent, yet disjoint visibility regions between two nodes dashed arcs overlapping visibility regions between two nodes

SCG operations Three operations: Construction Expansion Contraction

SCG construction Construct an EAG from the set of geometric entities each node of EAG has set of visible desired entities and visibility constraints Define objective function for each node Define sensor constraints G for each node Choose arbitrary initial camera pose I Link all nodes with solid arcs

Example: SCG construction {e1,e2} V2 {e3,e2} V3 {e4} V1 EAG {e1,e2} G2={V2,V2’}, O2,I2 {e3,e2} G3={V3,V3’}, O3,I3 {e4} G1={V1,V1’}, O1,I1 SCG

Passing Optimizations Stored in SLIST Node of SLIST defined as S={F,E,O} F camera pose O objective MSE (value) E set of entities SLIST = S

Similar Settings Nodes that have identical entity sets Are combined into new setting So such that Eo is set of entities Oo is MSE function Fo is the camera pose that results in smallest MSE Similar settings are replaced by So in the SLIST

Failing Optimizations There exist a set of passing entities and failing entities after an optimization Resolved using expansion and contraction operations

SCG expansion Creates subnodes Desired entity set is a subset of original node MSE function is defined in terms of such subset Sensor constraints are also defined in terms of such subsets Visibility constraints are the same as its original node ??

SCG expansion For multiple subnodes, the union of the desired entity set must be the same as the original nodes’ entity set All subnodes are connected by dashed arcs 1 2 3 SCG Expanded SCG 1 2 3a 3b

SCG contraction Creates supernodes Two or more similar nodes are contracted Supernode has the same entity set as original nodes Same MSE function Same sensor constraints Visibility constraints are the union of the visibility constraints of original nodes Initial camera pose that results in Min(MSE)

SCG contraction Neighboring nodes keep same relation with respect to supernode Example: contracting node 1 and 3b Contracting SCG 1-3b 2 3a Original SCG 1 2 3a 3b

Subnode strategies Two types of failing optimizations passing entities and failing entities in the set only failing entities in the set Five strategies to handle these cases Strategy 1: Pass/Fail Strategy 2: One less Strategy 3: Singleton Strategy 4: Similar node with one less Strategy 5: Similar node with singleton

Subnode strategies (optional) Pass/Fail Strategy create two subnodes, one with passing entities and one with failing entities One less Strategy for failing optimization with k entities create k subnodes containing k-1 entities each Singleton Strategy create k subnodes each with 1 entity

Subnode strategies (optional) Similar node with one-less strategy Select all nodes neighboring No whose entity set is a subset of No’s entity set Create a subnode that matches the entity set of each neighboring node If neighbors don’t share any common entities, then use one less strategy If the union of the neighbors’ entity set doesn’t include all the entities in No, then create an additional subnode with remaining entities

Subnode strategies (optional) Example case 1 Expanded SCG Contracted SCG {e1, e2} {e1,e2,e3} {e3} SCG {e1, e2} {e1,e2} {e3} {e1, e2} {e3}

Subnode strategies (optional) Example case 2 SCG expanded {e5, e4} {e1,e2,e3} {e6} SCG {e1, e2} {e1,e2} {e3} {e1,e3}

Subnode strategies (optional) Example case 3 Expanded SCG Contracted SCG {e1, e2} {e1,e2,e3, e4} {e3} SCG {e1, e2} {e1,e2} {e3} {e4} {e1, e2} {e3} {e4}

Subnode strategies (optional) Similar nodes with singleton same as before except using singleton strategy instead of one-less strategy

Subnode strategies Choosing between strategies S1: separates pass/fail entities to consider possibility that failing entities may be observable simultaneously S2: reduces number of camera poses S3: reduces computation time S4 and S5: reduces number of optimizations (supernodes)

Inspection Planning (so far) Determine optimal camera pose for current entity Determine optimal camera pose for current entity More entities? yes Passing optimization? Regroup entities Save camera pose no yes How do we process remaining entities?

Prioritizing nodes Prioritize nodes toward greater cardinality in entity sets to reduce number of optimizations Unprocessed nodes are included in priority queue known as NLIST

Prioritizing nodes NLIST = rest(NLIST) NLIST = rest(NLIST) Priority = 0 NLIST empty? Priority = #E(N’) Determine plan 1 yes Perform Optimization on N’ no N’ = first(NLIST) done Passing optimization? Save setting in SLIST #E(N’) < priority? no yes no yes Create subnodes Append NLIST Sort NLIST Find similar node with E# combine into supernode No replace similar nodes with No sort NLIST N’ = first(NLIST) 1

Inspection Plan The SLIST contains several camera poses Two options Minimize number of camera poses Determine smallest total MMSE Such that all desired entities are observed Integer program

Inspection Plan Let p be the number of camera poses in SLIST xi = 1 if ith camera pose Si is part of inspection plan for i=1 to p aij = 1 if entity Ej is in setting Si n be the total number of desired entities

Inspection Plan Case 2 Case 1

Visual Inspection Planning

Issues The generation of the visibility constraints is incomplete The software used to generate such constraints is obsolete The current nonlinear optimization algorithm is very sensitive to the initial conditions The process of generating a plan is not automated and is very tedious The test cases used to validate the software were very simple

Research Questions Will the previous results be affected if a complete entity aspect graph algorithm is used to determine the visibility constraints? Is there another nonlinear optimization algorithm that is more appropriate for this application? Is there a way to determine a good initial condition that will guarantee the optimal solution? Is the current error model good enough? How will the results change if a sub-pixel quantization model is used? Is the current objective function appropriate? How will the results change if the robustness approach is used instead of the minimum mean square error approach? Is there a way to characterize the different sub-node strategies? What observations can be made if objects that are more realistic are used?

Research Goal