Expert/rule based classification The topic of my research is Georferencing of images by exploiting geometric distortions in stereo images of UK DMC. And this is my preliminary defense presentation. Contact: mirza.waqar@seecs.edu.pk Mirza Muhammad Waqar

Lecture Overview Image classification basics Image and feature spaces Supervised vs Unsupervised classification Basic concepts of Expert classification Imagine’ Expert Classifier Knowledge Engineer Setting up of rules Output evaluation Final notes

Remote Sensing Process

Snow Vs. Clouds

Snow Vs. Clouds Clouds scatter at all wavelengths Snow absorbs at >1.4 mm c

Spatial Resolution Spatial Resolution / Pixel size Spectral Resolution / Number of bands Multispectral < 10 bands AVHRR: 1km Hyperspectral 10’s to 100’s of bands Landsat: 30m SPOT: 10m Quickbird: 2m Sub-meter Time

Generalized Workflow

Image Classification What is it ? Grouping of similar features Separation of dissimilar ones Assigning class label to pixels Resulting in manageable size of classes

Image Classification (2)

Image Classification (3)

Image Space

Feature Space

Feature Space (2) Two dimensional graph of scatter plot Formation of cluster of points representing DN values of two spectral bands Each cluster of points corresponds to a certain cover type on ground Scatter plot of two bands

Supervised Classification Satellite Image (Large Area) Ground Truth (Small Area) Thematic Map (Large Area)

Supervised vs Unsupervised

Limitations of Fundamental Methods One of the drawbacks of the fundamental methods is that the only information utilized is that contained in the image itself, in the form of one or more channels. Nothing else is considered. On the contrary, human interpreters of hard copy images can integrate many other types of information. While working on a image, they may also take into consideration such information as that from available thematic maps, personal field experience, common sense, etc.

Limitations of Fundamental Methods (2) When natural objectives, such as landscape and forest, are concerned, relations among the objects and their surroundings can be used by specialists with corresponding knowledge. As known from forestry and ecology, there exist certain relations between plants and their environmental conditions. Plants grow best where the environmental conditions are most favorable for their specific adaptabilities. Information about the distribution of the plant objects can be exploited from their environmental conditions.

Overview of Classification Methods

What is Expert, or Rule-based Classification? Expert knowledge can be represented in form of rules: if condition then inference Complex combinations of rules can be built (knowledge base) Can be applied on both pixel and object (region) base Although we use Erdas Imagine Expert Classifier as an example, the principles are generic

Rules Definition The base contains various "hard rules", i.e., the rules with no probability or belief weight. They are usually of the form of “IF···, THEN··· “ Another kind of rules are "fuzzy rules“, which go with a value of probability or belief weight. The hard rules are usually based on common knowledge or common sense; While the fuzzy rules are obtained through interviewing and questionnaires from experts

Expert Classification We can employ a similar approach to eCognition at the pixel level, using Imagine’s Expert Classifier It is argued that the quality of an image classification increases with the amount of information we have available Landcover classification

Expert Classification (2) We may further want to perform an analysis that goes beyond a mere identification of a landcover Example: analyze a terrain in terms of the mobility it allows. Identify different more or less easily traversable pixels, ranging from wide roads (easy) to forest (slow) to water or buildings (no go).

Expert Classification (3) This is what the Expert Classifier does, in a way that is more akin to GIS analysis than traditional classification The software uses a rule-based approach with a hierarchy of rules, user-defined variables, and information sources as varied as raster imagery, vector files, graphic models or external programs It is similar to the logic and structure of the graphic models that can be constructed in Imagine, but provides a better frame to follow complex rules and decisions

Expert Classifier The expert classifier is integrated into the Classifier module, and consists of 2 parts, the Knowledge Engineer and the Knowledge Classifier The Knowledge Engineer provides an interface for an expert, while the Knowledge Classifier can be used by an non-expert to execute an existing Classification Knowledge Base There is virtually no limit to complexity, although understanding the logic of a multitude of rules is not always easy, and neither is translating a research question into the appropriate hypotheses and rules

Expert Classifier - Layout Decision Tree Overview The main work (and close-up) window The knowledge base component

Decision Trees The classifier is composed of decision tree branches, i.e. a hypothesis, a rule (or more), and one or more conditions Hypothesis Rule Condition And Or

Decision Trees (2) Translating rules and hypotheses into algebra can be difficult

Example of Decision Tree Overview The main work (and close-up) window The knowledge base component

Example of Decision Tree (2) Note that the hypotheses don’t lead to a single answer, but rather to pixels of particular value in a new, resulting thematic image file. This means that the result of the analysis is a classified raster image The objective in the following example was to identify terrain suitable for easy traversing Input data consist of (i) the result of a supervised ML classification, (ii) a DEM, (iii) a map of major and minor roads, and (iv) a georeferenced aerial photograph. In addition, several existing graphic models are used

Decision Trees – A Small Exercise: Exercise 1: We want to find areas to grow grapes. Those need a southern exposure, preferably a hill side, and good rainfall. What data do we need and how do we construct the knowledge base? We need: Elevation data Rainfall information

Expert Classification Output So what do we get from Expert Classifier? A simple thematic map, where each pixel is assigned a class Satisfaction of one of several rules leads to acceptance of hypothesis – so how do we determine which data source to chose? The hypothesis with the highest confidence is chosen Sometimes we have data sources of different quality/reliability

We can selectively turn off individual classes Output Evaluation Expert Classifier provides a powerful way to evaluating the results Analysis can be run in ‘Test-mode’ We can selectively turn off individual classes We can also produce a confidence image to assess the quality of our classification

Confidence Image We can also create the confidence image Good to evaluate which rules may have to be revised

Output Evaluation (2) For testing we can also turn off individual hypotheses within the workspace

Output Evaluation (3) We can also evaluate the classified image, and “back-track” how a pixel came to be classified

Knowledge Classifier The knowledge base created can then also be used by non-experts in the Knowledge Classifier

Final Notes A rule-based system has obvious strengths and limitations. If the entire knowledge in a particular domain can be encoded in a finite set of rules, then a rule-based system is effective. On the other hand, if there are too many rules, it becomes difficult to maintain the system.

Final Notes (2) A well-constructed and useful knowledge base is a challenging and involved task Several refinements and iterations are likely required But these knowledge bases also serve as basic of expert systems where non-experts can do the analysis – getting it right is vital! The key is to understand the pros & cons of different classification methods, and then chose accordingly

Questions & Discussion