Witold Pedrycz Department of Electrical & Computer Engineering University of Alberta, Edmonton, Canada and Systems Research Institute, Polish Academy of Sciences Warsaw, Poland

Agenda Introduction: human-centricity of intelligent systems and information granules Conceptualization and realization of information granules Information content and its characterization Context-based information granules Successive refinements of information granules Information granules –based architectures Conclusions

Human-system interaction and system modeling Perception and processing processing realized at certain level of abstraction Acceptance of granular (non-numeric) data Effective two-way communication with the user at the level of information granules Adjustment of level of detail (abstraction) dependent upon the needs of the individual user (personalization); avoidance of unnecessary details and focus on essentials

Clustering and fuzzy clustering Discovery of structures and relationships in data Data analysis Construction of fuzzy sets Clustering in fuzzy modeling and modeling …

Information granules: from conceptualization to realization Implicit information granules Explicit (operational) information granules Explicit (operational) information granules Humans Computer realizations Various points of view (models) Fuzzy sets Rough sets Intervals (sets) Shadowed sets Probability functions Fuzzy sets Rough sets Intervals (sets) Shadowed sets Probability functions Information granules

Development of information granules Usage of all available experimental evidence (numeric and non-numeric; knowledge hints) Information granules capturing existing domain knowledge (especially guidance –knowledge hints provided by the designer/user) Results of information granulation dependent upon the underlying formalism of Granular Computing Information granules are context-dependent; the ensuing design framework should incorporate this aspect in an explicit way

Information granules: from conceptualization to realization DataInformation granules Construction of Intelligent systems auxiliary guidance mechanisms Content of information granules Formalisms of sets, fuzzy sets, rough sets

Clustering and fuzzy clustering Clustering 2,670,000 Fuzzy clustering 443,000 Rough clustering 268,000 Information granulation/granules 158,000 Granular Computing 104,000 Google Scholar, October 29, 2014

Objective function-based clustering {x 1, x 2,…, x N } Objective function  Minimize w.r.t. structure information granules G 1, G 2, …,G c Prototypes, medoids v 1, v 2,…, v c Partition matrix U

Fuzzy C-Means (FCM) as an example of fuzzy clustering v i – prototypes U- partition matrix

FCM – representation of information granules (granules) Partition matrix U prototypes v 1, v 2, …, v c Partition matrix U prototypes v 1, v 2, …, v c

Fuzzy Clustering: Fuzzy C-Means (FCM) Given data x 1, x 2, …, x N, determine its structure by forming a collection of information granules – fuzzy sets Objective function Minimize Q; structure in data (partition matrix and prototypes)

FCM – flow of optimization Minimize subject to (a) prototypes (b) partition matrix

Construction of information granules: Fuzzy C-Means (FCM) Data {x 1, x 2, …, x N } x k in R n. Performance index (objective function) Construct information granules (clusters) - fuzzy sets A 1, A 2, …, A c organized as partition matrix U Partition matrix Prototype v i

Quality of clustering Cluster validity indexes Cluster content Granulation-degranulation: reconstruction criterion

Information content of clusters Description of information content: Variability of data Classification content Variability with regard to auxiliary (output) data

Variability of data Description of data residing within a given i th cluster Variability of data around the prototype v i Variability in terms of membership grades of data

Classification content of clusters Applied to classification problems. Dominant class present in i th cluster Classification content: count index cumulative membership grades of classes

Granular mapping: an architecture Aggregation of contents of information granules and their activation levels

Auxiliary variable content Problems in which occur some additional variables (say output variable, y) whose values determine the content of the cluster. Clustering realized for data in the multivariable input space

Information granulation and degranulation: reconstruction criterion v 1 v 2 v c granulation u 1, u 2, …, u c v 1 v 2 v c degranulation Results of degranulation made more abstract (in the form of information granules): granular clustering

Key challenges of clustering Selection of distance function (geometry of clusters) Number of clusters Quality of clustering results

Landscape of clustering Graph-oriented and hierarchical (single linkage, complete linkage, average linkage..) Objective function-based clustering Variety of formalisms and optimization tools (e.g., methods of Evolutionary Computing) Diversity Commonality Data-driven methods

The dichotomy and a paradigm shift Human-centricity Guidance mechanisms

Knowledge –based clustering dataknowledge Partial supervision Context-based guidance Proximity –based Viewpoints

Domain Knowledge: categories of knowledge-oriented guidance Context-based guidance: clustering realized in a certain context specified with regard to some attribute Viewpoints: some structural information is provided Partially labeled data: some data are provided with labels (classes) Proximity knowledge: some pairs of data are quantified in terms of their proximity (resemblance, closeness)

Context-based clustering Clustering : construct clusters in input space X Context-based Clustering : construct clusters in input space X given some context expressed in output space Y Active role of the designer [customization of processing]

Context-based clustering: Conmputational considerations computationally more efficient, well-focused, designer-guided clustering process computationally more efficient, well-focused, designer-guided clustering process Data structure Data structure context

Context-based clustering: focus mechanism Determine structure in input space given the output is high Determine structure in input space given the output is medium Determine structure in input space given the output is low Input space (data)

Context-based clustering: examples Find a structure of customer data [clustering] Find a structure of customer data considering customers making weekly purchases in the range [$1,000 $3,000] Find a structure of customer data considering customers making weekly purchases at the level of around $ 2,500 Find a structure of customer data considering customers making significant weekly purchases who are young no context context (compound)

Context-based Fuzzy C-Means data(x k, y k ), k=1,2,…,N contexts: fuzzy sets W 1, W 2, …, W p defined in the output space w jk = W j (y k ) Context-drivenpartition matrix

Context-based clustering: the use of context xkxk Context W j ykyk W j (y k ) xkxk Context-based fuzzy clustering

Context-oriented FCM: Optimization flow Objective function Iterative adjustment of partition matrix and prototypes Subject to constraint U in U(W j )

Successive refinements of information granules Information granules constructed in a successive manner forming a hierarchy of refined constructs of higher specificity The refinement applied to information granules based on their information content Successive usage of context-based fuzzy clustering

Successive refinements of information granules information granule to be refined membership function A i [1] used as a context refinement process membership function A j [2] used as a context

Expansion formulas: Context-based FCM information granule to be refined membership function A i [1] used as context refinement process property of fuzzy partition membership function A j [2] used as context

Expansion formulas: Context-based FCM information granule to be refined membership function A i [1] used as context membership function A j [2] used as context

Successive fuzzy partitions

Fuzzy clustering with viewpoints

Viewpoints: definition Description of entity (concept) which is deemed essential in describing phenomenon (system) and helpful in casting an overall analysis in a required setting “external”, “reinforced” clusters

Viewpoints: examples viewpoint (a,b)viewpoint (a,?)

Viewpoints in fuzzy clustering B- Boolean matrix characterizing structure: viewpoints prototypes (induced by data)

Viewpoints in fuzzy clustering

Fuzzy clustering with proximity guidelines

Proximity hints Characterization in terms of proximity degrees: Prox(k, l), k, l=1,2, …., N and supervision indicator matrix B = [b kl ], k, l=1,2,…, N Prox(k,l) Prox(s,t)

Proximity measure Properties of proximity: (a)Prox(k, k) =1 (b)Prox(k,l) = Prox(l,k) Proximity induced by partition matrix U Linkages with kernel functions K(x k, x l )

Augmented objective function  > 0

Granular fuzzy clustering

Granular prototypes and reconstruction criterion prototypesgranular prototypes Selection/construction of prototypes Forming granular prototypes to capture existing structural variability and satisfying degranulation criterion (a) information granules of prototypes built around prototypes (b) optimization of allocation of granularity by minimizing reconstruction criterion

Formation of granular (interval) membership grades – details x ViVi u i - (x)=min(w 1 (x), w 2 (x)) u i + (x)=max(w 1 (x), w 2 (x))

Overview Information granules Blueprint of model content Model development, refinements, augmentations

Conclusions Fuzzy clustering as a conceptual and algorithmic backbone of design of information granules Human-centric (knowledge-oriented) design of information granules Emergence of higher type granular constructs Needs for further advancements in optimization frameworks of fuzzy clustering