Dealing with Uncertainty: A Survey of Theories and Practice Yiping Li, Jianwen Chen and Ling Feng IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 11, Nov Hyunsoo Park

This paper.. Introduce uncertainty Summary uncertainty handling methods Shows examples in various domain Main contribution –Cross-disciplinary view of uncertainty processing activities by different communities

Contents What is uncertainty? Uncertainty handling theories Uncertainty handling practices Uncertainty processing in database Challenges to uncertain data management Conclusion

What is uncertainty? Definition –“uncertainty is a general concept that reflects our lack of sureness about something or someone, ranging from just short of complete sureness to an almost complete lack of conviction about an outcome.” –Distinguish from degree of belief or confidence

Uncertainty category Aleatory uncertainty –Natural variability –Objective uncertainty –External uncertainty –Random uncertainty –Stochastic uncertainty –Inherent uncertainty –Irreducible uncertainty –Fundamental uncertainty –Real-world uncertainty –Primary uncertainty Epistemic uncertainty –Knowledge uncertainty –Subjective uncertainty –Internal uncertainty –Incompleteness –Functional uncertainty –Information uncertainty –Secondary uncertainty

Uncertainty category Aleatory uncertainty –Derives from nature variability of physical world –Reflects the inherent randomness in nature –Exists naturally regardless of human knowledge –Eg. Flipping a coin Epistemic uncertainty –Origins from human’s lack of knowledge of the physical world –lack of the ability of measuring and modeling physical world –Epistemic uncertainty can be reducible –Eg. Estimation distance between two cities

Eg. Flood frequency analysis Aleatory uncertainty –Probability distribution of frequency curve Epistemic uncertainty –Parameters of the frequency curve Two categories boundary is not clear!

Uncertainty management Reduce uncertainty –Not impossible, but hard Handling uncertainty –Bayesian inference, fuzzy sets, fuzzy logic, possibility theory, time Petri nets, evidence theory, rough sets, coherent lower and upper precisions, belief functions, information theory … In database community –null value –Probabilistic database –Fuzzy and possibilistic database

Uncertainty handling theories Probability theory –Monte Carlo method –Bayesian inference –Dampster-Shafer theory Fuzzy theory Info-gap theory Derived uncertainty theory

Uncertainty handling theories Probability theory –Monte Carlo method –Bayesian inference –Dampster-Shafer theory Fuzzy theory Info-gap theory Derived uncertainty theory

Probability theory The most well-established theory Originally, aims at random phenomena Can deal with both aleatory and epistemic uncertainty Currently, probability theory is a dominant position in uncertainty handling

Extensions of probability theory Monte-Carlo method –Simulate random sampling Bayesian method –Expresses relations between two or more events through conditional prob. and make inferences Dempster-Shafer evidence theory –Combination of two belief mass from different sources evidence –Eg. Different diagnosis from two doctors

Fuzzy sets Good way to deal with uncertainty arising from human linguistic labels Interface human conceptual categories and data –Eg. Youg, hard, warm  fuzzy sets

Info-Gap theory Models uncertainty for decision making –Model-based decisions involving severe uncertainty independent of probabilities –Severe uncertainty belongs epistemic uncertainty Reflect the information gap between what one does know and what one need to know Applies to the situation of limited information (not enough data for other methods)

Derived uncertainty theory Probability + fuzzy Uncertainty space –Uncertain measure, uncertain variable, uncertain distribution Eg. Distance between A and B –Uncertainty dist. 90km –Lower then 90km with uncertainty 0.3

Summary of the four uncertainty handling theories Probability theory Fuzzy theoryDerived uncertainty theory Info-Gap theory Managed uncertainty RandomnessAmbiguity with ill- defined boundary Human’s subjectiveness uncertainty Immeasurable factors of incomplete understanding Uncertainty measure Probability measure Membership function Uncertain measure A nested set Uncertainty distribution Probability density function and (PDF) cumulative density function (CDF) Possibility distribution 99-table-

Uncertainty handling practices Probability theoryFuzzy theoryDerived uncertainty theory Info-gap theory EconomicsDecision tree in budget making --Credit risk analysis EngineeringRisk analysis: Event Tree analysis, Fault Tree analysis, probabilistic life cycle assessment Fuzzy life cycle assessment Reliability analysis, risk analysis - EcologyPopulation forecasting --Conservation management Information science Social networking Project scheduling-

Economic risk analysis

Dam risk analysis

Uncertainty processing in databases Probabilistic data management –Classic probabilistic data management –Monte-Carlo-based data management –Evidence-based data management Fuzzy data management

Challenges to uncertain data management Leveled uncertainty representation Domain-driven uncertainty management Leveraging user knowledge Crowdsourcing for uncertain data management

Conclusion