1. A Semiotic Information Quality Framework: Applications and Experiments Mr Gregory Hill, Prof. Graeme Shanks and Dr Rosanne Price Prof. Graeme Shanks and Dr Rosanne Price Clayton School of IT Monash University, Australia

2. Overview Research ContextResearch Context Theoretical BasisTheoretical Basis –Semiotic Framework –Ontological Model –Information Theory ExperimentsExperiments –Impact of Data Quality Tagging –Impact of Data Quality Treatment

3. Research Context Semiotic Framework proposedSemiotic Framework proposed –Shanks and Darke (1998) Further theoretical and empirical developmentFurther theoretical and empirical development –Shanks and Price (2002-2005) (assessment) –Hill (2004) (measurement)

4. Theoretical Basis Semiotics SemioticsSemiotics –Theory of signs and symbols –Philosophy, linguistics, information systems Understand signs at different levelsUnderstand signs at different levels –Syntactic (form) –Semantic (meaning) –Pragmatic (use)

5. Theoretical Basis Semiotics - cont’d Syntactic QualitySyntactic Quality –Conformance to meta-data Semantic QualitySemantic Quality –Correspondence to external world Pragmatic QualityPragmatic Quality –Stakeholder assessment Ratings (scores)Ratings (scores) Utility (prices)Utility (prices)

6. Theoretical Basis Ontological Model Proposed by Wand and Wang (1996)Proposed by Wand and Wang (1996) –Incompleteness –Ambiguity –Incorrectness (garbling) –Meaninglessness Measurement?Measurement? W X State Transitions RepresentationExternal World

7. Theoretical Basis Information Theory Proposed by Shannon and Weaver (1949)Proposed by Shannon and Weaver (1949) –Quantifies amount of information –Information is “uncertainty removed” Entropy: H(X) = – E[log p(x)] = -  p(x) log p(x)Entropy: H(X) = – E[log p(x)] = -  p(x) log p(x) Mutual Information: I(X;Y) = H(X) - H(X|Y)Mutual Information: I(X;Y) = H(X) - H(X|Y) Used in information economics, psychology, genetics, game theory, cryptography, coding … but not information systems?Used in information economics, psychology, genetics, game theory, cryptography, coding … but not information systems?

8. Theoretical Basis Model Comparison Syntactic Semantic Pragmatic Empirical Subjective Assessment - Service-based Ontological Model Subjective Assessment - Product-based Integrity Rules Economic Subjective Measurement - Utility Theory Ontological Model Objective Measurement - Information Theory Integrity Rules Semiotic Theory

9. Experiment I Impact of Data Quality Tagging Data quality tags for human decision-makingData quality tags for human decision-making Prior data quality tagging experimentsPrior data quality tagging experiments –Chengular-Smith et al (1999) –Shanks and Tansley (2002) –Fisher et al (2003) Form of data quality tagsForm of data quality tags –Single criterion –Objective normalised score

10. Experiment I Impact of Data Quality Tagging - cont’d Context-dependent tagsContext-dependent tags –Semantic level criteria –Organisational role and task –Administrative/geographic context Form of tagsForm of tags –Subjective (Likert Scale ratings) –Objective (for comparison)

11. Experiment I Impact of Data Quality Tagging - cont’d Dependent Variables Independent Variables Decision Strategy Task Complexity Data Quality Tagging Decision Complacency Decision Consensus Decision Efficiency Decision Confidence Decision Time Confidence Rating Selected Apartment Measures

12. Experiment II Impact of Data Quality Treatments Treatment of “dirty data” in CRM processesTreatment of “dirty data” in CRM processes Simulation of “real-world” scenariosSimulation of “real-world” scenarios –Treatments (via garbling) –Outcomes (via pay-offs) Discover antecedents of value-creationDiscover antecedents of value-creation –Scenario (process, pay-offs, customer attributes) –Data quality treatment

13. Experiment II Impact of Data Quality Treatments - cont’d Treatment Process Customer Attributes Outcome Noise Process Pay-offs Decision Process External World Information System

14. Experiment II Impact of Data Quality Treatments - cont’d Value model of CRM processesValue model of CRM processes –Hill (2004) SIFT metrics for planning and monitoringSIFT metrics for planning and monitoring –Stake (pragmatic) –Influence (pragmatic) –Fidelity (semantic) –Tweak (semantic)

15. Experiment II Impact of Data Quality Treatments - cont’d Organisational Impact Independent Variables Dependent Variables Construc t Measur e ScenarioDecision Process Treatment InfluenceStakeFidelityTweak Value

16. Questions research@greg-hill.id.au