1. Predictive and Contextual Feature Separation for Bayesian Metanetworks Vagan Terziyan vagan@cc.jyu.fi Industrial Ontologies Group, University of Jyväskylä, Finland KES-2007, Vietri sul Mare, Italy 12 September 2007 Session IS03: Context-Aware Adaptable Systems and Their Applications (17:10, room D)

2. 2 Contents Bayesian Metanetworks Metanetworks for managing conditional dependencies Metanetworks for managing feature relevance Feature Separation for Bayesian Metanetworks Conclusions Vagan Terziyan Industrial Ontologies Group Department of Mathematical Information Technologies University of Jyvaskyla (Finland) http://www.cs.jyu.fi/ai/vagan This presentation: http://www.cs.jyu.fi/ai/KES-2007.ppthttp://www.cs.jyu.fi/ai/KES-2007.ppt

3. 3 Fixed conditional probability table: Conditional dependence between variables X and Y P(Y) =  X (P(X) · P(Y|X)) P(Y|X)y1y1 y2y2 …y m x1x1 p(x 1 | y 1 )p(x 1 | y 2 )p(x 1 | y m ) x2x2 p(x 2 | y 1 )p(x 2 | y 2 )p(x 2 | y m ) … x n p(x n | y 1 )p(x n | y 2 )… p(x n | y m ) Random variable Y {y 1, y 2, …, y m } Random variable X {x 1, x 2, …, x n }

4. 4 Bayesian Metanetwork Definition. The Bayesian Metanetwork is a set of Bayesian networks, which are put on each other in such a way that the elements (nodes or conditional dependencies) of every previous probabilistic network depend on the local probability distributions associated with the nodes of the next level network.

5. 5 Two-level Bayesian C-Metanetwork for Managing Conditional Dependencies

6. 6 Contextual Effect on Conditional Probability X x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 predictive attributes contextual attributes xkxk xrxr Assume conditional dependence between predictive attributes (causal relation between physical quantities)… xtxt … some contextual attribute may effect directly the conditional dependence between predictive attributes but not the attributes itself

7. 7 Contextual Effect on Conditional Probability (example) xkxk xrxr xtxt X k 1 : order flowers X k 2 : order wine X r 1 : visit football match X r 2 : visit girlfriend P 1 (X r |X k ) Xk1Xk1 Xk2Xk2 Xr1Xr1 0.30.9 Xr2Xr2 0.40.5 P 2 (X r |X k ) Xk1Xk1 Xk2Xk2 Xr1Xr1 0.10.2 Xr2Xr2 0.80.7 X t 1 : I am in Paris X t 2 : I am in Moscow X k : Order present X r : Make a visit

8. 8 Contextual Effect on Conditional Probability (example) xtxt P 1 (X r |X k ) Xk1Xk1 Xk2Xk2 Xr1Xr1 0.30.9 Xr2Xr2 0.40.5 P 2 (X r |X k ) Xk1Xk1 Xk2Xk2 Xr1Xr1 0.10.2 Xr2Xr2 0.80.7 X t 1 : I am in Paris X t 2 : I am in Moscow xrxr xkxk P( P (X r |X k ) | X t ) Xt1Xt1 Xt2Xt2 P 1 (X r |X k ) 0.70.2 P 2 (X r |X k ) 0.30.8

9. 9 Contextual Effect on Unconditional Probability X x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 predictive attributes contextual attributes xkxk Assume some predictive attribute is a random variable with appropriate probability distribution for its values… xtxt … some contextual attribute may effect directly the probability distribution of the predictive attribute x1x1 x2x2 x3x3 x4x4 X P(X)

10. 10 Contextual Effect on Unconditional Probability (example) xkxk xtxt XkXkXkXk P 1 (X k ) Xk1Xk1 Xk2Xk2 0.2 0.7 XkXkXkXk P 2 (X k ) Xk1Xk1 Xk2Xk2 0.5 0.3 X t 1 : I am in Paris X t 2 : I am in Moscow X k 1 : order flowers X k 2 : order wine X k : Order present P( P (X k ) | X t ) Xt1Xt1 Xt2Xt2 P 1 (X k ) 0.40.9 P 2 (X k ) 0.60.1

11. 11 Two-level Bayesian C-Metanetwork for managing conditional dependencies

12. 12 Two-level Bayesian R-Metanetwork for Modelling Relevant Features’ Selection

13. 13 Feature relevance modelling We consider relevance as a probability of importance of the variable to the inference of target attribute in the given context. In such definition relevance inherits all properties of a probability.

14. 14 General Case of Managing Relevance Probability P(XN)

15. 15 Example of Relevance Bayesian Metanetwork Conditional relevance !!!

16. 16 Example of Relevance Bayesian Metanetwork

17. 17 Separation of contextual and predictive attributes is based on: Part_of context Role-based context Interface-based context

18. 18 The nature of part_of context Machine Environment Sensors X x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 predictive attributes contextual attributes air pressure dust humidity temperature emission

19. 19 Context Description Framework (CDF) Basic Data Model Khriyenko O., Terziyan V., A Framework for Context-Sensitive Metadata Description, In: International Journal of Metadata, Semantics and Ontologies, Inderscience Publishers, ISSN 1744-2621, 2006, Vol. 1, No. 2, pp. 154-164.A Framework for Context-Sensitive Metadata DescriptionInternational Journal of Metadata, Semantics and Ontologies

20. 20 Part-of Context in CDF Resource k Resource i part_of Value_r Property_n Value_m Property_q Value_s Property_p Resource_k Property_nValue_r Resource_i Property_qValue_m Resource_i Property_pValue_s true_in_context Context_h RDF statement RDF container Predictive feature Contextual features

21. 21 Part-of Context example Kettle_1 Kitchen_1 part_of 89°C has_temperature 24°C has_temperature 16 sq/m has_size Kettle_1 has_temperature89°C Kitchen_1 has_temperature24°C Kitchen_1 has_size16 sq/m true_in_context Context_1 ResourcePredictive featuresContextual Features Kettle_1temperature environment environment_temperature environment environment_size

22. 22 Nested Part-of Context Resource_k Resource_i part_of Value_r Property_n Value_m Property_q Value_s Property_p Resource_f Value_x Property_a Value_y Property_b Value_z Property_c part_of Resource Predictive features Contextual Features Meta-Contextual Features Resource_kProperty_nProperty_qProperty_pProperty_aProperty_bProperty_c Resource_kProperty_n Value_r Resource_i Property_qValue_m Resource_i Property_pValue_s true_in_context Context_h true_in_context Resource_f Property_bValue_y Resource_f Property_cValue_z Context_g Resource_f Property_aValue_x

23. 23 Multiple Context Inheritance … John Golf_Club part_of 48 y. has_age Paris located_in 36 members_amount ResourcePredictive features Contextual Features (inherited from both parents) Johnage environment_1 environment_1_location environment_1 environment_1_members amount environment_2 environment_2_location environment_2 environment_2_belongs_to Symphonic_Orchestra Bagnolet located_in State belongs_to part_of

24. 24 Role-based context The example of the proactive object (human resource), which is member of several organization and which is playing different roles in each of them. The context of this object should include the description of these roles (duties, commitments, responsibilities, etc).

25. 25 Interface-based context The example of the domain object (aircraft) is shown in different interfaces: (a) Google Maps; (b) pilots’ control panel; (c) manufacturing design e-manual. Each interface is considered as a context, which affect on which parameters of the aircraft are to be shown a b c

26. 26 Summary We are considering a context as a set of contextual attributes, which are not directly effect probability distribution of the target attributes, but they effect on a “relevance” of the predictive attributes towards target attributes. Bayesian Metanetwork allows modelling such context- sensitive feature relevance. The model assumes that the relevance of predictive attributes in a Bayesian network might be a random attribute itself and it provides a tool to reason based not only on probabilities of predictive attributes but also on their relevancies. For Bayesian Metanetwork there is a need to distinguish predictive and contextual attributes and in this paper the separation of attributes is described based on three notions of a context: part_of context, role-based context and interface-based context.

27. 27 Read more about Bayesian Metanetworks in: Terziyan V., A Bayesian Metanetwork, In: International Journal on Artificial Intelligence Tools, Vol. 14, No. 3, 2005, World Scientific, pp. 371-384. http://www.cs.jyu.fi/ai/papers/KI-2003.pdf Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In: German Conference on Artificial Intelligence (KI-2003), LNAI, Vol. 2821, 2003, pp.370-384. http://www.cs.jyu.fi/ai/papers/IJAIT-2005.pdf Terziyan V., Vitko O., Learning Bayesian Metanetworks from Data with Multilevel Uncertainty, In: M. Bramer and V. Devedzic (eds.), Proceedings of the First International Conference on Artificial Intelligence and Innovations, Toulouse, France, August 22- 27, 2004, Kluwer Academic Publishers, pp. 187-196. http://www.cs.jyu.fi/ai/papers/AIAI-2004.ps