4/21/2017 Technisches Universität Wien Connecting User Interfaces and User Impairments for Semantically Optimized Information Flow in Hospital Information Systems Shuaib Karim1,2, A Min Tjoa1 1Institute of Software Technology and Interactive Systems Vienna University of Technology (http://www.ifs.tuwien.ac.at/) 2Computer Science Department Quaid-i-Azam University, Islamabad (http://www.qau.edu.pk/) September 05 / 2007

Need to connect diverse information domains 4/21/2017 Outline Need to connect diverse information domains Connecting Ontology approach Accessibility Framework Test Case (Connecting Ontology between User´s Impairments and User Interface Characteristics) Results Concluding Remarks

HIS (A mix of heterogeneous subsystems) 4/21/2017 Finance Food Section Labs. Pharmacy Operation Theatres Administration Clinical Support Medical Records Registration All subsystems are working to achieve a single goal. However, their individual goals are specific to their domain. Best Possible Patient Care

Sample Questions Requiring Connections 4/21/2017 Sample Questions Requiring Connections Disease ↔ Medicine Disease ↔ Diet Disease ↔ Clinical Procedure Disease ↔ Operative Procedure Procedure ↔ Resource Requirement Resource Requirement ↔ Cost User Interfaces ↔ User Profile Info. Semantics of Domain1 ↔ Info. Semantics of Domain2 These connections are often hard coded into the system, either in code or in data files. These are based upon some rules, which are normally not stored, because they exist as tacit knowledge. Encoded in software or in data → based upon some rules → ? Connection of heterogeneous domains =>

Significance of Connections (Ex. Food & Wine) 4/21/2017 Significance of Connections (Ex. Food & Wine) Suggests different foods and wine combinations → Domain expert´s knowledge encoded as OWL DL, such as; taste appetite or stimulation to eat digestion availability

Ex. Food & Wine Another Approach (“Connecting Ontology´´) 4/21/2017 Ex. Food & Wine Another Approach (“Connecting Ontology´´) Ontology of Food – Taxonomy with concepts such as; the calories with respect to the quantity the tendency to get digested on ist own (Quick, Medium duration, Long duration) … Ontology of Wine - Taxonomy with concepts such as; the appetizing effects (Nil, Low, Medium, High) the digestive effects (Nil, Low, Medium, High) Above knowledge normally available from Food and Wine producers

Food & Wine (new CQ) New CQ possible to fulfill: 4/21/2017 Food & Wine (new CQ) New CQ possible to fulfill: Food requiring no drinks Food requiring drinks of certain quality & brand Food & wine combination requiring least digestion time

O1 , O2 not related with same domains of discourse 4/21/2017 Characteristic Features of Connecting Ontology OC b / w two ontologies O1 and O2 O1 , O2 not related with same domains of discourse O1 , O2 developed using their own CODeP Similarity b / w CODeP indicate inter-connection Creation of new knowledge

Benefits of Connecting Ontology 4/21/2017 Benefits of Connecting Ontology Useful for top-down evolution of ontologies / applications Incompatibilities b / w the two ontologies are solved at the ontological level Helpful in code automation Possibility to determine the Cause - Effect relationship b / w the two ontologies Reliance on domain experts reduced due to encoding of domain knowledge The key is “How to capture and represent the domain knowledge ??´´

Connecting Ontology’s Domain Knowledge 4/21/2017 Connecting Ontology’s Domain Knowledge Available as; Structured documents Unstructured documents Tacit knowledge with domain experts Imaginative Or still imaginative.

Possible Line of Action 4/21/2017 Possible Line of Action Text processing of knowledge about domains of CO and the two participating ontologies Refinement of the above by aligning participating ontologies with global standard ontologies Exploiting valid queries on the two ontologies Exploiting queries result set and data mining Using Semantic Web Rules

Semantic Web – Architecture 4/21/2017 Semantic Web – Architecture

Generic Accessibility Pattern (CODeP) 4/21/2017 Generic Accessibility Pattern (CODeP)

Memory Recall Pattern (CODeP) 4/21/2017 Memory Recall Pattern (CODeP)

Perception Effect Pattern (CODeP) 4/21/2017 Perception Effect Pattern (CODeP)

Mobility Enhancement Pattern (CODeP) 4/21/2017 Mobility Enhancement Pattern (CODeP)

Persistence of Patterns 4/21/2017 Persistence of Patterns Formal description of semantics for each component Formal description of consequences and effects of potentially interacting component on each other

Visualization and Accessibility Plug-ins 4/21/2017 Visualization and Accessibility Plug-ins

Accessibility Plug-in 4/21/2017 Accessibility Plug-in A scalable framework using SemWeb Technology Stress on the development work in the framework. Sample ontologies for Impairments, Representation (UI) are developed. Then a classs (will be converted to Accessibility Service) is developed which manages the rules for connecting the Imapirment and UI ontologies, and generate a new connecting ontology. Next step is to pass this CO to stle sheet engine for automatically adapting the stylesheets according to the UI suggestions in the ImpUi connecting ontology. Next step is to make / use domain ontology from SLife and define rules to make connection between Domain ontology and Representation ontology. These rules will be managed by the Info-Viz Bridge Service.

User’s Impairments & UI Characteristics 4/21/2017 Test case Connecting Ontology for User’s Impairments & UI Characteristics

Ontology of Visualization Techniques 4/21/2017 Ontology of Visualization Techniques Ref: Chi E.H. A taxonomy of visualization techniques using the data state reference model. In INFOVIS, pages 69–75, 2000.

Ontology of Visualization Techniques 4/21/2017 Ontology of Visualization Techniques Ref: Chi E.H. A taxonomy of visualization techniques using the data state reference model. In INFOVIS, pages 69–75, 2000. UI / Visualization: An instance of UI is a Visualization. In above figure, Viz depends upon Information entity. Also, Information entity depends upon the task. So, Viz. Is also dependant upon the task in progress. We also say Viz. Is dependant upon User profile (specifically Impairments).

4/21/2017 Extension of ontology (Ref: “Hadzic M. and Chang E., Ontology-based Support for Human Disease Study, HICSS’05”) Human Disease Physiotherapy Psychotherapy Type Treatment Treatment Cause Type Symptom Drug Therapy Surgery Genetic Environmental Be careful while saying „Interface Adaptation“ is a Treatment. It is primarily Not. By our effort the working of user on computer and / or activities in general life will be performed more conveniently. But the Impairment will not be treated as such. Though it is partly true for cognitive impairments which is a further / side research agenda of our main research focus. So we should describe this figure as saying that our concept to incorporate Interface Adaptation for Impairment is analogous to Treatment for Human Disease in the given ontology by Hadzic M. Chemotherapy

Imapirment-User interface Connecting Ontology - Sample user scenarios 4/21/2017 Imapirment-User interface Connecting Ontology - Sample user scenarios Avoiding the confusing colors on an interface for a user with particular type of color blindness Font adjustments according to user‘s visual acuity Information presentation on the better part of the screen for a user suffering from Hemianopsia (absence of vision in half of visual field)

Impairment-User Interface Ontology (Ref: “Shuaib Karim and A Min Tjoa, Towards the Use of Ontologies for Improving User Interaction for People with Special Needs. ICCHP 2006: 77-84”) 4/21/2017

Impairment ontology - competency questions 4/21/2017 Impairment ontology - competency questions For a given impairment name; What is / are the related body parts ? What is the impaired side (right, left,...) ? What is its severity (on a predefined scale) ? What are the perception cues which are affected, and up to what degree (on a predefined scale) ? What is the effect of one impairment on another w.r.t. affected perception ?

Impairments Ontology 4/21/2017

Some derived concepts (1 / 2) 4/21/2017 Some derived concepts (1 / 2) LeftSidedImpairment RightSidedImpairment BothSidedImpairment

Derived concepts (2 / 2) AnySidedImpairment AnySidedImpairment 4/21/2017 Derived concepts (2 / 2) AnySidedImpairment AnySidedImpairment

User interface Ontology - competency questions 4/21/2017 User interface Ontology - competency questions Find the part-whole relationship of UI components Find the attributes of a component and their values (according to predefined usability scale for a normal user in normal conditions) For a given attribute, find the related UI components

User Interface Ontology 4/21/2017

Some derived concepts (1 / 2) 4/21/2017 Some derived concepts (1 / 2) GoodUsabilityComponent FairUsabilityComponent FairUserControlComponent

Rules to Connect Impairments and UI 4/21/2017 Rules to Connect Impairments and UI Low perception implies suggesting high usability components (VisualAcuityLow  uiLegibilityGood) (?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:Low) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Good)  (?x eg:suggests ?y).

Rules to Connect Impairments and UI 4/21/2017 Rules to Connect Impairments and UI High perception implies suggesting fair usability components (VisualAcuityHigh  uiLegibilityFair) (?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Fair)  (?x eg:suggests ?y).

Rules to Connect Impairments and UI 4/21/2017 Rules to Connect Impairments and UI High rheumatism implies suggesting easily operatable components (RheumatismHigh  uiUserControlGood) (?x rdf:type imp:Rheumatism) (?x imp:impairmentMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:userControl ui:Good)  (?x eg:suggests ?y).

4/21/2017 Generated RDF triples

4/21/2017

4/21/2017

4/21/2017

Consequences of Impairments-user Interface Connecting Ontology 4/21/2017 Consequences of Impairments-user Interface Connecting Ontology Helpful in automatically adapting UI for the user Helpful in deducing the best match of UI characteristics for a user with multiple impairments Possibility to use the ontology for diversity Historical data for studying the cause-effect relationship b/w the impairments and the computer interfaces Useful for rehabilitation purposes Possibility to extract impairment related semantics from user´s information stored in SemanticLIFE repository, and modify the impairments ontology accordingly Important slide. Make another slide for this. Speak more about it. Specify convincing examples/scenarios for each case.

4/21/2017 Concluding Remarks Integrating heterogeneous domains is always challenging The Connecting Ontology concept using the ontological rule-based approach paves the way for a generic solution Another abstraction level for persistence of rules The effort could be useful in HIS and other IMS Ontology for Impairments and Usability introduced The generic solution will ease in adaptation of accessibility by the industry within budget. It is one level above than UI from Haystack. They are generating semantic UI from infromation items. Our work is adaptation of this UI for people with special needs.

Future directions User testing 4/21/2017 Future directions User testing Instantiating the Impairments and UI ontologies Integrating capability measuring tools (MMSE) Ontology Elevation / Lifting Integration for ontology of visualization techniques

Thanks! Technisches Universität Wien 4/21/2017 Technisches Universität Wien Thanks! http://www.ifs.tuwien.ac.at/~skarim SemanticLIFE Project http://storm.ifs.tuwien.ac.at/ ASEA-UNINET(Asean European University Network) http://www.uibk.ac.at/asea-uninet/ HEC (Higher Education Commission of Pakistan) http://www.hec.gov.pk/