1. 4/21/2017
Technisches Universität Wien
Connecting User Interfaces and User Impairments for Semantically Optimized Information Flow in Hospital Information Systems
Shuaib Karim1,2, A Min Tjoa1
1Institute of Software Technology and Interactive Systems
Vienna University of Technology (
2Computer Science Department
Quaid-i-Azam University, Islamabad (
September 05 / 2007

2. Need to connect diverse information domains
4/21/2017
Outline
Need to connect diverse information domains
Connecting Ontology approach
Accessibility Framework
Test Case (Connecting Ontology between User´s Impairments and User Interface Characteristics)
Results
Concluding Remarks

3. HIS (A mix of heterogeneous subsystems)
4/21/2017
Finance
Food Section
Labs.
Pharmacy
Operation Theatres
Administration
Clinical Support
Medical Records
Registration
All subsystems are working to achieve a single goal. However, their individual goals are specific to their domain.
Best Possible Patient Care

4. Sample Questions Requiring Connections
4/21/2017
Sample Questions Requiring Connections
Disease ↔ Medicine
Disease ↔ Diet
Disease ↔ Clinical Procedure
Disease ↔ Operative Procedure
Procedure ↔ Resource Requirement
Resource Requirement ↔ Cost
User Interfaces ↔ User Profile
Info. Semantics of Domain1 ↔ Info. Semantics of Domain2
These connections are often hard coded into the system, either in code or in data files. These are based upon some rules, which are normally not stored, because they exist as tacit knowledge.
Encoded in software or in data →
based upon some rules → ?
Connection of heterogeneous domains =>

5. Significance of Connections (Ex. Food & Wine)
4/21/2017
Significance of Connections (Ex. Food & Wine)
Suggests different foods and wine combinations
→ Domain expert´s knowledge encoded as OWL DL, such as;
taste
appetite or stimulation to eat
digestion
availability

6. Ex. Food & Wine Another Approach (“Connecting Ontology´´)
4/21/2017
Ex. Food & Wine
Another Approach (“Connecting Ontology´´)
Ontology of Food – Taxonomy with concepts such as;
the calories with respect to the quantity
the tendency to get digested on ist own (Quick, Medium duration, Long duration) …
Ontology of Wine - Taxonomy with concepts such as;
the appetizing effects (Nil, Low, Medium, High)
the digestive effects (Nil, Low, Medium, High)
Above knowledge normally available from Food and Wine producers

7. Food & Wine (new CQ) New CQ possible to fulfill:
4/21/2017
Food & Wine (new CQ)
New CQ possible to fulfill:
Food requiring no drinks
Food requiring drinks of certain quality & brand
Food & wine combination requiring least digestion time

8. O1 , O2 not related with same domains of discourse
4/21/2017
Characteristic Features of Connecting Ontology OC b / w two ontologies O1 and O2
O1 , O2 not related with same domains of discourse
O1 , O2 developed using their own CODeP
Similarity b / w CODeP indicate inter-connection
Creation of new knowledge

9. Benefits of Connecting Ontology
4/21/2017
Benefits of Connecting Ontology
Useful for top-down evolution of ontologies / applications
Incompatibilities b / w the two ontologies are solved at the ontological level
Helpful in code automation
Possibility to determine the Cause - Effect relationship b / w the two ontologies
Reliance on domain experts reduced due to encoding of domain knowledge
The key is “How to capture and represent the domain knowledge ??´´

10. Connecting Ontology’s Domain Knowledge
4/21/2017
Connecting Ontology’s Domain Knowledge
Available as;
Structured documents
Unstructured documents
Tacit knowledge with domain experts
Imaginative
Or still imaginative.

11. Possible Line of Action
4/21/2017
Possible Line of Action
Text processing of knowledge about domains of CO and the two participating ontologies
Refinement of the above by aligning participating ontologies with global standard ontologies
Exploiting valid queries on the two ontologies
Exploiting queries result set and data mining
Using Semantic Web Rules

12. Semantic Web – Architecture
4/21/2017
Semantic Web – Architecture

13. Generic Accessibility Pattern (CODeP)
4/21/2017
Generic Accessibility Pattern (CODeP)

14. Memory Recall Pattern (CODeP)
4/21/2017
Memory Recall Pattern (CODeP)

15. Perception Effect Pattern (CODeP)
4/21/2017
Perception Effect Pattern (CODeP)

16. Mobility Enhancement Pattern (CODeP)
4/21/2017
Mobility Enhancement Pattern (CODeP)

17. Persistence of Patterns
4/21/2017
Persistence of Patterns
Formal description of semantics for each component
Formal description of consequences and effects of potentially interacting component on each other

18. Visualization and Accessibility Plug-ins
4/21/2017
Visualization and Accessibility Plug-ins

19. Accessibility Plug-in
4/21/2017
Accessibility Plug-in
A scalable framework using SemWeb Technology
Stress on the development work in the framework. Sample ontologies for Impairments, Representation (UI) are developed. Then a classs (will be converted to Accessibility Service) is developed which manages the rules for connecting the Imapirment and UI ontologies, and generate a new connecting ontology. Next step is to pass this CO to stle sheet engine for automatically adapting the stylesheets according to the UI suggestions in the ImpUi connecting ontology. Next step is to make / use domain ontology from SLife and define rules to make connection between Domain ontology and Representation ontology. These rules will be managed by the Info-Viz Bridge Service.

20. User’s Impairments & UI Characteristics
4/21/2017
Test case
Connecting Ontology for
User’s Impairments & UI Characteristics

21. Ontology of Visualization Techniques
4/21/2017
Ontology of Visualization Techniques
Ref: Chi E.H. A taxonomy of visualization techniques using the data state reference model. In INFOVIS, pages 69–75, 2000.

22. Ontology of Visualization Techniques
4/21/2017
Ontology of Visualization Techniques
Ref: Chi E.H. A taxonomy of visualization techniques using the data state reference model. In INFOVIS, pages 69–75, 2000.
UI / Visualization: An instance of UI is a Visualization. In above figure, Viz depends upon Information entity. Also, Information entity depends upon the task. So, Viz. Is also dependant upon the task in progress. We also say Viz. Is dependant upon User profile (specifically Impairments).

23. 4/21/2017
Extension of ontology (Ref: “Hadzic M. and Chang E., Ontology-based Support for Human Disease Study, HICSS’05”)
Human Disease
Physiotherapy
Psychotherapy
Type
Treatment
Treatment
Cause
Type
Symptom
Drug Therapy
Surgery
Genetic
Environmental
Be careful while saying „Interface Adaptation“ is a Treatment. It is primarily Not. By our effort the working of user on computer and / or activities in general life will be performed more conveniently. But the Impairment will not be treated as such. Though it is partly true for cognitive impairments which is a further / side research agenda of our main research focus. So we should describe this figure as saying that our concept to incorporate Interface Adaptation for Impairment is analogous to Treatment for Human Disease in the given ontology by Hadzic M.
Chemotherapy

24. Imapirment-User interface Connecting Ontology - Sample user scenarios
4/21/2017
Imapirment-User interface Connecting Ontology - Sample user scenarios
Avoiding the confusing colors on an interface for a user with particular type of color blindness
Font adjustments according to user‘s visual acuity
Information presentation on the better part of the screen for a user suffering from Hemianopsia (absence of vision in half of visual field)

25. Impairment-User Interface Ontology (Ref: “Shuaib Karim and A Min Tjoa, Towards the Use of Ontologies for Improving User Interaction for People with Special Needs. ICCHP 2006: 77-84”)
4/21/2017

26. Impairment ontology - competency questions
4/21/2017
Impairment ontology - competency questions
For a given impairment name;
What is / are the related body parts ?
What is the impaired side (right, left,...) ?
What is its severity (on a predefined scale) ?
What are the perception cues which are affected, and up to what degree (on a predefined scale) ?
What is the effect of one impairment on another w.r.t. affected perception ?

27. Impairments Ontology
4/21/2017

28. Some derived concepts (1 / 2)
4/21/2017
Some derived concepts (1 / 2)
LeftSidedImpairment
RightSidedImpairment
BothSidedImpairment

29. Derived concepts (2 / 2) AnySidedImpairment AnySidedImpairment
4/21/2017
Derived concepts (2 / 2)
AnySidedImpairment
AnySidedImpairment

30. User interface Ontology - competency questions
4/21/2017
User interface Ontology - competency questions
Find the part-whole relationship of UI components
Find the attributes of a component and their values (according to predefined usability scale for a normal user in normal conditions)
For a given attribute, find the related UI components

31. User Interface Ontology
4/21/2017

32. Some derived concepts (1 / 2)
4/21/2017
Some derived concepts (1 / 2)
GoodUsabilityComponent
FairUsabilityComponent
FairUserControlComponent

33. Rules to Connect Impairments and UI
4/21/2017
Rules to Connect Impairments and UI
Low perception implies suggesting high usability components (VisualAcuityLow  uiLegibilityGood)
(?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:Low) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Good)
 (?x eg:suggests ?y).

34. Rules to Connect Impairments and UI
4/21/2017
Rules to Connect Impairments and UI
High perception implies suggesting fair usability components (VisualAcuityHigh  uiLegibilityFair)
(?x rdf:type imp:VisualAcuity) (?x imp:perceptionMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:hasLegibility ui:Fair)
 (?x eg:suggests ?y).

35. Rules to Connect Impairments and UI
4/21/2017
Rules to Connect Impairments and UI
High rheumatism implies suggesting easily operatable components (RheumatismHigh  uiUserControlGood)
(?x rdf:type imp:Rheumatism) (?x imp:impairmentMeasure imp:High) (?y rdf:type ui:UiComponent) (?y ui:userControl ui:Good)
 (?x eg:suggests ?y).

36. 4/21/2017
Generated RDF triples

37. 4/21/2017

38. 4/21/2017

39. 4/21/2017

40. Consequences of Impairments-user Interface Connecting Ontology
4/21/2017
Consequences of Impairments-user Interface Connecting Ontology
Helpful in automatically adapting UI for the user
Helpful in deducing the best match of UI characteristics for a user with multiple impairments
Possibility to use the ontology for diversity
Historical data for studying the cause-effect relationship b/w the impairments and the computer interfaces
Useful for rehabilitation purposes
Possibility to extract impairment related semantics from user´s information stored in SemanticLIFE repository, and modify the impairments ontology accordingly
Important slide. Make another slide for this. Speak more about it. Specify convincing examples/scenarios for each case.

41. 4/21/2017
Concluding Remarks
Integrating heterogeneous domains is always challenging
The Connecting Ontology concept using the ontological rule-based approach paves the way for a generic solution
Another abstraction level for persistence of rules
The effort could be useful in HIS and other IMS
Ontology for Impairments and Usability introduced
The generic solution will ease in adaptation of accessibility by the industry within budget. It is one level above than UI from Haystack. They are generating semantic UI from infromation items. Our work is adaptation of this UI for people with special needs.

42. Future directions User testing
4/21/2017
Future directions
User testing
Instantiating the Impairments and UI ontologies
Integrating capability measuring tools (MMSE)
Ontology Elevation / Lifting
Integration for ontology of visualization techniques

43. Thanks! Technisches Universität Wien
4/21/2017
Technisches Universität Wien
Thanks!
SemanticLIFE Project
ASEA-UNINET(Asean European University Network)
HEC (Higher Education Commission of Pakistan)