1. Formalising and Evaluating Cultural User Experience
Semantic Media Adaptation and Personalization
initiative Workshop - SMAP
Special Session on Personalized Delivery of Cultural Heritage Content
9-10th July 2017, Bratislava, Slovakia
Formalising and Evaluating
Cultural User Experience
Authors: Markos Konstantakis, Konstantinos Michalakis,
John Aliprantis, Eirini Kalatha, George Caridakis

2. Intelligent Interaction Research Group Department of Cultural Technology and Communication University of the Aegean, Greece

3. Concept Background
Human
Computer Interaction
Digital Culture
User Experience
User Experience (UX) is a core concept of Human Computer Interaction (HCI).
Basic interest: augment user’s participation during his interaction with cultural objects by making him actor of his own cultural experience.

4. Challenges
Little attention has been paid to the evaluation of UX with technologies in cultural heritage environments.
UX frameworks have not been tested with visitors in realistic situations of cultural visits.
UX research in cultural services is limited only to understanding user needs and expectations through the use of limited use cases supporting limited interaction.

5. Related work (1/3)
2001: Ciolfi develop two scales to measure visitor’s museum exhibition and compare the traditional and conventional ways of interaction with the use of multimedia guides.
After questioning a large number of visitors, they produce 4 components to analyze their data and evaluate user’s cultural experience (engagement, meaningful experience, knowledge/learning and emotional connection) and in all of them, multimedia or audio guides had better results.

6. Related work (2/3)
2010: Walsh introduce the evaluation of cultural experience is a result of many different tools such as personal interviews, tests of knowledge and memory of the cultural artifacts, and video cameras which monitor the point of view of users.
2011: Othman emphasize that cultural background of the participants in evaluation is playing a critical role not only on how to perceive a question but also on the expression and the style of their answers, by performing a remote online UX evaluation across ten countries.

7. Related work (3/3) CHESS PROJECT
2014: CHESS aims at enriching the museum visit through personalized interactive storytelling experiences. CHESS implements a different method of evaluation by building and updating visitor’s profiles during their interaction with cultural artifacts multimedia guides, and adjusts accordingly the rest of the storytelling.

8. What is ux?
“​UX is a consequence of user’s internal state, the characteristics of designed system, and the context (or environment) within which the interaction occurs”. (Hassenzahl & Tractinsky ) .
Peter Morville’s UX honeycomb

9. From ux to cux
The interaction between different cultural objects and user’s cultural backgrounds defines the cultural user experience (CUX) as “The unique produced knowledge and experience from different cultural identities”.

10. Characteristics of a positive ux
Metrics discovered by Hassenzahl as criteria for evaluating UX.
Described under 2 dimensions of UX: pragmatic and hedonic.

11. Ux evaluation methods
A wide list of UX evaluation methods can be found in the literature:
15 UX evaluation methods from Jordan’s set of methods.
17 methods from the ENGAGE pool of methods.
8 methods from the HUMAINE set.
We identified 10 new methods found in the ACM Digital Library.
Assessing all those methods, we identified 15 common evaluation methods that are suited to cultural studies, as shown in the next Figure.

12. research methodology - steps

13. CUx evaluation
methods rating

14. Evaluation results - Positive cux windmill

15. Conclusion
We consider our CUX evaluation study a necessary step towards understanding user expectations.
We have extended the already existing work by formalizing and analyzing different UX evaluation methods.
It is imperative to understand the design implications on the user experiences at the early phases of the conceptual design and prototyping.

16. Future work
Understanding of what is the added value of Culture in the UX. This requires controlled experimental research setup with functional cultural services.
In the future, we will use this analysis as basis in designing new evaluation metrics. This will allow us to examine to what degree the expectations have been met and how significant the expected UX characteristics actually have become in cultural applications.

17. Contact Details Markos Konstantakis mkonstadakis@aegean.gr
University of the Aegean
Department of Cultural
Technology & Communication
Thank you for your attention!