1. UNED at iCLEF 2008: Analysis of a large log of multilingual image searches in Flickr Victor Peinado, Javier Artiles, Julio Gonzalo and Fernando López-Ostenero nlp.uned.es, Madrid CLEF workshop, 19 September 2008

2. iCLEF 2008 task Log analysis task. Data: logs from an online game (known item retrieval in Flickr) which implies searching in six languages simultaneously. Game interface implements reasonable baseline facilities for MLIA. Registered users with language profiles (native, active, passive, unknown languages)

3. Simultaneous search in six languages

4. Boolean search with translations

5. Relevance feedback

6. Assisted query translation

7. User profiles

8. User rank (Hall of Fame)

9. Group rank

10. Hint mechanism

11. 15 global session attributes

12. 84 search behaviour attributes

13. Harvested logs 109 topics 312 users / 41 teams 5101 complete search sessions Linguistics students, photography fans, IR researchers from industry and academia, monitored groups, other.

14. About users: procedence 312 users, 41 teams, 43 countries

15. About users: language skills

17. About user/image search session pairs The target image was annotated in a language that for the user was…

18. User’s behaviour and language skills Active Passive: ~ same success, more effort (~ +5%) Unknown: - 12% success, more effort (~ +10%)

19. You failed… what was the problem?

20. Overall… what were the challenges? Users learn that cross- linguality is a challenge!

21. Does learning improve success rate?

22. Cross-language interactive facilities: are they useful? At best, 1 out of 10 sessions manipulate translations provided by the system. So? Is that good or bad? Low or high?

23. Cross-language interactive facilities: are they useful? It’s good if compared with relevance feedback facilities!

24. Cross-Language Interactive facilities: are they perceived as useful?

25. Use of interactive facilities: Is there a learning effect?

26. Same with number of queries

27. Same with navigation beyond first page of results

28. Which interface facilities did you miss?

29. How were translations selected?

30. Conclusions Interactive CL features used more frequently than standard RF features. Occasional use of interactive features may have a big impact on the perceived usefulness of the interface! We finally have quantitatively relevant evidence on how much harder is to look for information in foreign (active/passive/unknown) languages… …And there is much space for improvement over baseline interface features!

31. Thank you!