31. Effectiveness and efficiency of Dragon as an IRS Efficiency: Speed of retrieving items vs speed of retrieving relevant items –Speed of retrieving items: 10/10. The system responses very quickly about users searches. –Speed of retrieving relevant items: 7/10. Even the speed of retrieval is fast, sometimes the result may not relevant to what users want. Possible solution  reconstruct search terms, or refine the search options. Efficiency: Recall vs Precision –Recall: 7/10. If the search term contains broader sense/disambiguated terms especially in keyword search and title search, the recall rate can be very high. In this sense, users may have a huge brunch of result for them to choose. However those results may not relevant to what users want. –Precision: 9/10. When search strategies are matched with the search terms in database, the result of precision can be very high. E.g. Putting similar/exact keywords in Subject field. However the precision is high only when users can type in exactly the same term as keywords in the database. Overall result: 8/10

36. Rate the effectiveness and efficiency of Naxos as an IRS Efficiency: Speed of retrieving items vs speed of retrieving relevant items –Speed of retrieving items: 10/10. The system responses very quickly on users searches. –Speed of retrieving relevant items: 9/10. The speed of retrieval is fast, and the result is relevant to what users want. However, it may give no result to terms that are too broad, or not matched with the keywords in database. Possible solution: refine search terms. Efficiency: Recall vs Precision –Recall: 9/10. The recall rate is medium, but users are still able to select what they want there. –Precision: 10/10. The precision can be very high whenever users put the full and part of author or title name. Therefore, users are easy to find what they want in the results. Overall result: 9.5/10