1. A Recommendation System for Software Function Discovery Naoki Ohsugi Software Engineering Laboratory, Graduate School of Information Science, Nara Institute of Science and Technology Tuesday 16 December, 2003. International Workshop on Community-Driven Evolution of Knowledge Artifacts

2. 2 of 14 Growth of Software Functions Application software is getting more complicated and providing more functions.  Total number of menu items (Microsoft Office)  Word 2000: 660  Word 2002: 772  Excel 2000: 705  Excel 2002: 792  PowerPoint 2000: 565  PowerPoint 2002: 646 Screenshot of MS-Word 2002 Users can’t find useful functions from too many functions.

3. 3 of 14 Users Could Not Find Some Useful Functions! Subjects: 32 users in our lab. Period: 22 months Total Number of Different Functions Maximum Number of Functions Used Minimum Number of Functions Used Average Number of Functions Used Excel2000Excel2002PPT2000PPT2002Word2000Word2002 Number of Functions 705 792 565 646 660 772 75 83 189 147 143 120 10 12 18 31 22 11 38 26 80 67 66 32 0 100 200 300 400 500 600 700 800 900 10.6%10.5%33.5%22.8%21.7%15.5%1.4%1.5%3.2%4.8%3.3%1.4%5.4%3.3%14.2%10.4%10.0%4.1%

4. 4 of 14 A Recommendation System for Software Function Discovery The system recommends individual users a set of candidate functions, which may be useful. Our solution is a Collaborative Filtering approach. Here’s my recommendation: Tools  Word Count…21 pts Insert  Date Time…20 pts Tools  Thesaurus…18 pts Insert  Footnote…18 pts Tools  Spelling…17 pts

5. 5 of 14 What is Collaborative Filtering (CF)? “Collaborative” means using some users’ knowledge for filtering. “Filtering” means selecting useful items from large amount of items. Using some users’ knowledge F K A B DE C G IJ H L NO M P Q ST R Large amount of items F is good!K is cool! ?? Selecting useful items F F K K

6. 6 of 14 Voting-based Recommendation Systems with CF The systems collect explicit votes as users’ knowledge. Amazon.com (Book recommendation system) http://www.amazon.com MovieLens (Movie recommendation system) http://www.movielens.umn.edu

7. 7 of 14 Logging Usage as Users’ Knowledge The proposed system automatically collects the records of executed functions (Usage logs) as users' knowledge. Usage logs are collected from some users via the Internet. Server of the System The Internet User Application Software e.g. MS-Word, Excel Usage log as shown below: 2002/02/03 18:50:41 Formatting->Font… 2002/02/03 18:50:45 File->Save As… Log Collector VBA Plug-In

8. 8 of 14 Function A Function B Function C Function D Function A Function B Function C Function D Dissimilar usersSimilar users Step1: Computing Similarities Computing similarities between the target user and the other users Function H Function I Function J Function K Function E Function F Function G Target userUser 1User 2User 3User 4 Function A Function B Function C Function A Function C Function A Function B Function C Function D Function A Function B Function C Function D Function A Function C

9. 9 of 14 Dissimilar usersSimilar users Step 2: Delivering Knowledge Delivering the useful functions candidate, which were frequently used by the similar users'. Function A Function B Function C Function D Function A Function B Function C Function D Function H Function I Function J Function K Function A Function B Function C Function E Function F Function G Target user Function A Function B Function C Function D Function A Function B Function C Function D Function A Function B Function C Function D User 1User 2User 3User 4 Function B Function D

10. 10 of 14 12345671234567 User 3 Undo Save Clear Cut Copy Paste Redo 60% 20% 6% 5% 4% 3% 2% 12345671234567 User 2 Save Undo Redo Copy Paste Cut Clear 55% 25% 10% 4% 3% 2% 1% Correlation based similarity +0.41+0.97 (Range of value [-1.00, +1.00]) Calculating Similarities by Correlation Coefficient  The dominant frequencies (e.g., “Undo” or “Save”) over-affect similarity computations. Conventional Similarity Calculation 12345671234567 Target user Undo Save Redo Copy Paste Cut Clear 60% 20% 10% 4% 3% 2% 1%

11. 11 of 14 Rank correlation based similarity+0.90+0.05 Better Similarity Calculation Calculating Similarities by Rank Correlation  The dominant frequencies ("Undo" & "Save") do not affect similarity computations. Correlation based similarity +0.41+0.97 12345671234567 User 3 Undo Save Clear Cut Copy Paste Redo 60% 20% 6% 5% 4% 3% 2% 12345671234567 User 2 Save Undo Redo Copy Paste Cut Clear 55% 25% 10% 4% 3% 2% 1% 12345671234567 Target user Undo Save Redo Copy Paste Cut Clear 60% 20% 10% 4% 3% 2% 1%

12. 12 of 14 Ndpm [0.0, 1.0] 0.0 is the best 1.0 is the worst Comparison Evaluating Accuracy of Recommendation Yao’s ndpm measure  * Y.Y. Yao, “Measuring Retrieval Effectiveness Based on User Preference of Documents”, J. of American Society for Information Science, 46, 2, 1995, pp.133-145. User’s Ideal Recommendation Interview for user 1. Function A 2. Function B 3. Function C 4. Function D System System’s Recommendation 1. Function A 2. Function B 3. Function C 4. Function D Usage logs 6 users 22 months

13. 13 of 14 Experimental Result 0.514 0.404 0.396 0.383 0.355 0.2 0.3 0.4 0.5 0.6 Ndpm Each user’s ndpm Average of ndpm 0.5 of ndpm User Count RandomBase Case Correlation based Similarity Rank Correlation based Similarity Algorithms Collected usage logs of Ms-Word 2000 Subjects: 6 users in our lab. Period: 22 months

14. 14 of 14 Conclusion I proposed a recommendation system to help users discover useful functions. I evaluated the accuracy of recommendation.  The result suggested the proposed system has a potential to provide useful recommendation for software function discovery.

15. 15 of 14 E

16. 16 of 14 N

17. 17 of 14 D