1. Copyright © 2005, SAS Institute Inc. All rights reserved. Cleaning up your Customer’s Dirty Language for Text Mining James A. Cox, Ph.D. Software Development Manager, SAS Text Miner Enterprise Miner Development

2. Copyright © 2005, SAS Institute Inc. All rights reserved. 2 Text Mining your customers: A window onto their world

3. Copyright © 2005, SAS Institute Inc. All rights reserved. 3 But what if your window is not clean?

4. Copyright © 2005, SAS Institute Inc. All rights reserved. 4 Out of the mouth of … customers *!@%$*!@%$ From: Joe Shmo To: whoever is interested I LUV you’re new prodict!keep makeing work beter! --- No man is an island (by whoever it was that said that)

5. Copyright © 2005, SAS Institute Inc. All rights reserved. 5 Text Parsing money supplyNoun GroupKeep +Clinton (Bill Clinton,…)PersonKeep +Clinton (Clinton, NJ,…)CityKeep +bill (bills, billed)VerbKeep +bill (bills)NounKeep regardless ofPrepDrop

6. Copyright © 2005, SAS Institute Inc. All rights reserved. 6 An organized view of your customers  Email, call centers, web surveys  Accessing info, merging it together  Eliminating run-on punctuation  Fixing misspellings  Discovering common themes using Text Miner!

7. Copyright © 2005, SAS Institute Inc. All rights reserved. 7 An example using Cell Phone data.  Access complaints from www.complaints.com  Eliminate extraneous text  Eliminate duplicates  Fix misspellings, abbreviations using %textsyn()  Analyze resulting data with SAS Text Miner  Compare to “uncleaned” text

8. Copyright © 2005, SAS Institute Inc. All rights reserved. 8

9. 9 Google interface

10. Copyright © 2005, SAS Institute Inc. All rights reserved. 10 KNOW what your customers are telling you  Competitive advantage!!!  If you don’t, your competitors might beat you to it.

11. Copyright © 2005, SAS Institute Inc. All rights reserved. 11 Macros  %textsyn and %puncexpand (renamed %tmpunct) will be available in EA release this month  These, along with other tips and techniques, and the google interface can be downloaded from ftp://ftp.sas.com/techsup/download/EMiner/M2004cd/

12. Copyright © 2005, SAS Institute Inc. All rights reserved. 12 Text Mining at SUGI 30  Terry Woodfield, Patricia Cerrito, John Wallace -- - whoops  BOF session, Room 111 at 7 PM tomorrow night  Manya Mayes --- predicting titles, tomorrow 3:30, Room 112

13. Copyright © 2005, SAS Institute Inc. All rights reserved. 13 Copyright © 2005, SAS Institute Inc. All rights reserved. 13