1. Introduction to Grid Computing Felix Hageloh Roberto Valenti Deployment of a Language Detector Grid Service University of Amsterdam, 02-11-2005

2. Overview Introduction Introduction Required Steps Required Steps Our Service Our Service Introduction Introduction The basic idea The basic idea Use Case Use Case Interface Interface Implementation Implementation Problems Encountered Problems Encountered Future Work Future Work Conclusions Conclusions Questions Questions

3. Introduction Our chosen task: Grid Services Our chosen task: Grid Services Task Goals: Task Goals: Build a grid service. Build a grid service. Aggregate the service with another to provide additional, higher-level services Aggregate the service with another to provide additional, higher-level services

4. Steps Get access to the systems Get access to the systems Authentication Authentication Security issues Security issues Obtain User Certificate Obtain User Certificate Obtain Host Certificate Obtain Host Certificate Implement the service Implement the service Create required files Create required files WSDL WSDL QNames QNames WSDD WSDD JNDI JNDI Compile and create GAR file Compile and create GAR file As Globus user: As Globus user: Deploy service Deploy service Start container Start container

5. But you all know this… So… we jump to our service.

6. Our Service

7. Our Service: Introduction We were requested to implement a useful service which could be integrated on other services We were requested to implement a useful service which could be integrated on other services We are AI students so… We are AI students so… Let’s Merge AI and Grid Computing!!

8. Our Service: The Basic Idea Idea: Language Detection Is a necessary first step in a multitude of applications Idea: Language Detection Is a necessary first step in a multitude of applications Useful Web Service Examples: Useful Web Service Examples: Email filtering Email filtering Information retrieval Information retrieval Spell checkers Spell checkers Can also be component of an aggregated grid service Can also be component of an aggregated grid service

9. Our Service: The Basic Idea What about creating a Language Detector on the Grid? What about creating a Language Detector on the Grid? Training and Testing can be extremely time consuming running on a single machine Training and Testing can be extremely time consuming running on a single machine Data difficult to obtain -> can be shared on the Grid Data difficult to obtain -> can be shared on the Grid Duplicate data for parallel computing Duplicate data for parallel computing

10. Our Service: Use Case Simple Interface: Receives a piece of text Receives a piece of text Returns a string indicating the language Returns a string indicating the language

11. Our Service: Adding States Grid services can have states (as opposed to web services) Grid services can have states (as opposed to web services) Not necessary for our service but for the learning factor Not necessary for our service but for the learning factor Added “dummy” states to our service: Added “dummy” states to our service: Last Operation Last Operation Times Used Times Used

12. Our Service: Statefull Use Case

13. Our Service: Interface Requests and Responses Requests and Responses

14. Our Service: Interface Port Types Port Types

15. Our Service Implementation

16. Language Detection: Basic Idea Essentially based on probabilities of character combinations Essentially based on probabilities of character combinations Every language has typical character combinations that are very frequent in that language Every language has typical character combinations that are very frequent in that language “th” in english “th” in english “ij” in dutch “ij” in dutch Easy for humans to detect a language even when we don’t know that specific language Easy for humans to detect a language even when we don’t know that specific language

17. Language Learning: Standard Process Standard machine learning process Standard machine learning process

18. Language Learning: Markov Models Basic Markov Model Basic Markov Model k th order Markov Model k th order Markov Model

19. Language Detection: Classification Transitional probabilities estimated as Transitional probabilities estimated as Classification Classification

20. Language Detection: Example The training text for a language consists of the string The training text for a language consists of the string Learned model: Learned model: test text ( ^^, t, 1.0 ) ( ^t, e, 1.0 ) ( te, s, 0.5 ) ( es, t, 1.0 ) ( st,, 1.0 ) ( te, x, 0.5 ) ( ex, t, 1.0 ) ( xt,, 1.0 ) the probability of the string the probability of the string would be: test P(test|L) = P(t|^^)*P(e|^t)*P(s|te) *P(t|es)*P(_|st) = 1*1*0.5*1*1=0.5

21. Language Detection: Performance

22. Problems Encountered Necessary tools had to be installed (ANT) Necessary tools had to be installed (ANT) Problems on our machine (GRAM) Problems on our machine (GRAM) Conflicts with other team Conflicts with other team Buggy shell script to build gar file Buggy shell script to build gar file Sensitive to path lengths/ names Sensitive to path lengths/ names

23. Future Work Connect with other services Connect with other services Make training and evaluation a grid service Make training and evaluation a grid service Make it part of a multi lingual retrieval engine Make it part of a multi lingual retrieval engine Web interface (interactive) Web interface (interactive)

24. Conclusions Successfully managed to create and deploy our own web service Successfully managed to create and deploy our own web service Broke loose from the tutorial web service structure Broke loose from the tutorial web service structure Merged Grid Computing with AI Merged Grid Computing with AI Got hands on experience with Grid applications and structure Got hands on experience with Grid applications and structure A lot of possibilities to integrate and/or extend the implemented service A lot of possibilities to integrate and/or extend the implemented service

25. Questions ?