Introduction to Grid Computing Felix Hageloh Roberto Valenti Deployment of a Language Detector Grid Service University of Amsterdam,

Overview Introduction Introduction Required Steps Required Steps Our Service Our Service Introduction Introduction The basic idea The basic idea Use Case Use Case Interface Interface Implementation Implementation Problems Encountered Problems Encountered Future Work Future Work Conclusions Conclusions Questions Questions

Introduction Our chosen task: Grid Services Our chosen task: Grid Services Task Goals: Task Goals: Build a grid service. Build a grid service. Aggregate the service with another to provide additional, higher-level services Aggregate the service with another to provide additional, higher-level services

Steps Get access to the systems Get access to the systems Authentication Authentication Security issues Security issues Obtain User Certificate Obtain User Certificate Obtain Host Certificate Obtain Host Certificate Implement the service Implement the service Create required files Create required files WSDL WSDL QNames QNames WSDD WSDD JNDI JNDI Compile and create GAR file Compile and create GAR file As Globus user: As Globus user: Deploy service Deploy service Start container Start container

But you all know this… So… we jump to our service.

Our Service

Our Service: Introduction We were requested to implement a useful service which could be integrated on other services We were requested to implement a useful service which could be integrated on other services We are AI students so… We are AI students so… Let’s Merge AI and Grid Computing!!

Our Service: The Basic Idea Idea: Language Detection Is a necessary first step in a multitude of applications Idea: Language Detection Is a necessary first step in a multitude of applications Useful Web Service Examples: Useful Web Service Examples: filtering filtering Information retrieval Information retrieval Spell checkers Spell checkers Can also be component of an aggregated grid service Can also be component of an aggregated grid service

Our Service: The Basic Idea What about creating a Language Detector on the Grid? What about creating a Language Detector on the Grid? Training and Testing can be extremely time consuming running on a single machine Training and Testing can be extremely time consuming running on a single machine Data difficult to obtain -> can be shared on the Grid Data difficult to obtain -> can be shared on the Grid Duplicate data for parallel computing Duplicate data for parallel computing

Our Service: Use Case Simple Interface: Receives a piece of text Receives a piece of text Returns a string indicating the language Returns a string indicating the language

Our Service: Adding States Grid services can have states (as opposed to web services) Grid services can have states (as opposed to web services) Not necessary for our service but for the learning factor Not necessary for our service but for the learning factor Added “dummy” states to our service: Added “dummy” states to our service: Last Operation Last Operation Times Used Times Used

Our Service: Statefull Use Case

Our Service: Interface Requests and Responses Requests and Responses

Our Service: Interface Port Types Port Types

Our Service Implementation

Language Detection: Basic Idea Essentially based on probabilities of character combinations Essentially based on probabilities of character combinations Every language has typical character combinations that are very frequent in that language Every language has typical character combinations that are very frequent in that language “th” in english “th” in english “ij” in dutch “ij” in dutch Easy for humans to detect a language even when we don’t know that specific language Easy for humans to detect a language even when we don’t know that specific language

Language Learning: Standard Process Standard machine learning process Standard machine learning process

Language Learning: Markov Models Basic Markov Model Basic Markov Model k th order Markov Model k th order Markov Model

Language Detection: Classification Transitional probabilities estimated as Transitional probabilities estimated as Classification Classification

Language Detection: Example The training text for a language consists of the string The training text for a language consists of the string Learned model: Learned model: test text ( ^^, t, 1.0 ) ( ^t, e, 1.0 ) ( te, s, 0.5 ) ( es, t, 1.0 ) ( st,, 1.0 ) ( te, x, 0.5 ) ( ex, t, 1.0 ) ( xt,, 1.0 ) the probability of the string the probability of the string would be: test P(test|L) = P(t|^^)*P(e|^t)*P(s|te) *P(t|es)*P(_|st) = 1*1*0.5*1*1=0.5

Language Detection: Performance

Problems Encountered Necessary tools had to be installed (ANT) Necessary tools had to be installed (ANT) Problems on our machine (GRAM) Problems on our machine (GRAM) Conflicts with other team Conflicts with other team Buggy shell script to build gar file Buggy shell script to build gar file Sensitive to path lengths/ names Sensitive to path lengths/ names

Future Work Connect with other services Connect with other services Make training and evaluation a grid service Make training and evaluation a grid service Make it part of a multi lingual retrieval engine Make it part of a multi lingual retrieval engine Web interface (interactive) Web interface (interactive)

Conclusions Successfully managed to create and deploy our own web service Successfully managed to create and deploy our own web service Broke loose from the tutorial web service structure Broke loose from the tutorial web service structure Merged Grid Computing with AI Merged Grid Computing with AI Got hands on experience with Grid applications and structure Got hands on experience with Grid applications and structure A lot of possibilities to integrate and/or extend the implemented service A lot of possibilities to integrate and/or extend the implemented service

Questions ?