1. Matching Conflicts: Functional Validation of Agents
George Cybenko and Guofei Jiang
Thayer School of Engineering
Dartmouth College
18th July,1999, AAAI’99
Orlando, Florida
Acknowledgements:
This work was partially supported by AFOSR grants F , NSF grant CCR and DARPA contract F
Well, Today, what I’m going to talk is about “matching conflicts : Functional validation of agents. “ The work was partially supported by AFOSR, NSF and
DARPA.
In most multiagent systems, A broker or matchmaker is used to identify and locate agent that can provide specific services. And basically These system use keywords or ontologies to specify the agent’s service.
However, we think keywords and ontologies can not be defined precisely enough to make the brokering or matchmaking among agents sufficiently
robust in a truly distributed, heterogeneous , multiagent computing environments.
So matching conflicts exists between client agents’ requested functionality and a server agent’s real functionality. So I’m going to talk about how we are trying to use a new form of interagent communication named “functional validation” to solve this problem.

2. Multiagent Grids
Several efforts are underway to build multiagent grids such as Globus, Infospheres, Sciagents and DARPA CoABS.
Basic structure:
User
Interface
Request
Client
Server
Service
Well, now around US, several efforts are under way to build multiagent “grids” ,
such as the Globus project at Argonne National Lab, the Infospheres project at Cal-tech, Sciagent project at Purdue univ. and the muri Darpa Coabs project.
All these projects are trying to provide basic software infrastructure for computations that integrate geographically disparate resources.
Though these projects have different and complicated structures, let’s just take a look on the most basic one:
1. Server here means the distributed, remote service provider. It can offer some specific services such as database, mathematical package , online simulation and
so on. It advertise its service catalog on a matchmaker or broker.
2. So the user(client) can use some language such as KQML to invoke a service location request. And this request is described by Keywords and ontologies.
3. Matchmaker or broker will use these keywords to query it’s service catalog
and return the possible server agent’s name to the client.
4. Then the client can negotiate with the remote server and ask for the service.
Service
Location
Request
Reply
Advertise
Matchmaker

3. Matching Conflicts
Brokers and matchmakers use keywords and domain ontologies to specify services.
Keywords and ontologies cannot be defined and interpreted precisely enough to make brokering or matchmaking between grid agents’ services robust in a truly distributed, heterogeneous computing environment.
Matching conflicts exist between client’s requested functionality and service provider’s actual functionality.
So just like what I say, …..
Here Ontologies specify a domain and key words specify functionality within that domain. E.g. Signal processing is a ontology , and FFT.is a keyword among this domain.

4. An Example
A client requires a three-dimensional FFT. A request is made to a broker or matchmaker for a FFT service based on the keywords and possibly parameter lists.
The broker or matchmaker uses the keywords to retrieve its catalog of services and returns with several candidate remote services.
The client must validate the actual functionality of these remote services before the client commits to use it.
Question: Do all parties involved in the computation agree on the actual agent functionality?
Let me give a example now. 1.
2. But these service candidates often has different implementation , different accuracy , different conditions , different computing speed, different database format and so on. 3.
4. The fundamental question here is that: …….
5. In order to answer this question, we think functional validation is needed.,

5. Functional Validation
Functional validation means that a client presents to a prospective service agent a sequence of challenges. The service agent replies to these challenges with corresponding answers. Only after the client is satisfied that the service agent’s answers are consistent with the client’s expectations is an actual commitment made to using the service.
Three steps:
Service agent identification and location.
Service agent functional validation.
Commitment to the service agent.
1. It is especially important in mission critical applications.
2. So now we think there are three steps during a service deal.
Let’s take a look.

6. Basically : Service identification and location.
Functional validation
Service commitment.

7. Our approach
Challenge the service agent with some test cases x1, x2, ..., xk . The remote service agent R provides the corresponding answers fR(x1), fR(x2), ..., fR(xk). The client C may or may not have independent access to the answers fC(x1), fC(x2), ..., fC(xk).
Some basic questions:
How large should k be?
What if fC(x) is not known by the client agent?
What if the service agent is fee based and so answers, fR(x), are not given away freely before commitment?
How to implement the validation process online?
Actually I think we can find the same idea everywhere in this commercial society.
E.g. You can download a lot of free demo software . By the demo, you can functionally validate whether the software is that you need and then make a decision whether to buy it.

8. Possible situations C = Client , R = Remote service
1. C “knows” fC(x) and R provides fR(x).
E.g. Simple test case(easy to be computed by C itself). Matrix inverse operation. Comparing fC(x) and fR(x).
PAC Learning theory.
2. C “knows” fC(x) and R does not provide fR(x).
E.g. Fee-based service. R offers hashed results g(fR(x)), where g is a secure hash function. E.g , y is a singular matrix. Comparing and
Zero-Knowledge proofs.
Traditional Software testing
Zero-knowledge proofs have been used to securely exchange information between corresponding parties without giving away any unnecessary knowledge.
E.g. the challenge here would be to “prove” that a service agent “knows” how to compute matrix inversion without giving away any information that could be useful to the client agent.

9. Possible situations
3. C does not “know” fC(x) and R provides fR(x).
E.g. Weather forecast; Some control systems.
E.g. C doesn’t know but can verify R’s service by
Simulation-based learning and reinforcement learning etc.
4. C does not “know” fC(x) and R does not provide fR(x).
Basically this situation can be reduced to a combination of the above two.
E.g. R offers hashed result , C can verify R’s service by
Some complicated example. Control system. Indirectly

10. Mathematical framework
The goal of PAC learning is to use few examples as possible, and as little computation as possible to pick a hypothesis concept which is a close approximation to the target concept.
Define a concept to be a boolean mapping X is the input space. c(x)=1 indicates x is a positive example , i.e. the service agent can offer the “correct” service for challenge x.
Define an index function
Now define the error between the target concept c and the hypothesis h as
C(x)=1 positive
C(x)=0 negative

11. Mathematical framework
The client agent can randomly pick m samples to PAC learn a hypothesis h about whether the service agent can offer the “correct” service .
Theorem(Blumer et.al.) Let H be any hypothesis space of finite VC dimension d contained in , P be any probability distribution on X and the target concept c be any Borel set contained in X. Then for any , given the following m independent random examples of c drawn according to P , with probability at least , every hypothesis in H that is consistent with all of these examples has error at most .
VC dimension is a combinational measure of concept class complexity which assigns to each concept class C a single number that chracterizes the sample size need to pac learn c.
Step: By determining VC dimension , select EBUXITONG AND Delta,
we can randomly pick m samples to learn a hypothesis about whether the service is “correct

12. Protocol Testbeds
public Client() { try {index=object.validation("actcomm.dartmouth.edu","1997","Plus","plus.data"); } catch(Exception e) {String err=e.toString(); System.out.println(err); } if(index==true) {System.out.println("Correct service"); } else {System.out.println("Incorrect service"); }
public class Validation { public boolean validation(String sname, String pname, String fname, String dname) { DataInputStream instream=null; String inline; Socket socket; PrintStream out; String Outline; ……}}
Well, let me talk something about what we are doing now. We are also doing some work to figure out the protocol of functional validations.
Using Java, we have built the multiagent protocol testbeds for functional validtaion.

13. Mobile Functional Validation Agent
Send
Create
Mobile
Agent
User
Interface
User
Agent
A’s Service
Correct
B’s Service
Correct
Jump
Incorrect
Incorrect
Interface
Agent
Interface
Agent
Also at the same we are integrating functional validation idea into our airforce muri active communication project. The user agent integrate some test cases
into the mobile agent and the mobile agents jump around the possible service providers and functional verify whether they can offer the correct service.
The advantage is :
1. More efficient. Functional validation is implemented in the server machine.
2. Reduce network traffic. Communication in the same machine. Especially
for a complicated service, user doesn’t need to keep the link for along time.
3. Autonomous. Mobile functional validation agents will jump around to verify and the user doesn’t need to interference.
Machine
C, D, E, ...
Computing
Server B
Computing
Server A
C, D, E, …..
Machine B
Machine A

14. Future work and open questions
PAC learning framework is very pessimistic, need a different approach.
Make agent functional validation a standard agent service.
Thanks!!
1.PAC learning framework is very pessimistic, now the test bounds is still too big for real application. So we need to find a different approach to formulize this problem.
2. We are trying to make functional validation as a standard agent service
and hope it can be integrated into some “grids” project, such as Globus.
3. Thanks. Any question??