PGP Project Viktor Yarmolenko Lewis Mackenzie Paul Cockshott Ewan Borland

 Background  Motivation  Proposed Solution  Status  Questions Contents

Background Setup

Background Algorithm View from 3 cameras

Background Algorithm X warp Y warp Correlation

Background Algorithm Reconstructed model viewed from a Virtual perspective position.

Background Highly parallelizable problem – the following can be done independently:  8 Matching jobs per group  N process groups per sequence  ~1min on Matching  ~1min on Building (2GHz) The problem

Motivation Dynamic data graphs S M T S M 1 1

Motivation S M T S M B Dynamic data graphs

Motivation S M T S M B M M Dynamic data graphs

Motivation S T S B M M Dynamic data graphs

Motivation The search for silver bullet Allows dynamic creation of processes (arbitrary code) Allows creation of communications channels between processes. Allows channels to be dynamically reconfigurable. 1.To develop a highly parallel system for 3D computer vision algorithms, 2.Which also can be a general framework for distributed processing. Our Aim: Primary Requirements:

Motivation The search for silver bullet  Use of GRID as a transport layer sounds good  The requirements could not be readily met by existing GRID protocols  No current API (MPI, RMI, PVM, DSM) can readily meet the requirements  We need a conceptually new parallel architecture

Solution The key concepts taken from Milner’s calculus are the ability to dynamically create processes and communication channels and to transmit communication channels along other channels. What is J  ? J  is a Java interface loosely modelled on the primitives of the  -calculus (Milner) to be used as a substratum for GRID based parallel computing.  "The Polyadic  -calculus, a tutorial“, Milner, R., (1991)  “A calculus of mobile processes”, Milner, R., Parrow, J., Walker, D., Information and Computation, 100:1-77, (1992) (Milner)

JPieTask – implements Runnable JPieFunnel – extends OutputStream JPieTap – extends InputStream JPiePipe – contains connected JPieTap and JPieFunnel J  primitives Solution 1000km

VM 21 1 J  example Solution

ILS model Initiator Locator Servent  Initiator which Starts job Locator – currently at EPCC Runs web services Maintains MySQL database of servents Servent runs JPie daemon Register With locator Query locator Reply with Available servents Any servent can also become an initiator during the course of a computation and spawn more tasks

ILS model Initiator Locator Servent Initiator Locator keeps track of available memory and performance of servents Servent Register performance and memory Queries locator For machines of Above performance Level X Reply with list of suitalble servents Java Grande Benchmark used To rate Servent performance Starts job

Status Need Your Thoughts  Performed network tests using the data demanding part of the algorithm. Data transfer is not a bottleneck in this problem.  Completed the multi processor implementation of J , using sockets.  Currently installing resource locator using Web services to find free cpus.  Would like to implement J  transport using a GRID layer. Acknowledgements: NeSC – Sponsors of the project EPCC – Discussions

Examples Conformed sequence

Examples Conformed mesh

Examples Landmark grid

Examples Textured

Examples Textured + mesh

Stream down the stream in pictures VM 21 1

Preliminary Tests Exp Total CPUs Local CPUs Remote CPUs Time (min) Comments Four local PCs local PCs (4 Dual + 4 Single, in the same room) remote PCs (16xCPU IBM machine 60 miles away) remote + 12 local remote + 12 local + 15 other PCs from the department  A part of algorithm was used  Total data transfer is over 5GB  Processed 12 sec of 3D video  That is 3600 images (12  25  4  3)  The bandwidth at it’s bottleneck 100Mbits  Virtually theoretical speedup  Can be improved by using J  Client 1 Thread 1 Host 1 HDD Thread 2 Thread 3 Thread n Client 4 Thread 1 HDD Thread 2 Thread 3 Thread n Host 2 Host 3 Host 4 Host 5 Host 6 Host 7 Host N