EGEE is a project funded by the European Union under contract IST Outstanding design issues Stephen Hicks 23/06/04

23/06/ Contents API PrimaryProducers Streaming Synchronization Security Other issues

23/06/ API undeclareTable semantics  should data remain available for retention period? Requires extra threads API is still open so user could just call undeclareTable later! dropTable  how to propagate schema change to consumers & producers periodic calls to Schema to check status of table notification by Schema/Registry when a table is dropped Security  Provide list of VOs when creating Consumer, Producer etc.  All subsequent commands executed in context of those VOs

23/06/ PrimaryProducers: Retention periods etc. Either two types:  history  history & latest Or just one: history & latest  Simpler for users  Simpler for mediator (?)  Less “lite”-weight! (but not much) Retention period (or min/max RP?)  Stored in Registry & used by mediator  History “queue” uses RP, latest “queue” does not New Consumer query  HISTORY(TimeInterval): get tuples published within the TimeInterval.  HISTORY query: use producer with longest RP

23/06/ PrimaryProducers: Tuple storage Tuple storage needs to be able to:  Store tuples  Return tuples for streaming  Answer SQL (history or latest) queries Two flavours:  Database Uses standard SQL over JDBC for all inserts and queries Each tuple requires a unique ID (auto-incremented INT) for streaming  Memory Inserts and streaming handled using in-memory structures –e.g. HashMap for LATEST, LinkedList for HISTORY For non-trivial SELECTs, dump tuples into IMDB then execute SELECT using JDBC –Performance not too badly degraded (?) –SELECT on in-memory producer will be rare –Suitable IMDB available? e.g. HSQLDB

23/06/ Streaming Which thread(s) should initiate the streaming?  Currently, HTTPProcessor thread for “/insert” call  New design allows for one thread or pool of threads to stream Potential problems with firewalls, timeouts etc. Security  In JDK1.5, security support for NIO is provided Presumably requires modification for grid use  JRA3? Described as an “advanced API”  (Slow) alternatives to NIO Notification using Web Services – security provided by JRA3 Standard Java I/O – security (presumably) provided by JRA3 JMS

23/06/ Synchronization / deadlocks Asynchronous communications  Messenger classes no longer applicable  Alternatives Asynch. stub implementation using separate thread(s) Asynch. transport e.g. SOAP/SMTP Identifying / diagramming deadlocks

23/06/ Security Requirements  Dynamic access lists  Access restrictions based on columns of a table rows of a table  “Immediate” propagation of changes to access rights Solutions  parameterized view-based rules defined for table, stored in Schema access rights for view defined using –name/value pairs of credentials –reference to ACL  reference to ACL published with tuple could ACL contain permissions for separate columns?  authorized users published in separate table and JOINed to data table

23/06/ Other issues Design documents  Sequence diagrams Questions…