COMP 415, Spring 2008

T ABLE OF C ONTENTS 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

P ROBLEM F ORMULATION 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

P ROBLEM F ORMULATION Messages Take Time To Travel Across a Network How Can We Find Bottlenecks? When Should We Cancel a Message?

P ROBLEM F ORMULATION | BASIC S OLUTION Catch Messages Correlate Store Associations Retrieve Latencies Display System Map

S OLUTION F EATURES 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

S OLUTION F EATURES Relocate-Ready Components Process Abstraction Encapsulated Optimization Points Variable Correlation Multiple Visualizations

S OLUTION A RCHITECTURE 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

S OLUTION A RCHITECTURE Data Flow Scalability Data Recipient Correlation Engine

D ATA F LOW D IAGRAM o JPM Service o Controller o Database o Manipulation o View Client

S CALABILITY D IAGRAM o Data Mining o Correlation o Database

D ATA R ECIPIENT D IAGRAM o Possibilities o Parsing o API Calls o Batching o Destinations

C ORRELATION E NGINE D IAGRAM o Data Reader o Data Writer

M AJOR S YSTEMS 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

M AJOR S YSTEMS Correlation Messaging View

Correlation Engine C ORRELATION A correlation engine holds sets of messages and performs matching between the sets. Data Reader Input Messages Output Messages Server Log Data Reader Input Messages Output Messages Server Log Correlation Rules Association Store

A SSOCIATION G RAPH Xml Configuration File Stores configuration of association graph Information about correlation rules to use Formats of data logs Server A Server B Server C InOut InOut InOut

D ESIGN C HOICES In-memory correlation Correlation Rules can create their own data structures to expedite their matching Sliding Time Window Features Scalability Flexibility Efficiency

M ESSAGING Components not colocated Ensure recoverability Handle large volumes of data

V IEW Eclipse RCP framework External graphing package Wireframes

V IEW | E CLIPSE RCP F RAMEWORK Integration with JPMorganChase Modular design Standardized system

V IEW | E XTERNAL G RAPHING P ACKAGE

V IEW | W IREFRAMES M ULTIPLE S ERVER V IEW

V IEW | W IREFRAMES S INGLE S ERVER V IEW

P ROJECT T IMELINE 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

P ROJECT T IMELINE Five Phases Phase 1: Due February 1 st Phase 2: Due February 15 th Phase 3: Due March 1 st Phase 4: Due April 1 st Phase 5: Due May 1 st Front-Loaded Early Integration

P ROJECT T IMELINE Full Interface Skeleton Stub Modules Admin API (Control) AddMessage API Correlation (Default Rules) View Path Latency Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Admin Client (Control) Correlation (Arbitrary) Message Batching Message Datastore Association Datastore Real-Time View Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Engine Scheduling Correlation Strength Data Client Admin API Graph Node Datastore View Graph Latency Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Zero & Many Correlation Parsing Module Single Message View Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Phase IPhase IIPhase IIIPhase IVPhase V

P ROJECT T IMELINE Re-Correlation Recoverability Control Path Datastore Collapse Nodes (View) Phase IPhase IIPhase IIIPhase IVPhase V

C ONCLUSION 1. Problem Formulation 2. Solution Features 3. Solution Architecture 4. Major Systems 5. Project Timeline 6. Conclusion

C ONCLUSION

Thank You For Listening!