1. Distributed Trading System SWE 622 Charles Beach James Wolfe Scott Jones Sharita Green

2. Agenda Prototype Design Code Snippets Evaluations Logistic Measurements Demo

3. Evaluator... network Exchange1 Trader1 System Design ExchangeX TraderX... Logs

4. TraderClient Goods Map ExchangeEngine network Exec Buy/Sell Echo Pub/Sub Exchange GUI Trader Exchange Design

5. Exec GUI network Exchange ConfigFile ExchangeInterface Buy/Sell Receive Echoes Trader TraderEngine Trader Design

6. Communication Traders communicate with Exchanges via RMI Register Buy Sell Exchanges echo transactions to Traders via Pub/Sub.

7. Trader Registration Configuration FileTraderEngine ExchangeInterface ExchangeInterface ExchangeEngine network TraderClient TraderClient Exchange1 Exchange2 RMI: register() Trader1

8. Exchange Transaction Echoes Exchange network ExchangeInterface TraderEngine TraderGUI ExchangeInterface TraderEngine TraderGUI Trader1TraderN RMI: buy/sell TraderX Pub/Sub

9. Trader Registration Date tempDate = new Date(); long time1 = tempDate.getTime(); // RMI Register method ArrayList tempList = _bRemote.registerTrader(traderName); tempDate = new Date(); long time2 = tempDate.getTime(); // Calculate the clock offset long mid = (time2 + time1)/2; offset = ((Long)tempList.get(0)).longValue() - mid; // Calculate the network latency long netLat = (time2 - time1)/2; TraderLogger.instance().logExchangeTimeOffset(getExchangeName(), offset netLat); Calculation of exchange/trader clock offset and network latency

10. Trader processing Echo // Determine if transaction is the next one expected if(sequenceNumber == client.getSequence()+1) { long transactionTime = timeStamp + client.getOffset(); client.incrementSquence(); // Determine if the data is stale if((_time - transactionTime) >= 10000) { updateGUI(exchangeName, function, product, quantity, price, true); } else { updateGUI(exchangeName, function, product, quantity, price, false); } Detection out of order echo sequences and stale data

11. Exchange Processing of Registrations Update registration times as traders re-register Continuously check registration times for “failed” traders public void registerTrader(String traderName) {... // timestamp registration Date d = new Date(); long registerTime = d.getTime(); // re-register by updating registration time client.setRegisterTime(registerTime); ExchangeLogger.instance().logTraderRegistration(traderName);... } class CheckTraders extends Thread {... Iterator itr = _traders.iterator(); Date d = new Date(); long time = d.getTime(); while(itr.hasNext()) { TraderClient client = (TraderClient)itr.next(); if (time > client.getRegisterTime() + 10000) { ExchangeLogger.instance().logTraderFailure(client.getTraderName()); itr.remove(); }

12. RMI register TraderExchange (ET) Tt1 Tt2 Trader Calculations Tte = (Tt1 + Tt2) / 2 Offset = Et - Tte Et Tte Offset Calculation

13. Problem: Need to identify an un-received message as stale before subsequent “on time” message becomes stale itself. Solution: Include in echo message sequence number, time of message and time of previous message. Registration response provides Trader with the sequence number of last message sent. Hold “out of order message” in queue until missing message(s) arrive or until previous message(s) are stale. Previous message staleness calculated by taking “time of previous message” from on hold message, adding offset and comparing against current time. When current time > 10 sec more than calculated time of previous message, then all previous un-received messages are noted as stale and on hold message can be released (and guaranteed to not be stale). Staleness and Out of Order Messages

14. Evaluations Test Scenario included two Exchanges and two Traders across two machines connected by a local LAN PC1 network Exchange1 Trader1 PC2 Exchange2 Trader2

15. Evaluation 1 Latency between starting to send an order and proceeding with other activities. Exchange logs a timestamp before echoing order and when finished notifying all clients ExchangeNumber of Orders Processed Average Latency Exchange12319ms Exchange22119ms

16. Evaluation 2 Latency between issuing an order and the last trader receiving it’s echo Exchange logs a timestamp before echoing order Each trader logs a timestamp when receiving an order Evaluator computes traders timestamp relative to exchanges based on offset of clocks. ExchangeNumber of Orders Processed Average Latency Exchange12391ms Exchange22199ms

17. Evaluation 3 Latency between trader failure and the last exchange stopping echoing orders. Trader logs a timestamp when going offline Exchange logs a timestamp when detection of trader failure Evaluator computes traders timestamp relative to exchanges based on offset of clocks. Amount of time between trader failure and its next registration. At most 15 seconds

18. Evaluation 4 Storage footprint on exchange required to echo orders as a function of the number of traders With our pub/sub approach, the exchange only sends out ONE message that reaches all Traders. Therefore the storage footprint required to echo orders to traders was not expected to grow significantly with the number of Traders. Function remains CONSTANT

19. Evaluation 4 Memory Footprint The exchange used about: 31.3MB with no traders at the very start, but then drops to 28.29MB if nothing happens 28.33MB when one trader was started 30.59MB on first sell request (still 1 trader) 30.72MB on second sell request (still 1 trader) 31.02MB on seven buy requests (still 1 trader) 31.75MB on resizing (bigger) the server window 32.14MB on resizing (smaller) the window 32.14MB when second trader started 32.16MB on first buy from second trader 32.21MB on first sell of new product from second trader 32.21MB when third trader started 32.28MB after a while with no input (it fluctuates at this point 32.27 - 32.33, but mainly 32.28) 32.38MB on first sell of new product from third trader Conclusions Additional Traders have minimal impact on system memory

20. Evaluation 5 Average and max clock skew Trader logs each calculated “time offset” relative to each exchange over 10 second intervals TraderExchangeMax SkewAvg Skew Trader1Exchange13ms0ms Exchange22ms0ms Trader2Exchange12ms0ms Exchange24ms0ms

21. Evaluation 6 Average and max network latency Trader logs each calculated network latency during re- registration. TraderExchangeMax Latency Avg Latency Trader1Exchange18ms4ms Exchange27ms5ms Trader2Exchange19ms5ms Exchange28ms5ms

22. Development Measurements SLOC count: 2009 Exchange: 473 Trader: 557 Logger: 175 Evaluator: 804 Design Effort: 110 man-hours Design: 30hrs Coding/Integrating: 75hrs Experiments/Evaluation: 5hrs

23. Demo

24. Questions