SWE 622- Distributed Systems Project Phase III Eric Barnes, David Chang, David Nelson Fisayo Oluwadiya, Xiang Shen

Initial Design

Final Design

Initial Class Diagram

Final Class Diagram

public void init() throws Exception { _serverSocket = null; try { _serverSocket = new ServerSocket(_exchangePort); } catch (IOException e) { log.error("Could not listen on port:" + _exchangePort); System.exit(1); } while (true) { Socket clientSocket = null; try { log.info("WAITING"); clientSocket = _serverSocket.accept(); log.info("ACCEPTING"); } catch (IOException e) { log.error("Accept failed."); System.exit(1); } TraderConnectionManager manager = new TraderConnectionManager(clientSocket, this); traderManagers.add(manager); } Class Exchange init() method

public TraderConnectionManager(Socket clientSocket, Exchange _exchange) { this.exchange = _exchange; this.outMsgQueue = new ArrayBlockingQueue (50); try { out = new PrintWriter(clientSocket.getOutputStream(), true); in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream())); } catch (IOException e) { } Thread tin = new Thread(new TraderInput(clientSocket, outMsgQueue)); tin.start(); Thread tout = new Thread(new TraderOutput(clientSocket, outMsgQueue)); tout.start(); //set last keepalive so that we don't time out right away lastKeepaliveReceived = System.currentTimeMillis(); //schedule keepalives keepaliveTimer = new Timer("keepallive"); keepaliveTimer.scheduleAtFixedRate(new CheckTimeout(), KEEPALIVE_INTERVAL, KEEPALIVE_INTERVAL); } Class TraderConnectionManager constructor

public void run() { boolean connectedToExchange = initSocket(exchangAddr, port); if (connectedToExchange) { // set last keepalive so that we don't time out right away lastKeepaliveReceived = System.currentTimeMillis(); // schedule keepalives keepaliveTimer = new Timer("keepalive-" + _timeMsgQueue); keepaliveTimer.scheduleAtFixedRate(new SendKeepalive(), KEEPALIVE_INTERVAL, KEEPALIVE_INTERVAL); // schedule clock syncs clockSyncTimer = new Timer("clockSync-" + _timeMsgQueue); clockSyncTimer.scheduleAtFixedRate(new SyncClock(), 0, CLOCK_OFFSET_CHECK_INTERVAL); } Note: KEEPALIVE_INTERVAL = 5000; MAX_MISSED_KEEPALIVES = 2; //check every 1 hour (1000 ms/sec * 60 sec/min * 60 min/hour CLOCK_OFFSET_CHECK_INTERVAL = 1000 * 60 * 60; Class ExchangeConnectionManager run() method

TradeTransaction t = new TradeTransaction(fromServer); boolean outOfOrder = t.getMessageID() < newestMessageProcessed; //if the message was out of order, flag it, but don't set it as the last ID recieved to // ensure that subsequent out of order messages are also flagged. e.g. if IDs arrive // in the order 5 3 4, both 3 and 4 should be flagged if(outOfOrder) { log.error("Out of order"); t.setOutOfOrder(true); } else { newestMessageProcessed = t.getMessageID(); log.info("Message sequence:" + newestMessageProcessed); } t.setStale(_clockDelta + System.currentTimeMillis()); processedTAQueue.put(t); Just in case the echo messages are out of order

TradeTransaction t = new TradeTransaction(fromServer); boolean outOfOrder = t.getMessageID() < newestMessageProcessed; //if the message was out of order, flag it, but don't set it as the last ID recieved to // ensure that subsequent out of order messages are also flagged. e.g. if IDs arrive // in the order 5 3 4, both 3 and 4 should be flagged if(outOfOrder) { log.error("Out of order"); t.setOutOfOrder(true); } else { newestMessageProcessed = t.getMessageID(); log.info("Message sequence:" + newestMessageProcessed); } t.setStale(_clockDelta + System.currentTimeMillis()); processedTAQueue.put(t); Just in case the echo messages are out of order

private class SyncClock extends TimerTask public void run() { long requestTime; long receiveTime; long oneWay; long exchangeTime; long totalDelta = 0; int tryTimes = 3; String timeStr = ""; for (int i = 0; i < tryTimes; i++) { requestTime = System.currentTimeMillis(); log.info("requestTime " + i + "=" + requestTime); requestTime(requestTime); try { timeStr = _timeMsgQueue.take(); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } exchangeTime = Long.parseLong(timeStr.replace("time:", "")); log.info("exchangeTime " + i + "=" + exchangeTime); receiveTime = System.currentTimeMillis(); log.info("receiveTime " + i + "=" + receiveTime); oneWay = (receiveTime - requestTime) / 2; totalDelta += exchangeTime + oneWay - receiveTime; } _clockDelta = totalDelta / tryTimes; log.info("clockDelta=" + _clockDelta); } How to figure how to the clock delta between exchange and trader

All SLOC: SLOC Directory SLOC-by-Language (Sorted) 435 gui java= exchange java= trader java= common java=147 Totals grouped by language (dominant language first): java: 1325 (100.00%) Reused SLOC (GUI code provided by Prof. Sousa): SLOC Directory SLOC-by-Language (Sorted) 311 gui java=311 Totals grouped by language (dominant language first): java: 311 (100.00%) Total New SLOC: SLOC Directory SLOC-by-Language (Sorted) 1014 src java=1014 Totals grouped by language (dominant language first): java: 1014 (100.00%) Total Physical Source Lines of Code (SLOC) = 1,014 Development Effort Estimate, Person-Years (Person-Months) = 0.20 (2.44) (Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05)) Schedule Estimate, Years (Months) = 0.29 (3.51) (Basic COCOMO model, Months = 2.5 * (person-months**0.38)) Total Estimated Cost to Develop = $ 27,415 (average salary = $56,286/year, overhead = 2.40). SLOCCount, Copyright (C) David A. Wheeler Source Lines Of Code

Effort spent in design: 120 Effort spent in design coding: 100 Effort spent in design integrating: 40 Effort spent in design experiments: 60 Total effort: 320 Note: All of them are estimates Effort spent for the project (man-hour)

SUSE Linux 10 SP1 (i586) –Kernel: bigsmp #1 SMP Thu May 17 14:00:09 UTC 2007 –Memory: 4GB –CPU: 2 x Intel(R) 3.40GHz –Java: java version "1.6.0_12“ Solaris 10 – Kernel: SunOS mason 5.10 Generic_ sun4u sparc SUNW,Sun-Fire-V890 – Memory: 32GB – CPU: 8 x Sparc 1350 MHz – Java: java version "1.6.0_11“ Microsoft Windows XP SP3 –Java: java version "1.6.0_01“ Microsoft Windows Vista SP1 –Java: java version "1.6.0_12“ Testing Environment

LAN: Ping < 1ms Direct connected by a network switch WAN: 10ms < Ping < 30ms Pinging mason.gmu.edu [ ] with 32 bytes of data: Reply from : bytes=32 time=13ms TTL=246 Reply from : bytes=32 time=14ms TTL=246 Reply from : bytes=32 time=19ms TTL=246 Reply from : bytes=32 time=12ms TTL=246 Ping statistics for : Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 12ms, Maximum = 19ms, Average = 14ms Many network hops Tracing route to mason.gmu.edu [ ] over a maximum of 30 hops: 1 1 ms 1 ms 1 ms ms 9 ms 8 ms ms 9 ms 9 ms ip dc.dc.cox.net [ ] 4 9 ms 10 ms 9 ms mrfddsrj01-ge110.rd.dc.cox.net [ ] 5 13 ms 10 ms 10 ms ashbbbrj01-ae0.0.r2.as.cox.net [ ] 6 13 ms 64 ms 11 ms ae car1.Washington1.Level3.net [ ] 7 16 ms 13 ms 12 ms GEORGE-MASO.car1.Washington1.Level3.net [ ] 8 * * * Request timed out. 9 * * * Request timed out ms 15 ms 12 ms mason.gmu.edu [ ] Testing Network and network latency

LAN: Ping statistics for : Approximate round trip times in milli-seconds Minimum = 0ms, Maximum = 0ms, Average = 0ms WAN: Ping statistics for (mason.gmu.edu): Approximate round trip times in milli-seconds: Minimum = 17ms, Maximum = 20ms, Average = 18ms

Avg=86.7

Between each Exchange and Trader there is a TCP connection If the Trader fails, each Exchange will know immediately –Read failed error –Depends on the network latency If the network also fails, each Exchange will disconnect the failed Trader after the time-out –MAX_MISSED_KEEPALIVES* KEEPALIVE_INTERVAL In the log file: ERROR TraderConnectionManager:130 - Read failed. Client seems gone or the exchange is closing. INFO TraderConnectionManager:68 - Disconnecting INFO TraderConnectionManager:85 - disconnecting trader INFO TraderConnectionManager:137 - EXIT INPUT INFO TraderConnectionManager:177 - EXIT OUTPUT Latency between a trader failure and the last exchange stopping echoing orders

Linux and the Sun box all have ntpd running All Windows boxes all have the Windows time service running Clock Skew we observed on the Linux box (from the NTP log): …… 25 Apr 04:56:57 ntpd[3151]: time reset s 25 Apr 05:01:13 ntpd[3151]: synchronized to LOCAL(0), stratum Apr 05:02:18 ntpd[3151]: synchronized to , stratum 1 25 Apr 05:28:11 ntpd[3151]: time reset s 25 Apr 05:32:28 ntpd[3151]: synchronized to LOCAL(0), stratum Apr 05:33:31 ntpd[3151]: synchronized to , stratum 1 25 Apr 05:47:29 ntpd[3151]: time reset s …… Testing Scenario: –Stop ntpd on Linux –Change time, for example: date +%T -s "17:07“ –Test Clock Skew