QA Function for Low Latency Trading Platform

Agenda Latency : Definition and Importance  Latency : Definition and Importance Low Latency Application Architecture - QA checkpoints Risks involved - QA significance Global QA Team - Effective Test Management Global QA Team – Key Benefits Best Practices – QA Methodology QA Service Offering Appendix - I : QA Metrics Q & A.

Latency : Definition and Importance Latency - a synonym for delay, is an expression of how much time it takes for a packet of data to get from one designated point to another When talking about latency in the context of capital markets consider the round trip between trade initiation event and trade execution event. Latency involves Processing time and Propagation time Buy Side Client Brokerage Firms Trading Venue/ Exchanges A B C D H G F E Propagation Time Processing Time Propagation Time Processing Time Importance Buy Side firms want to be closer to the liquidity (exchange matching engine) Faster access to market benefits the Arbitrage Trader. Cancelling Open Orders quickly - those who cancel first will reduce their potential losses High frequency Traders hold the positions for as low as milliseconds employing computerized algorithms and hence very sensitive of the processing speed of markets and of their own access to the market.

Low Latency Application Architecture - QA checkpoints Client Exchange Booking Apps OMS Backfiller Component Client Facing Component (FIX Engine) Market Facing Component Proprietary Protocol FIX Protocol Native / Exchange Protocol Proprietary Protocol 1 2 3 4 5 1 2 3a 3b 4 5 Validating the message received from the client. Validating Proprietary Protocol conversion within the application Validating the message sent to the exchange/ destination trading venues. Responses received from destination venues. Reports sent back to the client. Message sent to internal OMS systems for trade reporting and settlement.

Risks involved - QA significance Risks/Challenges involved emphasizing strong QA involvement  Incorrect messages/orders sent to exchanges can lead to compliance and regulatory breaches which can also imply monetary penalties Incorrect report/trade execution not complying with FIX/specific protocol can lead to messages not being delivered to client or even denial of trade executions. This can lead to huge financial losses as the trader can be exposed to unaccounted positions Errors in trade artifacts sent to downstream order management systems may result in erroneous trade settlement and clearance Delays in messages sent to exchanges can result in huge direct losses to clients who stand a chance to have substantial gains from arbitrage or even marginal fluctuation in stock price changes hence emphasizing on the importance of low latency. Delays in cancelling the open orders in market can also implicate serious monetary losses on account of gaining unwanted positions. The most famous examples are “6th May Flash Crash” in US markets

Case Study: QA Function for Low Latency Platform of a Global Bank

Global QA Team - Effective Test Management Before US Team US Off-shore QA Team UK Team UK APAC Team APAC After US Global Team UK Off-shore QA Team APAC Improving quality of work by building a strong team Removing redundancies and duplicity. Leveraging Regional Synergies Key Objective

Global QA Team – Key Benefits Sharing of Issues/ Defects Training and Knowledge sharing Leveraging regional best practices Streamlined reporting Test Case reusability Global QA Team Effective Resource utilization FIX protocol testing Building a strong SME team People Benefits

Best Practices – QA Methodology Aligning with Rapid Deployment Process “Follow the sun” model Automated regression suite Real time production scenarios testing Build tracking mechanism Exploratory Testing Value Added Testing

Capacity/Load Testing QA Service Offering Regression Testing Regression testing is performed to ensure any new functionality/bug fixes do not impact the existing functionality Regression test suites are maintained, updated and validated periodically to ensure that critical bug fixes/changes make it to the test suite which would be executed on future releases.  Functional Testing Functional Testing to test new functionalities/enhancements/ requirements and bug fixes in the new release/ build provided by Development team.  Automated Testing In-house automation tools are used to automate a substantial amount of test cases. The tool utilizes the record and replay functionality, parsing the logs and replaying against simulators. As the FIX logs have similar pattern this helped in reusing the same component for automation efforts in other regions also.  Performance Testing Performance team engaged to provide performance figures to calculate the round trip delays in QA environment against available simulators/exchange sessions. The performance figures even though not realistic provide benchmarks for QA team for further deployments.  Capacity/Load Testing Capacity and Load testing performed to identify the benchmark capacity figures for the application and also to test the application behavior under variable loads, like multiple client connections. 

Appendix -1 : QA Metrics  QA Coverage: Exchanges Covered: 25+ exchanges covered across US,UK and APAC Protocols Tested: FIX 4.2 protocol Customized FIX protocol Proprietary protocol Exchange specific native protocols Indicative QA Metrics for Global QA team supporting low latency applications Total Team Size: Around 7 Total QA Drops: 200+ Yearly Total Defects Raised: 290+ Yearly Test Case Executed: Around 17000+ Yearly

Q & A