A UNIFIED ECOSYSTEM FOR MARKET DATA VISUALIZATION Kishore Ramachandran, Director Janaki Parameswaran, Project Manager

What We Do? By the Numbers We oversee more than 3,900 securities firms with approximately 640,795 brokers. Every day, we watch over nearly 6 billion shares traded in U.S. listed equities markets— using technology powerful enough to detect potential abuses. In fact, FINRA handles more Big Data on a daily basis than the Library of Congress or Visa®—processing approximately 6 terabytes of data and up to 75 billion events every day to build a complete, holistic picture of market trading in the U.S. In 2015, FINRA: Referred over 800 fraud cases for prosecution Levied more than $191 million in fines & restitution

Turning Data Into Insights Business Challenges Data discovery Bring disparate sources of data together Generate meaningful insights Learn from past behavior of others and discover new reports Technological Challenges Providing users with interactive capabilities on big data Time to market Adapt system behavior to user behavior

Legacy System

Varied Requirements Trend Analysis Audit Trail Outliers in Data Comparison Analysis

Approach to a unified analytics platform? Support decision making through actionable and insightful reports Offer self-service capabilities for identifying anomalous trading patterns Provide flexible data visualization tools Enable complex statistical modeling without needing to master the algorithms Optimize performance to reduce time spent on repetitive tasks and data manipulation Provide a common gateway to rationalized data Create a learning system that adapts based on usage patterns

Unified Platform Solution

Data Exploration Simple Scenario

Data Exploration Advanced Scenario 1

Data Exploration Advanced Scenario 2

Unified System

Architecture

Technology Stack – Data

BI Platform

Scalability & Automation AWS code deploy for application deployments and releases Open source Data Management of S3 Elastic Load balancing Auto-scaling group Automated off-loading and on-loading of data Health-checks to detect data gaps & new versions Restful services for data availability Dashboard for usage metrics

Security & Access Control Security is our #1 priority Dev, QC and Prod environments separated Virtual Private Cloud (VPC) encryption Domain based Access Control Security groups and ISSO Restrictions on confidential data Usage metrics and dashboards for total transparency

Cloud Operations Cost Management Splunk AWS Billing App Spot/Reserved Pricing Monitoring Cloudwatch Splunk Forever Continuous Delivery Puppet Cloud Formation

Agile for Analytics Deliver value early & often following an iterative approach System validation by continuously seeking business feedback Pivot direction quickly to avoid scope creep issues User stories to match deliverables to purpose Develop software components as micro services that are reusable Each software deliverable is a product by itself that is self documenting Automated testing from early stages so testing is incremental and doesn’t add up in the end

Our Story, Our Success Positive Feedback from our Business community:

Key Learnings

ABOUT THE PRESENTERS Kishore Ramachandran Janaki Parameswaran Director, FINRA Data Analytics Kishore.Ramachandran@finra.org LinkedIn Janaki Parameswaran Project Manager, FINRA Data Analytics Janaki.Parameswaran@finra.org LinkedIn

QUESTIONS? Learn more at http://technology.finra.org/ FINRA Technology is hiring http://technology.finra.org/careers.html