1. A UNIFIED ECOSYSTEM FOR MARKET DATA VISUALIZATION
Kishore Ramachandran, Director Janaki Parameswaran, Project Manager

3. What We Do? By the Numbers
We oversee more than 3,900 securities firms with approximately 640,795 brokers.
Every day, we watch over nearly 6 billion shares traded in U.S. listed equities markets— using technology powerful enough to detect potential abuses.
In fact, FINRA handles more Big Data on a daily basis than the Library of Congress or Visa®—processing approximately 6 terabytes of data and up to 75 billion events every day to build a complete, holistic picture of market trading in the U.S.
In 2015, FINRA:
Referred over 800 fraud cases for prosecution
Levied more than $191 million in fines & restitution

4. Turning Data Into Insights
Business Challenges
Data discovery
Bring disparate sources of data together
Generate meaningful insights
Learn from past behavior of others and discover new reports
Technological Challenges
Providing users with interactive capabilities on big data
Time to market
Adapt system behavior to user behavior

5. Legacy System

6. Varied Requirements Trend Analysis Audit Trail Outliers in Data
Comparison Analysis

7. Approach to a unified analytics platform?
Support decision making through actionable and insightful reports
Offer self-service capabilities for identifying anomalous trading patterns
Provide flexible data visualization tools
Enable complex statistical modeling without needing to master the algorithms
Optimize performance to reduce time spent on repetitive tasks and data manipulation
Provide a common gateway to rationalized data
Create a learning system that adapts based on usage patterns

8. Unified Platform Solution

9. Data Exploration Simple Scenario

10. Data Exploration Advanced Scenario 1

11. Data Exploration Advanced Scenario 2

12. Unified System

13. Architecture

14. Technology Stack – Data

15. BI Platform

16. Scalability & Automation
AWS code deploy for application deployments and releases
Open source Data Management of S3
Elastic Load balancing
Auto-scaling group
Automated off-loading and on-loading of data
Health-checks to detect data gaps & new versions
Restful services for data availability
Dashboard for usage metrics

17. Security & Access Control
Security is our #1 priority
Dev, QC and Prod environments separated
Virtual Private Cloud (VPC) encryption
Domain based Access Control
Security groups and ISSO
Restrictions on confidential data
Usage metrics and dashboards for total transparency

18. Cloud Operations Cost Management Splunk AWS Billing App
Spot/Reserved Pricing
Monitoring
Cloudwatch
Splunk
Forever
Continuous Delivery
Puppet
Cloud Formation

19. Agile for Analytics
Deliver value early & often following an iterative approach
System validation by continuously seeking business feedback
Pivot direction quickly to avoid scope creep issues
User stories to match deliverables to purpose
Develop software components as micro services that are reusable
Each software deliverable is a product by itself that is self documenting
Automated testing from early stages so testing is incremental and doesn’t add up in the end

20. Our Story, Our Success
Positive Feedback from our Business community:

21. Key Learnings

22. ABOUT THE PRESENTERS Kishore Ramachandran Janaki Parameswaran
Director, FINRA
Data Analytics
LinkedIn
Janaki Parameswaran
Project Manager, FINRA
Data Analytics
LinkedIn

23. QUESTIONS?
Learn more at
FINRA Technology is hiring