Case study: A DSL for normalization of financial data sets Software Development Automation 2014 Edwin Hautus The Science of Finance

\ 2 Agenda —Problem Domain —First iteration —Second iteration —Lessons learned

\ 3 We are a leading global diversified provider of financial information services We help our customers reduce risk, improve operational efficiency and benefit from enhanced transparency Our customers include investment banks, hedge funds, asset managers, central banks, regulators, auditors, fund administrators and insurance companies Problem Domain 2003 Founded 3,000+ Employees 3,000+ Customers 20+ Offices

\ 4 Problem Domain

\ 5 Normalization questions —Identification - is it the same object? —Field mapping - do these fields have the same semantics? —Structural changes – how can the data be transformed? Problem Domain

\ 6 Workflow & Roles

\ 7 Problem Domain Problem Statement Define a framework to improve efficiency and quality of normalization

\ 8 First iteration (2007) —Standardize writing of normalization specifications —Create a domain model to describe the data structures —Create a XML-based DSL to define mappings —Use a runtime engine to execute the mappings First iteration

\ 9 Results —Good improvement in efficiency and quality —Applied in over 75 projects —The XML DSL is too technical for data analysts —Quite a few iterations required to make sure mappings meet the requirements —Mappings can become really complicated! First iteration

\ 10 Second Iteration (2013) —Have analysts write mappings directly —Create parsed language to replace XML (based on ANTLR) —Simplify business logic —Allow analyst to verify mappings with runnable tests before handover Second iteration

\ 11 Field name mapping Second iteration Instrument isin Bond isinCode input Bond bond Instrument instrument = from bond { isin = isinCode }

\ 12 Field mapping with simple transformation Second iteration InstrumentType cfiSedolcfiCode input Sedol sedol InstrumentType instrumentType = from sedol { cfi = cfiCode.substring(0,2) + “XXXX” }

\ 13 Field mapping via map Second iteration InstrumentType cfiAnnadebtEquityCode Map input Anna anna InstrumentType instrumentType = from Anna { cfi = debtEquityCodeToCfi[debtEquityCode] } Map debtEquityCodeToCfi = { "D" = "DBXXXX" "E" = "ESXXXX" >> "MMXXXX“ >> "MMXXXX“}

\ 14 Structural change: unfold Second iteration input MarkitMap markitMap Company company = from markitMap { from lei { lei = leiCode }} Company leiMarkitMaplei Lei leiCode

\ 15 Structural change: fold Second iteration input Company company MarkitMap markitMap = from company { lei = { leiCode = lei }} Company leiMarkitMap Lei lei Lei leiCode

\ 16 Structural change: fold list field Second iteration input Instrument instrument Sedol sedol = from instrument { details = from sedols { sedolCode = it actionIdentifier = “I” }} Instrument sedols[]Sedoldetails[] SedolDetails sedolCode actionIdentifier

\ 17 Conditional mapping Second iteration input Bond bond Instrument instrument = from bond { if isin { identifier = isin }else{ identifier = cusip }} Instrument identifier Bond isin cusip ?

\ 18 Annotations Second iteration Instrument isin Bond isinCode namename input Bond bond Instrument instrument = from bond { isin = name = name }

\ 19 Additional language features —Asserts —Functions —Imports —Comments Second iteration

\ 20 Lessons learned on DSL development —Using a syntax based approach is worth the effort —DSLs need to be periodically refined as you learn about the domain —Provide integrated testing solution Lessons learned

\ 21 Key factors in adoption of the DSL by business users —Requires more effort from users in order to be precise —Provides feeling of empowerment —Immediate feedback loop through test functionality is crucial Lessons learned

mines data pools intelligence surfaces information enables transparency builds platforms provides access scales volume extends networks & transforms business.

The information contained in this presentation is confidential. Any unauthorised use, disclosure, reproduction or dissemination, in full or in part, in any media or by any means, without the prior written permission of Markit Group Holdings Limited or any of its affiliates ("Markit") is strictly prohibited. Opinions, statements, estimates and projections in this presentation (including other media) are solely those of the individual author(s) at the time of writing and do not necessarily reflect the opinions of Markit. Neither Markit nor the author(s) has any obligation to update this presentation in the event that any content, opinion, statement, estimate or projection (collectively, "information") changes or subsequently becomes inaccurate. Markit makes no warranty, expressed or implied, as to the accuracy, completeness or timeliness of any information in this presentation, and shall not in any way be liable to any recipient for any inaccuracies or omissions. Without limiting the foregoing, Markit shall have no liability whatsoever to any recipient, whether in contract, in tort (including negligence), under warranty, under statute or otherwise, in respect of any loss or damage suffered by any recipient as a result of or in connection with any information provided, or any course of action determined, by it or any third party, whether or not based on any information provided. The inclusion of a link to an external website by Markit should not be understood to be an endorsement of that website or the site's owners (or their products/services). Markit is not responsible for either the content or output of external websites. Copyright ©2014, Markit Group Limited. All rights reserved and all intellectual property rights are retained by Markit. Disclaimer