Comparing security identifiers for business databases Phil Reed @PhilReedData The University of Manchester phil.reed@manchester.ac.uk

Overview Introduction to me and The University of Manchester The problem of company and other security identifiers Suggestions Going further What’s next? What do you do? Summary

1. Introduction to me and Manchester

Library restructure 2012, from subject-based to service area-based Research and Learning Services Teaching, Learning and Students Research Services Academic Engagement 2016 review Business Data Service No more subject librarians, although business data… HANDS UP who knows about our structure? Who has something similar or is considering it?

Subject expert Technical expert Business School: Research Asst. Convergent evolution? Teaching/Research Librarian Research Assistant Software Consultant Data Specialist Subject expert Technical expert Library: Data Specialist Business School: Research Asst. Business Librarian <> Software Consultant “Vertebrate wings are homologous as forelimbs, being derived from the same organs; however, as organs of flight in pterosaurs (1), bats (2) and birds (3) they are analogous; they resemble each other in some ways, and they fulfill similar functions, but their roles in flight have evolved separately.” Me By John Romanes (1892): Darwin and after Darwin - Picture from John Romanes': Darwin and after Darwinhttp://www.talkorigins.org/faqs/precursors/images/homology.jpg, Public Domain, https://commons.wikimedia.org/w/index.php?curid=1324636

2. The problem of company and other security identifiers So many competing products!

Many kinds of security identifiers GVKEY Native Vendor’s own PERMNO CIK Ticker Many kinds SEDOL CUSIP 6, 8, 9-digit ISIN The Problem: so many company and financial databases, each has its own ID system, competitors. We want to use data from one combined with data from another. For example, BoardEx director biography with company performance from BvD Amadeus or Compustat. They have their own IDs, patchy coverage of anything shared.

Other things to consider Codes can change with time Codes can be recycled Uniqueness not guaranteed Private and sovereign codes may differ Formatting hell in Excel Apple’s CUSIP 037833100 ≠ 4E+07 Also the CUSIP cost/charge, are code’s the implementer’s intellectual property? And parts of the world have different coverage Issue vs company code

3. Suggestions A light, perhaps a bit dim Proposed two tools. (1) A list of techniques and tools for converting between various pairs. (2) A table comparing platforms and ID types, at least for securities. By Sun Ladder - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=9684092

Suggestions: conversion techniques List of tools and what they convert from and to Tool name From To Found Translate to PERMNO Ticker, CUSIP PERMNO WRDS CRSP Tools Capital IQ template Name (loose), ISIN CIQ ID, Ticker Capital IQ Excel Capital IQ hand-coding CIQ ID, ISIN, GVKEY, CIK Datastream hand-coding CUSIP, SEDOL DS Local Code Datastream Excel TRSL hand-coding CUSIP, SEDOL, ISIN, DS Code, Ticker Thomson ONE Excel CUSIP Converter CUSIP 8/9 CUSIP 6/8/9 WRDS Compustat Give detail of (1) A list of techniques and tools for converting between various pairs.

CRSP to Datastream From 7525108 260543103 e.g. CUSIP To U00752510 U26054310 Local Code Requires Datastream Advance for Office Excel add-in and some formula editing

CRSP to Eventus From CUSIP To PERMNO

Capital IQ identifier converter Paste Values / Paste Special here If you see “No Result”, try a different identifier type. Capital IQ identifiers (CIQ ID) and Ticker are produced.

Capital IQ can convert… From Name (fuzzy) CIQ ID ISIN SEDOL CIK GVKEY To Name Requires Capital IQ Excel add-in and some formula editing

Suggestions: comparison table Green = easy to match Yellow = requires conversion etc. Red = not available/usable For securities, what codes are known to what databases? (2) A table comparing platforms and ID types, at least for securities.

4. Going further A case study: Markit CDS & Capital IQ, combining deals info with company info Case study: Markit CDS, very briefly! Include benefits of cost saving. This wasn’t really in my job spec, now I do more of this in RA role.

Markit Map Markit CDS Capital IQ Markit Ticker ISIN Markit CDS Capital IQ Company data Markit has deals info, Capital IQ has company info – why pay again for Markit’s company info? CDS data Other data No common ID!

Given an incomplete mapping… The “mapping” provided to us: Summary of which securities had activity that day, their Markit Ticker and ISIN code Summary put in multiple XML files in a zip file Zip file of most recent 8 days put on FTP server …Developed an intermediate database Eventually had enough data to create a partial mapping table Markit Map Case study: Markit CDS, very briefly! Include benefits of cost saving. This wasn’t really in my job spec, now I do more of this in RA role. Markit Ticker  ISIN

5. What next? What next? Fuzzy text matching Other reference tables Software Sustainability Institute Digital Humanities What next? Fuzzy text matching Table for bonds/people/deals SSI DH

What are the requirements? 6. What do you do? What do you do? What are your needs? What are the requirements? Ask audience for ideas to make the results more usable/available and their experiences. Your ideas: Other visualisations or methods to explain ID matching? What do you do?

7. Summary Many different databases, many types of security identifier Not to mention other entity types (deals, people, bonds…) Can summarise major identifiers and their common use Can list existing tools to map or convert between them Harder: can write software tools to do more of this Longevity: nothing stays the same, keep systems up to date

EXTRA SLIDES

Future tables

BoardEx, Bureau van Dijk (Europe companies) Bureau van Dijk (Amadeus) (Europe Very Large & Large companies) BoardEx ID 6,987 ISIN 2,849 41% of BoardEx ISIN [1..108] 5,424 BvD ID and ISIN 12,605 Starting with a list of 6,987 BoardEx European/UK company IDs where 5,424 have one or more ISIN codes (average 1.06, up to 108 ISIN for one!) Got a list of 12,605 Bureau van Dijk (Amadeus) [Very Large + Large] company IDs where all have one ISIN code Using the first and second ISIN codes from BoardEx list, match the Bureau van Dijk list on ISIN code There are 2,849 matches (41%)

Custom intermediate database USER END Markit converter overview Markit via WRDS Zipped XML maps XML maps Custom intermediate database (XML, Java) Markit FTP Server XML maps of 8 most recent days markitTicker, redEntityCode, name, ISIN(s) Download CDS data identified by ticker, a.k.a. markitTicker Code Converter (Java application) WinSCP (later Java): Load zipped XML maps from FTP Server (weekly) Java application: Unzip and organise XML maps BaseX GUI: Load XML maps into master database Given file with column of markitTicker markitTicker BaseX/Java application/XQuery: For given markitTicker, return all ISIN(s) For each match, write ISIN(s) to new column ISIN(s) But what about Sovereign or Private firms if given ISINs? What about multiple ISINs?