1. Comparing security identifiers for business databases
Phil Reed @PhilReedData
The University of Manchester

2. Overview Introduction to me and The University of Manchester
The problem of company and other security identifiers
Suggestions
Going further
What’s next?
What do you do?
Summary

3. 1. Introduction to me and Manchester

4. Library restructure 2012, from subject-based to service area-based
Research and Learning Services
Teaching, Learning and Students
Research Services
Academic Engagement
2016 review
Business Data Service
No more subject librarians, although business data…
HANDS UP who knows about our structure? Who has something similar or is considering it?

5. Subject expert Technical expert Business School: Research Asst.
Convergent evolution?
Teaching/Research Librarian
Research Assistant
Software Consultant
Data Specialist
Subject expert
Technical expert
Library: Data Specialist
Business School: Research Asst.
Business Librarian <> Software Consultant
“Vertebrate wings are homologous as forelimbs, being derived from the same organs; however, as organs of flight in pterosaurs (1), bats (2) and birds (3) they are analogous; they resemble each other in some ways, and they fulfill similar functions, but their roles in flight have evolved separately.” Me
By John Romanes (1892): Darwin and after Darwin - Picture from John Romanes': Darwin and after Darwinhttp:// Public Domain,

6. 2. The problem of company and other security identifiers
So many competing products!

7. Many kinds of security identifiers
GVKEY
Native
Vendor’s own
PERMNO
CIK
Ticker
Many kinds
SEDOL
CUSIP
6, 8, 9-digit
ISIN
The Problem: so many company and financial databases, each has its own ID system, competitors. We want to use data from one combined with data from another. For example, BoardEx director biography with company performance from BvD Amadeus or Compustat. They have their own IDs, patchy coverage of anything shared.

8. Other things to consider
Codes can change with time
Codes can be recycled
Uniqueness not guaranteed
Private and sovereign codes may differ
Formatting hell in Excel Apple’s CUSIP ≠ 4E+07
Also the CUSIP cost/charge, are code’s the implementer’s intellectual property?
And parts of the world have different coverage
Issue vs company code

9. 3. Suggestions A light, perhaps a bit dim
Proposed two tools. (1) A list of techniques and tools for converting between various pairs. (2) A table comparing platforms and ID types, at least for securities.
By Sun Ladder - Own work, CC BY-SA 3.0,

10. Suggestions: conversion techniques
List of tools and what they convert from and to
Tool name
From To
Found
Translate to PERMNO
Ticker, CUSIP
PERMNO
WRDS CRSP Tools
Capital IQ template
Name (loose), ISIN
CIQ ID, Ticker
Capital IQ Excel
Capital IQ hand-coding
CIQ ID, ISIN, GVKEY, CIK
Datastream hand-coding
CUSIP, SEDOL
DS Local Code
Datastream Excel
TRSL hand-coding
CUSIP, SEDOL, ISIN, DS Code, Ticker
Thomson ONE Excel
CUSIP Converter
CUSIP 8/9
CUSIP 6/8/9
WRDS Compustat
Give detail of (1) A list of techniques and tools for converting between various pairs.

11. CRSP to Datastream
From
e.g.
CUSIP To U U
Local Code
Requires Datastream Advance for Office Excel add-in and some formula editing

12. CRSP to Eventus
From
CUSIP To
PERMNO

13. Capital IQ identifier converter
Paste Values / Paste Special here
If you see “No Result”, try a different identifier type.
Capital IQ identifiers (CIQ ID) and Ticker are produced.

14. Capital IQ can convert…
From
Name (fuzzy)
CIQ ID
ISIN
SEDOL
CIK
GVKEY To
Name
Requires Capital IQ Excel add-in and some formula editing

15. Suggestions: comparison table
Green = easy to match
Yellow = requires conversion etc.
Red = not available/usable
For securities, what codes are known to what databases?
(2) A table comparing platforms and ID types, at least for securities.

16. 4. Going further
A case study: Markit CDS & Capital IQ, combining deals info with company info
Case study: Markit CDS, very briefly! Include benefits of cost saving. This wasn’t really in my job spec, now I do more of this in RA role.

17. Markit Map Markit CDS Capital IQ
Markit Ticker
ISIN
Markit CDS
Capital IQ
Company data
Markit has deals info, Capital IQ has company info – why pay again for Markit’s company info?
CDS data
Other data
No common ID!

18. Given an incomplete mapping…
The “mapping” provided to us:
Summary of which securities had activity that day, their Markit Ticker and ISIN code
Summary put in multiple XML files in a zip file
Zip file of most recent 8 days put on FTP server
…Developed an intermediate database
Eventually had enough data to create a partial mapping table
Markit Map
Case study: Markit CDS, very briefly! Include benefits of cost saving. This wasn’t really in my job spec, now I do more of this in RA role.
Markit Ticker  ISIN

19. 5. What next? What next? Fuzzy text matching
Other reference tables
Software Sustainability Institute
Digital Humanities
What next?
Fuzzy text matching
Table for bonds/people/deals
SSI DH

20. What are the requirements?
6. What do you do?
What do you do?
What are your needs?
What are the requirements?
Ask audience for ideas to make the results more usable/available and their experiences.
Your ideas:
Other visualisations or methods to explain ID matching?
What do you do?

21. 7. Summary Many different databases, many types of security identifier
Not to mention other entity types (deals, people, bonds…)
Can summarise major identifiers and their common use
Can list existing tools to map or convert between them
Harder: can write software tools to do more of this
Longevity: nothing stays the same, keep systems up to date

22. EXTRA SLIDES

23. Future tables

24. BoardEx, Bureau van Dijk
(Europe companies)
Bureau van Dijk (Amadeus)
(Europe Very Large & Large companies)
BoardEx ID
6,987
ISIN
2,849
41% of BoardEx
ISIN [1..108]
5,424
BvD ID and ISIN 12,605
Starting with a list of 6,987 BoardEx European/UK company IDs
where 5,424 have one or more ISIN codes (average 1.06, up to 108 ISIN for one!)
Got a list of 12,605 Bureau van Dijk (Amadeus) [Very Large + Large] company IDs
where all have one ISIN code
Using the first and second ISIN codes from BoardEx list, match the Bureau van Dijk list on ISIN code
There are 2,849 matches (41%)

25. Custom intermediate database
USER END
Markit converter overview
Markit via WRDS
Zipped
XML maps
XML maps
Custom intermediate database
(XML, Java)
Markit FTP Server
XML maps of 8 most recent days
markitTicker, redEntityCode, name, ISIN(s)
Download CDS data identified by ticker, a.k.a. markitTicker
Code Converter
(Java application)
WinSCP (later Java):
Load zipped XML maps from FTP Server (weekly)
Java application:
Unzip and organise XML maps
BaseX GUI:
Load XML maps into master database
Given file with column of markitTicker
markitTicker
BaseX/Java application/XQuery:
For given markitTicker, return all ISIN(s)
For each match, write ISIN(s) to new column
ISIN(s)
But what about Sovereign or Private firms if given ISINs?
What about multiple ISINs?