Discovering Companies we Know Closed Set Extraction Discovering Companies we Know Rani Shlivinski March 2019

USE CASE: This is Refinitiv Eikon Refinitiv Eikon is a platform that enables professionals in the financial industry to analyze data and make smarter investments

USE CASE: Eikon Company Pages Investors usually have a portfolio of companies they keep track on

PROBLEM: Extracting and Resolving Company Mentions

PROBLEM: Resolving using Organization Authority / PermID.org Definitely not a bank!

SOLUTION: What’s the Deal with Closed Set? We need to identify company mentions in news stories and assign them with the correct identifier If we only care about resolveable entities, we can create in advance a list of the companies we should extract

We are using an ensemble of Machine Learning models on top of an extremely large lexicon

We are using OA to extract company aliases: Apple Inc. Problem (1) We are using OA to extract company aliases: Apple Inc. Apple Incorporated Apple Apple Computers, Inc. But which apple is it?

And what about this list of companies from Linkedin? Problem (2) And what about this list of companies from Linkedin? Could Wednesday, Spring or Milk ever be used in texts as company names?

Our current lexicon supports about 36 million company aliases Problem (3) Our current lexicon supports about 36 million company aliases In customary prefix-tree implementations, such lexicon will consume gigabytes of RAM

Solutions Meet our technology stack

This way we improve memory consumption by a factor of 5-10 Bloom Filter Lexicon A Bloom Filter is a space-efficient probabilistic data structure which answers: whether an element is a member of a set We load the filter with company names and then use it to identify those names in the texts This way we improve memory consumption by a factor of 5-10 Maybe No

Filtering Noise: CSE Ensemble

CSE Ensemble We take a 360 approach where we have different models for different aspects of the problem: Alias aware Context aware World knowledge aware

Alias Aware: Alias Cleaner Based on alias attributes, data from Wikipedia, lists of geographies, people names, etc.

Alias Aware: Word2Vec Word2Vec is an algorithm for training word embeddings, i.e. mapping a fixed vocabulary into a vector space such that semantically close words will have similar vectors We have trained word2vec on 10M documents of our news data and use the resulting embeddings as feature vectors for each alias

Each one of these models results are scored using Random Forest Alias Aware: Scoring Each one of these models results are scored using Random Forest The models may agree or disagree. Final result is taken at a later stage An interesting attribute of these two models is that their results can be precalculated and thus reduce overall latency

Context Aware: NLP Tagger Each word in the local context is a feature Company name is masked for the model

World Knowledge Aware: Signature Model Signature model looks for pertinent contextual clues from OA in the document body

Final decision is taken with yet another Random Forest model Ensemble For each company instance the results of the for models previously mentioned are taken into account before outputting the final score Final decision is taken with yet another Random Forest model

Quality Figures For the Eikon use case, we test end-2-end quality, which means a combined figure of Extraction, Resolution and Relevance Precision: The ratio of correct instances out of all identified instances Recall: The ratio correct instances out of all existing instances Precision Recall Public Companies 88% (extraction only=97%) currently under tests Private Companies 76% (extraction only=97%) 74%-86%

What about Deep Learning? The system so far presented is that state of the art of classical machine learning We are currently experimenting with the same network topology that is being successfully used for Person extraction. Results look promising This configuration is trained with CSE Ensemble as its teacher

The End