Knowledge Representation and Reasoning into Machine and Deep Learning David Newman SVP, Innovation R&D Innovation Group, Wells Fargo Bank david.newman@wellsfargo.com linkedin.com/in/davidsnewman1 September 27, 2017

What is Knowledge Representation and Reasoning (KRR)? Presentation Topics The Challenge of Data What is Knowledge Representation and Reasoning (KRR)? What are Some Use Cases? What are some ways that KRR via Ontologies can be Applied to Improve Machine and Deep Learning?

Why are we so Impacted by Data Challenges? Collecting, Cleaning, and Organizing Data Consumes at Least 80% of Data Scientists Time. Why are we so Impacted by Data Challenges? *Cleaning Big Data: Most Time-Consuming, Least Enjoyable Data Science Task, Survey Says, Forbes, March 2016

What are Some of these Key Data Challenges? Data challenges include: reconciling and harmonizing disparate data across line of business silos linking and aggregating data validating, curating, and classifying data for operational processing and creating good feature sets for machine learning We need an effective data management capability that can ensure that data: is harmonized and integrated across all data sources is curated and aligned to common meaning is utilized and understood not only by humans, but by machines as well

Conventional Data Management Capabilities are Not Sufficiently Fulfilling our Data Needs! Bad Data Bad Data

Applying Knowledge Representation and Reasoning using Semantic Technology is a Way to Meet our Complex Data Needs!

Ontologies Express the Meaning of Concepts Concept of “Corporate Control” Subject Corporation Predicate controls Object Corporation type is kind of type controls majority voting shares London Bank is inverse of plays role plays role inference inference Parent Company is majority controlled by Subsidiary “Things” not Strings

For Finance the EDM Council is Developing a Free and Open Source Common Financial Data Standard Using Ontologies

FIBO Ontologies Provide a Scaffolding for Concept Reuse and Extension… Supporting Efficiency, Effectiveness and Governance Industry Standard Ontology Enterprise Standard Ontology Enterprise Knowledge Graph

Ontologies can Play a Critical Role to Better Enable AI Artificial Intelligence Knowledge Representation and Reasoning Machine Learning Deep Learning Natural Language Processing

Progressing Up the AI Stack Ontologies are a Bridge Between Human Induced Knowledge and Machine Induced Knowledge Machine Induced Knowledge Cognitive Computing Machine and Deep Learning Foundational knowledge for cognitive computing to learn and build stronger associative connections Progressing Up the AI Stack Enterprise Ontology Semantically curated and inferred data will give greater lift to machine learning Prior knowledge from ontologies used for supervised ML training sets for data quality, correctness and better NLP Conceptual scaffolding for enterprise ontology Validate, curate, link and harmonize legacy data Industry Ontology Human Induced Knowledge

Ontologies can be Used for Better Feature Engineering Disparate Data Sources Data Linkage and Harmonization Data Validation and Consistency Data Classification and Inference Create Feature Set From Multiple Disparate Data Sources and Formats Identify instances of incorrect or inconsistent data Smart Dimensionality Reduction

Ontologies Enable Data Harmonization and Alignment Needed for Consolidation of Disparate Data Based on Common Meaning Global Bank owns > 50% voting shares of London Bank Semantic Mapping Semantic Adapters FIBO Operational Ontologies FIBO RDBMS Big Data

Ontologies Better Position us to Manage Linked Data and Reduce Data Redundancy “Things Not Strings”

Using Ontologies … Automated Logical Consistency Checks can be Performed to Identify Violations of Semantic Rules in Features Definition of Aunt in Ontology Supervised Machine Learning Data in Message Payload Logical Inconsistency! Reasoning

Human Facing Definition Machine Facing Definition Ontologies can be Used to Reason over Data for Classification and Dimensionality Reduction An interest rate swap in which fixed interest payments on the notional are exchanged for floating interest payments. Human Facing Definition Fixed Float IR Swap InterestRateSwap and hasLeg some FixedRateLeg and hasLeg some FloatingRateLeg Machine Facing Definition Fixed Float IR Swap Business Entity Business Entity Interest Rate Swap London Bank Atlas Bank type Inferred Fixed Float IR Swap identifies identifies type Inferred LEI5001 Swap1001 LEI7777 LEI has party has Leg Swap has Leg has party LEI Floating Rate Leg Inferred Swap Leg 1 Swap Leg 2 Inferred Fixed Rate Leg type type has principle has rate has rate has principle 10000000 LIBOR 5% 10000000 has currency has currency USD USD

Knowledge Graph is the Convergence of Ontologies with Machine Learning Insights Probabilistic Associations and Classifications all counterparties for interest rate swap trades that are likely (> 99.5%) to default given a x% rise in LIBOR Operational Semantic Graph Data Knowledge Graph

Machine Learning can Generate Probabilistic Associations that can be Expressed by the Knowledge Graph as New Relations Address Explanation Person A Other Name Cosine Other Relation Customer A Similarity Infer predicate relationships (same as, colludes, knows, et.al.) Household Customer B Address Enterprise Entity Resolution Dedupe/Link Customers, People, Companies, Addresses From Strings to Things KYC, Fraud, Credit Risk … Person B Name Other Relation

Ontologies can be Used to Transform Unstructured Content into a Structured Form for Machine Learning and in silico Memory intended use plans to expand “I am planning to buy a truck to expand my construction business” capture content James Jackson semantically parse and classify text Natural Language Understanding store customer communications Intent: purchase truck Context: business expansion Action: recommend Auto Loan Action: offer to increase credit line Knowledge Graph understanding machine learning recommendation

Ontology Concept Iteration for Text Generation and Extraction Ontologies can also Help Generate More Precise Neural Word Embeddings to Better Optimize Natural Language Understanding Concept Definition Concept Web Link Ontology Concept(s) Relation Relation Definition Concept Annotation Web Link Ontology Concept Iteration for Text Generation and Extraction Understanding contracts that describe financial instruments Merge human knowledge with machine knowledge Train with less but more precise data Generate Neural Word Embeddings for each Concept in Ontology Neural Word Embeddings Neural Network

Important Advances in Ontology and Machine Learning Research Research from the University of Mannheim, Germany [Paulheim, Stuckenschmidt] show how ontologies can help ML perform validation of data at least 50x faster than a state of the art semantic reasoner by training a binary classifier. Research from Rensselaer Polytechnic Institute [Makni, Hendler] shows how we can use ontologies to train deep learning neural networks to perform reasoning to materialize semantic inferences efficiently at scale. Research from the University of Oregon [Wang, Dou, Lowd] shows how we can use ontologies to improve ML insights by building a lattice of deep neural networks that reflects the structure of taxonomies. They call this Semantic Deep Learning.

Thank You! david.newman@wellsfargo.com linkedin.com/in/davidsnewman1