Speeding Up Batch Alignment of Large Ontologies Using MapReduce Uthayasanker Thayasivam and Prashant Doshi Dept. of Computer Science University of Georgia

Introduction  Ontology: formalize the knowledge of a domain by means of defining concepts and properties that relate them

Introduction: Ontology Alignment

Problem Definition: Ontology Alignment find a set of correspondences between two ontologies O 1 = and O 2 =. The ontology alignment problem:

Ontology Alignment Challenges  Improving the Alignment Quality  Structural & lexical disparity  Improving the Alignment Efficiency  Quickly producing quality alignment  Improving the Scalability Ontology Sizes Efficiency / Quality Resources Efficiency / Quality

Space of Alignments m11m12…m1|V 2 | m21m22…m2|V 2 | ………… m|V 1 |1m|V 1 |2…m|V 1 ||V 2 | x1 x2.. x|V 1 | y1y2…y|V 2 | Alignment between many-to-many Alignment Space Size: one-to-manyone-to-one Evaluating An Alignment: Cartesian Product of entities

Space of Alignments m11m12…m1|V 2 | m21m22…m2|V 2 | ………… m|V 1 |1m|V 1 |2…m|V 1 ||V 2 | x1 x2.. x|V 1 | y1y2…y|V 2 | Alignment between many-to-many Alignment Space Size: one-to-manyone-to-one Evaluating An Alignment: Cartesian Product of entities Bipartite graph

Large Ontology Matching  Reduction of alignment space  Early pruning of dissimilar element pairs  aflood (Hanif and Masaki ‘09)  Partition based matching  Falcon-AO (Jian et. al. ‘05)  Parallel matching  MapPSO (Bock and Hettenhausen ‘10)  VDoc+ (Zhang ‘12) O2O2 O1O1 P11P11 P12P12 P13P13 P21P21 P22P22 P23P23 4 blocks

Batch Alignment of Large Ontologies  Scalability is challenging  OAEI Very Large Biomedical Ontology Track  8 out of 21 tools completed  Ontology repositories (e.g., NCBO at Stanford)  Batch alignment of ontologies  New ontologies posted  Ontologies get updated Approach allows any alignment algorithm to be utilized on a MapReduce architecture

Contributions: Batch Alignment of Large Ontologies General & Novel Approach To speed up batch alignment of large ontologies using MapReduce  No impact to alignment quality for some algorithms  Benefits ontology repositories

MapReduce Framework

output Key-> Value Key-> Key-> Output Value Key identifies a subproblem

MapReduce Framework O1O1 O2O2 O11O11 O21O21 O31O31 O12O12 O22O22

O1O1 O2O2 O11O11 O21O21 O31O31 O12O12 O22O22 …

O1O1 O2O2 O11O11 O21O21 O31O31 O12O12 O22O22

O1O1 O2O2 O11O11 O21O21 O31O31 O12O12 O22O22

Mapper & Reducer Algorithms

Identifying Alignment Subproblems  Approach: Hamdi et al  Identify anchors: entity pairs with identical names or labels  Cluster concepts around the anchors  Using structural neighborhood Entities from one cluster are predominantly in correspondence with entities in one other cluster

Merging Subproblem Alignments

Performance Evaluation  Datasets  Conference track from OAEI (120 pairs)  Large ontologies from OAEI (SNOMED, NCI,... 5 pairs)  New biomedical ontology testbed (50 pairs from NCBO)  Algorithms  Compare F-measure & runtime  Default setup on a single node  MapReduce setup using Hadoop (12 nodes each with 24 2GB & 2GHz Intel Xeon processors) Falcon-AOOptima+LogMapYAM++

Results – 3 Datasets Algos.Speedup Confer.LargeOAEIBiomed Falcon2155 LogMap9165 Optima Yam ConferenceLarge OAEI Biomedical

Results – Large OAEI ontologies  Conference Track  No partitioning  No change in output Ontology Pairs MapRed./Def. Falcon-AO MapReduce LogMap MapRed./Def. Optima+ MapReduce YAM++ Default LogMap Default YAM++ PRFPRFPRFPRFPRFPRF mouse, human STW, TheSoz fma,nci fma, snomed snomed, nci  Other Datasets  LogMap & Yam++ :  Tradeoff is in the alignment quality  Falcon-AO & Optima+:  No change in output

Speedup with # of nodes in the Hadoop cluster

Discussion  First inter-matcher parallelization approach  Especially using MapReduce  Exhibits significant speedup for batch alignment  Some algorithms may find small reduction in alignment quality due to the partitioning  Significant speedup for single ontology pair  Falcon-AO, Optima+ & YAM++  Any alignment algorithm can fit in our framework

Thank you Questions ?

Parallel Alignment of Large Ontologies on A Computing Cluster  Current Divide and Conquer Approaches  Heavily rely on structure  Size based partitioning techniques are not effective  Current Parallel Matching algorithms  Parallelize the process within the algorithms  Do not support multi node – cluster architecture