Sangam: A Transformation Modeling Framework Kajal T. Claypool (U Mass Lowell) and Elke A. Rundensteiner (WPI)

The Era of Electronic Information Age of electronic information Age of electronic information Data exists in many different formats Data exists in many different formats  Different data models  Different schemas Users need to Users need to  Publish data in many formats  Integrate and transform data  Query and expect results in common format Underlying problem Underlying problem  Need to express mapping of data from one format to another  Need to perform transformation of data based on expressed mappings.

Schema Translation: State of the Art Naïve approach [Zhang01,Shanmugasundram99] Naïve approach [Zhang01,Shanmugasundram99]  Write specific programs to translate data from one format to another  Examples:  Algorithms: translate XML documents into relational data [zhang01,shanmugasundram99]  Latex2html: convert latex into HTML documents

Schema Translation: State of Art Matching approach [milo98] Matching approach [milo98]  Automatically discover the semantic correspondences between two schemas  Generate translations based on discovered matches Modeling approach [bernstein00,atzeni96] Modeling approach [bernstein00,atzeni96]  Transform local schema into common data model  Translation language to express mappings between schemas in middle layer

The Sangam Framework Goals: Flexible, extensible, and re-usable Goals: Flexible, extensible, and re-usable Allow users to: Allow users to:  Explicitly model translations between schemas  Compose translations from an existing library of modeled translation patterns  Choose from a library of translation operators  Generate translation model from based on schema match process  For all modeled translations: transform the data based on translation

Overview of Sangam Framework Matcher Schema S1 Schema S2 Matches Transformation Framework Transformation Model Pattern Interface TranInterfaceTranInterface User feedback Displayed to User Evaluator Data D2 Data D1 Transformed Data Transformed Schema Transform- ation Patterns User Input System generated output System Input Legend : ToolSet

Outline Sangam graphs Sangam graphs Cross algebra operators Cross algebra operators Composition techniques Composition techniques Cross algebra graphs Cross algebra graphs Execution strategies Execution strategies Architecture Architecture Conclusions Conclusions

Sangam Graphs Sangam Sangam 1. Common data model:  Sangam graph model 2. Translation language  Algebra-based RDB Import XML Export Sangam graph Cross Algebra for translation

Requirements for a Common Data Model Graph-based Graph-based  Common denominator for most data models Expressiveness Expressiveness  Represent schemas from different data models Fundamental constraints Fundamental constraints  Represent constraints such as quantifier, order and key constraints Existing data models not completely suitable Existing data models not completely suitable  Relational, and OO cannot represent order in clean manner  XML (older spec) can not represent key constraints

Sangam Graph Model Satisfies requirements Satisfies requirements  Graph based  Based on SIGs[ Miller93 ]  Can model schemas from different data models  Can represent quantifier, order, key and foreign key constraints Graph Graph  Nodes represent entities  Eg. Relation, attribute, element  Edge relationships between them  Eg. Containment relationship between relation and attribute

<!ATTLIST item id ID #REQUIRED featured CDATA #IMPLIED> Example: Sangam Graph

Cross Algebra Requirements for a transformation language Requirements for a transformation language  Node and edge manipulations  Minimal granularity  Eg: Relation has name and attributes  Allow composition Unique contribution: algebra-based translation language Unique contribution: algebra-based translation language  Translate from one Sangam graph to another Four Operators: Four Operators:  Represent core set of graph linear transformations [ GBook00 ]  Can be composed to formulate more complex operations such as a join operation – not our focus

Cross Algebra Operators cross, connect, smooth and subdivide cross, connect, smooth and subdivide

Composition of Operators Context Dependency Context Dependency  Output = union of output of all operators Derivation Derivation  Output = output of root operator

Cross Algebra Graphs

Evaluating a Cross Algebra Graph Function EvaluateCAT (input: Operator op, Sangam Graph G, output: Sangam Graph G’) { if (!op.hasChildren ()) G’  p.evaluate (G, G’) op.markDone () out  G’ // cached local output return G’ while (op.hasChildren()) { operator opC  op.getNextChild () if (e: == derivation) G_local  EvaluateCAT (opC, G, G’) G’  op.evaluate (G_local, G’) op.markDone () out  G’ // cached local output return G’ elseif (e: == context dependency) G_local  EvaluateCAT (opC, G, G’) G’_local  op.evaluate (G, G’) G’  G_local U G’_local op.markDone () out  G’_local // local cached output return G’

Architecture

Conclusions Sangam: Flexible, extensible, and re-usable transformation modeling framework Sangam: Flexible, extensible, and re-usable transformation modeling framework Key contributions: Key contributions:  Concept of Sangam  Cross Algebra  An algebra for modeling linear transformations  Composition techniques to deal with finer granularity  Evaluation techniques Future work Future work  Modeling the data model layer  Optimization of evaluation strategies  Non-linear transformations