Context Observation Measurement Relationship Entity Characteristic Value Standard hasContextRelationship ofEntity hasValue ofCharacteristic usesStandard hasMeasurement hasContext hasContextObservation 0..* * * * * Annotation Examples (12/18/2009) OBOE Conceptual Model

Annotation Examples (12/18/2009) Annotation Syntax observation "o1” entity ”TemporalRange” measurement "m1” characteristic ”Year” standard ”DateTime” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” characteristic “TaxonomicTypeName” standard “ITIS” measurement "m4” characteristic “EntityName” standard “LocalTreeNames” context observation “o1” relationship “Within” map “yr" to “m1” map “diam” to “m2" if diam > 0 map “spec" to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” * Code exists to read/write annotations using this XML format

Annotation Examples (12/18/2009) AnnotationDataset Materialize Define (view def.) OBOE Model (individuals/triples) OBOE Concepts instantiates uses terms from observation-based representation of Query* * Conceptually, we want to query datasets via annotations

Annotation Examples (12/18/2009) yrspecsppdbh 20071piru piru abba33.2 observation "o1” entity ”TemporalRange” measurement "m1” characteristic ”Year” standard ”DateTime” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” characteristic “TaxonomicTypeName” standard “ITIS” measurement "m4” characteristic “EntityName” standard “LocalTreeNames” context observation “o1” relationship “Within” map “yr" to “m1” map “dbh” to “m2" if dbh > 0 map “spec" to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset * Basic idea: go row-by-row through dataset, generating individuals/triples : Obs : Meas : Year : DateTime 2007 : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : Year : DateTime 2008 : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : Year : DateTime 2008 : Obs : Meas : EntN : LocTN. 2 : Meas : TaxN : ITIS Abie. : Meas : DBH : Centim : Tree : Tempral Range : Tree : Tempral Range : Tree : Tempral Range hasContext

Annotation Examples (12/18/2009) yrspecsppdbh 20071piru piru abba33.2 observation "o1” entity ”TemporalRange” measurement "m1” characteristic ”Year” standard ”DateTime” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” characteristic “TaxonomicTypeName” standard “ITIS” measurement "m4” characteristic “EntityName” standard “LocalTreeNames” context observation “o1” relationship “Within” map “yr" to “m1” map “dbh” to “m2" if dbh > 0 map “spec" to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset Same Trees!! (both have name = 1) Same Year and year observation!! : Obs : Meas : Year : DateTime 2007 : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : Year : DateTime 2008 : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : Year : DateTime 2008 : Obs : Meas : EntN : LocTN. 2 : Meas : TaxN : ITIS Abie. : Meas : DBH : Centim : Tree : Tempral Range : Tree : Tempral Range : Tree : Tempral Range hasContext

Annotation Examples (12/18/2009) yrspecsppdbh 20071piru piru abba33.2 observation "o1” distinct yes entity ”TemporalRange” measurement "m1” key yes characteristic ”Year” standard ”DateTime” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” characteristic “TaxonomicTypeName” standard “ITIS” measurement "m4” key yes characteristic “EntityName” standard “LocalTreeNames” context observation “o1” relationship “Within” map “yr" to “m1” map “dbh” to “m2" if dbh > 0 map “spec" to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset : Obs : Meas : Year : DateTime 2007 : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : EntN : LocTN. 1 : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : Year : DateTime 2008 : Obs : Meas : EntN : LocTN. 2 : Meas : TaxN : ITIS Abie. : Meas : DBH : Centim : Tree : Tempral Range : Tree : Tempral Range Every observation has an implicit “distinct” attribute (set to “no”) … and every measurement has an implicit “key” attribute (set to “no”) hasContext

Observation measurement keys – Like a primary key constraint – States that observation instances with the same measurement key values are of the same entity instance – Does not imply the same observation instance, unless the observation is declared distinct – All key measurements of an observation together form the primary key Distinct observations – Only applies if at least one key measurement is defined – States that observation instances with the same entity instance are the same observation instance Annotation Examples (12/18/2009)

pltsppdbh Apiru35.8 Apiru36.2 Bpiru33.2 observation "o1” distinct yes entity ”Plot” measurement "m1” key yes characteristic ”EntityName” standard ”Nominal” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” key yes characteristic “TaxonomicTypeName” standard “ITIS” context observation “o2” relationship “Within” map “plt" to “m2” map “dbh” to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset : Obs : Meas : EntN : Nominal A : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : EntN : Nominal B : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Tree : Plot hasContext Here we don’t have unique ids for trees But, assume each spp name within a plot uniquely identifies a tree … i.e., at most one tree of a particular type was measured (possibly multiple times) in each plot

Annotation Examples (12/18/2009) pltsppdbh Apiru35.8 Apiru36.2 Bpiru33.2 observation "o1” distinct yes entity ”Plot” measurement "m1” key yes characteristic ”EntityName” standard ”Nominal” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” key yes characteristic “TaxonomicTypeName” standard “ITIS” context observation “o2” relationship “Within” map “plt" to “m2” map “dbh” to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset : Obs : Meas : EntN : Nominal A : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : EntN : Nominal B : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Tree : Plot hasContext The Tree entity instance should depend on the plot it is in!!! (context)

Annotation Examples (12/18/2009) pltsppdbh Apiru35.8 Apiru36.2 Bpiru33.2 observation "o1” distinct yes entity ”Plot” measurement "m1” key yes characteristic ”EntityName” standard ”Nominal” observation "o2” entity “Tree” measurement "m2" precision: "0.1” characteristic “DBH” standard ”Centimeter” measurement "m3” key yes characteristic “TaxonomicTypeName” standard “ITIS” context identifying yes observation “o2” relationship “Within” map “plt" to “m2” map “dbh” to “m3” map “spp" to “m4" if spp == “piru” value=“Picea rubens” map “spp" to “m4" if spp == “abba” value=“Abies balsamea” AnnotationDataset : Obs : Meas : EntN : Nominal A : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Obs : Meas : EntN : Nominal B : Obs : Meas : TaxN : ITIS Picea. : Meas : DBH : Centim : Tree : Plot hasContext Every context relationship has an “identifying” qualifier (set to “no”) Uniqueness within context observation Similar to a weak-entity constraint (ER) : Tree

Representing instances … Annotation(AnnotId, Resource) Observation(ObsId, AnnotId, EntId) Measurement(MeasId, ObsId, MeasType, Value) Context(ObsId1, ObsId2, Rel) Relationship(RelId, RelType) Entity(EntId, EntType) This could be queried itself and/or mapped to triples Note that ObsIds are unique across annotations Context.ObsId’s must be for the same annotation Annotation Examples (12/18/2009) * Simple relational schema for OBOE models (individuals/triples)

Representing annotations … Annotation(AnnotId, Res) ObservationType(ObsTypeId, AnnotId, EntType, Unique) MeasType(MeasTypeId, ObsTypeId, CharType, StdType, ProtType, Precision, Value, Key) ContextType(ObsTypeId1, ObsTypeId2, RelType) Map(ResAttribute, MeasType, Condition, Value) Annotation Examples (12/18/2009)

Materialization Algorithm Start with simple case of no key, unique, and identifying constraints Add these incrementally Define algorithm so that it works one row at a time Can we also define the algorithm as a view, to enable querying through views (rewriting)? – This was what the prolog code did … Annotation Examples (12/18/2009)

MapRow(Row : Dom(A 1 )×Dom(A 2 ) ⋯ Dom(A n ), AnnotId : int) let D = [] /* D is a dictionary (ObsTypeId, Keys) → ObsId */ foreach A i, MeasTypedId, Cond, Val in Map where satisfies(Row, A i, Cond) select MeasTypeId, ObsTypeId, Std, Key from MeasType for MeasTypeId let MeasId = CreateNewId() let Keys = GetObsTypeKeys(ObsTypeId, Row, AnnotId) let ObsId = CreateObsId(ObsTypeId, Keys, D, Row, AnnotId) D = D ∪ [(ObsTypeId, Keys) → ObsId] let A i Val = GetValue(Row[A i ], Cond, Val) insert MeasId, ObsId, Std, A i Val into Meas end MapRow Annotation Examples (12/18/2009)