1 Doing Ontology Over Images Barry Smith

What ontologies are for

3 what molecular function ? what disease process ? need for semantic annotation of data

4 through labels (nouns, noun phrases) which are algorithmically processable

5 natural language labels to make the data cognitively accessible to human beings

6 compare: legends for maps

7 compare: legends for cartoons

8

9 ontologies are legends for data

10 ontologies are legends for images

11 what lesion ? what brain function ?

12 x i = vector of measurements of gene i k = the state of the gene ( as “on” or “off”) θ i = set of parameters of the Gaussian model... ontologies are legends for mathematical equations

13 The OBO Foundry Idea MouseEcotope GlyProt DiabetInGene GluChem sphingolipid transporter activity

14 annotation using common ontologies yields integration of databases MouseEcotope GlyProt DiabetInGene GluChem Holliday junction helicase complex

15 annotation using common ontologies can yield integration of image data

16 annotation using common ontologies can support comparison of image data

17 truth

18 simple representations can be true

19 there are true cartoons

20 a cartoon can be a veridical representation of reality

21 Cartographic Projection

22 maps may be correct by reflecting topology, rather than geometry

23 a fully labeled image can be an even more veridical representation of reality an image can be a veridical representation of reality

24

25

26 cartoons, like maps, always have a certain threshold of granularity

27 grain resolution

28 grain resolution serves cognitive accessibility we transform true images into true cartoons

29 there are also true cartoon sequences

30

31 Pathway diagrams are annotated dynamic cartoons

32 pathways can be represented at different levels of granularity

33 the jaw

34

35 Joint capsule Netter

36 Mandible and condyle movement

37 Condyle position in fossa wrt location of disc

38 TMJ in jaw open and closed positions

39 Parts 1 head of condyle F 2 neck of condyle F 3 disc B 4 retrodiscal tissue B 7 articular eminence F 8 zygomatic arch F 10 upper head of lateral pterygoid muscle F 11 lower head of lateral pterygoid muscle F Holes 5 lower joint compartment B 6 upper joint compartment B Holes and Parts

40 ANTERIOR Temporomandibular Joint (TMJ) from Thomas Bittner and Louis Goldberg, KR-MED 2006

41 adjacency relations No connectedness Only (temporary) adjacency Connectedness adjacency graph Adjacency relations

42 Frames of reference Rigid = do not change shape (bones) A B C D E F The extension of the axis of the condyle intersects the fossa in region D

43 instances vs. types

44 two kinds of annotations

45 names of instances

46 names of types

47 pathway maps are representations of complexes of types

48 molecular images and radiographic images are representations of instances

49 MIAKT system

50

51

52

53

54 Patient #47920

55

56 Mammography #31667

57 Mammography #31667 Medical-Image #44922

58 MRI-Exam #32388 Medical-Image #44922 Mammography #31667 Patient #47920 Breast #1388 Abnormality #86023

59 SNAP and SPAN in brain imaging SNAP CT Computer Tomography PET Positron emission tomography SPECT Single Photon Emission CT MRT fMRT MRS SPAN EKP event correlate potential quantitative electroencephalography qEEG

60 canonicity ! fiatness ! granularity !

61 digital representations of analogue reality