Seeing Things in the Clouds over concept lattices with tag clouds browsing semi-structured data Bernd Fischer object attribute context table relation Galois connection knowledge discovery mining software repositories join focus visualization information retrieval meet navigation Stellenbosch Computer Science

How do you find stuff on the Internet? concept-based browsing query

How do you find stuff on the Internet? Yikes! results!

How do you find stuff on the Internet? concept-based browsing query     lattice

How do you find stuff you didn’t look for? Retrieval: extract objects that satisfy a pre-defined criterion query describes criterion main operation is matching: check satisfaction against query main goal is precision: show only relevant objects Browsing: spontaneously explore a collection focus describes current position and selection main operation is navigation: change the focus main goal is recall: show all relevant objects

How do you browse? (hierarchical) navigation structure focus selection

How do you browse semi-structured data? What is semi-structured data? What is structured data? Structured data has a very high degree of regularity... an explicit, tight format (schema) Typical examples: spreadsheets relational databases (SQL: structured query language)

How do you browse semi-structured data? What is semi-structured data? Semi-structured data contains both free-text and formatted fields... has large structural variance... is implicitly formatted Typical examples: product reviews newspaper articles + meta-data revision control logs

Approach: find a suitable abstract data representation –bag-of-words, graphs, binary relations, RDF triples, XML,... find a suitable hierarchy –metric spaces, graphs, concept lattices,... find a suitable visual representation –lists, graphs, tag clouds, city scapes,... find a navigation algorithm How do you browse semi-structured data?

How do you represent data? Structured data is represented by n-ary relations or tables: each object becomes a row each column represents an attribute type text remains unstructured authortitleyearvenue FischerSpecification-based browsing J. ASE van ZijlSupernondeterministic finite CIAA

How do you represent data? Structured data is represented by n-ary relations or tables: each object becomes a row each column represents an attribute type text remains unstructured set-valued attributes require normalization authortitleyearvenue FischerSpecification-based browsing J. ASE van ZijlSupernondeterministic finite CIAA GreeneConceptCloud: A Tag-cloud FSE FischerConceptCloud: A Tag-cloud FSE

How do you represent data? Structured data is represented by n-ary relations or tables: each object becomes a row each column represents an attribute type text remains unstructured set-valued attributes require normalization Semi-structured data can be represented by binary relations: text is split into words each occurring value and word becomes an attribute build context table: add cross if attribute applies to object –word appears in document, meta-data, references... idtitleyearvenue 08Specification-based browsing J. ASE 15Supernondeterministic finite CIAA 42ConceptCloud: A Tag-cloud FSE idauthor 08Fischer 15van Zijl 42Greene 42Fischer Greenevan Zijlbrowsingtag ××× 15 ×× 42 ×××××

How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ×××××

How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) common attributes: α(O) = { a ∈ A | ∀ o ∈ O : o ~ a } α({08, 42} = FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× α({08, 42} = {Fischer, browsing}

How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) common attributes: α(O) = { a ∈ A | ∀ o ∈ O : o ~ a } common objects: ω(A) = { o ∈ O | ∀ a ∈ A : o ~ a } concept: (O, A) s.t. α(O) = A ∧ ω(A) = O FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× α({08, 42} = {Fischer, browsing} ω({Fischer, browsing} FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× ω({Fischer, browsing} = {08, 42} extentintent

FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) common attributes: α(O) = { a ∈ A | ∀ o ∈ O : o ~ a } common objects: ω(A) = { o ∈ O | ∀ a ∈ A : o ~ a } concept: (O, A) s.t. α(O) = A ∧ ω(A) = O {08} {F,browsing,’00} {42} {F,G,browsing,tag,’14} {08, 42} {F,browsing} {42} {tag}   extentintent

FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) common attributes: α(O) = { a ∈ A | ∀ o ∈ O : o ~ a } common objects: ω(A) = { o ∈ O | ∀ a ∈ A : o ~ a } concept: (O, A) s.t. α(O) = A ∧ ω(A) = O sub-concept ordering: (O ₁, A ₁ ) ≤ (O ₂, A ₂ ) iff O ₁ ⊆ O ₂ iff A ₁ ⊇ A ₂ concept lattice: concepts of a context form a complete lattice {08} {F,browsing,’00} {42} {F,G,browsing,tag,’14} {08, 42} {F,browsing} {42} {tag}  

Are we there yet? Nope. Concept lattices induce enough structure for navigation but too much to show directly!

How do you visualize concept lattices? Approach: don’t show the lattice use concepts as focus visualize only focus concept –but in relation to lattice

How do you visualize concepts? Approach: don’t show the lattice use concepts as focus visualize only focus concept –but in relation to lattice use extent to derive tag cloud

How do you build tag clouds for concepts? What is a tag cloud? visual representation of text data – summarize large data set – emphasize important tags single words or short phrases importance reflected as size – frequency in document – number of tagged items – number of page hits different layout methods

How do you build tag clouds for concepts? intent looks like tag cloud but is common to all objects ⇒ all tags same size instead: collect all attributes from all objects in extent –can be expressed in concept lattice: –also add extent via object identifiers intent shown as largest tags –smaller tags are related information {08, 42} {Fischer,browsing} FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× browsing Fischer Greene tag  

The ConceptCloud Browser by: Gillian Greene, US file message date author controls

The ConceptCloud Browser most prolific contributor

How do you navigate with tag clouds? Navigation modes: refinement: narrow the selection –select a new tag widening: extend the selection –remove a selected tag

How do you navigate with concept lattices? Navigation modes: refinement: narrow the selection –select a new tag: f’ = f ∧ δ(t) widening: extend the selection –remove a selected tag (ω({t}), α(ω({t}))) if t ∈ A δ(t) = (α(ω({t})), ω({t})) if t ∈ O focus concept tag concept focus concept focus concept

How do you navigate with concept lattices? Navigation modes: refinement: narrow the selection –select a new tag: f’ = f ∧ δ(t) widening: extend the selection –remove a selected tag: f’ = f ∨ δ(t) (ω({t}), α(ω({t}))) if t ∈ A δ(t) = (α(ω({t})), ω({t})) if t ∈ O tag concept focus concept tag concept focus concept focus concept focus concept

How do you navigate with concept lattices? Navigation modes: refinement: narrow the selection –select a new tag: f’ = f ∧ δ(t) widening: extend the selection –remove a selected tag: f’ = f ∨ δ(t) f’ = ∧ i ∈ π(f) \ {t} δ(i) –join-based widening can be useful as well (ω({t}), α(ω({t}))) if t ∈ A δ(t) = (α(ω({t})), ω({t})) if t ∈ O tag concept focus concept tag concept focus concept  

Navigation in the ConceptCloud Browser

The Percept Browser by: Carl Kritzinger, Fireworks

Semi-structured data is common but hard to analyze Tag clouds are a good visualization approach and the combination with concept lattices makes it easy to navigate and find related information Flexible approach, generic tool –different data sets –different types of contexts ( ⇒ different types of analysis) Scalability –DBLP, IMDb, Wikipedia? Customizability –context extraction –tool scripting Conclusions & Future Work

Semi-structured data is common but hard to analyze Tag clouds are a good visualization approach and the combination with concept lattices makes it easy to navigate and find related information Flexible approach, generic tool –different data sets –different types of contexts ( ⇒ different types of analysis) Scalability –DBLP, IMDb, Wikipedia? Customizability –context extraction –tool scripting

How do you find stuff you didn’t look for? Retrieval: extract objects that satisfy a pre-defined criterion query describes criterion main operation is matching: check satisfaction against query main goal is precision: show only relevant objects Browsing: spontaneously explore a collection focus describes current position and selection main operation is navigation: change the focus main goal is recall: show all relevant objects “That’s funny...” “Bingo!”

How do you find hierarchy in relations? Formal concept analysis: formal context: (O, A, ~) common attributes: α(O) = { a ∈ A | ∀ o ∈ O : o ~ a } common objects: ω(A) = { o ∈ O | ∀ a ∈ A : o ~ a } concept: (O, A) s.t. α(O) = A ∧ ω(A) = O FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× α({08, 42} = {Fischer, browsing} ω({Fischer, browsing} FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× FischerGreenevan Zijlbrowsingtag ××× 15 ×× 42 ××××× ω({Fischer, browsing} = {08, 42} extentintent “That’s funny...”

What is a tag cloud? How do you build tag clouds for concepts?

What is a tag cloud? visual representation of text data – summarize large data set – emphasize prominent tags single words or short phrases importance reflected as size – frequency in document – number of tagged items – number of page hits different layout methods

How do you navigate? Navigation modes: refinement: narrow the selection widening: extend the selection

How do you navigate with concept lattices? Navigation modes: refinement: narrow the selection –select a new tag: f’ = f ∧ δ(t) widening: extend the selection –remove a selected tag: f’ = f ∨ δ(t) (ω({t}), α(ω({t}))) if t ∈ A δ(t) = (α(ω({t})), ω({t})) if t ∈ O tag concept focus concept tag concept focus concept focus concept   focus concept

How do you navigate with concept lattices? Navigation modes: refinement: narrow the selection –select a new tag: f’ = f ∧ δ(t) widening: extend the selection –remove a selected tag: c’ = ∧ i ∈ π(c) \ {t} δ(i) –join-based widening can be useful as well (ω({t}), α(ω({t}))) if t ∈ A δ(t) = (α(ω({t})), ω({t})) if t ∈ O tag concept focus concept tag concept focus concept

Navigation in the ConceptCloud Browser