1. Graph Search with Indexing
Hakan Kardes

2. Research Papers Covered in this Talk
X. Yan and J. Han, gSpan: Graph-Based Substructure Pattern Mining, ICDM'02
X. Yan and J. Han, CloseGraph: Mining Closed Frequent Graph Patterns, KDD'03
X. Yan, P. S. Yu, and J. Han, Graph Indexing: A Frequent Structure-based Approach, SIGMOD'04 (also in TODS’05, Google Scholar: ranked #1 out of 63,300 entries on “Graph Indexing”)
X. Yan, P. S. Yu, and J. Han, “Substructure Similarity Search in Graph Databases”, SIGMOD'05 (also in TODS’06)
F. Zhu, X. Yan, J. Han, and P. S. Yu, “gPrune: A Constraint Pushing Framework for Graph Pattern Mining”, PAKDD'07 (Best Student Paper Award)
C. Chen, X. Yan, P. S. Yu, J. Han, D. Zhang, and X. Gu, “Towards Graph Containment Search and Indexing”, VLDB'07, Vienna, Austria, Sept. 2007
October 14, 2018

3. Graph, Graph, Everywhere
Aspirin
Yeast protein interaction network
An Internet Web
Co-author network
October 14, 2018

4. Why Graph Mining and Searching?
Graphs are ubiquitous
Chemical compounds (Cheminformatics)
Protein structures, biological pathways/networks (Bioinformactics)
Program control flow, traffic flow, and workflow analysis
XML databases, Web, and social network analysis
Graph is a general model
Trees, lattices, sequences, and items are degenerated graphs
Diversity of graphs
Directed vs. undirected, labeled vs. unlabeled (edges & vertices), weighted, with angles & geometry (topological vs. 2-D/3-D)
Complexity of algorithms: high complexity of current algorithms!
October 14, 2018

5. Outline Graph Search: Querying Graph Databases Graph indexing methods
Path based indexing
gIndex
Experiments
Conclusion
October 14, 2018

6. Graph Search: Querying Graph Databases
Given a graph database and a query graph, find all graphs containing this query graph
query graph
graph database
October 14, 2018

7. Scalability Issue Sequential scan Disk I/O
Query graph
Sequential scan
Disk I/O
Subgraph isomorphism testing
An indexing mechanism is needed
DayLight: Daylight.com (commercial)
GraphGrep: Dennis Shasha, et al. PODS'02
Grace: Srinath Srinivasa, et al. ICDE'03
Sample database
Two issues have to be solved: Disk I/O for false positive and subgraph isomorphism testing for false positive.
Therefore, an indexing mechanism is needed.
The list just shows a partial list of the previous research.
DayLight system is a commercial product targeted for chemical information system. The following is copied from
Once a large graph database is formed, how can we do database management, and further data warehouse and data mining.
2. GraphGrep uses a path-based approach to do graph indexing. Its implementation is available publicly.
3. Grace develops a hierarchical vector space method. It tries to extract features from the graphs in a hierarchical way.
DayLight System
“At Infinity Pharmaceuticals we use the Daylight toolkits for in-house development and have implemented DayCart (as a central component of our chemical registration system. This system has been fully integrated within all aspects of our drug discovery processes and software applications. We have successfully "pushed" the registration process to the chemists; they can quickly and easily register compounds within the context of their electronic notebooks. This has provided a significant enhancement to both the efficiency of data capture and the quality of the data within the chemistry database(s). DayCart integrates well into our overall application and data environment providing a flexible chemical data model which facilitates a data architecture that is tailored to Infinity's processes and scientific approach while at the same time allows speed and scalability. The benefit delivered to Infinity is measured in the strong capability to rapidly and efficiently develop intellectual property and access corporate knowledge effectively. “
October 14, 2018

8. Two steps in processing graph queries
Framework
Two steps in processing graph queries
Step 1. Index Construction
Enumerate structures in the graph database, build an inverted index between structures and graphs
Step 2. Query Processing
Enumerate structures in the query graph
Calculate the candidate graphs containing these structures
Prune the false positive answers by performing subgraph isomorphism test
October 14, 2018

9. Indexing Strategy Query graph (Q) Graph (G)
If graph G contains query graph Q, G should contain any substructure of Q
Substructure
Our work, also with all the previous work follows this indexing strategy.
Remarks
Index substructures of a query graph to prune graphs that do not contain these substructures
October 14, 2018

10. Cost Analysis Query Response Time Disk I/O time
Isomorphism testing time
Graph index access time
T_io is the time of fetching a graph from a disk, which may result in one disk block IO. I did not show it in the paper. If the graph dataset can not be held in the main memory, then we need also count the IO time
Our goal is to minimize |Cq|
Size of candidate answer set
Remark: make |Cq| as small as possible
October 14, 2018

11. Outline Graph Search: Querying Graph Databases Graph indexing methods
Path based indexing
gIndex
Experiments
Conclusion
October 14, 2018

12. Path-Based Approach Sample database (a) (b) (c) Paths
0-length: C, O, N, S
1-length: C-C, C-O, C-N, C-S, N-N, S-O
2-length: C-C-C, C-O-C, C-N-C, ...
3-length: ...
Built an inverted index between paths and graphs
October 14, 2018

13. Problems of Path-Based Approach
Sample database
(a)
(b)
(c)
Query graph
Only graph (c) contains this query graph. However, if we only index paths: C, C-C, C-C-C, C-C-C-C, we cannot prune graph (a) and (b).
Only graph (C) contains the query graph. Unfortunately, if we use path only, we cannot prune graphs (a) and (b)
The question is why we use path. The reason is path is very simple, easier to manipulate.
October 14, 2018

14. gIndex: Indexing Graphs by Data Mining
Another methodology on graph index:
Identify frequent structures in the database, the frequent structures are subgraphs that appear quite often in the graph database
Prune redundant frequent structures to maintain a small set of discriminative structures
Create an inverted index between discriminative frequent structures and graphs in the database
The reason of doing frequent graphs is just because the number of frequent graphs is much less than the number
of all subgraphs in the graph database although this number is still too large for indexing.
October 14, 2018

15. IDEAS: Indexing with Two Constraints
discriminative (~103)
frequent (~105)
structure (>106)
October 14, 2018

16. Why Discriminative Subgraphs?
Sample database
(a)
(b)
(c)
All graphs contain structures: C, C-C, C-C-C
Why bother indexing these redundant frequent structures?
Only index structures that provide more information than existing structures
October 14, 2018

17. Discriminative Structures
Pinpoint the most useful frequent structures
Given a set of structures f1, f2, …, fn and a new structure x, we measure the extra indexing power provided by x,
P (x|f1, f2, …, fn), where fi is contained in x
When P is small enough, x is a discriminative structure and should be included in the index
Index discriminative frequent structures only
Reduce the index size by an order of magnitude
October 14, 2018

18. Why Frequent Structures?
We cannot index (or even search) all of substructures
Large structures will likely be indexed well by their substructures
Size-increasing support threshold
minimum
support threshold
support
size
October 14, 2018

19. Why Frequent Structures?
We cannot index (or even search) all of substructures
Large structures will likely be indexed well by their substructures
Size-increasing support threshold
minimum
support threshold
support
size
October 14, 2018

20. Outline Graph Search: Querying Graph Databases Graph indexing methods
Path based indexing
gIndex
Experiments
Conclusion
October 14, 2018

21. Experimental Setting
The AIDS antiviral screen compound dataset from NCI/NIH, containing 43,905 chemical compounds
Query graphs are randomly extracted from the dataset.
GraphGrep: maximum length (edges) of paths is set at 10
gIndex: maximum size (edges) of structures is set at 10
The fingerprint set size is set as large as possible until the program takes 1G memory. Here it is 10K.
Other parameter setting is provided in the paper
October 14, 2018

22. Experiments: Index Size
# OF FEATURES
Remarks:
It shows the raw feature set of path-based approach and structure-based approach. Of course, we can compress the paths into an arbitrary size of index with performance loss. In our experiment setting, we do not compress the path-based index in order to illustrate the best performance it can achieve.
It is also possible to apply discriminative filtering on paths. We have not done experiments on it.
The number of frequent structures or discriminative frequent structures does not change over the different
size of databases.
DATABASE SIZE
October 14, 2018

23. Experiments: Answer Set Size
# OF CANDIDATES
1. That is, the number of graphs containing the query graph is less than 50.
2. Query size are about the number of edges in a query.
3. Answer set size is the number of candidate graphs that may contain the query graph.
4. The actual match is the lower bound that all algorithms can achieve. That is, it is the actual number of graphs that really contain the query graph.
5. The performance of queries whose support is less than 50
QUERY SIZE
October 14, 2018

24. Experiments: Incremental Maintenance
The fact that the number of frequent structures does not change over the different size of databases inspires us to design an algorithm which construct the index incrementally.
2. The black line shows the performance of indexing from the scratch, for example, from a database with 2,000 graphs, 4,000 graphs, 6,000 graphs, 8,000 graphs and 10,000 graphs.
3. The red line shows the performance of incremental indexing, for example, we first built an index for 2,000 graphs, then we add another 2,000 graphs into the database, update the indexing of the original discriminative frequent graphs, then we add another 2,000, … and so on
Frequent structures are stable to database updating
Index can be built based on a small portion of a graph database, but be used for the whole database
October 14, 2018

25. Conclusions Graph indexing has wide applications
Daylight and GraphGrep: path indexing
gIndex: Graph indexing
Frequent and discirminative subgraphs are high-quality indexing fatures
There are some more indexing methods but they are similar to the gIndex
Grafill: Similairty (subgraph) search in graph databases
Graph indexing and feature-based approximate matching
cIndex: Containment graph indexing
A contrast feature-based indexing model
October 14, 2018

26. QUESTIONS