Visual exploratory data analysis: Big Data Analysis and Data Mining, Paris, 7-8 September, 2017 Visual exploratory data analysis: data embedding (DE) & graph visualization (GV) Witold Dzwinel

Visual data mining (VDM) [Felizardo et al. 2012] Hypotheses verification ML algorithms adaptation and tuning Matching the best data representation

The problem How to preserve in 2-D the main topological features of these data representations? the neigborhood (fine grained) the cluster structure (coarse grained) Visualization of two data representations in 2-D (3-D) Euclidean space: high dimensional data (HD) ↔ M, N-D feature vectors Y (data embedding DE) complex networks G(V,E,W) (graph visualization GV) N, M, #V, #E → are huge

HD Embedding: Y → X dissimilarity matrix representation of data

Bottlenecks Storage and Computational complexities Manifold problem Curse of dimensionality (O(M2) & O(MlogM) e.g. based on stochastic neighbor embedding: bh-SNE, q-SNE, w-SNE, LargeVis etc. and forceatlas based GV algorithms )

Computational complexity Existing VE and GV methods based on distances are strongly overdetermined. in 2-D at least: ~2 ◦ M distances can define the stable solution for rigid graphs Which distances?????

Manifold problem

HDD ↔ graph representation k-nearest neighbor graphs ↔ DE k-NN graph is not rigid!! Other distances are necessary for k-NN graph visualization

Computational complexity We propose a drastic simplification of distances matrix i (i data vector or graph vertex), find the small sets of for DE: NN(i) of the k-nearest and RN(i) r - random neighbors for GV: all-connected NN(i) and r-disconnected RN(i) vertices We assume that k+r ~ N (dimensionality of Y) It gives O(M) linear-time & memory complexity of both DE and GV algorithms

Curse of dimensionality 1. Increase the contrast between the nearest (connected) and the random neighbors (vertices) 2. Use force-directed method for minimization of the stress function

Examples: MNIST MNIST N=784 M= 70000 C= 10 T=11 min T=30 min T<1 min MNIST N=784 M= 70000 C= 10 Well balanced set of gray-scale handwritten digit images (http://yann.lecun.com/exdb/mnist/).

Examples: NORB (small) M=43600, N=2048 The NORB dataset (NYU Object Recognition Benchmark) contains stereo image pairs of 50 uniform-colored toys under 18 azimuths, 9 elevations, and 6 lighting conditions

DBN - autoencoder 30 min [Snoek et al., 2012]

Autoencoder, Snoek et al.2012 NORB: 1m Van der Maaten, 2014

Examples: Reuters t-SNE (M~58000, N=2000) 5h Reuters 2000 266931 8 Strongly imbalanced text corpus known as RCV1. We used a subset of this repository consisting of 8 clusters (http://about.reuters.com/researchandstandards/corpus/).

Examples: Reuters 5 min

Complex networks visualization Historic articles from Wikipedia and links between them.

Fine structure of historic graph

Big graphs (social networks)

State_of_the_art 2167.88 sec. 250 sec. http://yifanhu.net/index.html AT&T Labs -- Research

Internet topology

Internet topology

Patent database

Patents database

Conclusions 1. Low memory complexity O(nM) 2. Low computational complexity O((n+r)M) 3. High level of parallelization (PM) 4. Easy implementation on Big data platforms (Hadoop, Apache Spark) 5. Near neighbors (NeN) instead of NN! 6. Big graphs visualization

We have ... 1. Desktop versions with GUI for interactive visualization of large HD data (IVTA) and GV (IVGA). 2. Ultrafast methods for k-NN neighbor search implemented in CUDA. 3. GV parallel (CUDA, MPI) software employing B- matrices and algebraic graph representations. 4. Feature extraction software (CUDA) based on DBNs.

Future work 1. Developing VE and GV systems for distributed data visualization involving big data architectures (Hadoop, Spark …). 2. Employing algebraic descriptors for data analytics, and new data manipulation techniques 3. Using our DBN software for data preprocessing, i.e., feature extraction for big distributed data repositories

Acknowledgments. This research is supported by the Polish National Center of Science (NCN) project #DEC-2013/09/B/ST6/01549.