1. Viz: A Visual Analysis Suite for Explaining Local Search Behavior Steven Halim (School of Computing, NUS) Roland Yap Hock Chuan (School of Computing, NUS) Lau Hoong Chuin (School of Information Systems, SMU) User Interface Software and Technology 2006 Montreux, Switzerland http://www.comp.nus.edu.sg/~stevenha/viz

2. Outline Short Background: Local Search (LS) for attacking NP-hard Combinatorial Optimization Problems (COPs), The problem: Understanding and Tuning LS… Our Approach: Explaining Local Search using Visualization (Viz) Viz DEMO Q & A

3. Local Search e.g. Tabu Search, Iterated Local Search, Simulated Annealing …is commonly used for attacking NP-hard (intractable) Combinatorial Optimization Problem (COP): e.g. Traveling Salesman Problem (TSP) Simplified explanation of Local Search: Background – Local Search

4. Local Search algorithms have tunable configurations: Parameter values, Components, and/or Search strategies. These configurations affects its behavior  performance… To make a LS perform well in context is not straightforward. Its heuristic/stochastic behavior is not well understood. Empirical analysis is needed. Background – Tuning Problem

5. Bottleneck !!! Common Practice: Ad Hoc Tuning Implement the local search Evaluate its performance Textually/Statistically… Good or Give up Not good Stop Modify the local search using something?? (“ad hoc”)

6. We propose: Visual Diagnosis Implement the local search Evaluate its performance Using Visualization Tool… Good or Give up Not good Stop Modify the local search after understanding what is happening. The tool helps us to think more systematically.

7. Why Use Visualization? Visualization is a cognitive tool. Human brain is a powerful pattern finding machine. As long as the data is visualized in a way that conform with the rules of human visual perception. Information Visualization: “The use of interactive visual representations of abstract data to amplify cognition.” Facilitates hypothesis formation in empirical analysis. Allows users to answer questions they didn't know they had.

8. Explaining Local Search Behavior via Visualization http://www.comp.nus.edu.sg/~stevenha/viz

9. Explaining Local Search Behavior It is more useful to understand how and why something works, Rather than just simply know that something works… By understanding the underlying principles, We can adjust properly if presented with different scenarios! To understand LS, we need to answer these questions: 1. Does it behave like as what we intended? 2. How good is the local search in intensification? 3. How good is the local search in diversification? 4. Is there any sign of cycling behavior? 5. How does the local search algorithm make progress?

10. Explaining Local Search Behavior 6. Where in the search space does the search spend most of its time? 7. How far is the starting (initial) solution to the global optima/best known solution? 8. Does the search quickly find the global optima/best known solution region or does it wander around in other regions? 9. How wide is the local search coverage? 10. What is the effect of modifying a certain search parameter/component/strategy w.r.t the search behavior? 11. How do two different algorithms compare? The behavior of heuristic and stochastic local search is hard to explain without appropriate tools!

11. How to Understand LS Behavior? Various local search analysis tool: Data Analysis (text-based): Solution Quality Robustness Speed/Running Time Run Time/Length Distribution [Hoos 98; Hoos & Stuetzle, 05] Fitness Landscape (Statistical) Analysis [Jones, 95; Watson et al., 05] Information Visualization: Visualization of Fitness Landscape/properties/search behavior

12. Visualizations for Local Search Existing Visualizations: Objective Value over Time Fitness Distance Correlation [Jones, 95; Watson et al. 05] Problem Specific, e.g. TSP [Klau et al., 02] 2-D Animation of 2 Dimensional Problem [Syrjakow & Szczerbicka, 99] N-to-2-Space Mapping [Kadluczka, 04] Search Trajectory Analysis [Lau et al., 05; Halim et al., 06a,b]

13. Analogy of Search Trajectory Visualization: Finding the Highest Mountain

14. 1. Without anchor points, the behavior of the pink trajectory is hard to be explained. 2. Do several local search runs with different configurations, record diverse local optima/anchor points (circled). 3. With anchor points, the behavior of the pink trajectory is as follows: trapped in region that contains red/blue anchor points, thus failed to visit good solutions, the green/orange anchor points. 4. The behavior of the pale blue trajectory is as follows: after reaching a local optima, it diversifies to another place. It manages to reach green and orange anchor points, and thus its performance is better than the pink trajectory in Figure 3.

15. Search Trajectory Visualization In Practice Layout the points in Abstract 2-D Space Points that are close in N-dimensional space in terms of distance metric (hamming, permutation distance, etc) are laid out close to each other in the abstract 2-D space and vice versa. This utilizes human strength in discerning 2-D spatial information. Layout First Phase: The anchor points are measured with each other using distance metric. The anchor points are installed greedily in abstract 2-D space Re-optimize using the Spring Model layout algorithm. Layout Second Phase: Again, using Spring Model algorithm, the points along search trajectory are aid out in abstract 2-D space using these anchor points as reference. Presentation Aspects: Color coding is used to enhance our understanding: blue: good, green: medium, brown: poor anchor points. The search trajectory is animated over time. Anchor Points are quite close to each other. Trapped in cycling near Anchor Point C Only covers regions near Anchor Point D and E Anchor Points are quite close to each other, but their quality are different. Trapped in cycling near Anchor Point C because it is attractive (green) Only covers regions near Anchor Point D and E, which are good regions (blue and green)

16. Explaining Local Search Behavior using Viz: Local Search Visual Analysis Suite http://www.comp.nus.edu.sg/~stevenha/viz Note: we use the updated version of Viz for the demo.

17. Design Goals Enable algorithm designer (human) to understand LS behavior. Must obey Information Visualization Rules: Overview first, Zoom and Filter, then Details on Demand. Utilize human visual perception strengths. Interactive. Responsive. Aim for widespread use: Generic: Support various (/all) Local Searches. Support various (/all) Combinatorial Optimization Problems. Viz as Analysis Suite. Facilitates information sharing with other algorithm designers.

19. Using Viz

20. Viz Features (Overview) Multi-Source Visualizations Visual Comparison Animated Search Playback Information Visualization + Data Analysis Color and Highlighting Multiple Level of Details Viz GUI is: User friendly, use familiar widgets Interactive Responsive Customize-able

21. Multi-Source Visualizations The local search playback are viewed from several angles: Generic Search Trajectory Objective Value Fitness Distance Correlation Event Bar Algorithm Specific Problem Specific Coordinated Visualizations

22. Human is creature of comparison. Actually, human is not a perfect judge. But, good at relative comparison when given a baseline. Two comparison modes: Juxtapose Superimpose Useful for understanding: Behavior: Same LS & configuration, different parts of search Robustness: Same LS & configuration, two runs Effect of Modifications: Same LS, different configurations Overall behavior: Two totally different LS Visual Comparison

23. Animations Local Search info is too big: cannot be presented at one time. But rather: animated over time. (the search trail is drawn using alpha blending). Local Search typically runs in thousand iterations… Analyzing from start to end may be overwhelming We use Event Bar for quick reference to interesting regions. Viz computes highlights/index points for generic events (e.g. new best found solution) Animated Search Playback

24. Info Visualization + Data Analysis We combine strengths from both worlds. Information Visualization Data Analysis (Textual/Statistics)

25. Viz DEMO http://www.comp.nus.edu.sg/~stevenha/viz

26. Viz is Useful for: Local Search Algorithm Analysis tool: Individual Addressing Tuning Problem Debugging Improving algorithm Collaborative Researchers can annotate and publish *.viz file for information exchange. Benchmarking Pedagogical tool: For Studying/Teaching Local Search Algorithms. Presentation tool: To illustrate behavior of a new Local Search Algorithm.

27. Summary We have seen that proper visualization can be used to help explaining/understanding local search behavior. Some of these are hard to derive without proper visualizations. A necessary step for tuning/improving local search. To be useful, a visualization tool should utilize human visual perception strengths. Viz’s capability in explaining local search is so far promising. Lessons for UIST: Regardless whether your research is on Local Search or not. Hopefully you can get something from what I share here with my visualization tool Viz Careful application and extrapolation of existing techniques in a new domain can result in fascinating extensions. (multidisciplinary). Many other fields can be enhanced using proper Information Visualization tools, developing specific visualization for these domains may be fruitful.

28. End of Talk Time for Q & A Thank you for your attention For further details and to download Viz, please visit: http://www.comp.nus.edu.sg/~stevenha/viz http://www.comp.nus.edu.sg/~stevenha/viz