1. RAIK 283: Data Structures & Algorithms
Brute Force *
Dr. Ying Lu
September 20, 2012
*slides referred to
Design and Analysis of Algorithms – Chapter 3

2. Design and Analysis of Algorithms – Chapter 3
Brute force
A straightforward approach usually based on problem statement and definitions
Examples:
Selection Sort
Graph Traversal
Simple Computational Tasks
Exhaustive Search
Design and Analysis of Algorithms – Chapter 3

3. Design and Analysis of Algorithms – Chapter 4
Graph traversal
Many problems in A.I. and operations research require the systematic processing of vertices and edges of graphs
Graph traversal algorithms:
Depth-first search
Breadth-first search
First, graph representations!
Design and Analysis of Algorithms – Chapter 4

4. Graph representations using data structures
Adjacency Matrix Representation
Let G = (V, E), n = |V|, m = |E|, V = {v1, v2, …, vn)
G can be represented by an n  n matrix C
Design and Analysis of Algorithms – Chapter 4

5. Adjacency list representation
Design and Analysis of Algorithms – Chapter 4

6. Design and Analysis of Algorithms – Chapter 4
Traversing graphs
Depth-First Search (DFS) and Breadth-First Search (BFS)
Two elementary traversal strategies
Both provide efficient ways to “visit” each vertex and edge of a graph
Both work on directed and undirected graphs
They differ in the order of “visiting” vertices
Design and Analysis of Algorithms – Chapter 4

7. Design and Analysis of Algorithms – Chapter 4
Depth-first search
Explore graph always moving away from last visited vertex
Pseudocode for Depth-first-search of graph G=(V,E)
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

8. Example – undirected graphb e f c d g h
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Depth-first traversal:
Design and Analysis of Algorithms – Chapter 4

9. Design and Analysis of Algorithms – Chapter 4
Question
How to rewrite the procedure dfs(v), using a stack to eliminate recursion
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

10. Non-recursive version of DFS algorithm
Algorithm dfs(v)
s.createStack();
s.push(v);
count := count + 1
mark v with count
while (!s.isEmpty()) {
let x be the node on the top of the stack s;
if (no unvisited nodes are adjacent to x)
s.pop(); // backtrack
else {
select an unvisited node u adjacent to x;
s.push(u);
mark u with count }
Design and Analysis of Algorithms – Chapter 4

11. Time efficiency analysis
DFS can be implemented with graphs represented as:
Adjacency matrices:
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

12. Time efficiency analysis
DFS can be implemented with graphs represented as:
Adjacency linked lists:
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

13. Time efficiency analysis
DFS can be implemented with graphs represented as:
Adjacency matrices: Θ(V2)
Adjacency linked lists: Θ(V+E)
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

14. Design and Analysis of Algorithms – Chapter 4
Application: checking graph connectivity and finding connected components
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

15. Application: checking acyclicity
DFS(G)
count :=0
mark each vertex with 0 (unvisited)
for each vertex v V do
if v is marked with 0
dfs(v)
dfs(v)
count := count + 1
mark v with count
for each vertex w adjacent to v do
if w is marked with 0
dfs(w)
Design and Analysis of Algorithms – Chapter 4

16. Design and Analysis of Algorithms – Chapter 4
Breadth-first search
Explore graph moving across to all the neighbors of last visited vertex
Similar to level-by-level tree traversals
Applications: same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges
Design and Analysis of Algorithms – Chapter 4

17. Example – undirected graphb e f c d g h
Breadth-first traversal:
Design and Analysis of Algorithms – Chapter 4

18. Example – undirected graphb e f c d g h
Depth-first search could be implemented on a stack
How about breadth-first search
Design and Analysis of Algorithms – Chapter 4

19. Breadth-first search algorithm
bfs(v)
count := count + 1
mark v with count
initialize queue with v
while queue is not empty do
a := front of queue
for each vertex w adjacent to a do
if w is marked with 0
mark w with count
add w to the end of the queue
remove a from the front of the queue
BFS(G)
count :=0
mark each vertex with 0
for each vertex v V do
if v is marked with 0
bfs(v)
Design and Analysis of Algorithms – Chapter 4

20. Breadth-first search: Notes
BFS has same efficiency as DFS and can be implemented with graphs represented as:
Adjacency matrices: Θ(V2)
Adjacency linked lists: Θ(V+E)
Design and Analysis of Algorithms – Chapter 4

21. Design and Analysis of Algorithms – Chapter 3
In-class exercise
Exercise Gadget testing
A firm wants to determine the highest floor of its n-story headquarters from which a gadget can fall with no impact on the gadget’s functionality. The firm has two identical gadgets to experiment with. Design an algorithm in the best efficiency class you can to solve this problem.
Design and Analysis of Algorithms – Chapter 3

22. Brute force polynomial evaluation
Problem: Find the value of polynomial p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Design and Analysis of Algorithms – Chapter 3

23. Brute force polynomial evaluation
Problem: Find the value of polynomial p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Algorithm:
x := x0
p := 0.0
for i := n down to 0 do
power := 1
for j := 1 to i do
power := power * x
p := p + a[i] * power
return p
Efficiency:
Design and Analysis of Algorithms – Chapter 3

24. Brute force polynomial evaluation
Problem: Find the value of polynomial p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Algorithm:
x := x0
p := 0.0
for i := n down to 0 do
power := 1
for j := 1 to i do
power := power * x
p := p + a[i] * power
return p
Efficiency: (n2)
Design and Analysis of Algorithms – Chapter 3

25. Brute force polynomial evaluation
Problem: Find the value of polynomial p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Algorithm:
x := x0
p := 0.0
for i := n down to 0 do
power := 1
for j := 1 to i do
power := power * x
p := p + a[i] * power
return p
Efficiency: (n2)
Can we design a linear algorithm for this problem
Design and Analysis of Algorithms – Chapter 3

26. Polynomial evaluation: improvement
We can do better by evaluating from right to left: p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Algorithm:
x := x0
p := a[0]
power := 1
for i := 1 to n do
power := power * x
p := p + a[i] * power
return p
Efficiency: (n)
Design and Analysis of Algorithms – Chapter 3

27. Polynomial evaluation: improvement
We can do better by evaluating from right to left: p(x) = anxn + an-1xn-1 +… + a1x1 + a at a point x = x0
Algorithm:
x := x0
p := a[0]
power := 1
for i := 1 to n do
power := power * x
p := p + a[i] * power
return p
Efficiency: (n)
Can we design a better than linear algorithm for this problem
Design and Analysis of Algorithms – Chapter 3

28. Brute force closest-pair algorithm
Problem: find the closest pair among n points in k-dimensional space
Design and Analysis of Algorithms – Chapter 3

29. Brute force closest-pair algorithm
Problem: find the closest pair among n points in k-dimensional space
Algorithm: Compute distance between each pair of points and identify the pair resulting in the shortest distance
What is (or should be) the basic operation of the algorithm?
Design and Analysis of Algorithms – Chapter 3

30. Brute force closest-pair algorithm
Problem: find the closest pair among n points in k-dimensional space
Algorithm: Compute distance between each pair of points and identify the pair resulting in the shortest distance
Basic operation (squaring):
Design and Analysis of Algorithms – Chapter 3

31. Brute force closest-pair algorithm
Problem: find the closest pair among n points in k-dimensional space
Algorithm: Compute distance between each pair of points and identify the pair resulting in the shortest distance
Basic operation:
How many different pairs of points?
Design and Analysis of Algorithms – Chapter 3

32. Principle of counting: Product Rule
The Product Rule
Suppose that a procedure can be broken down into a sequence of two tasks. If there are n1 ways to do the first task and for each of these ways of doing the first task, there are n2 ways to do the second task, then there are n1n2 ways to do the procedure.
Design and Analysis of Algorithms – Chapter 3

33. Principle of counting: Product Rule
The Product Rule
Suppose that a procedure can be broken down into a sequence of two tasks. If there are n1 ways to do the first task and for each of these ways of doing the first task, there are n2 ways to do the second task, then there are n1n2 ways to do the procedure.
Calculation
Applying the product rule, there are n*(n-1) pairs among n points
Considering (a, b) and (b, a) as the same, we divide the above number by 2 and thus, there are n*(n-1)/2 different pairs of points
Design and Analysis of Algorithms – Chapter 3

34. Brute force closest-pair algorithm
Problem: find the closest pair among n points in k-dimensional space
Algorithm: Compute distance between each pair of points and identify the pair resulting in the shortest distance
Basic operation:
Efficiency:
(n2)
Design and Analysis of Algorithms – Chapter 3

35. Design and Analysis of Algorithms – Chapter 3
In-class exercise
Can you design a faster algorithm than the one based on the brute-force strategy to solve the closest-pair problem for n points x1, x2, … , xn on the real line?
What is the time efficiency of your algorithm?
Design and Analysis of Algorithms – Chapter 3

36. Design and Analysis of Algorithms – Chapter 3
In-class exercise
Can you design a faster algorithm than the one based on the brute-force strategy to solve the closest-pair problem for n points x1, x2, … , xn on the real line?
Algorithm:
Step1: Sort the numbers in ascending order, O(nlogn)
Step 2: Compute the differences between adjacent numbers in the sorted list, (n)
Step 3: Find the smallest such difference, (n)
Running time of the entire algorithm:
O(nlogn) + (n) + (n) = O(nlogn)
Design and Analysis of Algorithms – Chapter 3

37. Design and Analysis of Algorithms – Chapter 3
Convex hull problem
Convex hull
Problem:
Find smallest convex polygon enclosing n points on the plane
Convex:
A geometric figure with no indentations.
Formally, a geometric figure is convex if every line segment connecting interior points is entirely contained within the figure's interior.
convex
Non-convex
Design and Analysis of Algorithms – Chapter 3

38. Example: Convex Hull Input: p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11 p9
Output: p2,p9,p11,p4,p5,p6,p8,p2
p11 p4 p1 p2 p7 p3 p5
p10 p8 p6

39. Convex hull: application domain*
Computer visualization, ray tracing
(e.g. video games, replacement of bounding boxes)
Path finding
(e.g. embedded AI of Mars mission rovers)
Geographical Information Systems (GIS)
(e.g. computing accessibility maps)
Visual pattern matching
(e.g. detecting car license plates)
Verification methods
(e.g. bounding of Number Decision Diagrams)
Geometry
(e.g. diameter computation)
*slide refer to
Given a finite set of points P in R^d, the diameter of P is defined as the maximum distance between two points of P.
Design and Analysis of Algorithms – Chapter 3

40. Convex hull brute force algorithm
Extreme points of the convex polygon
Consider all the points in the polygon as a set. An extreme point is a point of the set that is not a middle point of any line segment with end points in the set. p9
p11 p4 p1 p2 p7 p3 p5
p10 p8 p6
Design and Analysis of Algorithms – Chapter 3

41. Convex hull brute force algorithm
Extreme points of the convex polygon
Consider all the points in the polygon as a set. An extreme point is a point of the set that is not a middle point of any line segment with end points in the set. p9
p11 p4 p1 p2 p7 p3 p5
p10 p8 p6
Which pairs of extreme points need to be connected to form the boundary of the convex hull?
Design and Analysis of Algorithms – Chapter 3

42. Convex hull brute force algorithm
A line segment connecting two points Pi and Pj of a set of n points is a part of its convex hull’s boundary if and only if all the other points of the set lies on the same side of the straight line through these two points.
Design and Analysis of Algorithms – Chapter 3

43. Convex hull brute force algorithm
The straight line through two points (x1, y1), (x2, y2) in the coordinate plane can be defined by the following equation
ax + by = c
where a = y2 – y1, b = x1 – x2, c = x1y2 - y1x2
Such a line divides the plane into two half-planes: for all the points in one of them: ax + by > c, while for all the points in the other, ax + by < c.
Design and Analysis of Algorithms – Chapter 3

44. Convex hull brute force algorithm
Algorithm: For each pair of points p1 and p2 determine whether all other points lie to the same side of the straight line through p1 and p2, i.e. whether ax+by-c all have the same sign
Efficiency:
Design and Analysis of Algorithms – Chapter 3

45. Convex hull brute force algorithm
Algorithm: For each pair of points p1 and p2 determine whether all other points lie to the same side of the straight line through p1 and p2, i.e. whether ax+by-c all have the same sign
Efficiency: (n3)
Design and Analysis of Algorithms – Chapter 3

46. Exhaustive search: definition
A brute force solution to a problem involving search for an element with a special property, usually among combinatorial objects such as a permutations, combinations, or subsets of a set.
Design and Analysis of Algorithms – Chapter 3

47. Exhaustive search: method
Construct a way of listing all potential solutions to the problem in a systematic manner
all solutions are eventually listed
no solution is repeated
Evaluate solutions one by one, perhaps disqualifying infeasible ones and keeping track of the best one found so far
When search ends, announce the winner
Design and Analysis of Algorithms – Chapter 3

48. Example 1: Traveling salesman problem
Given n cities with known distances between each pair, find the shortest tour that passes through all the cities exactly once before returning to the starting city.
Alternatively: Find shortest Hamiltonian circuit in a weighted connected graph.
Example: a b c d 8 2 7 5 3 4
Alternatively, the problem can be stated as finding shortest Hamiltonian circuit of a graph, where the graph’s vertices representing the cities and the edge weights specifying the distances. Hamiltonian circuit is defined as a cycle that passes through all the vertices of the graph exactly once.
Design and Analysis of Algorithms – Chapter 3

49. Traveling salesman by exhaustive search
Tour Cost
a→b→c→d→a = 17
a→b→d→c→a = 21
a→c→b→d→a = 20
a→c→d→b→a = 21
a→d→b→c→a = 20
a→d→c→b→a = 17
Efficiency:
Design and Analysis of Algorithms – Chapter 3

50. Traveling salesman by exhaustive search
Tour Cost
a→b→c→d→a = 17
a→b→d→c→a = 21
a→c→b→d→a = 20
a→c→d→b→a = 21
a→d→b→c→a = 20
a→d→c→b→a = 17
Efficiency: (n-1)!/2
Design and Analysis of Algorithms – Chapter 3

51. Design and Analysis of Algorithms – Chapter 3
0-1 Knapsack problem
Given a knapsack with maximum capacity W, and a set S consisting of n items
Each item i has some weight wi and benefit value vi
Problem: How to pack the knapsack to achieve maximum total value of packed items?
Design and Analysis of Algorithms – Chapter 3

52. 0-1 Knapsack problem: a picture
W = 20 wi vi 10 9 8 5 4 3 2
Weight
Benefit value
This is a knapsack
Max weight: W = 20
Items
Design and Analysis of Algorithms – Chapter 3

53. Design and Analysis of Algorithms – Chapter 3
0-1 Knapsack problem
Problem, in other words, is to find
The problem is called a “0-1” problem, because each item must be entirely accepted or rejected.
In the “Fractional Knapsack Problem,” we can take fractions of items.
Design and Analysis of Algorithms – Chapter 3

54. 0-1 Knapsack problem: brute-force approach
Let’s first solve this problem with a straightforward algorithm
We go through all combinations (subsets) and find the one with maximum value and with total weight less or equal to W
Design and Analysis of Algorithms – Chapter 3

55. Example 2: Knapsack Problem
Given n items:
weights: w1 w2 … wn
values: v1 v2 … vn
a knapsack of capacity W
Find the most valuable subset of the items that fit into the knapsack
Example:
item weight value Knapsack capacity W=16
$20
$30
$50
$10
Design and Analysis of Algorithms – Chapter 3

56. Knapsack by exhaustive search
Subset Total weight Total value
 $0
{1} $20
{2} $30
{3} $50
{4} $10
{1,2} $50
{1,3} $70
{1,4} $30
{2,3} $80
{2,4} $40
{3,4} $60
{1,2,3} not feasible
{1,2,4} $60
{1,3,4} not feasible
{2,3,4} not feasible
{1,2,3,4} not feasible
Most valuable subset?
Efficiency:
Design and Analysis of Algorithms – Chapter 3

57. 0-1 Knapsack problem: brute-force approach
Algorithm:
We go through all combinations and find the one with maximum value and with total weight less or equal to W
Efficiency:
Since there are n items, there are 2n possible combinations of items.
Thus, the running time will be O(2n)
Design and Analysis of Algorithms – Chapter 3

58. Brute force strengths and weaknesses
wide applicability
simplicity
yields reasonable algorithms for some important problems
sorting; matrix multiplication; closest-pair; convex-hull
yields standard algorithms for simple computational tasks and graph traversal problems
Design and Analysis of Algorithms – Chapter 3

59. Brute force strengths and weaknesses
rarely yields efficient algorithms
some brute force algorithms unacceptably slow
e.g., the recursive algorithm for computing Fibonacci numbers
not as constructive/creative as some other design techniques
Design and Analysis of Algorithms – Chapter 3