1. Algorithms +
L. Grewe

2. Algorithms and Programs
Algorithm: a method or a process followed to solve a problem.
A recipe.
An algorithm takes the input to a problem (function) and transforms it to the output.
A mapping of input to output.
A problem can have many algorithms.

3. Algorithm Properties
An algorithm should (ideally) possess the following properties:
It must be correct.
It must be composed of a series of concrete steps.
There can be no ambiguity as to which step will be performed next.
It must be composed of a finite number of steps.
It must terminate.
A computer program is an instance, or concrete representation, for an algorithm in some programming language.
“Correct” means computes the proper function.
“Concrete steps” are executable by the machine in question.
We frequently interchange use of “algorithm” and “program” though they are actually different concepts.

4. Algorithm Efficiency
There are often many approaches (algorithms) to solve a problem. How do we choose between them?
At the heart of computer program design are two (sometimes conflicting) goals.
To design an algorithm that is easy to understand, code, debug.
To design an algorithm that makes efficient use of the computer’s resources.

5. Algorithms
Any problem can have a large number of algorithms that could be used to solve it.
……but there are some algorithms that are effecitve in solving many problems.

6. Common Algorithms / Algorithm Methods
Greedy Algorithms
Divide and Conquer
Dynamic Programming
Backtracking
Branch and Bound

7. Other (common) Algorithms / Algorithm Methods
Linear Programming
Integer Programming
Neural Networks
Genetic Algorithms
Simulated Annealing
Typically these are covered in application specific courses that use them (e.g. Artificial Intelligence)

8. Algorithms specific to a data structure
Also algorithms can be specifically designed for common operations on a particular data structure.
Example - Graph Algorithms
Graph matching
Find shortest path(s)

9. Defining the Problem
Before even trying to design or reuse an existing algorithm, you must define your problem.
……many problems can be defined as an optimization problem.

10. Optimization Problem Problem = Function to X + Constraints
This is where you describe the problem as a formula/function. Example, “find the shortest path” can be stated as Sum (distances) is minimum. Here the X is to “minimize”. The Function is the “Sum(distances)”.
Sometimes these are called “COST functions”
Constraints
In our shortest path problem this might be to never visit the same node in the path twice.

11. One Solution to any Problem…..The Brute Force Algorithm
Try every solution!
Exponential Time, because exponentially many
This is WHY we discuss algorithms!!!!

12. Optimization Problem Elements
Instances: The possible inputs to the problem.
Solutions for Instance: Each instance has an exponentially large set of solutions.
Cost of Solution: Each solution has an easy to compute cost or value.
Specification
<preCond>: The input is one instance.
<postCond>: An valid solution with optimal cost. (minimum or maximum)

13. Greedy Algorithms
Class of algorithms that solve Optimization Problems in a “greedy way”
Some greedy algorithms will generate an optimal solution, others only a “good (enough)” solution.
Greedy Method = at each point in the algorithm a decision is make that is best at that point. Decisions made are not changed at a later point.
Greedy Criterion = criterion used to make a greedy decision at each point.

14. Divide and Conquer
Problem = Set of Several Independent (smaller) sub-problems.
Divide problem into several independent sub-problems and solve each sub-problem. Combine solutions to derive final solution.
Can work well on parallel computers.
Many times the sub-problems are the same problem and need to only develop one algorithm to solve them and then the algorithm to combine the results.

15. Divide and Conquer Example: Starting Thoughts:
You are given a bag with 16 coins and told one is counterfeit and lighter than the others. Problem = determine if bag contains the counterfeit coin. You have a machine that compares the weight of two sets of coins and tells you which is lighter or if they are the same.
Starting Thoughts:
You could start with comparing coin 1 and 2. If one is lighter you are done. You can then compare coin 3 and 4 and so on. If you have N coins this can take N/2 times.
The Divide and Conquer Way:
If you have N coins divide into two N/2 groups. Weigh them. If one is lighter then we are done and the bag does contain a counterfeit coin. If they are the same, there is no counterfeit coin. This takes ONLY 1 operation.
What happens if you want to find the counterfeit coin?

16. Dynamic Programming
dynamic programming is a method of solving complex problems by breaking them down into simpler steps.
Bottom-up dynamic programming simply means storing the results of certain calculations, which are then re-used later because the same calculation is a sub-problem in a larger calculation.
Top-down dynamic programming involves formulating a complex calculation as a recursive series of simpler calculations.

17. More on Dynamic Programming
Some programming languages can automatically memorize the result of a function call with a particular set of arguments
Some languages make it possible portably (e.g. Scheme, Common Lisp or Perl), some need special extensions (e.g. C++, see [2]). Some languages have automatic memoization built in. In any case, this is only possible for a referentially transparent function.

18. Backtracking
Backtracking is a general algorithm for finding all (or some) solutions to some computational problem, that incrementally builds candidates to the solutions, and abandons each partial candidate c ("backtracks") as soon as it determines that c cannot possibly be completed to a valid solution
problems which admit the concept of a "partial candidate solution“
and a relatively quick test of whether it can possibly be completed to a valid solution.

19. Branch and Bound
Divides a problem to be solved into a number of subproblems, similar to the strategy backtracking.
Systematic enumeration of all candidate solutions, where large subsets of fruitless candidates are discarded en masse, by using upper and lower estimated bounds of the quantity being optimized
Efficiency of the method depends strongly on the node-splitting procedure and on the upper and lower bound estimators. All other things being equal, it is best to choose a splitting method that provides non-overlapping subsets.