1. Complexity Computer Science and Software Engineering
© 2014 Project Lead The Way, Inc.

2. Computers Have Finite Resources
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Computers Have Finite Resources
Constraints:
Limited time to execute
Limited bandwidth
Limited memory to store the program
Limited memory for input/output data
These constraints trade off against each other
Often readability is more important than any of these
What's the "best" program you can create to solve a particular problem? You can't make one program the "best" at everything at once. For many programs, the best algorithm is the one that executes the fastest; in other cases, it's the one that uses the least memory. In most cases it doesn't matter how good the algorithm unless the code is easy to read so that other programmers can use and maintain your code.

3. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
How Fast Is a Solution?
Time efficiency: How much time does a program take to run?
Time complexity of a solution: How much more time will a program take to run if given more bytes as input?
A program that solves a problem quickly is efficient. If the input gets bigger, the algorithm takes longer. Complexity asks "how much longer?"

4. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
How Fast Is a Solution?
Time complexity described by the type of curve showing time vs. input bytes
Brute force, an algorithm in which you try all possibilities, takes exponentially longer with longer passwords
As the input gets bigger, the algorithm takes longer. We describe how much longer with the type of curve that we get if we graph time vs. input length. Brute force, for example, is exponential.

5. Why Is Brute Force Exponential?
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Why Is Brute Force Exponential?
Consider passwords with a-z,A-Z,0-9 which allows =62 characters
bytes
passwords to try 1 62 2
622 3
623 4
624 n
62n
As the input gets bigger, the algorithm takes longer. We describe how much longer with the type of curve that we get if we graph time vs. input length. Brute force, for example, is exponential.

6. Exponential Curve Gets Ahead of Others
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Exponential Curve Gets Ahead of Others
Exponential growth will outpace any other kind of growth
Make brute force impractical by using long passwords
Constant, linear, and quadratic complexity are called "polynomial". If the time to solve a problem grows like a polynomial as the input gets longer, we say the problem can be solved in polynomial time. Problems that take exponential time will take longer to solve.

7. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
How Hard Is a Problem?
Time complexity of a problem is the time complexity of its best solution
The algorithm might be quick with special input even if it is long. We describe difficulty using the worst-case input.
We've talked about the time needed for a particular solution to a problem, for a particular program. Computer scientists can sometimes prove that a solution is *the* most efficient solution. That solution says how hard the problem was. For input of a given number of bytes, it describes the worst running time.

8. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Computability
Some problems are so hard the program might never finish.
Computable problems can be solved by a computer
Analyzing programs is often uncomputable
Halting Theorem: "Will this program ever finish?" is sometimes uncomputable
Security analysis: "Does any input crash this program?" is often uncomputable
Computable problems are solvable – in finite time – by a "modern" computer. We won't get into the details of what a "modern" computer is, but all current commercial computers fit this definition. Some problems are uncomputable. Does a program have a security vulnerability? That is often an uncomputable problem.

9. Encryption and Time Complexity
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Encryption and Time Complexity
One-way functions are more time-consuming to undo than to do
RSA encryption keys (used for HTTPS) are hard to crack because multiplying is faster than factoring
Multiplying big prime numbers is a one-way function

10. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Time Complexity
Linear time complexity: Twice as much input takes twice as much time
Addition has linear time complexity. 20-digit numbers take twice as long to add as 10-digit numbers. This graph shows the time complexity of different kinds of problems. The green curve shows how the running time grows for an algorithm with linear time complexity.

11. Time Complexity Quadratic time complexity:
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Time Complexity
Quadratic time complexity:
Double the input  4x the time
Triple the input  9x the time
Multiplying two numbers has quadratic complexity. Algorithms to multiply numbers take more and more time as the numbers get bigger. The time to multiply numbers grows more than the time to add numbers.

12. Encryption Keys Broken by Factoring
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Encryption Keys Broken by Factoring
Factoring takes longer than quadratic time. So prime numbers can keep our secrets safe maybe.
Factoring numbers takes even more time than multiplying them. Even for the very best algorithms for factoring, we know that the time grows more than the brown (quadratic) curve shown here, but we don't know how much more. If we discover a fast factoring algorithm, it would affect our usual methods for sending credit cards over the Internet!

13. Use heuristics when problems can't be solved in reasonable time
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Use heuristics when problems can't be solved in reasonable time
P problems can be solved in polynomial time, also called reasonable time.
For hard problems, heuristics can give us good results but won't guarantee we found the best solution.
What is the P=NP question? There is a group of problems in a set called P. These are the polynomial-time problems. They can be solved in a reasonable amount of time.
For example, "Is <a> one of the factors of <b>?" is a P problem. This is because you can write a function that answers the yes/no question within a reasonable amount of time. "Reasonable" means that if you use bigger numbers a and b, the function might take longer to decide, but only a limited amount of time longer. Increasing the number of digits in <a> or <b> to seven times as many digits might take 7 times longer, or even 7^2 times longer, or even 7^100 times longer. But it will not take 2^7 times longer, because that would be exponential time. For long enough input, exponential time will always be more time polynomial time, no matter how big the power of the polynomial.

14. An Adventure to the Edge of Math
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
An Adventure to the Edge of Math
Feel free to appreciate the beauty of these three problems without getting into details
The rest of this presentation is about interesting, important computer science problems that no one has solved. Three famous problems are pictured at the bottom of this slide: the travelling salesperson problem, the knapsack problem, and the 3-coloring problem.

15. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Does P=NP?
One of seven six unsolved Millennium Prize "Million Dollar Questions"
Are the hardest "NP problems" (pictured below) solvable in polynomial time?
If proved (or disproved), this hypothesis would tell us a lot about what computers can or cannot do. We don't know "how hard" some problems are.
The Clay Foundation offered $1 million to anyone who answered one of the seven questions they selected from the biggest questions in mathematics as of year One is the P=NP question, which asks "How much time does it take a computer to solve these certain kinds of problems?" (Another of the Clay Foundation's questions-- to prove the Poincare Hypothesis-- was solved in 2008, but Grigory Perelman turned down the prize saying it would cheapen his work. Another of the questions-- to prove the Riemann Hypothesis-- might reveal the mysterious pattern of the prime numbers.)

16. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Does P=NP? The Details
P problems can be solved in polynomial time, also called reasonable time.
NP problems include P problems, but they also include the NP-complete problems that we think are harder to solve than P. We know that we can check a proposed solution to an NP problem in reasonable time.
Optional: The second bullet point on this slide goes beyond beyond AP CS Principles. It is included here because the the three examples of NP problems are fun and are easy to understand. Mathematicians and computer scientists are still searching for better algorithms for NP problems.
From the College Board's AP CS Principles Curriculum Framework:
Learning Objective LO Explain the difference between algorithms that run in reasonable time and those that do not run in a reasonable time. Exclusion: Any discussion of nondeterministic polynomial (NP) is beyond the scope of the AP CS Principles course framework and the AP exam.
Essential Knowledge EK4.2.1A Many problems can be solved in a reasonable time.
EK4.2.1B Reasonable time means that as the input size grows, the number of steps the algorithm takes is proportional to the square (or cube, or fourth power, fifth power, etc.) of the size of the input.
EK4.2.1C Some problems cannot be solved in a reasonable time, even for small input sizes.
EK4.2.1D Some problems can be solved but not in a reasonable time. In these cases, heuristic approaches may be helpful to find solution in reasonable time.

17. Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Does P=NP? We Don't Know!
If a problem can be checked in reasonable time, can it be solved in reasonable time?
NP-Complete problems are the hard ones. They've been shown to be equivalent to each other.
Three famous NP-Complete problems with many applications
Travelling Salesperson
Knapsack Problem
3-Color a Graph

18. NP-Complete Problems: Travelling Salesperson Problem
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
NP-Complete Problems:
Travelling Salesperson Problem
What is the shortest route visiting all these cities?

19. NP-Complete Problems: Knapsack Problem
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
NP-Complete Problems:
Knapsack Problem
Choice of items that have weight and value
Maximize value without exceeding maximum weight
3 lbs
$40
20 lbs
limit
10 lbs
$150
3 lbs
$40
3 lbs
$40
The "knapsack problem" is another problem that shows up in many applications. In this picture the burglar can only carry a certain weight, but he wants to carry away the highest total value possible. For example, what is the greatest value you can get for under 20 pounds? What algorithm did you follow? How would your algorithm scale look if there were 100 times as many items to choose from?
3 lbs
$40
5 lbs
$70
15 lbs
$220
5 lbs
$70
5 lbs
$70
5 lbs
$70

20. NP-Complete: 3-Color a Graph
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
NP-Complete: 3-Color a Graph
Color all nodes using 3 colors total
Adjacent nodes must not share color
The "3-coloring" problem is another problem that shows up in many applications. It uses "graphs", a way of visualizing data that are linked to each other.
Graphs have nodes and edges. The circles shown here are called "nodes." Nodes that are connected by a line (called an "edge") are adjacent nodes.
"Coloring" a graph means filling in every node with a color.
"3-coloring a graph" means using 3 colors but not using the same color on adjacent nodes.
This graph has been 3-colored successfully. No two adjacent nodes have the same color.
Like the travelling salesperson problem and the knapsack problem, this problem is equivalent to many real problems that people write programs to solve. A state government, for example, might want to spend money on its counties in different categories like education, environment, and health care, and they might want to spread it out around the state without spending money on the same thing in adjacent areas. Before developing a solution for a large data set, a programmer has to know how their solution will scale up.
These graphs model how neurons are connected in brains, and proving whether P=NP might tell us whether the modern computer can eventually do what a human brain can do.

21. Another 3-Coloring Problem
Presentation Name
Course Name
Unit # – Lesson #.# – Lesson Name
Another 3-Coloring Problem
Here's another 3-coloring problem. It has more edges than the previous problem. The old solution doesn't 3-color it anymore; do you see the adjacent nodes that have the same color? Can you 3-color it?

22. Can You 3-Color It? Presentation Name Course Name
Unit # – Lesson #.# – Lesson Name
Can You 3-Color It?