HKOI 2005 Training Introduction to Algorithms Alan, Tam Siu Lung

What is Algorithm? Algorithm –A set of data manipulations instructions to solve a problem (informatics problem) Problem –A mapping from each input (in a specific format) to an output (in a specific format) Pseudo-Code –Realization of Algorithm in the current domain Mapping from problem input to algorithm input Mapping from algorithm output to problem output Program –Realization of Pseudo-Code given A compiler and the language supported Data in plain text format and the IO routines supported

Visualization of Program AB Algorithm1 Coding Pseudo-Code Source-Code int main() { A a = Read(); B b = Algorithm1(a); C c = Algorithm2(b); D d = Algorithm3(c); Write(d); } BC Algorithm2 CD Algorithm3

Algorithm Example Sorting Problem –Given an array of N comparable elements with comparable keys –Output a permutation of the elements –such that no element is has a key greater than that of the one there before Note –Array = a sequence of elements –Element = a key-value pair –Permutation =... –Greater than = …

Realization Example To print in sorted order a set of points in a cetacean plane by the distance from origin: Realization contains –Key = distance from origin –Value = coordinates = (x, y) –Comparison of 2 elements = comparison of the numerical values of distances Pseudo-Code contains –Store the set in an array –Sort them –Output the array in order

Why Algorithms? Computational Biology Database Cryptography Quantitative Analysis Artificial Intelligence Scheduling

Example – Multiple Sequence Alignment Input –GCTAGCTAAC –AACTAGC –AGCGCTAGCTA –TAACGACTAT Find a string with minimum which contains all these sub-strings

Example – Multiple Sequence Alignment Find a string with minimum which contains all these sub-strings Output AACTAGCGCTAGCTAACGACTAT GCTAGCTAAC AACTAGC AGCGCTAGCTA TAACGACTAT

Example – Sorting By Reversal Given a sequence of integers 1..N Every step you can reverse a continuous sub-subsequence Find the minimum number of steps to sort it

Example – Sorting By Reversal Input – Output – – – –

To know what algorithms are Mathematics –Symbols and notations –Methods Data Structures –Specification of input and output –Internal storage during processing You can recite them. It is science. You can also derive them.

To know how to use algorithms Techniques –Common ways that algorithms are built upon Formulations –How to map the problem domain into entities in algorithms It is art. We have no way to teach you! That’s why you need exercises.

Techniques Recursion –Reduction into sub-problems Divide and Conquer –Reduction into >1 sub-problems –Collect the results Exhaustion –Expand all reductions into sub-problems Greedy –Use only 1 reduction into sub-problem(s) Dynamic Programming –Try all and remember the one

To know why algorithms work Complexity Analysis Proof of correctness Mathematics-oriented Out of our scope, but … To “invent” new algorithms …

HKOI concerns All basics and techniques specified above –Mathematics, Data Structures –Recursion, Exhaustion, Greedy, Algorithms which implement the techniques –Sorting, Searching, DFS, BFS, Prim, Kruskal –Dijkstra, Bellman-Ford, Warshall-Floyd Famous IO models used by algorithms –Tree, Graph –Search Space (Graph Searching) –State Space (Constraint Satisfaction)

Introduction to Algorithms Questions?

HKOI also concerns Modifications to algorithms –To accommodate the situation Multi-state BFS –For speed Branch & Bound –For memory Bit-wise States Other paradigms –Open test data (known input) –Interactive (hidden input)

Complexity An approximation to the runtime and memory requirement of a program. Note that there are –Best-case complexity –Average-case complexity –Worst-case complexity In most cases, we concern runtime only.

Measuring Runtime Definition –An operation –Problem size Consider bubble sort: for i := N - 1 downto 1 do for j := 1 to i do if A[j] > A[j+1] then swap(A[j], A[j+1]); In worst case how many “if” operations? “swap” operations? “for j” operations? “for j” operations? Integer comparisons?

Complexity Give a name to a group of functions with similar growth property, e.g. use O(n 2 ) to mean n 2 + n, 2n 2 – 10n, 10n 2, … Since the constant won’t be too big practically, it is a good indicator of expected runtime. We uses “worst case runtime complexity” (a.k.a. order) extensively to describe runtime of a program

Graph

WA NT SA Q NSW V T

Graph A directed graph G=(V,E), where –V = the set of Node/Vertex –E = the set of Edge/Arc, where E=(v 1,v 2 ) where v 1 and v 2 are in V Extreme: |E| ≤ |V| 2 So we have real algorithms with order: –O(|V| 3 ) –O(|V| + |E|) –O(|V| log |V| + |E|) –O(|E| log(log* |E| – log* |E|/|V|))

Difficulty of Problem Definitions (INCORRECT) –A problem with order being a polynomial is called polynomial-time solvable (P) –A problem whose solution is verified in polynomial time is said to be polynomial-time verifiable (NP) –A problem with no known polynomial-time solution to date is called NP-hard Difficulty of problems are roughly classified as: –Easy: in P (of course all P problems are also in NP) –Hard: in NP but not in P (NP-complete) –Very Hard: not even in NP

Complexity When you have a problem, given input (output) constraints and runtime constraints, you can guess whether an algorithm works or not. But sometimes the stated order of an algorithm depends on some tricky data structures, which we won’t have skills or time to implement!

IOI/NOI Problems Requires Understanding of basic algorithms Innovation on ways to realize the algorithms Technical capability of the realization Remember: Different people care different things in different situations

Finally Information, What Knowledge, How Understanding, Why Induction Deduction