Brief Introduction of Algorithm

What is Algorithm A method with several definite steps to effectively complete a task. In general, it starts from the initial state with input data and ends at final states (not only one!!) with some output information. Typically, it can be described with a flow chart.

An Example of Flow Chart Flow chart is also a basic mechanism of language. There is a kind of “flow chart” which can be much more complicated and we are not going to talk about that in detail.

If You Want to Learn Algorithm… This is a book for those without any fundamental knowledge of algorithm. However, some coding techniques and concept is prerequisite.

An Example of Algorithm Sorting some disordered data  Bubble sort  Quick sort

Example – Bubble Sort A step-by-step example

Example – Quick Sort A step-by-step example

Example – Comparison In general, quick sort can be much more faster than bubble sort. However, there are some issues when making comparison  A quantitative comparison?  Is there a worst case?

Computational Complexity Big O notation (infinite asymptotics)  Ex: What does O(n 2 ) means?  Ex: What about O(2 n )?  Ex: If an algorithm takes several steps (listed below) to finish, how do we denote its complexity using big O notation?

Computational Complexity Bubble sort: O(n 2 ) Quick sort: O(n log n)

Worst case For some algorithms, there are some cases will terribly downgrade their performance. For example, complexity of quick sort will become O(n 2 ) when encountering worst cases.

Other Considerations Memory consumption  Usually, we must strike a balance between computation speed and memory consumption. Programming difficulty  Some legendary algorithm performs great. However, writing the program can be a nightmare.

Other Considerations More detailed complexity  Doing multiplication is much slower than doing addition.  Some algorithms replace parts of multiplication operations with additions. Ex: Conventional matrix multiplication: O(n 3 ) Strassen’s matrix multiplication: O(n ) Is it truly much better than conventional method?

Algorithm and Computation Theory There are huge number of algorithms designed for different kinds of problem.  Common mathematical problem, numerical evaluation, networking, etc. However, all of them can be discussed in view of computation theory.  It is doubtless that we should take care of their computation behavior and concept. Hence, we can further improve them.

Some Methodology of Algorithm Design Divide and conquer Dynamic programming Greedy method Linear programming Genetic algorithm

For Further Reading… “The Art of Computer Programming” by Donald Knuth Creator of TeX Turing award Be sure to have some programming knowledge before reading it!