Review of Basic Computer Science Background

Computer Science You’ll Need Some models of computationSome models of computation –Finite state machines –Recursive functions / Turing machines –Logic circuit models –Cellular automata Computability theoryComputability theory –Universal vs. non-universal models Computational complexity theoryComputational complexity theory –Polynomial vs. exponential problem classes Computer architecture/engineering conceptsComputer architecture/engineering concepts –functional units, CPUs, instruction sets, etc...

Models of Computation Non-universal models:Non-universal models: –Finite state machines (w. read-only inputs) Related to regular expression languagesRelated to regular expression languages –Pushdown automata (FSM w. a stack) Relation to context-free grammar languagesRelation to context-free grammar languages Universal models:Universal models: –Recursive function theory (Church) –“File clerk model” / Turing machines (Turing) FSM with unbounded-size read-write external memoryFSM with unbounded-size read-write external memory –Logic circuits (Shannon) –RAM machines (von Neumann) –Cellular automata (von Neumann)

Church-Turing thesis All the “universal” models were proved early on to be able to simulate each otherAll the “universal” models were proved early on to be able to simulate each other –Straightforward to write “emulator” programs “Church-Turing thesis”:“Church-Turing thesis”: –Anything that can be computed physically (in any way whatsoever) can be computed by machines in these models Hence the term “universal”Hence the term “universal” –This is in the form of a physical postulate. Note that universality & the Church-Turing thesis say nothing about efficiency.Note that universality & the Church-Turing thesis say nothing about efficiency.

Computational Complexity Theory Characterizes the “complexity” (defined in terms of resource usage) of computationsCharacterizes the “complexity” (defined in terms of resource usage) of computations –This “complexity” can be viewed as essentially the reciprocal of computational efficiency efficiency would be amount of computation completed per unit resources consumedefficiency would be amount of computation completed per unit resources consumed Traditional focus in complexity theory only on either of two resources, in isolation:Traditional focus in complexity theory only on either of two resources, in isolation: –“Time” (most popular) Number of primitive ops. or parallel steps performedNumber of primitive ops. or parallel steps performed –“Space” Max. number of bits of storage needed at any timeMax. number of bits of storage needed at any time

Cost-Efficiency Resources (time, space, hardware, energy, etc.) have some associated cost for their expenditureResources (time, space, hardware, energy, etc.) have some associated cost for their expenditure –Resource usage should be minimized A 100% efficient process is one that wastes no resources. Want to maximize efficiency.A 100% efficient process is one that wastes no resources. Want to maximize efficiency. –Let $ min be the minimum cost required to carry out a process, $ the actual cost. –So % $ = $ min /$, or cost-efficiency, is the fraction of cost that is well spent. Note that % $ is inversely proportional to cost $ (or “complexity”), even if $ min is unknown.Note that % $ is inversely proportional to cost $ (or “complexity”), even if $ min is unknown.

Complexity Classes In complexity theory, it is useful to classify computational problems or tasks on a given machine in terms of the asymptotic order of growth of the complexity.In complexity theory, it is useful to classify computational problems or tasks on a given machine in terms of the asymptotic order of growth of the complexity. –Allows ignoring the effects of minor implementation details Further, similar orders of growth are often grouped together, e.g.:Further, similar orders of growth are often grouped together, e.g.: –P - The class of problems solvable in polynomial (O(n k ) for some k) time This class was thought to be independent of the machine model used, until the discovery of quantum computing!This class was thought to be independent of the machine model used, until the discovery of quantum computing!

Order of Growth Notation