CS 480 ARTIFICIAL INTELLIGENCE Instructor: B. Ravikumar Computer Science Department 116 I Darwin Hall Class meets: Fridays 9 to 12

Course details Catalog Description: A survey of techniques that simulate human intelligence. Topics may include: pattern recognition, general problem solving, adversarial game-tree search, decision-making, expert systems, neural networks, fuzzy logic, and genetic algorithms. Prerequisite: CS 315 or consent of instructor. Background Expected: Programming and data structures (CS 315) Discrete mathematics (CS 242) Linear algebra Some background in logic and probability will be helpful, but not necessary.

Course details Course Goals: AI covers wide range of topics: understanding language vision and speech processing problem solving, planning common sense reasoning. AI techniques: combinatorial (searching, A* algorithm etc.) logical (prove assertion in formal framework) probabilistic (decision tree, Bayesian network) machine learning (neural network, evolutionary technique)

Course details Other references: N. Nillsson, AI: A new synthesis. Winston, Artificial Intelligence.

Course details

Short Quizzes (5 – 10%) Two Mid-Term tests (20%) – Both tests will be in class and will be about 75 miutes long. The tests will be open book/open notes. Home Work and Projects ( %) – There will be some common programming projects and a final project. The final project will be done individually. You can choose a problem from a list that will be provided early in the semester. The project is due the last week of the semester. You are to write a report summarizing your contributions to the chosen problem. Some selected project work will be presented in the department colloquium. Final Examination ( %) – The final examination will be comprehensive and will take place at the scheduled time posted in the web page /classsched/ finals_sched.pdf (not updated for Fall 09 as of August 15, 2009.) /classsched/ finals_sched.pdf

Lecture 1 Outline Course overview What is AI? A brief history The state of the art Slides adapted from Russell and Norvig, AIAMA

Course overview Introduction (chapters 1,2) Techniques Combinatorial (search) approach to AI (chapters 3,4,5,6) Symbolic (logical) approach to AI (chapters 7,8,9) Probabilistic approach to AI (chapters 13,14) Learning approach to AI (chapters 18,20) Applications Natural Language Processing (chapter 22,23) Computer vision (Chapter 24)

What is AI? Authors think AI falls into four categories: Thinking humanlyThinking rationally Acting humanlyActing rationally The textbook advocates "acting rationally"

What is AI? Before attempting a definition, we will state some major contemporary applications of AI: business: advertising, financial decision making web: identifying objects in images, social network models etc. medical: image classification (belign vs. malignant tumor), image analysis using functional MRI multiple field: language translation, semantic analysis, speech synthesis, speech to text conversion. industrial: vision, robotics

Acting humanly: Turing Test Turing (1950) "Computing machinery and intelligence": "Can machines think?"  "Can machines behave intelligently?" Operational test for intelligent behavior: the Imitation Game Predicted that by 2000, a machine might have a 30% chance of fooling a lay person for 5 minutes Anticipated all major arguments against AI in following 50 years Suggested major components of AI: knowledge, reasoning, language understanding, learning

Thinking humanly: cognitive modeling 1960s "cognitive revolution": information- processing psychology Requires scientific theories of internal activities of the brain How to validate? Requires 1) Predicting and testing behavior of human subjects (top- down) or 2) Direct identification from neurological data (bottom- up) Both approaches (roughly, Cognitive Science and Cognitive Neuroscience) now distinct from AI

Thinking rationally: "laws of thought" What are correct arguments/thought processes? Several Greek schools developed various forms of logic: notation and rules of derivation for thoughts; may or may not have proceeded to the idea of mechanization Direct line through mathematics and philosophy to modern AI Problems: 1.Not all intelligent behavior is mediated by logical deliberation 2.What is the purpose of thinking? What thoughts should I have?

Acting rationally: rational agent Rational behavior: doing the right thing The right thing: that which is expected to maximize goal achievement, given the available information Doesn't necessarily involve thinking – e.g., blinking reflex – but thinking should be in the service of rational action

AI techniques Combinatorial search problems –state space (over which search is performed) –finite state space (discrete) –how to move from one state to another (transition rules) Applications –Games (one player or two players) –Navigation (robotics) Solution –Search tree exploration

techniques Combinatorial search approach Sliding piece puzzle: Start: goal: Legal moves: slide a piece next to empty slot. Many AI problems can be modeled as search problems.

A portion of a search tree for the 8-puzzle.

Combinatorial search Uninformed search depth-first breadth-first iterative deepening breadth-depth informed search best-first

Combinatorial search Depth-first search What are the ways to speed-up DFS?

Combinatorial search Breadth-first search

Combinatorial search heuristic search for each node, a heuristic provides an estimate of its distance from the goal. for sliding-piece puzzle, Manhattan distance is one such estimate. estimate for other search problems? (e.g. queen placement)

Combinatorial search Consider placement in the 5 th row.

techniques Symbolic (logical) approach to AI intelligent problem solving requires reasoning and deduction. Knowledge is represented as a set of logical assertions A 1, …, A n, and a conclusion to be drawn is also expressed as an assertion. Can we deduce F from A 1, …, A n ?

Knowledge bases Knowledge base = set of sentences in a formal language Declarative approach to building an agent (or other system): –Tell it what it needs to know Then it can Ask itself what to do - answers should follow from the KB Agents can be viewed at the knowledge level i.e., what they know, regardless of how implemented Or at the implementation level –i.e., data structures in KB and algorithms that manipulate them

Why knowledge-base The state of the world –may require lots of information.. The agent knowledge of the state of the world: – If s is world state K(s) is what the agent knows. For economy: –Not everything explicitly specified. Some facts can be inferred. –Agent may infer whatever he does not know explicitly. Nillson: Constraints on feature values –Block A is not on the floor Issues: –In what language to express what the agent knows about the world. How explicit to make this knowledge. How to infer. Description of the world Agent knowledge of state Agent explicit specification of what he knows

The party example If Alex goes, then Beki goes: A  B If Chris goes, then Alex goes: C  A Beki does not go: not B Chris goes: C Query: Is it possible to satisfy all these conditions? This is called satisfiability problem.

Example of languages Programming languages: –Formal languages, not ambiguous, but cannot express partial information. Not expressive enough. Natural languages: –Very expressive but ambiguous: ex: small dogs and cats. Good representation language: –Both formal and can express partial information, can accommodate inference Main approach used in AI: Logic-based languages. Predicate-logic with Horn clauses

Deduction algorithms Example: Given P  R, and Q  ~R Can we deduce ~(P & Q)? Applications expert systems (Mycin, dendral are early examples) logic programming automatic theorem proving (software validation) Resolution strategy

Example:  X (  Y ((mother(X)  child_of(Y,X))  loves(X,Y))) mother(mary) child_of(tom,mary) Can we deduce? loves(tom, mary) Logical deduction in predicate logic

techniques Probabilistic approach to AI Knowledge representation models uncertainties. Example: H = “Have a headache” F = “Coming down with Flu” P(H) = 1/10 P(F) = 1/40 P(H|F) = ½ Given that you have a headache, what is the probability that you have flu? This kind of modeling is widely used in various prediction problems, e.g., in determining the insurance premium for car etc.

Probabilistic approach to AI Some games are inherently probabilistic. Financial markets backgammon

techniques Training set New applicant: (young, has job, does not own house, good credit). Will (s)he default? We can build a probabilistic model to answer.

techniques Machine learning approach to AI: self-improving algorithms solution obtained without explicit programming Closer to modeling human intelligence or natural intelligence (we learn many things by observing even if step by step procedure absent) Prominent examples: Neural networks Genetic algorithms, evolutionary method

techniques Neuron (very roughly modeled by neurons in human brains.

techniques An algorithm called back propagation algorithm is used to adjust the weights of neurons based on the discrepancy between correct output and computed output.

techniques Evolutionary algorithms: encoding of the collection of solutions as strings. goal is to evolve the “best” solution. use cross-over and mutation and iterate. Example of cross-over and mutation

AI prehistory PhilosophyLogic, methods of reasoning, mind as physical system foundations of learning, language, rationality MathematicsFormal representation and proof algorithms, computation, (un)decidability, (in)tractability, probability Economicsutility, decision theory Neuroscience physical substrate for mental activity Psychology phenomena of perception and motor control, experimental techniques Computer building fast computers engineering Control theorydesign systems that maximize an objective function over time Linguisticsknowledge representation, grammar

Abridged history of AI 1943 McCulloch & Pitts: Boolean circuit model of brain 1950 Turing's "Computing Machinery and Intelligence" 1956Dartmouth meeting: "Artificial Intelligence" adopted 1950sEarly AI programs, including Samuel's checkers program, Newell & Simon's Logic Theorist, Gelernter's Geometry Engine 1965Robinson's complete algorithm for logical reasoning 1966—73AI discovers computational complexity Neural network research almost disappears 1969—79Early development of knowledge-based systems AI becomes an industry Neural networks return to popularity AI becomes a science, probabilistic techniques dominate The emergence of intelligent agents

State of the art Deep Blue defeated the reigning world chess champion Garry Kasparov in 1997 Proved a mathematical conjecture (Robbins conjecture) unsolved for decades No hands across America (driving autonomously 98% of the time from Pittsburgh to San Diego) During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling program that involved up to 50,000 vehicles, cargo, and people NASA's on-board autonomous planning program controlled the scheduling of operations for a spacecraft Proverb solves crossword puzzles better than most humans