Cognitive Science and its Applications R. Karthikeyan, Associate Professor/CSE PSNACET 5/7/2019

Cognitive Pertaining to the act or process of knowing, perceiving, remembering etc. Pertaining to the mental processes of perception, memory, judgment and reasoning 5/7/2019

Cognitive Science Interdisciplinary scientific study of the mind and its processes Examines what cognition is, what it does and how it works. Includes research on intelligence and behavior, especially focusing on how information is represented, processed, and transformed within nervous systems and machines Multiple research disciplines – Psychology, AI, Philosophy, Neuroscience, linguistics, and Anthropology. 5/7/2019

Psychology Cognitive science Social psychology Educational psychology 5/7/2019

Levels of analysis Computational theory, specifying the goals of the computation Representation and algorithms, giving a representation of the inputs and outputs and the algorithms which transform one into the other Hardware implementation, how algorithm and representation may be physically realized. 5/7/2019

Example Task: Multiplication. Input numbers x and y. Output x times y. Algorithm: Given inputs x and y, look up number z on row x and column y. Output z. Implementation: human being and paper. 5/7/2019

Alternative algorithm How to calculate x times y: If x =1, then answer is y. If x>1, add y to itself (x-1) number of times. The result is the answer. Example: 3x5 = 5+5+5 = 15 5/7/2019

Two points The same task can be performed with different algorithms. Two different systems can do the same task in very different ways. The same algorithm can be implemented with different hardware. 5/7/2019

Scope “Cognitive“ is used for "any kind of mental operation or structure that can be studied in precise terms“ Large field and covers a wide array of topics on cognition Social and cultural factors emotion, consciousness  Animal cognition, comparative and evolutionary approaches 5/7/2019

Artificial intelligence Major contribution of AI and cognitive science to psychology - information processing model of human thinking Involves the study of cognitive phenomena in machines. Computers are also widely used as a tool with which to study cognitive phenomena.  Computational modeling uses simulations to study how human intelligence may be structured. 5/7/2019

Information processing everywhere Perception acquiring real-time information about the surrounding environment. Language use making use of information about syntax, semantics and phonology. Reasoning combining different sources of information, deriving new information, testing consistency of information, etc. Action making use of information in action planning and guidance. Memory storing and retrieving information 5/7/2019

Attention Selection of important information The human mind is bombarded with millions of stimuli and it must have a way of deciding which of this information to process. Experiments that support this metaphor include the dichotic listening task and studies of inattentional blindness. 5/7/2019

Attention(Contd.) In the dichotic listening task, subjects are bombarded with two different messages, one in each ear, and told to focus on only one of the messages. At the end of the experiment, when asked about the content of the unattended message, subjects cannot report it. 5/7/2019

Knowledge and processing of language Ability to learn and understand language is an extremely complex process Driving research questions in studying how the brain itself processes language To what extent is linguistic knowledge innate or learned? Why is it more difficult for adults to acquire a second-language than it is for infants to acquire their first-language? How are humans able to understand novel sentences? 5/7/2019

Knowledge and processing of language(contd.) Example: Syntactic Disambiguation “We shall discuss violence on TV.” Two interpretations : VP V NP N discuss violence PP on TV P VP V NP PP N discuss violence on TV P 5/7/2019

Learning and Development Learning and development are the processes by which we acquire knowledge and information over time. Infants are born with little or no knowledge (depending on how knowledge is defined), yet they rapidly acquire the ability to use language, walk, and recognize people and objects. Research in learning and development aims to explain the mechanisms by which these processes might take place. 5/7/2019

Memory Memory allows us to store information for later retrieval. Memory is often thought of consisting of both a long-term and short-term store. Long-term memory allows us to store information over prolonged periods (days, weeks, years). We do not yet know the practical limit of long-term memory capacity. Short-term memory allows us to store information over short time scales (seconds or minutes). 5/7/2019

Memory(contd.) Memory is also often grouped into declarative and procedural forms Declarative memory—grouped into subsets of semantic and episodic forms of memory-refers to our memory for facts and specific knowledge, specific meanings, and specific experiences(e.g. "Who was the first president of the U.S.A.?", or "What did I eat for breakfast four days ago?"). Procedural memory allows us to remember actions and motor sequences (e.g. how to ride a bicycle) and is often dubbed implicit knowledge or memory 5/7/2019

Perception and action Perception is the ability to take in information via the senses, and process it in some way.  Vision and hearing are two dominant senses that allow us to perceive the environment Action is taken to refer to the output of a system. In humans, this is accomplished through motor responses. Spatial planning and movement, speech production, and complex motor movements are all aspects of action. 5/7/2019

Applications of Cognitive Science 5/7/2019

Virtual Human- Human Computer Interaction Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them 5/7/2019

Human Humans are limited in their capacity to process information. This has important implications for Design Information is received and responses given via a number of input and output channels: Visual Channel Auditory Channel Haptic Channel Movement Information is stored in memory: Sensory Memory Working Memory Long-Term Memory 5/7/2019

Computer There is not much difference in Human and Computer Computer consist of Input Devices Output Devices Memory Processing Computer can be Mobile, Spacecraft Cockpit, etc. 5/7/2019

Interaction HCI is concerned with joint performance of Task by Human & Computer Communication between The User and The System Physical Interaction Interaction Devices Conceptual Interaction Interaction Styles 5/7/2019

Goals The main goal of HCI is Usability “It is a measure of the effectiveness, efficiency and satisfaction with which specified user can achieve specified goals in a particular environment” A usable System is: Easy to Learn Easy to Remember how to Use Effective to Use Efficient to Use Safe to Use Enjoyable To Use 5/7/2019

The Model Human Processor The model human processor consists of three interacting systems. Each has its own memory and processor. Perceptual processor Outputs into audio storage Outputs into visual storage Cognitive processor Outputs into working memory. Has access to: Working memory Long term memory Motor processor Carries out actions 5/7/2019

User Interaction Design Data Collection Data Analysis User Modeling Design Prototyping Evaluation 5/7/2019

User Interaction Design Flow (Re)Design Identify needs/establish requirements Evaluate Build an interactive version Final product 5/7/2019

Interaction Styles Command Line Interface Menus Question & Answer Dialogue Form-Fill Interface WIMP Direct Manipulation Natural Language 5/7/2019

Interaction Devices Input Devices Keyboards Mouse, Trackballs, Touch Pads Touch Screens Speech Input 3D trackers, 3D Mouse Output Devices Screens Printers Sound Output 5/7/2019

Layered Virtual Human 5/7/2019

Layered Virtual Human(contd.) Cognitive Layer Inner layer where cognitive components exist One cognitive level per virtual human Mind of the virtual human that makes decisions based on input, goals, and desired behavior. Virtual Human Layer Consists of components that make up the virtual human Input : Vision, Speech and Smell Output : Verbal speech, Body gestures, Actions This layer can be used by one or more virtual human. Closely connected to cognitive layer . 5/7/2019

Layered Virtual Human(contd.) Simulation Layer This is everything else that has to do with the environment that the virtual humans exist in. Include the game engine that creates the world that the characters are displayed in and interact with a world. All input from the real world, like gesture recognition, object or human positions, microphones or cameras would feed into the simulation layer. 5/7/2019

Sense-Think-Act Cycle 5/7/2019

Sense-Think-Act Cycle(contd.) Virtual humans follow the same paradigm as Belief- Desire-Intention (BDI) style agents with a sense-think- act cycle The cognitive and virtual human layers correspond to the right side of this diagram Simulation and real world correspond to the left side The virtual human research attempts to integrate all of these layers and attempts to answer some fundamental questions about how they should be built, and how they can be effectively used in training environments. 5/7/2019

Virtual Human Integrated Architecture 5/7/2019

Virtual Human Integrated Architecture(contd) Cognitive Layer Intelligent Agent - major reasoning engine of the agent, based on the SOAR Cognitive Architecture. SOAR is a symbolic reasoning system that includes concepts such as long term and short term memory, goal directed behavior and a decision procedure for selecting rules. Dialog Manager – reason about trainee’s utterances Emotional Modeling - appraising current events and the current situation and comparing them with the agent's beliefs and goals to determine emotional state and appropriate coping behavior. 5/7/2019

Virtual Human Integrated Architecture(contd) Virtual Human Layer Speech recognition Natural Language Understanding Natural Language Generation Non verbal Behavioral Generator Smart Body Text to Speech 5/7/2019

Virtual Human Integrated Architecture(contd) Simulation Layer Unreal Tournament and Integrated Architecture Gaze/Gesture Tracking/Input system 5/7/2019

SASO-ST [Stability and Support Operations - Simulation and Training] 5/7/2019

SASO-ST A research prototype demonstrating advanced virtual human technologies in a new negotiation domain. Trainees are to communicate in real-time with an embodied virtual human doctor to negotiate and convince Dr. Perez to move the clinic out of harms way 5/7/2019

ELECT-Bilat 5/7/2019

ELECT-Bilat Enhanced Learning Environment with Creative Technologies is a game based simulation for soldiers to practice and conduct bilateral engagements in a cultural context that includes virtual humans that verbally respond to the selected questions. Uses a menu based system instead of speech recognition 5/7/2019

Cognition reigns but does not rule Paul Valéry 5/7/2019

Thank You 5/7/2019