1. Cognitive Science and its Applications
R. Karthikeyan,
Associate Professor/CSE
PSNACET
5/7/2019

2. Cognitive
Pertaining to the act or process of knowing, perceiving, remembering etc.
Pertaining to the mental processes of perception, memory, judgment and reasoning
5/7/2019

3. Cognitive Science
Interdisciplinary scientific study of the mind and its processes
Examines what cognition is, what it does and how it works.
Includes research on intelligence and behavior, especially focusing on how information is represented, processed, and transformed within nervous systems and machines
Multiple research disciplines – Psychology, AI, Philosophy, Neuroscience, linguistics, and Anthropology.
5/7/2019

4. Psychology Cognitive science Social psychology Educational psychology
5/7/2019

5. Levels of analysis
Computational theory, specifying the goals of the computation
Representation and algorithms, giving a representation of the inputs and outputs and the algorithms which transform one into the other
Hardware implementation, how algorithm and representation may be physically realized.
5/7/2019

6. Example Task: Multiplication. Input numbers x and y. Output x times y.
Algorithm: Given inputs x and y, look up number z on row x and column y. Output z.
Implementation: human being and paper.
5/7/2019

7. Alternative algorithm
How to calculate x times y:
If x =1, then answer is y.
If x>1, add y to itself (x-1) number of times. The result is the answer.
Example: 3x5 = = 15
5/7/2019

8. Two points The same task can be performed with different algorithms.
Two different systems can do the same task in very different ways.
The same algorithm can be implemented with different hardware.
5/7/2019

9. Scope
“Cognitive“ is used for "any kind of mental operation or structure that can be studied in precise terms“
Large field and covers a wide array of topics on cognition
Social and cultural factors
emotion, consciousness 
Animal cognition, comparative and evolutionary approaches
5/7/2019

10. Artificial intelligence
Major contribution of AI and cognitive science to psychology - information processing model of human thinking
Involves the study of cognitive phenomena in machines.
Computers are also widely used as a tool with which to study cognitive phenomena. 
Computational modeling uses simulations to study how human intelligence may be structured.
5/7/2019

11. Information processing everywhere
Perception
acquiring real-time information about the surrounding environment.
Language use
making use of information about syntax, semantics and phonology.
Reasoning
combining different sources of information, deriving new information, testing consistency of information, etc.
Action
making use of information in action planning and guidance.
Memory
storing and retrieving information
5/7/2019

12. Attention Selection of important information
The human mind is bombarded with millions of stimuli and it must have a way of deciding which of this information to process.
Experiments that support this metaphor include the dichotic listening task and studies of inattentional blindness.
5/7/2019

13. Attention(Contd.)
In the dichotic listening task, subjects are bombarded with two different messages, one in each ear, and told to focus on only one of the messages.
At the end of the experiment, when asked about the content of the unattended message, subjects cannot report it.
5/7/2019

14. Knowledge and processing of language
Ability to learn and understand language is an extremely complex process
Driving research questions in studying how the brain itself processes language
To what extent is linguistic knowledge innate or learned?
Why is it more difficult for adults to acquire a second-language than it is for infants to acquire their first-language?
How are humans able to understand novel sentences?
5/7/2019

15. Knowledge and processing of language(contd.)
Example: Syntactic Disambiguation
“We shall discuss violence on TV.”
Two interpretations : VP V NP N
discuss
violence PP on TV P VP V NP PP N
discuss
violence on TV P
5/7/2019

16. Learning and Development
Learning and development are the processes by which we acquire knowledge and information over time.
Infants are born with little or no knowledge (depending on how knowledge is defined), yet they rapidly acquire the ability to use language, walk, and recognize people and objects.
Research in learning and development aims to explain the mechanisms by which these processes might take place.
5/7/2019

17. Memory
Memory allows us to store information for later retrieval. Memory is often thought of consisting of both a long-term and short-term store.
Long-term memory allows us to store information over prolonged periods (days, weeks, years). We do not yet know the practical limit of long-term memory capacity.
Short-term memory allows us to store information over short time scales (seconds or minutes).
5/7/2019

18. Memory(contd.)
Memory is also often grouped into declarative and procedural forms
Declarative memory—grouped into subsets of semantic and episodic forms of memory-refers to our memory for facts and specific knowledge, specific meanings, and specific experiences(e.g. "Who was the first president of the U.S.A.?", or "What did I eat for breakfast four days ago?").
Procedural memory allows us to remember actions and motor sequences (e.g. how to ride a bicycle) and is often dubbed implicit knowledge or memory
5/7/2019

19. Perception and action
Perception is the ability to take in information via the senses, and process it in some way. 
Vision and hearing are two dominant senses that allow us to perceive the environment
Action is taken to refer to the output of a system. In humans, this is accomplished through motor responses.
Spatial planning and movement, speech production, and complex motor movements are all aspects of action.
5/7/2019

20. Applications of Cognitive Science
5/7/2019

21. Virtual Human- Human Computer Interaction
Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them
5/7/2019

22. Human
Humans are limited in their capacity to process information. This has important implications for Design
Information is received and responses given via a number of input and output channels:
Visual Channel
Auditory Channel
Haptic Channel
Movement
Information is stored in memory:
Sensory Memory
Working Memory
Long-Term Memory
5/7/2019

23. Computer There is not much difference in Human and Computer
Computer consist of
Input Devices
Output Devices
Memory
Processing
Computer can be Mobile, Spacecraft Cockpit, etc.
5/7/2019

24. Interaction
HCI is concerned with joint performance of Task by Human & Computer
Communication between The User and The System
Physical Interaction
Interaction Devices
Conceptual Interaction
Interaction Styles
5/7/2019

25. Goals The main goal of HCI is Usability
“It is a measure of the effectiveness, efficiency and satisfaction with which specified user can achieve specified goals in a particular environment”
A usable System is:
Easy to Learn
Easy to Remember how to Use
Effective to Use
Efficient to Use
Safe to Use
Enjoyable To Use
5/7/2019

26. The Model Human Processor
The model human processor consists of three interacting systems. Each has its own memory and processor.
Perceptual processor
Outputs into audio storage
Outputs into visual storage
Cognitive processor
Outputs into working memory.
Has access to:
Working memory
Long term memory Motor processor
Carries out actions
5/7/2019

27. User Interaction Design
Data Collection
Data Analysis
User Modeling
Design
Prototyping
Evaluation
5/7/2019

28. User Interaction Design Flow
(Re)Design
Identify needs/establish requirements
Evaluate
Build an interactive version
Final product
5/7/2019

29. Interaction Styles Command Line Interface Menus
Question & Answer Dialogue
Form-Fill Interface
WIMP
Direct Manipulation
Natural Language
5/7/2019

30. Interaction Devices Input Devices Keyboards
Mouse, Trackballs, Touch Pads
Touch Screens
Speech Input
3D trackers, 3D Mouse
Output Devices
Screens
Printers
Sound Output
5/7/2019

31. Layered Virtual Human
5/7/2019

32. Layered Virtual Human(contd.)
Cognitive Layer
Inner layer where cognitive components exist
One cognitive level per virtual human
Mind of the virtual human that makes decisions based on input, goals, and desired behavior.
Virtual Human Layer
Consists of components that make up the virtual human
Input : Vision, Speech and Smell
Output : Verbal speech, Body gestures,
Actions
This layer can be used by one or more virtual human. Closely connected to cognitive layer .
5/7/2019

33. Layered Virtual Human(contd.)
Simulation Layer
This is everything else that has to do with the environment that the virtual humans exist in.
Include the game engine that creates the world that the characters are displayed in and interact with a world.
All input from the real world, like gesture recognition, object or human positions, microphones or cameras would feed into the simulation layer.
5/7/2019

34. Sense-Think-Act Cycle
5/7/2019

35. Sense-Think-Act Cycle(contd.)
Virtual humans follow the same paradigm as Belief- Desire-Intention (BDI) style agents with a sense-think- act cycle
The cognitive and virtual human layers correspond to the right side of this diagram
Simulation and real world correspond to the left side
The virtual human research attempts to integrate all of these layers and attempts to
answer some fundamental questions about how they should be built, and how they can be effectively used in training environments.
5/7/2019

36. Virtual Human Integrated Architecture
5/7/2019

37. Virtual Human Integrated Architecture(contd)
Cognitive Layer
Intelligent Agent - major reasoning engine of the agent, based on the SOAR Cognitive Architecture.
SOAR is a symbolic reasoning system that includes concepts such as long term and short term memory, goal directed behavior and a decision procedure for selecting rules.
Dialog Manager – reason about trainee’s utterances
Emotional Modeling - appraising current events and the current situation and comparing them with the agent's beliefs and goals to determine emotional state and appropriate coping behavior.
5/7/2019

38. Virtual Human Integrated Architecture(contd)
Virtual Human Layer
Speech recognition
Natural Language Understanding
Natural Language Generation
Non verbal Behavioral Generator
Smart Body
Text to Speech
5/7/2019

39. Virtual Human Integrated Architecture(contd)
Simulation Layer
Unreal Tournament and Integrated Architecture
Gaze/Gesture Tracking/Input system
5/7/2019

40. SASO-ST [Stability and Support Operations - Simulation and Training]
5/7/2019

41. SASO-ST
A research prototype demonstrating advanced virtual human technologies in a new negotiation domain.
Trainees are to communicate in real-time with an embodied virtual human doctor to negotiate and convince Dr. Perez to move the clinic out of harms way
5/7/2019

42. ELECT-Bilat
5/7/2019

43. ELECT-Bilat Enhanced Learning Environment with
Creative Technologies is a game based simulation for soldiers to practice and conduct bilateral engagements in a cultural context that includes virtual humans that verbally respond to the selected questions.
Uses a menu based system instead of speech recognition
5/7/2019

44. Cognition reigns but does not rule Paul Valéry
5/7/2019

45. Thank You
5/7/2019