1 Computer Science 129 Science, Computing and Society Week 4 Chapter 3

2 GRE/GMAT WORDS Homogeneous – common origin Loquacious – talkative Pragmatic – practical values and results Volatile – explosive, unstable

3 Chapter 3 INFORMATION IS PHYSICAL Edward Fredkin: “the universe is a computer” Consider flight simulators Fredkin believes computer running the program of our universe is not located in our universe

4 Chapter 3 Strange theory….but…. Processing information is a common feature of the universe Brains do it! All communication is just information processing

5 Chapter 3 The main idea of this is: Information is something real and important in the description of nature Information is not abstract Information is physical

6 Chapter 3 Consider Complex systems: the universe is full of them Chemicals in our bodies - Black holes –Summer thunderstorms All are information-processing mechanisms They acquire information about the environment and use it to decide what to do next.

7 Chapter 3 Information processing systems are essentially “observers” But, in classical physics, observers are irrelevant Laws of nature should be unaffected by observation – they are there whether anyone sees them or not

8 Chapter 3 19 th century Physicist James Maxwell proposed a hypothetical observer known as his demon This demon related to the story of Thermodynamics

9 Chapter 3 The 1 st and 2 nd Laws of Thermodynamics Thermodynamics – study of energy (heat) change (Empirical Laws) 1 st - Total energy of a system plus the surroundings is constant – or The Law of Conservation of Energy 2 nd – Entropy (disorder) of the universe increases – energy can’t even break even

10 Chapter 3 The Second Law requires any use of energy to produce some “waste heat” that is too degraded to do further work Order gives way to disorder - or entropy

11 Chapter 3 James Maxwell – known for explaining Electricity and Magnetism using mathematics (Maxwell’s Equations) Maxwell believed the 2 nd Law was a statistical law – based on probabilities Consider speed of molecules (temperature)

12 Chapter 3 In measuring temperature you can precisely measure the average speed of the molecules not the exact speed of any one molecule

13 Chapter 3 Maxwell’s demon – 2 rooms with same temperatures – door between is controlled by demon – demon lets fast moving molecules into one room and slow into the other One room hot, other very cold – demon could heat or air-condition a house Demon gets useful energy from waste heat – breaking 2 nd Law Or did it?

14 Chapter 3 No – it didn’t The Demon, must be made, then it must find out how fast a molecule is going – measuring it, all of which requires energy – more energy that it could get out of the system demon.freeonlinegames.com/

15 Chapter 3 Shannon’s Entropy Shannon’s math measured information in bits (information theory : 2 1 =1 bit and two possibilities, 2 2 =2 bits and 4 possibilities, 2 3 =3 bits and 8 possibilities and so on)

16 Chapter 3 Remember our base 2 system? 1 bit = 2 possibilities ; zero=0, one=1 2 bits = 4 possibilities ; zero=00, one=01, two=10, three=11 3 bits = 8 possibilities ; Zero=000, one=001, two=010, three=011, four=100, five=101, six=110, seven=111

17 Chapter 3 Entropy in computer science measures uncertainty Using bits to quantify weather uncertainty:

18 Chapter 3 In the morning there are the following possibilities (pretend these are like math givens. There are only these possibilities, no more, no less): Sunny all dayCloudy then Sunny Cloudy all daySunny then Storm Storm all dayCloudy then Storm Sunny then CloudyStorm then Cloudy 8 possibilities require 3 bits (2 3 =8) or 3 bits of uncertainty

19 Chapter 3 Jill is the weather person on channel Z Her weather forecast is essentially: Good Bad This can be represented with 1 bit (2 1 =2) If we have 3 bits of uncertainty and 1 bit of information – missing 2 bits – even if she is always right, we still have more uncertainty than we do information

20 Chapter 3 Shannon’s equations look very similar to the 2 nd Law of Thermodynamics This raises the question: Is there a real connection between the entropy of physics and the entropy of information?

21 Chapter 3 Information is always embodied in some physical representation – ink on paper – holes in punch cards, magnetic patterns on a floppy disk – arrangement of atoms in DNA Information is Physical

22 What You Should Know WHAT IS THE FIRST LAW OF THERMODYNAMICS? WHAT IS THE SECOND LAW OF THERMODYNAMICS?

23 What You Should Know SHORT ESSAY: EXPLAIN MAXWELL’S DEMON AND HOW DOES MAXWELL’S DEMON RELATE TO THE SECOND LAW OF THERMODYNAMICS?

24 What You Should Know HOW DOES INFORMATION THEORY WORK?

25 What You Should Know SHORT ESSAY: EXPLAIN HOW SHANNON’S ENTROPY WORKS. GIVE EXAMPLES.