1. IA32 Linux Memory Layout Stack Heap Data Text not drawn to scale FF
Runtime stack (8MB limit)
Heap
Dynamically allocated storage
When call malloc(), calloc(), new()
Data
Statically allocated data
E.g., arrays & strings declared in code
Text
Executable machine instructions
Read-only
8MB
Heap
Data
Upper 2 hex digits = 8 bits of address
Text 08 00

2. Memory Allocation Example
not drawn to scale
Memory Allocation Example FF
Stack
char big_array[1<<24]; /* 16 MB */
char huge_array[1<<28]; /* 256 MB */
int beyond;
char *p1, *p2, *p3, *p4;
int useless() { return 0; }
int main() {
p1 = malloc(1 <<28); /* 256 MB */
p2 = malloc(1 << 8); /* 256 B */
p3 = malloc(1 <<28); /* 256 MB */
p4 = malloc(1 << 8); /* 256 B */
/* Some print statements ... */ }
Heap
Data
Text
Where does everything go? 08 00

3. IA32 Example Addresses address range ~232 not drawn to scale FF Stack
$esp 0xffffbcd0
p3 0x
p1 0x
p4 0x1904a110
p2 0x1904a008
&p2 0x
beyond 0x
big_array 0x
huge_array 0x
main() 0x080483c6
useless() 0x
final malloc() 0x006be166 80
Heap
Data
malloc() is dynamically linked
address determined at runtime
Text 08 00

4. Pointers Sure. Try How about some 0x0000A4F5, pointers to deal
with the new boss?
Sure. Try
0x0000A4F5,
0x00008EEF and
0x0000B32A.

5. C operators -> has very high precedence () has very high precedence
Operators Associativity
() [] -> left to right
! ~ * & (type) sizeof right to left
* / % left to right
left to right
<< >> left to right
< <= > >= left to right
== != left to right
& left to right
^ left to right
| left to right
&& left to right
|| left to right
?: right to left
= += -= *= /= %= &= ^= != <<= >>= right to left
, left to right
-> has very high precedence
() has very high precedence
monadic * just below

6. C Pointer Declarations: Test Yourself!
int *p p is a pointer to int
int *p[13] p is an array[13] of pointer to int
int *(p[13]) p is an array[13] of pointer to int
int **p p is a pointer to a pointer to an int
int (*p)[13] p is a pointer to an array[13] of int
int *f() f is a function returning a pointer to int
int (*f)() f is a pointer to a function returning int
int (*(*f())[13])() f is a function returning ptr to an array[13]
of pointers to functions returning int
int (*(*x[3])())[5] x is an array[3] of pointers to functions
returning pointers to array[5] of ints

7. C Pointer Declarations
int *p p is a pointer to int
int *p[13] p is an array[13] of pointer to int
int *(p[13]) p is an array[13] of pointer to int
int **p p is a pointer to a pointer to an int
int (*p)[13] p is a pointer to an array[13] of int
int *f() f is a function returning a pointer to int
int (*f)() f is a pointer to a function returning int
int (*(*f())[13])() f is a function returning ptr to an array[13]
of pointers to functions returning int
int (*(*x[3])())[5] x is an array[3] of pointers to functions
returning pointers to array[5] of ints

8. Avoiding Complex Declarations
Use typedef to build up the declaration
Instead of int (*(*x[3])())[5] :
typedef int fiveints[5];
typedef fiveints* p5i;
typedef p5i (*f_of_p5is)();
f_of_p5is x[3];
x is an array of 3 elements, each of which is a pointer to a function returning an array of 5 ints

9. Internet Worm and IM War
November, 1988
Internet Worm attacks thousands of Internet hosts.
How did it happen?
July, 1999
Microsoft launches MSN Messenger (instant messaging system).
Messenger clients can access popular AOL Instant Messaging Service (AIM) servers
AIM
client
MSN
server
MSN
client
AIM
server
AIM
client

10. Internet Worm and IM War (cont.)
August 1999
Mysteriously, Messenger clients can no longer access AIM servers.
Microsoft and AOL begin the IM war:
AOL changes server to disallow Messenger clients
Microsoft makes changes to clients to defeat AOL changes.
At least 13 such skirmishes.
How did it happen?
The Internet Worm and AOL/Microsoft War were both based on stack buffer overflow exploits!
many Unix functions do not check argument sizes.
allows target buffers to overflow.

11. String Library Code Implementation of Unix function gets()
No way to specify limit on number of characters to read
Similar problems with other Unix functions
strcpy: Copies string of arbitrary length
scanf, fscanf, sscanf, when given %s conversion specification
/* Get string from stdin */
char *gets(char *dest) { int c = getchar();
char *p = dest;
while (c != EOF && c != '\n') {
*p++ = c;
c = getchar(); }
*p = '\0';
return dest;

12. Vulnerable Buffer Code
/* Echo Line */ void echo() { char buf[4]; /* Way too small! */ gets(buf); puts(buf); }
int main() { printf("Type a string:"); echo(); return 0; }
unix>./bufdemo
Type a string:
unix>./bufdemo
Type a string:
Segmentation Fault
unix>./bufdemo
Type a string: ABC
Segmentation Fault

13. Buffer Overflow Disassembly
080484f0 <echo>:
80484f0: push %ebp
80484f1: 89 e mov %esp,%ebp
80484f3: push %ebx
80484f4: 8d 5d f lea 0xfffffff8(%ebp),%ebx
80484f7: 83 ec sub $0x14,%esp
80484fa: 89 1c mov %ebx,(%esp)
80484fd: e8 ae ff ff ff call b0 <gets>
: 89 1c mov %ebx,(%esp)
: e8 8a fe ff ff call
804850a: 83 c add $0x14,%esp
804850d: 5b pop %ebx
804850e: c leave
804850f: c ret
80485f2: e8 f9 fe ff ff call f0 <echo>
80485f7: 8b 5d fc mov 0xfffffffc(%ebp),%ebx
80485fa: c leave
80485fb: 31 c xor %eax,%eax
80485fd: c ret

14. Buffer Overflow Stack Before call to gets Stack Frame for main
/* Echo Line */ void echo() { char buf[4]; /* Way too small! */ gets(buf); puts(buf); }
Return Address
Saved %ebp
%ebp
Stack Frame
for echo
[3]
[2]
[1]
[0]
buf
echo:
pushl %ebp # Save %ebp on stack
movl %esp, %ebp
pushl %ebx # Save %ebx
leal -8(%ebp),%ebx # Compute buf as %ebp-8
subl $20, %esp # Allocate stack space
movl %ebx, (%esp) # Push buf on stack
call gets # Call gets
. . .

15. Buffer Overflow Stack Example
unix> gdb bufdemo
(gdb) break echo
Breakpoint 1 at 0x
(gdb) run
Breakpoint 1, 0x in echo ()
(gdb) print /x $ebp
$1 = 0xffffc638
(gdb) print /x *(unsigned *)$ebp
$2 = 0xffffc658
(gdb) print /x *((unsigned *)$ebp + 1)
$3 = 0x80485f7
Before call to gets
Before call to gets
0xffffc658
Stack Frame
for main
Stack Frame
for main
Return Address 08 04 85 f7
Saved %ebp ff ff c6 58
0xffffc638
Stack Frame
for echo
Stack Frame
for echo
[3]
[2]
[1]
[0]
buf xx xx xx xx
buf
80485f2: call 80484f0 <echo>
80485f7: mov 0xfffffffc(%ebp),%ebx # Return Point

16. Buffer Overflow Example #1
Before call to gets
Input
0xffffc658
0xffffc658
Stack Frame
for main
Stack Frame
for main 08 04 85 f7 08 04 85 f7 ff ff c6 58
0xffffc638 ff ff c6 58
0xffffc638
Stack Frame
for echo 00
Stack Frame
for echo 37 36 35 xx xx xx xx
buf 34 33 32 31
buf
Overflow bufs, but no problem

17. Buffer Overflow Example #2
Before call to gets
Input
0xffffc658
0xffffc658
Stack Frame
for main
Stack Frame
for main 08 04 85 f7 08 04 85 f7 ff ff c6 58
0xffffc638 ff ff c6 00
0xffffc638
Stack Frame
for echo 38
Stack Frame
for echo 37 36 35 xx xx xx xx
buf 34 33 32 31
buf
Base pointer corrupted
. . .
804850a: 83 c4 14 add $0x14,%esp # deallocate space
804850d: 5b pop %ebx # restore %ebx
804850e: c leave # movl %ebp, %esp; popl %ebp
804850f: c ret # Return

18. Buffer Overflow Example #3
Before call to gets
Input
0xffffc658
0xffffc658
Stack Frame
for main
Stack Frame
for main 08 04 85 f7 08 04 85 00 ff ff c6 58
0xffffc638 43 42 41 39
0xffffc638
Stack Frame
for echo 38
Stack Frame
for echo 37 36 35 xx xx xx xx
buf 34 33 32 31
buf
Return address corrupted
80485f2: call 80484f0 <echo>
80485f7: mov 0xfffffffc(%ebp),%ebx # Return Point

19. Malicious Use of Buffer Overflow
Stack after call to gets()
void foo(){
bar();
... }
foo stack frame
return
address A B
pad
int bar() {
char buf[64];
gets(buf);
...
return ...; }
data written
by gets()
exploit
code
bar stack frame B
Input string contains byte representation of executable code
Overwrite return address with address of buffer
When bar() executes ret, will jump to exploit code

20. Exploits Based on Buffer Overflows
Buffer overflow bugs allow remote machines to execute arbitrary code on victim machines
Internet worm
Early versions of the finger server (fingerd) used gets() to read the argument sent by the client:
finger
Worm attacked fingerd server by sending phony argument:
finger “exploit-code padding new-return-address”
Exploit code: executed a root shell on the victim machine with a direct TCP connection to the attacker.

21. Exploits Based on Buffer Overflows
Buffer overflow bugs allow remote machines to execute arbitrary code on victim machines
IM War
AOL exploited existing buffer overflow bug in AIM clients
Exploit code: returned 4-byte signature (the bytes at some location in the AIM client) to server.
When Microsoft changed code to match signature, AOL changed signature location.

22. Example Virus: Code Red Worm
History
June 18, Microsoft announces buffer overflow vulnerability in IIS Internet server
July 19, over 250,000 machines infected by new virus in 9 hours
White house must change its IP address. Pentagon shut down public WWW servers for day
When CS:APP Web Site was set up
Received strings of form
GET /default.ida?NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN....NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090%u9090%u8190%u00c3%u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a
HTTP/1.0" "-" "-"

23. Code Red Exploit Code Starts 100 threads running Spread self
Generate random IP addresses & send attack string
Between 1st & 19th of month
Attack
Send 98,304 packets; sleep for 4-1/2 hours; repeat
Denial of service attack
Between 21st & 27th of month
Deface server’s home page
After waiting 2 hours

24. Avoiding Overflow Vulnerability
/* Echo Line */ void echo() { char buf[4]; /* Way too small! */ fgets(buf, 4, stdin); puts(buf); }
Use library routines that limit string lengths
fgets instead of gets
strncpy instead of strcpy
Don’t use scanf with %s conversion specification
Use fgets to read the string
Or use %ns where n is a suitable integer

25. System-Level Protections
Randomized stack offsets
At start of program, allocate random amount of space on stack
Makes it difficult for hacker to predict beginning of inserted code
Non-executable code segments
In traditional x86, segments are either “read-only” or “writeable”
Can execute anything readable
Better: add explicit “execute” permission
Stack corruption detection
Compiler inserts “canary” just beyond array and code to check it before function return
Code aborts with error if canary changed
unix> gdb bufdemo
(gdb) break echo
(gdb) run
(gdb) print /x $ebp
$1 = 0xffffc638
$2 = 0xffffbb08
$3 = 0xffffc6a8

26. Web-aside: Assembly & C
Requirements to have asm + C integrated together?
In assembly, follow register usage, parameter passing, and return value conventions
Problems with inline assembly
Not portable
Knowing what registers are safe to use
Advantages of GCC’s extended asm
You inform compiler of source values required, results your code generates, and registers that will be overwritten
GCC generates code to correctly set up source values, execute desired instructions, and correctly use computed results

27. BONUS SLIDES

28. IA32 Floating Point (x87) History Summary Floating Point Formats
8086: first computer to implement IEEE FP
separate 8087 FPU (floating point unit)
486: merged FPU and Integer Unit onto one chip
Becoming obsolete with x86-64
Summary
Hardware to add, multiply, and divide
Floating point data registers
Various control & status registers
Floating Point Formats
single precision (C float): 32 bits
double precision (C double): 64 bits
extended precision (C long double): 80 bits
Instruction
decoder and
sequencer
Integer
Unit
FPU
Memory

29. FPU Data Register Stack (x87)
FPU register format (80 bit extended precision)
FPU registers
8 registers %st(0) - %st(7)
Logically form stack
Top: %st(0)
Bottom disappears (drops out) after too many pushs 79 78 64 63 s
exp
frac
“Top”
%st(0)
%st(1)
%st(2)
%st(3)

30. FPU instructions (x87)
Large number of floating point instructions and formats
~50 basic instruction types
load, store, add, multiply
sin, cos, tan, arctan, and log
Often slower than math lib (!)
Sample instructions:
Instruction Effect Description
fldz push 0.0 Load zero
flds Addr push Mem[Addr] Load single precision real
fmuls Addr %st(0)  %st(0)*M[Addr] Multiply
faddp %st(1)  %st(0)+%st(1);pop Add and pop

31. FP Code Example (x87) Compute inner product of two vectors
Single precision arithmetic
Common computation
pushl %ebp # setup
movl %esp,%ebp
pushl %ebx
movl 8(%ebp),%ebx # %ebx=&x
movl 12(%ebp),%ecx # %ecx=&y
movl 16(%ebp),%edx # %edx=n
fldz # push +0.0
xorl %eax,%eax # i=0
cmpl %edx,%eax # if i>=n done
jge .L3
.L5:
flds (%ebx,%eax,4) # push x[i]
fmuls (%ecx,%eax,4) # st(0)*=y[i]
faddp # st(1)+=st(0); pop
incl %eax # i++
cmpl %edx,%eax # if i<n repeat
jl .L5
.L3:
movl -4(%ebp),%ebx # finish
movl %ebp, %esp
popl %ebp
ret # st(0) = result
float ipf (float x[],
float y[],
int n) {
int i;
float result = 0.0;
for (i = 0; i < n; i++)
result += x[i]*y[i];
return result; }

32. Inner Product Stack Trace
eax = i ebx = *x ecx = *y
Initialization
1. fldz
0.0
%st(0)
Iteration 0
Iteration 1
2. flds (%ebx,%eax,4)
5. flds (%ebx,%eax,4)
0.0
%st(1)
x[0]*y[0]
%st(1)
x[0]
%st(0)
x[1]
%st(0)
3. fmuls (%ecx,%eax,4)
6. fmuls (%ecx,%eax,4)
0.0
%st(1)
x[0]*y[0]
%st(1)
x[0]*y[0]
%st(0)
x[1]*y[1]
%st(0)
4. faddp
7. faddp
0.0+x[0]*y[0]
%st(0)
x[0]*y[0]+x[1]*y[1]
%st(0)

33. x87 Floating Point Problems
One of “least elegant features” of x86
Remnants of original co-processor design
Compiler technology is much better today
Stack registers are problematic
All must be treated as caller-saved, too much memory traffic
Can’t treat as true stack over multiple function calls
Separate FP status word is problematic
Set by FP comparison instructions, outcome used by branches
Must transfer to int word, test bits explicitly
“Even experienced programmers find this code arcane and difficult to read.”

34. SSE Floating Point Architecture
Better compiler target than x87
Includes new registers and instructions
In Intel CPUs since Pentium III
Default for x86-64, but not IA32

35. Vector Instructions: SSE Family
SIMD (single-instruction, multiple data) vector instructions
New data types, registers, operations
Parallel operation on small (length 2-8) vectors of integers or floats
Example:
Floating point vector instructions
Available with Intel’s SSE (streaming SIMD extensions) family
SSE starting with Pentium III: 4-way single precision
SSE2 starting with Pentium 4: 2-way double precision
All x86-64 have SSE3 (superset of SSE2, SSE)
“4-way” + x

36. Intel Architectures (Focus Floating Point)
Processors
Architectures
Features
8086
286
x86-16
time
386
486
Pentium
Pentium MMX
Pentium III
Pentium 4
Pentium 4E
x86-32
MMX
SSE
SSE2
SSE3
4-way single precision FP
2-way double precision FP
Pentium 4F
Core 2 Duo
x86-64 / em64t
Our focus: SSE3
used for scalar (non-vector) floating point
SSE4

37. SSE3 Registers All caller saved %xmm0 for floating point return value
128 bit = 2 doubles = 4 singles
%xmm0
Argument #1
%xmm8
%xmm1
Argument #2
%xmm9
%xmm2
Argument #3
%xmm10
%xmm3
Argument #4
%xmm11
%xmm4
Argument #5
%xmm12
%xmm5
Argument #6
%xmm13
%xmm6
Argument #7
%xmm14
%xmm7
Argument #8
%xmm15

38. SSE3 Registers Different data types and associated instructions
Integer vectors:
16-way byte
8-way 2 bytes
4-way 4 bytes
Floating point vectors:
4-way single
2-way double
Floating point scalars:
single
double
128 bit
LSB

39. SSE3 Instructions: Examples
Single precision 4-way vector add: addps %xmm0 %xmm1
Single precision scalar add: addss %xmm0 %xmm1
%xmm0 +
%xmm1
%xmm0 +
%xmm1

40. SSE3 Instruction Names addps addss addpd addsd our focus
packed (vector)
single slot (scalar)
addps
addss
single precision
addpd
addsd
our focus
double precision

41. SSE3 Basic Instructions
Moves
Usual operand form: reg → reg, reg → mem, mem → reg
Arithmetic
Single
Double
Effect
movss
movsd
D ← S
Single
Double
Effect
addss
addsd
D ← D + S
subss
subsd
D ← D – S
mulss
mulsd
D ← D x S
divss
divsd
D ← D / S
maxss
maxsd
D ← max(D,S)
minss
minsd
D ← min(D,S)
sqrtss
sqrtsd
D ← sqrt(S)

42. x86-64 FP Code Example Compute inner product of two vectors
float ipf (float x[],
float y[],
int n) {
int i;
float result = 0.0;
for (i = 0; i < n; i++)
result += x[i]*y[i];
return result; }
Compute inner product of two vectors
Single precision arithmetic
Uses SSE3 instructions
ipf:
xorps %xmm1, %xmm1 # result = 0.0
xorl %ecx, %ecx # i = 0
jmp .L8 # goto middle
.L10: # loop:
movslq %ecx,%rax # icpy = i
incl %ecx # i++
movss (%rsi,%rax,4), %xmm0 # t = y[icpy]
mulss (%rdi,%rax,4), %xmm0 # t *= x[icpy]
addss %xmm0, %xmm1 # result += t
.L8: # middle:
cmpl %edx, %ecx # i:n
jl L10 # if < goto loop
movaps %xmm1, %xmm0 # return result
ret

43. SSE3 Conversion Instructions
Conversions
Same operand forms as moves
Instruction
Description
cvtss2sd
single → double
cvtsd2ss
double → single
cvtsi2ss
int → single
cvtsi2sd
int → double
cvtsi2ssq
quad int → single
cvtsi2sdq
quad int → double
cvttss2si
single → int (truncation)
cvttsd2si
double → int (truncation)
cvttss2siq
single → quad int (truncation)
double → quad int (truncation)

44. x86-64 FP Code Example
double funct(double a, float x, double b, int i) {
return a*x - b/i; }
a %xmm0 double
x %xmm1 float
b %xmm2 double
i %edi int
funct:
cvtss2sd %xmm1, %xmm1 # %xmm1 = (double) x
mulsd %xmm0, %xmm1 # %xmm1 = a*x
cvtsi2sd %edi, %xmm0 # %xmm0 = (double) i
divsd %xmm0, %xmm2 # %xmm2 = b/i
movsd %xmm1, %xmm0 # %xmm0 = a*x
subsd %xmm2, %xmm0 # return a*x - b/i
ret

45. Constants Here: Constants in decimal format
compiler decision
hex more readable
double cel2fahr(double temp) {
return 1.8 * temp ; }
# Constant declarations
.LC2:
.long # Low order four bytes of 1.8
.long # High order four bytes of 1.8
.LC4:
.long # Low order four bytes of 32.0
.long # High order four bytes of 32.0
# Code
cel2fahr:
mulsd .LC2(%rip), %xmm0 # Multiply by 1.8
addsd .LC4(%rip), %xmm0 # Add 32.0
ret

46. Comments SSE3 floating point x87 still supported
Uses lower ½ (double) or ¼ (single) of vector
Finally departure from awkward x87
Assembly very similar to integer code
x87 still supported
Even mixing with SSE3 possible
Not recommended
For highest floating point performance
Vectorization a must (but not in this course)
See next slide

47. Vector Instructions
Starting with version 4.1.1, GCC can auto-vectorize to some extent
-O3 or –ftree-vectorize
No speed-up guaranteed – limited capability
Intel’s C++ compiler (icc) currently much better
Spice machines have installed (May 2010)
For highest performance vectorize yourself using intrinsics
Intrinsics = C interface to vector instructions
Future
Intel AVX announced: 4-way double, 8-way single