Lecture 2 Memory management

Perspective of memory services Memory is a central shared resource in a multiprocessing RTE. We will compare memory models in C++ and Java. We can look at memory operations at different levels: Hardware level: operations like READ and WRITE. Operating System level: sharing the physical memory among processes. Programming Language: how the language relates to memory operations at runtime and at compile time.

Memory Technologies On chip storage: small and fast. Expensive. registers and cache storage. Reside on the processor’s chip. Random Access Memory (RAM). Main memory. Quite large and still fast. Persistent online storage. The disks (including SSD). Based on magnetic/electromagnetic technology. Persistent offline storage. Tapes/DVD/CD are magnetic or optical offline storage technologies

Properties of a memory Memory space: Word size: how much main memory the processor can address (maximum potential size). Word size: Words are the native units of data the CPU operates on. Usually, registers can store one word, and writes/reads from main memory are at least the size of a word. Memory is viewed as array of words. The index of a word in the array is called address. Each READ/WRITE is performed on a single word.

Hardware perspective The memory stores all the data a process needs for its computation (e.g., parameters, variables, objects, etc.).  To perform an operation on a word of data, we follow: Load the data from memory into internal registers. Apply the operation on the registers and put the result in another register. Optionally, write the result from the register into the relevant memory location.

Hardware primitives Read (Address), (Register): copy the content of the word stored at (Address) into (Register). Write (Register), (Address): copy the content of the (Register) to the memory cell at (Address).

Example Assume the Java code: int i = 10; int j = 20; i = i + j; This will be compiled to: ([var] stands for the address of the variable var). ;; Initialize the memory cells for i and j WRITE $10, [i] ;; 10 is a constant value WRITE $20, [j] ;; 20 is a constant value ;; Execute the sum READ [i], R1 ;; copy the content of i in register 1 (R1) READ [j], R2 ;; copy the content of j in register 2 (R2) ADD R1, R2, R3 ;; execute the operation: R1 + R2 and store the result in R3 WRITE R3,[i] ;; store the result in the memory location of i

Programming Language Perspective Which memory-related services we already used in Java? Explicit: Object allocation – using new. Initialization – by calling constructors or initializing primitive variables. Copying - Copying data from one variable to another. Implicit: Object deletion - done by the Garbage Collector Variable allocation - Local (primitive) variables allocation Variable deallocation - Local variable deallocation when leaving scope

Memory Access In high level languages, we usually access memory through a variable. A variable in Java is characterized by: A name. A type. A scope: the program region in which the name is recognized and refers to the same variable Variables are bound to values at runtime. A value is characterized by: A type A position in memory which is the value address A size - how much memory does the value requires in memory

Memory Access When a variable is bound to a value, the type of the value must be compatible with the type of the variable. Types are compatible when: They are identical. The type of the value is more specific than the type of the variable. The type of the value implements the type of the variable (when the type of the variable is an interface).

Stack model vs. Heap model Programming languages employ two different memory models: The stack (call-stack). The heap. Procedures High level languages usually employ the concept of procedures. A program is composed of several procedures. Each procedure can receive arguments, define variables, perform operations and optionally return a value.  Even object oriented programs are, under the hood, procedural.

The Call Stack Function calls are nested. One function calls the other, and then, after the call, continue its run. Each function call requires a dedicated frame on a stack: Activation Frame. Each time a procedure is called, a new activation frame is generated on the stack. Activation Frames holds the information: The parameters passed to the procedure Local primitive variables declared in the procedure The return address - Where to return to when the procedure ends

The Call Stack Consider the following code: public static void foo(int a, bool b){ int var_a; int var_b; . } When an AF is created, a binding between local variables and their location on the stack is made. When a procedure terminates, its associated activation frame is popped from the stack, and discarded.

Stack Implementation To realize the concept of procedures and activation frames, computer systems support a call stack data structure. For each process/thread there exists a memory region which is called the call stack. The OS allocates the memory for the stack. Each process/thread manages its own stack.

Stack Implementation The stack is defined by two CPU registers: esp: the stack pointer, initialized by the OS to point to the top of the stack. ebp: the base pointer, points to the start of the current stack frame (the address immediately before the first local variable). These registers my be directly modified by the process/thread. By assigning values directly. By pushing/popping words to/from the stack. Push X: advance esp by one address, write X in the address in esp. By calling or returning from a function.

Stack Implementation The way that the stack is manipulated in order to perform function calls is called the "Calling convention". It is a low-level scheme for how functions receive parameters from their caller and how they return a result. In C calling convention, parameters are loaded to the stack from right to left by the caller and the return value is commonly stored in the eax register. The caller is also responsible for removing the pushed parameters from the stack after the invoked function returns.

Example – on the board!

Stack – advantages and disadvantages Fast management of memory. Deallocation by changing esp. The stack is often limited in size. One cannot use variables once they are popped. All allocation are done at compile time – the allocations must be static and their size must be known at compile time.

Heap Sometimes, programs need to store information which is: relevant across function calls. too big to fit on the stack. or of size that is unknown at compile time. We need to be able to allocate a block of memory of a given size. We do not care where the memory is coming from – we just want it. We abstract this service as a heap, where blocks of memory are heaped in a pile

Heap in Java In Java, there is a clear distinction between what is saved on the stack and what is stored on the heap. Local variables are stored on the stack, and Objects are stored on the heap. Example: int i = 5; Object o = new Object(); Both i and o are stored on the stack. The object itself allocated by new, is stored on the heap. What’s in the variable o on the stack?

Heap in Java In Java, there is a clear distinction between what is saved on the stack and what is stored on the heap. Local variables are stored on the stack, and Objects are stored on the heap. Example: int i = 5; Object o = new Object(); Both i and o are stored on the stack. The object itself allocated by new, is stored on the heap. What’s in the variable o on the stack? It holds the location of the object in the heap! The section of o on the stack holds a reference (an address) to the start of the memory new allocated to hold the instance of the new Object.

Heap implementation The OS (physically) allocates a large block of memory to each process, as the process starts, to serve as its heap. This block is shared by all the threads in the same process, and may be enlarged by using special system calls (e.g., brk(2)). Allocating blocks from the heap, keeping track of “live” blocks, and requesting the OS to enlarge the heap are done by the RTE. Managing the heap is done by the RTE while managing the stack is done with code instructions inserted by the compiler.  

Memory deallocation in Java Each time we call the new operator, a block of memory is allocated, large enough to hold the Object we are instantiating. What happens to an Object once we have no further use for it? E.g. we allocated an array inside some function, and then returned without leaking its reference outside the scope of the function. This array can no longer be accessed, and should be released. This is done automatically for us, by an active entity which is called the “garbage collector”. In a programming language like C, allocating and deallocating memory is a manual process.

The JVM Garbage Collection Process A “used object”: an object that some variable holds its reference. A “unused object”: an object that is not referenced by any variable. Garbage collection: the process identifying which objects are not in use and deleting them. Marking: identifying which pieces of memory are in use and which are not. Composing a set of all Objects referenced from the stack and from registers. For each Object in the set, adding all Objects it references. Repeat step (1) until the set remains unchanged.

The JVM Garbage Collection Process Deletion: Removes unreferenced objects leaving referenced objects and pointers to free space. Compacting: to further improve performance, the garbage collector compacts the remaining referenced objects. By compacting the objects, this makes new memory allocation and access much easier and faster.

Heap: advantages and disadvantages The Heap model allows for dynamic memory allocation - i.e., the size of the allocation does not have to be known at compile time. Memory on the heap stays on the heap until it is freed. The heap is much larger than the stack and holds most of the process’s memory. Heap allocation costs more than stack allocation. If the user is responsible to free the allocated memory - the program can have memory leaks and access to non-allocated memory. If the runtime is responsible - the performance of the program may degrade as CPU time will be dedicated for the garbage collection execution.

Stack in C++ Stack implementation is similar to Java. Objects in C++ can be allocated on the stack. They can be passed by value to functions! When objects are released, a function called “destructor” is called. Destructor: a function which is in charge of cleaning after the object: releasing the object state, and any resources the object might have acquired. There are also constructor, copy constructor, assignment operator.

Stack allocation in C++ int x(0); // initializer is in parentheses int y = 0; // initializer follows "=" int z{ 0 }; // initializer is in braces ObjectType o(10); // call ObjectType's ctor with argument 10 ObjectType o2; // call ObjectType's default constructor ObjectType o2{10}; // call ObjectType's ctor with argument 10 ObjectType o2{}; // call ObjectType's default constructor

May not do what we expect // create a new ObjectType on the stack by calling ObjectType's ctor // with argument 10 then call the copy constructor to copy the new object into o3 ObjectType o3 = ObjectType(10); // this will NOT call the default constructor! // but instead declare a function named o4 that accept no arguments and return ObjectType ObjectType o4();

Objects on the stack, passed by value At runtime: A place to hold class Foo is reserved on the stack The default constructor is called An activation frame for doSomthing is created. A new object of class Foo is instantiated on the stack, by using the copy constructor of class Foo and passing foo1 as the argument. The method changeMe() of the object foo2 is called. The destructor of foo2 is called. The doSomething frame is popped from the stack and the program continues from the return address. changes made on foo2 are not reflected in foo1. void doSomething(Foo foo2){ foo2.changeMe(...); } Foo foo1; doSomthing(foo1);

Objects on the stack, passed by value At runtime: A place to hold class Foo is reserved on the stack for foo1. A place to hold class Foo is reserved on the stack for foo3. The default constructor is called for foo1. An activation frame for doSomthing is created. A new object of class Foo is instantiated on the stack, by using the copy constructor of class Foo and passing foo1 as the argument. The method changeMe() of the object foo2 is called. foo2 is copied by the copy constructor of class Foo to the location of foo3. The destructor of foo2 is called. The doSomething frame is popped from the stack and the program continues from the return address. changes made on foo2 are not reflected in foo1. Foo doSomething(Foo foo2) { foo2.changeMe(...); return foo2; } ... Foo foo1; Foo foo3 = doSomething(foo1);

Another example

Objects on the stack, passed by value At runtime, the following takes place: A place to hold class Counter is reserved on the stack and is bound with the variable counter1. A place to hold class Counter is reserved on the stack and is bound with the variable counter3. The default constructor is called for counter1 An activation frame for increment is created, with enough space to hold the return address and an object of class Counter. A new object of class Counter is instantiated on the stack, by using the copy constructor of class Counter and passing counter1 as the argument. The method inc() of the object counter2, is called. counter2 is copied by the copy constructor of class Counter to the location of counter3. The destructor of counter2 is called. The increment frame is popped from the stack and the program continues from the return address. Note that: Changes made on counter2 are not reflected in counter1. 

Objects on the stack, passed by value Output: default constructor copy constructor destructor counter1: 0, counter3: 1 increment implicitly created 2 additional Counter objects, instantiating, copying and destructing them!

Heap in C++ As everything in C++ can be allocated on the stack, so can everything be allocated on the heap! Even primitive types!!! Example: int *i = new int(8); Foo *foo = new Foo(42);

New and Delete Operators Allocating on the heap is achieved by calling the new operator. new is an operator, not a function, which allocates space on the heap. Initializes the space by calling a constructor. The returned value of new a_type(vars) is a_type*, which is called a pointer to a_type. The memory is initialized using the constructor we requested (which must exists).

New and Delete Operators It is the responsibility of the programmer to free the memory. For example: delete i; delete foo; Each time we delete an object, the destructor is called

Constructors and desctructors // Define the constructor. String::String( char *ch ) { sizeOfText = strlen( ch ) + 1; // Dynamic allocation of memory. _text = new char[ sizeOfText ]; // If the allocation succeeds, // copy the initialization string. if( _text ) strcpy_s( _text, sizeOfText, ch ); } // Define the destructor. String::~String() { // Deallocate the memory. if (_text) delete[] _text; #include <string.h> class String { public: // Declare constructor String( char *ch ); // and destructor. ~String(); private: char *_text; size_t sizeOfText; };

Allocating and Deleting Arrays in C++ Arrays in C++ are allocated by using new []. Parameter: the size of the array. Example: Foo *foo_array = new Foo[100]; The return type of new[] operator is, again, Foo*. We allocated a block of memory on the heap, for 100 Foo objects. Each object is initialized using the default constructor of class Foo.

Allocating and Deleting Arrays in C++ How can we release the memory? delete[] foo_array; Which, in turn, calls the destructor of each object in the array, and then frees the memory. Note that new and new[] are not the same operators. Same holds for delete and delete[]. You must pay attention to call delete[] on memory allocated by new[] and delete for memory allocated by new.