Manipulating Bit Fields in C COMP 40: Machine Structure and Assembly Language Programming (Fall 2016) Manipulating Bit Fields in C Noah Mendelsohn Tufts University Email: noah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah

Goals for this presentation Learn to use C language to pack and extract bit fields Also: learn exact width C integer types

Warning: Because it makes the examples easier to understand, some of the code in these slides uses the syntax: 0b11001001 for literals. This is a GNU extension to the C language that is not allowed in COMP 40 code. You may use hex literals like this: 0xC9 as they are standard.

Exact Width Integer Types

Why exact width integer types? The problem: C integer types aren’t fixed size Size of char, int, long, etc. depends on platform and compiler Sometimes we need to get a known size The solution: stdint.h defines fixed size integers int32_t: 32 bit signed integer uint32_t: 32 bit unsigned integer int16_t: 16 bit signed integer uint64_t: 64 bit unsigned integer Etc. When writing bit packing code, you need to know or account for the size of the integers you’re manipulating

Why Bother with Bit Fields?

Why use bit fields? Save space: Storing 10 million values each ranging from 0-10 If each is a 32 bit int: 40 megabytes If each is a 4 bit int: 5 megabytes Manipulate standard file formats and network packets Sometimes, you have to (write flags into a device register)

Example: Internet Protocol Packet How can we extract this 8 bit protocol number from the 32 bit field? Example: Internet Protocol Packet 32 bits V HDLN SVC TYPE LENGTH ID FLGS FRAG OFFSET TTL PROTOCOL HDR CHECKSUM SOURCE ADDRESS DESTINATION ADDRESS OPTIONS THE TCP OR UDP DATA (VARIABLE LEN)

Extracting Bit Fields

Extracting a bit field * * Note that extra spaces have been used in these boolean literals to make them, easier to read. Such spaces are not allowed in code (and, as previously noted, the 0b1001 syntax is a GNU extension not allowed in COMP 40 submissions anyway). uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0;

Extracting a bit field “and” together i and mask Create mask to select bits we need Extracting a bit field uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0; “and” together i and mask (i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0;

Extracting a bit field “and” together i and mask? uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0; “and” together i and mask? (i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0; … and shift to finish the job (i & mask) >> 5 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

Be careful with signed shifts uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; In this example, masking leaves on high order bit (u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; (i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; Signed shifts usually propagate that bit! u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

Be careful with signed shifts uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; Unsigned is correct Signed has unwanted leading 1’s In this example, masking leaves on high order bit (u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; (i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; Signed shifts usually propagate that bit! u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

Be careful with signed shifts Huh? Be careful with signed shifts Turns out that right shifting of a signed integer is implementation define…but why? C wants these operations to be super-efficient, and on some hardware there’s no efficient signed shift. On our systems, the sign will propagate and you may count on that in your homework submissions. uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; In this example, masking leaves on high order bit u &= mask: 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 1; i &= mask == 0b 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1; Signed shifts usually propagate that bit! u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1;

How can we choose the bits to extract at runtime? unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; int width = 3; int offset = 5; This time we’ll have to compute the mask unsigned short mask = ~0; /* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */ /* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */ mask = mask >> (16 - width) << (offset); (i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; Now, finish the job as before

We can adapt to integer size at runtime unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; int width = 3; int offset = 5; This time we’ll have to compute the mask unsigned short mask = ~0; /* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */ /* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */ mask = mask >> (number_of_bits - width) << (offset); (i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; Now, finish the job as before const number_of_bits = sizeof(unsigned short) * 8;

Setting Bit Fields

Putting new value in a bit field uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

Putting new value in a bit field Important: we have zeros where new value is going Putting new value in a bit field uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask… (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

Putting new value in a bit field uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask… (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0; … shift the new value and use “or” to combine ((i & mask)| (new_value << 5)) == 0b 1 0 0 1 1 1 0 0 0 1 0 0 1 1 1 0;

Flag Bits

Very common idiom – single bit flags One bit each /* Flag definitions */ const uint16_t HI_PRIORITY = 0x8000; const uint16_t SECURE = 0x4000; const uint16_t ARCHIVED = 0x2000; … const uint16_t BEAUTIFUL = 0x0002; const uint16_t SPECTACULAR = 0x0001;

Very common idiom – single bit flags Can use binary instead of hex, but hard to count the zeros! Very common idiom – single bit flags /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001;

Initializing flags Use | to combine bits /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; uint16_t myflags = SECURE | BEAUTIFUL; /* secure and beautiful */

Test flags with &  remember C treats anything != 0 as true! Testing flags /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; if (myflags & BEAUTIFUL) {…}; /* if beautiful */

Testing multiple flags on Testing flags Testing multiple flags on /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; if ((myflags & (BEAUTIFUL | SECURE)) == (BEAUTIFUL | SECURE)) {…}; /* if beautiful and secure */

Us | to turn on additional flags Turning flags on /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; myflags |= ARCHIVED; /* now it’s archived too */

To turn off, & with all the other flags! Turning flags off /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; myflags &= ~BEAUTIFUL; /* but not beautiful anymore  */