1. Manipulating Bit Fields in C Noah Mendelsohn Tufts University Email: noah@cs.tufts.edunoah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah COMP 40: Machine Structure and Assembly Language Programming (Spring 2014)

2. © 2010 Noah Mendelsohn Goals for this presentation  Learn to use C language to pack and extract bit fields  Also: learn exact width C integer types 2

3. © 2010 Noah Mendelsohn Warning: Because it makes the examples easier to understand, some of the code in these slides uses the syntax: 0b11001001 for literals. This is a GNU extension to the C language that is not allowed in COMP 40 code. You may use hex literals like this: 0xC9 as they are standard.

4. © 2010 Noah Mendelsohn Exact Width Integer Types

5. © 2010 Noah Mendelsohn Why exact width integer types?  The problem: C integer types aren’t fixed size –Size of char, int, long, etc. depends on platform and compiler –Sometimes we need to get a known size  The solution: stdint.h defines fixed size integers – int32_t : 32 bit signed integer – uint32_t : 32 bit unsigned integer – int16_t : 16 bit signed integer – uint64_t : 64 bit unsigned integer –Etc.  When writing bit packing code, you need to know or account for the size of the integers you’re manipulating 5

6. © 2010 Noah Mendelsohn Why Bother with Bit Fields?

7. © 2010 Noah Mendelsohn Why use bit fields?  Save space: –Storing 10 million values each ranging from 0-10 –If each is a 32 bit int: 40 megabytes –If each is a 4 bit int: 5 megabytes  Manipulate standard file formats and network packets 7

8. © 2010 Noah Mendelsohn 32 bits V HDLN SVC TYPE LENGTH IDFLGSFRAG OFFSET TTLPROTOCOLHDR CHECKSUM SOURCE ADDRESS DESTINATION ADDRESS OPTIONS THE TCP OR UDP DATA (VARIABLE LEN) Example: Internet Protocol Packet 8 How can we extract this 8 bit protocol number from the 32 bit field?

9. © 2010 Noah Mendelsohn Extracting Bit Fields

10. © 2010 Noah Mendelsohn Extracting a bit field 10 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; * * Note that extra spaces have been used in these boolean literals to make them, easier to read. Such spaces are not allowed in code (and, as previously noted, the 0b1001 syntax is a GNU extension not allowed in COMP 40 submissions anyway).

11. © 2010 Noah Mendelsohn Extracting a bit field 11 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0; “and” together i and mask (i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0; Create mask to select bits we need

12. © 2010 Noah Mendelsohn Extracting a bit field 12 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0; “and” together i and mask? (i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0; … and shift to finish the job (i & mask) >> 5 (i & mask) >> 5 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

13. © 2010 Noah Mendelsohn Be careful with signed shifts 13 uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; Signed shifts usually propagate that bit! u >> 13 u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1; In this example, masking leaves on high order bit (u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; (i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0;

14. © 2010 Noah Mendelsohn Be careful with signed shifts 14 uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; Signed shifts usually propagate that bit! u >> 13 u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1; In this example, masking leaves on high order bit (u &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; (i &= mask) == 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; Unsigned is correct Signed has unwanted leading 1’s

15. © 2010 Noah Mendelsohn Be careful with signed shifts 15 uint16_t u = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0; int16_t i = 0b 1 0 1 1 1 1 0 0 1 0 1 0 1 1 1 0; Signed shifts usually propagate that bit! u >> 13 u >> 13 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; i >> 13 i >> 13 == 0b 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1; In this example, masking leaves on high order bit u &= mask: 0b 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0 1; i &= mask == 0b 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1; Huh? Turns out that right shifting of a signed integer is implementation define…but why? C wants these operations to be super- efficient, and on some hardware there’s no efficient signed shift. On our systems, the sign will propagate and you may count on that in your homework submissions.

16. © 2010 Noah Mendelsohn How can we choose the bits to extract at runtime? 16 unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; int width = 3; int offset = 5; /* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */ mask = mask >> (16 - width) << (offset); This time we’ll have to compute the mask unsigned short mask = ~0; /* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */ (i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; Now, finish the job as before

17. © 2010 Noah Mendelsohn We can adapt to integer size at runtime 17 unsigned short i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; int width = 3; int offset = 5; /* mask is now 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 */ mask = mask >> (number_of_bits - width) << (offset); This time we’ll have to compute the mask unsigned short mask = ~0; /* mask is now 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 */ (i & mask) >> offset == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1; Now, finish the job as before const number_of_bits = sizeof(unsigned short) * 8;

18. © 2010 Noah Mendelsohn Setting Bit Fields

19. © 2010 Noah Mendelsohn Review: Extracting a bit field 19 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t mask = 0b 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0; “and” together i and mask? (i & mask) == 0b 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0; … and shift to finish the job (i & mask) >> 5 (i & mask) >> 5 == 0b 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1;

20. © 2010 Noah Mendelsohn Putting new value in a bit field 20 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0;

21. © 2010 Noah Mendelsohn Putting new value in a bit field 21 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask… (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0; Important: we have zeros where new value is going

22. © 2010 Noah Mendelsohn Putting new value in a bit field 22 uint16_t i = 0b 1 0 0 1 1 1 0 0 1 0 1 0 1 1 1 0; uint16_t new_value = 0b 0 1 0; /* value we want * uint16_t mask = 0b 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1; “and” together i and mask… (i & mask) == 0b 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 0; … shift the new value and use “or” to combine (i | (new_value << 5)) == 0b 1 0 0 1 1 1 0 0 0 1 0 0 1 1 1 0;

23. © 2010 Noah Mendelsohn Flag Bits

24. © 2010 Noah Mendelsohn Very common idiom – single bit flags 24 /* Flag definitions */ const uint16_t HI_PRIORITY = 0x8000; const uint16_t SECURE = 0x4000; const uint16_t ARCHIVED = 0x2000; … const uint16_t BEAUTIFUL = 0x0002; const uint16_t SPECTACULAR = 0x0001; One bit each

25. © 2010 Noah Mendelsohn Very common idiom – single bit flags 25 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; Can use binary instead of hex, but hard to count the zeros!

26. © 2010 Noah Mendelsohn Initializing flags 26 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; Use | to combine bits uint16_t myflags = SECURE | BEAUTIFUL; /* secure and beautiful */

27. © 2010 Noah Mendelsohn Testing flags 27 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; Test flags with &  remember C treats anything != 0 as true! if (myflags & BEAUTIFUL) {…}; /* if beautiful */

28. © 2010 Noah Mendelsohn Testing flags 28 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; Testing multiple flags on if ((myflags & (BEAUTIFUL | SECURE)) == (BEAUTIFUL | SECURE)) {…}; /* if beautiful and secure */

29. © 2010 Noah Mendelsohn Turning flags on 29 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; Us | to turn on additional flags myflags |= ARCHIVED; /* now it’s archived too */

30. © 2010 Noah Mendelsohn Turning flags off 30 /* Flag definitions */ const uint16_t HI_PRIORITY = 0b1000000000000000; const uint16_t SECURE = 0b0100000000000000; const uint16_t ARCHIVED = 0b0010000000000000; … const uint16_t BEAUTIFUL = 0b0000000000000010; const uint16_t SPECTACULAR = 0b0000000000000001; To turn off, & with all the other flags! myflags &= ~BEAUTIFUL; /* but not beautiful anymore  */