1. Operating System Security
Dr. Neminath Hubballi
IIT Indore © Neminah Hubballi

2. IIT Indore © Neminah Hubballi
Outline
Functions of Operating System
Security concerns in OS
Process security
File Security
Booting security
Hibernation security
Password based security
Event log management in windows
IIT Indore © Neminah Hubballi

3. Functions of Operating System
Is a mediator between user applications and hardware
Handles lot many complex tasks
Memory management
Process management
Handling deadlocks
File system support
Multitasking
Multi user support
IIT Indore © Neminah Hubballi

4. IIT Indore © Neminah Hubballi
What Can Go Wrong ?
Consider a situation where there is shortage of systems in school of CSE IITI
If you are asked to share same PC with your peers
What do you want to do
Users have different level of access
Based on role
Multiple users and multi tasking requires a level of protection
One user from interfering other users
One program from interfering other users
Sharing of resources
Optimization
IIT Indore © Neminah Hubballi

5. Organization of Computer Hardware and Software
Many devices
Each device has a driver
Provides APIs to access
Kernel
Heart of OS
Manages the low level h/w resources
Non essential component of OS
Printing program
User application
Access the service provided by OS
System calls
User Applications
Non Essential OS
Applications
OS Kernel
Hardware
IIT Indore © Neminah Hubballi

6. IIT Indore © Neminah Hubballi
Process
Kernel defines the notion of a process
Programs are stored in persistent storage
Can multiple copies of same program run simultaneously ?
Time slicing
Process tree
Fork system call
Parent and child
Peers
In Linux system init is the root of process tree
Meaning all other processes are created by it
Its PID is 0
IIT Indore © Neminah Hubballi

7. IIT Indore © Neminah Hubballi
Process Tree Diagram
IIT Indore © Neminah Hubballi

8. IIT Indore © Neminah Hubballi
Process Privileges
To grant appropriate access restrictions on the process operating system associates privilege information to a process
This privilege is same as privilege of user who is running the process
Each process has a user id called uid, group id gid
The uid is a number between 0 to which uniquely identifies each user
Typically uid 0 is assigned to the root user
Similarly gid is also a number in the same range
Effective user id eid is the user id whose privileges are used to access a resource
IIT Indore © Neminah Hubballi

9. File Ownership and Permissions
Assigned to each file/directory
Provides security
Ability to manage users and their files.
Needed to access file/directory
Usually granted to groups
In multi-user operating systems like Linux, access is given only to authorized users
Super (root) user:
Has special privileges –
In a sense owns everything
Can change file ownerships
Bypass permissions that owner of file may have set
Uses root account to provide administrative functions
IIT Indore © Neminah Hubballi

10. File Ownership and Permissions
Unix systems treat everything as a file
Special files
Devices- a piece of hardware either part of system or an external unit
Sockets- a means of communicating with other processes
Permissions can be of
Read
Write and
Execute
File system
ext 2 and ext 3 – permissions apply
FAT – no means of ownership
IIT Indore © Neminah Hubballi

11. IIT Indore © Neminah Hubballi
Sticky Bit in Unix
Sticky Bit: Mainly used to avoid some other user deleting a file though she has a write permission on the folder
If Sticky bit is enabled on a folder, the folder contents are deleted by only owner who created them and the root user.
This is a security measure to avoid deletion of critical folders and their content(sub-folders and files), though other users have full permissions.
Setting sticky bit
chmod +t /opt/dump/
+t indicates sticky bit setting\
Check sticky bit
ls –l : a t will appear in the listing
Revoking sticky bit
Chmod -t /opt/dump/
IIT Indore © Neminah Hubballi

12. IIT Indore © Neminah Hubballi
Memory Management
Process granularity
Each process upon creation is allocated some memory called as address space
This memory is organized in segments
.text, .data, .bss, and heap and stack segments
Each segment has its own access permissions
Readable, writable and executable
Operating system protects one process from other by not allowing access to others address space
Global granularity
Kernel address space
User address space
IIT Indore © Neminah Hubballi

13. Booting Sequence and Security
Typical booting sequence is
There is a chain of trust in booting process
An attacker can subvert booting process by altering or modifying something in any of these components
In order to protect system from such changes most systems have a BIOS password
BIOS
Secondary Boot Loader
Operating System
IIT Indore © Neminah Hubballi

14. Hibernation and Security
Hibernation is a concept of saving state of system into disk
Typically in a file state information is stored
Entire main memory is copied into a file (in a compressed format)
Since entire state is copied onto disk
All passwords and other sensitive information carry danger of being exposed
Researchers have shown the feasibility of extracting such information by mounting a live CD attack
On a windows machine state is stored in a file c:\hiberfil.sys
IIT Indore © Neminah Hubballi

15. Password and User Account Management in Operating System
Naive approach
Create a file password where all users passwords are stored
Neminath : pass156
Gourinath : test234
Somnath : temp123
Save the file password in a place in the system
What if a thief gets access to this file ?
All user accounts are compromised
What can we do to prevent it ?
Encrypt the file containing password file
Seems a good idea but not enough
There is a key used to encrypt the file
How does the OS verifies the password ?
Key needs to be somewhere in the system
Key needs to be stored
If file containing password can be stolen key also can be stolen
Use one way hashing and salting – most flavors of unix systems use this method
IIT Indore © Neminah Hubballi

16. Password and User Account Management in Operating System
One way hashing is a function f
Characteristic of this function is
when supplied x computes f(x) easily
But inverse is extremely complex i.e., given f(x) it is difficult to calculate x.
An example
Convert all the characters into their ASCII values and XOR them
Resultant is a small number derived out of XOR operation
Store the hash value in password file
Note we do not store the password anywhere
The idea is make it impossible to guess the password even if hash value is known
Now the password file looks like
Neminath : a12hf
Gourinath : b4a2e
Somanath : d34ef
IIT Indore © Neminah Hubballi

17. Password and User Account Management in Operating System
From a thief's perspective
She can start guessing passwords one by one and compare it to the hash values in password file
She needs to know which hash function to use
There are only handful of good one-way hash functions implemented
So its easy to guess one or find out one
Using a table called rainbow table (which is a list of common pre-computed password hashes )it is easy to break it
Even if one user uses a weak password which maches with that of dictionary system security is compromised
This is addressed through a technique called as salting
IIT Indore © Neminah Hubballi

18. Password and User Account Management in Operating System
Salting
Do not hash passwords as it is add something to it and then hash
Unix system uses an additional 12 bit number to hash password
How to chose value of salt
Deterministic across the users: not a good idea !
Unique for each user : sounds good but how to get a unique one
Take the current timestamp of system and divide with a predetermined number to get a 12 bit remained use it as a salt.
Meta data as salt : use the birthday or PAN number of user as salt
IIT Indore © Neminah Hubballi

19. Unix Password Management
A file in the directory /etc/passwd stores the Unix users password
Unix password is hashed and salted
Salting and hashing effectively increase the amount of work done to break into the target system
Algorithm
Take the first 8 ASCI characters of the user password and encrypt a 64 bit constant character consisting of all 0’s
If the user password is less than 8 characters suitably pad it to make it 56 bits
Encrypt the 64 bit length 0’s 25 times with DES using user password as key
Resultant encrypted version is stored in the file
IIT Indore © Neminah Hubballi

20. IIT Indore © Neminah Hubballi
Windows Event Logging
IIT Indore © Neminah Hubballi

21. IIT Indore © Neminah Hubballi
Windows Event Logs
Types of Logs
Application logs : events from applications
Security logs : login and logout details, failed logins
System logs: events from system components\
The event header contains
Date and time
User and Computer
Event ID 
Level
Source 
Category 
IIT Indore © Neminah Hubballi

22. IIT Indore © Neminah Hubballi
Windows Event Logs
Types of Events
Information : Normally indicates successful operation of something
Warning: Indicates not a severe issue but in future it may be troublesome
Error: Describes a significant problem
Success Audit (Security log) : When a user successfully logs into system
Failure Audit (Security log) : A failed login attempt
IIT Indore © Neminah Hubballi

23. IIT Indore © Neminah Hubballi
An Example
IIT Indore © Neminah Hubballi

24. IIT Indore © Neminah Hubballi
An Example
Level time and day source event id task category
Warning 12/6/ :34:21 PM Tcpip 4228 None
Details: TCP/IP has chosen to restrict the scale factor due to a network condition. This could be related to a problem in a network device and will cause degraded throughput.
Issue is related to communication
Scale factor is related to receive window size
By using the window scale option, the receive window size may be increased up to a maximum value of 1,073,725,440 bytes. This is done by specifying a one byte shift count in the header options field.
The true receive window size is left shifted by the value in shift count. A maximum value of 14 may be used for the shift count value.
IIT Indore © Neminah Hubballi

25. IIT Indore © Neminah Hubballi
Logging at Many Places
Windows event logs
Antivirus programs
Firewalls
Radius server
DHCP server
IDS and IPS ……..
IIT Indore © Neminah Hubballi

26. IIT Indore © Neminah Hubballi
Event Correlation
Courtesy: SANS Analyst Program Document
IIT Indore © Neminah Hubballi

27. IIT Indore © Neminah Hubballi
Sequence of Events
Courtesy: SANS Analyst Program Document
IIT Indore © Neminah Hubballi