1. Operating System Protection Through Program Evolution
By Dr. Frederick B. Cohen
Presented by William Lu

2. The Ultimate Attack How to defeat defensive measures of a system?
Gain physical access to the system
Reverse engineer defenses
Find weak link and exploit

3. The Ultimate Defense How to defend against attackers
Make attacks extremely complex
Make costs too high to be worth attacking
i.e. passwords
Large space
Spread out probability density (diffusion)
Obscuring stored information (confusion)
How to defend against attackers
In practice, password space is huge but human methods of password selection leads to a high probability of select words.

4. The Ultimate Defense Current operating systems
Space is enormous (all programs that fit in memory)
High probability subspace (very small number of versions)
No confusion
No confusion – the part of the program that performs any given operation is apparent

5. The Ultimate Defense How to increase operating system defenses?
Reducing coherence
How?
Unique defense for each system
Feasibility?
Too many unique defenses to design
Compromise?
Implement a fixed number of defenses

6. The Ultimate Defense More practical solution? Goal?
Evolutionary defenses
Goal?
Produce a large search space
Provide confusion
Provide diffusion
With more evolution, there’s less performance, but higher cost of attack

7. (some) Techniques for Program Evolution
Equivalent instruction sequences
Instruction reordering
Variable substitution

8. Equivalent Instruction Sequences
What does it do?
Replaces instruction sequences with equivalent sequences
i.e. add 17 is equivalent to add 20 and subtract 3
How does it help defend against attacks?
As evolution increases so does time and space
Potentially infinite evolutions
Creates enormous possible executions

9. Instruction Reordering
What does it do?
Reorders instructions without altering program execution
Order does not matter
How does it help defend against attacks?
Typical system call involves setting a series of values.
Values are independent and can be set in any order.
Assigning independent values can be reordered
Increases complexity of attacks to n! different orderings (n = # of different instructions)

10. Instruction Reordering
3 different instructions
6 different forms …
I=3
J=5
K=8

11. Variable Substitution
What does it do?
Alters the location of memory storage areas
How does it help defend against attacks?
Prevents static examination and analysis of parameters
Move variables to different locations to prevent patterns without affecting program execution

12. (and other) techniques?
What to do with these
(and other) techniques?

13. Providing Evolution in Defenses
How to evolve?
Select a mix of evolution techniques
Increase complexity while minimizing impact on end users

14. Providing Evolution in Defenses
When to evolve?
At the factory?
Uniquely identify each disk sent out
Reduced efficiency
At installation?
Unique tracking number, unique registration numbers
Evolution at installation can take place while installer is asking for user input or swapping of disks.
Crucial to have unique and confidential evolution
Takes time at end user level

15. Providing Evolution in Defenses
When to evolve?
After installation?
Cannot assure against corruption
Cannot trust internal checking
Attack that succeeds on one day may fail the next
Unique tracking number, unique registration numbers
Evolution at installation can take place while installer is asking for user input or swapping of disks.

16. Attacks on Program Evolution
Points of Attack
Tracing Attack

17. Point of Attack How? How to defend against it?
Find original entry point and exploit it
Gain direct access to hardware or operating system internals
How to defend against it?
Evolve the core of the operating system
Evolving the calling mechanism
Calls that bypass protection may be of the wrong form
Used by viruses

18. Tracing Attacks How? How to defend against it?
Trace programs at execution or simulation
How to defend against it?
Use redundancy
Force attacker to use tracing on each attack

19. Conclusion
Program evolution can increase the complexity for an attacker
Create a large search space to make attacks infeasible
Need more study to reach maturity