2. Cyber Grand Challenge
“Cyber Grand Challenge (CGC) is a contest to build high-performance computers capable of playing in a Capture-the-Flag style cyber-security competition.”
The Cyber Grand Challenge is a competition to pit computers against each other in a CTF.
But why?

3. Humans > Robots * Yes, even at software
Well, it’s because in terms of CTF, humans are still much better than robots. So DARPA created a robot-only competition where these systems can compete against each other in a feedback loop. This animation on the right is actually from a similar competition to make soccer playing robots.
* Yes, even at software

4. Competition Setup Teams build “Cyber Reasoning Systems”
Fully autonomous systems
Given a bundle of Challenge Binaries (CBs)
CB := “Pwnables”
Teams create Proof of Vulnerabilities (PoV)
PoVs are inputs that trigger vulnerable code paths (via a crash)
Two rounds: CQE (Qualifying) and CFE (Final)
For CQE, the competition was against reference PoVs and polls
For CFE, CRSs competed against each other and validated or invalidated competitor patches
So, the competition is set up where a dozen or so teams build Cyber Reasoning Systems. These systems will be expected to automatically process custom created executables, identify software vulnerabilities in them, and develop proof of vulnerabilties – which are inputs that tickle the vulnerable code path to prove its existence and reachability.
These CRSs will eventually compete head to head against each other.

5. DECREE OS: DECREE Removes complexity Very clever implementation
DARPA Experimental Cyber Research Evaluation Environment
Similar to Linux x86
Executables are statically-linked ELFs
Implemented as a new binfmt
Removes complexity
Very clever implementation #
Syscall
Linux Eq. 1
_terminate
exit 2
transmit
send 3
receive
recv 4
fdwait
select 5
allocate
mmap 6
deallocate
munmap 7
random
n/a
To facilitate the competition and reduce the complexity, DARPA developed DECREE. A custom ABI with only 7 system calls. You can’t open files, you can’t open sockets, you can’t fork. You can’t really write real software on it but it’s a great starting point for the competition.
The syscalls are listed here.

6. Challenge Sets 247 C & C++ network services
Implementing things like mail servers, like ftp servers, etc
Except all have custom protocols, no real RFCs allowed
All have 1 or more exploitable or crashing vulnerability.
You’re expected to develop a “Proof of Vulnerability” and a “Replacement Binary”
Replacement binaries are distributed to all competitors for analysis

7. Challenge Sets (Continued)
Each challenge has:
A detailed readme
Vulnerability Description
Vulnerability CWE
A “Challenges” section
Polls (aka input generators) with high code coverage
One or more Proof of Vulnerability triggers
Included patches, guarded by compile-time #ifdefs

8. Example Challenge: CADET_0001
Here’s an example challenge. We’d actually be given this as a compiled executable, but it’s easier to visualize with the source code. This implements a network service that tells you if a string is a palindrome or not.
Can you spot the bug?
It reads too much data into a fixed size stack buffer, resulting in a stack smash.
Example Challenge: CADET_0001

9. Proof of Vulnerability
Type 1:
A crashed with demonstrated control of a register and EIP
Negotiated register and mask
void cgc_NegotiateType1Pov(unsigned int ipmask, unsigned int regmask, unsigned int regnum, unsigned int *type1_ip, unsigned int *type1_reg);
// ipmask -- mask of CB/RB instruction pointer
// regmask -- mask of CB/RB register (specified by third argument)
// 0 - eax, 1 - ecx, 2 - edx, 3 - ebx, 4 - esp, 5 - ebp, 6 - esi, 7 - edi
238
char szReadBuf[256];
240
unsigned int type1_ip;
unsigned int type1_reg;
cgc_NegotiateType1Pov( 0xfefefefe, 0xfefefefe, EBP_REG_NUM, &type1_ip, &type1_reg );

10. Proof of Vulnerability
Type 2:
Read secret data from static page and exfil it on the network
Negotiated register and mask
Page was mapped into every process at 0x4347C000 “CGC”
void cgc_NegotiateType2Pov(unsigned int *type2_addr, unsigned int *type2_size, unsigned int *type2_length);
unsigned int type2_addr;
unsigned int type2_size;
unsigned int type2_length; 97
// negotiate a Type2 POV with the Competition Framework
cgc_NegotiateType2Pov(&type2_addr, &type2_size, &type2_length);

11. Network IDS A snort-like IDS
Rules submitted are distributed to all competitors
block (name:"ch_sec directory traversal"; side:client; regex:"^ch_sec \x7c\x2b\x2b\x7c\x2b\x2b\x7c";)

12. Scoring Availability * Security * Evaluation Availability: Security:
Performance: Non-liner function based on file size increase, execution time, and memory usage
Retained functionality: Ratio of passed polls
Security:
Reference: Reference PoV patched
Consensus: No further vulnerabilities were introduced
Evaluation:
Proof of Vulnerability Generated