1. Automatically Detecting Error Handling Bugs using Error Specifications Suman Jana 1, Yuan Kang 1, Samuel Roth 2, Baishakhi Ray 3 1 Columbia University 2 Ohio Northern University 3 University of Virginia

2. Incorrect error handling: a major source of security vulnerabilities CVE-2014-0092 CVE-2015-0208 CVE-2015-0288 CVE-2015-0285 CVE-2015-0292 One of OWASP’s top 10 sources of security vulnerabilities CVE-2015-8617 CVE-2015-7941 CVE-2014-9826 CVE-2015-8340...

3. Error handling is especially problematic in C int ret = func1(...); if (ret == 0) { /* pass the error upstream or exit */ } char *ptr = func2(...); if (!ptr) { /* pass the error upstream or exit */ } No built-in exception handling mechanism Developers design custom error protocols (e.g., 0 means error)

4. What can go wrong? int check_if_ca (…) {... if (result <0) { goto cleanup; }... result = 0; cleanup: return result; } int _gnutls_certificate_verify2 (…) {... if (check_if_ca(...)==0) { result = 0; goto cleanup; }... result = 1; cleanup: return result; } Completely breaks SSL/TLS security guarantees against an active man-in-the-middle attacker GnuTLS CVE-2014-0092

5. Error conditions are rare Error handling bugs are often silent (no crashes) but produce incorrect results ○ Hard to detect without test oracles ○ How to design program-independent error handling oracles? Bugs manifest far away from their sources ○ Hard to localize Error handling bugs are hard to detect and localize

6. EPEx: automated detection of error handling bugs

7. Error path exploration Too many paths 1. Under-constrained symbolic execution at caller 2. Identify and only explore error paths

8. Generic error handling oracle Error handling oracle Pass the error upstream Log the error Terminate the program Buggy error handling if none of these happen

9. Generic error handling oracle Error handling oracle Pass the error upstream What about error code transformations? Programs have valid error ranges: transformations within those ranges are fine Error spec

10. Bug localization Pinpoints function call with buggy error handling Error handling oracle

11. Minimize false positives Error specs/path exploration might not be perfect Compare error handling of the same function across call sites

12. EPEx architecture Source code Error specs Bugs Error handling oracle Error path exploration Comparison across call sites EPEx

13. error spec check_if_ca: < 0 GnuTLS error range: != 1 int _gnutls_certificate_verify2 (…) {... if (check_if_ca(...)==0) { result = 0; goto cleanup; }... result = 1; cleanup: return result; } How does EPEx work? int check_if_ca (…) {... if (result <0) { goto cleanup; }... result = 0; cleanup: return result; } GnuTLS CVE-2014-0092 Not an error path

14. int _gnutls_certificate_verify2 (…) {... if (check_if_ca(...)==0) { result = 0; goto cleanup; }... result = 1; cleanup: return result; } How does EPEx work? int check_if_ca (…) {... if (result <0) { goto cleanup; }... result = 0; cleanup: return result; } An error path result = 1 (not an error) incorrect error propagation GnuTLS CVE-2014-0092 error spec check_if_ca: < 0 GnuTLS error range: != 1

15. Most functions in a library/application share the same error specs ○ 256 tested functions had only 38 unique specs How hard is creating error specs? ProgramError code(s)Non-error code(s) GnuTLS[-403, -1][0, 1] OpenSSL≤ 01 cURL[1, 91]0

16. Error specs of API functions can be reused across applications We also have an automated way of inferring API function error specs from their usages How hard is creating error specs? 31st IEEE/ACM International Conference on Automated Software Engineering 2016

17. Leveraged Clang static analyzer engine for under-constrained symbolic execution Implemented as a custom Clang checker ○ Detect error paths with checkPostCall ○ Check logging or termination with checkPreCall ○ Check error code propagation with checkPreStmt 617 lines of C++ code and 227 lines of Python code Implementing EPEx in Clang

18. Evaluation subjects GnuTLS httpd 867,000 lines of C code Mutt mbed TLS GNU wget

19. Reported Bugs130 Real Bugs102 False Positives28 Accuracy (78%) (22%)

20. Running time ProgramNo checkerDiv-by-zero checkerEPEx GnuTLS1.85m13.28m12.82m OpenSSL8.25m186.90m132.33m cURL0.18m13.96m12.95m httpd0.04m4.68m4.51m

21. static int dtls1_add_cert_to_buf(BUF_MEM *buf, unsigned long *l, X509 *x) { int n; unsigned char *p; … p = (unsigned char *)&(buf->data[*l]); l2n3(n, p); i2d_X509(x, &p); *l += n + 3; return 1; } Exhibit 1: OpenSSL Missing error check → data corruption

22. Exhibit 2: OpenSSL int PEM_ASN1_write_bio(...) { int ret = 0; … /* Generate a salt */ if (RAND_pseudo_bytes(iv, enc->iv_len) < 0) goto err; … ret = 1; err: OPENSSL_cleanse(iv, sizeof(iv)); … return ret; } RAND_pseudo_bytes can return 0 or -1 on error Wrong error check → weak randomness

23. BIGNUM *SRP_Calc_server_key(BIGNUM *A, BIGNUM *v, BIGNUM *u, BIGNUM *b, BIGNUM *N) { if ((bn_ctx = BN_CTX_new()) == NULL || (tmp = BN_new()) == NULL || (S = BN_new()) == NULL) goto err; if (!BN_mod_exp(tmp, v, u, N, bn_ctx)) goto err;... err: BN_CTX_free(bn_ctx); BN_clear_free(tmp); return S; } Exhibit 3: OpenSSL Incorrect error propagation → weak key

24. int gnutls_ocsp_resp_get_single (...time_t *this_update) { ret = asn1_read_value(...); if (ret != ASN1_SUCCESS) { *this_update = (time_t) (-1); }... return GNUTLS_SUCCESS; } static int check_ocsp_response(...) { ret = gnutls_ocsp_resp_get_single(...&vtime); if (ret < 0) {... } if (now - vtime > MAX_OCSP_VALIDITY_SECS) {... } Exhibit 4: GnuTLS Missing propagation → incorrect time check

25. AUTHORITY_KEYID *v2i_AUTHORITY_KEYID(...) { serial = ASN1_INTEGER_dup(X509_get_serialNumber(cert)); if (!isname || !serial) { X509V3err(...); goto err; } ASN1_STRING *ASN1_STRING_dup(ASN1_STRING *str) { ASN1_STRING *ret; if (!str) return NULL; } A sample false positive Clang’s symbolic analysis explores paths in each source file independently openssl/crypto/x509v3/v3_akey.c openssl/crypto/asn1/asn1_lib.c

26. Conclusions Error handling bugs are dangerous and hard to find with existing tools EPEx can scalably find and localize such bugs by ○ Only exploring error paths ○ Using a error handling oracle at the caller function https://github.com/yujokang/EPEx Start using it now!