Security Vulnerabilities in Third Party Code: FIX ALL THE THINGS!KYMBERLEE PRICEBUGCROWD

whoami?

Senior Director of a Red Team PSIRT Case Manager Data Analyst Internet Crime Investigator Security Evangelist Behavioral Psychologist Lawful Good @kym_possible

Agenda

Quick overview of problem space A deeper look at 7 specific libraries Library Management SDL Recommendations Case study

Development Realities

Can only pick two!

security

Hint!

Where the Vulns Are

“Third-party programs are responsible for 76% of the vulnerabilities discovered in the 50 most popular programs in 2013, say the results of Secunia's Vulnerability Review 2014”

http://www.net-security.org/secworld.php?id=16448

When reviewing this report, you find that it is flawed and

not referring to 3rd Party Libraries but third party

software i.e. non-Microsoft programs.

Vulnerabilities by Type

Source: VulnDBJuly, 14 2015

Logjam

bashHOLE?

Shell Shock.. meh

BashBleed

2013-01-12 – GNU C Library Function Heap Buffer Overflow (GHOST)

2005-06-08 – Microsoft IE Script Code Obfuscation (Ghost)

How many vulnerabilities do you think there have been in OpenSSL

since Heartbleed?(please don’t use the Secunia counting method!)

Lets Play Another Game!

IDåç Disc Date CVSSTitle124300 7/9/2015 4 OpenSSL crypto/x509/x509_vfy.c X509_verify_cert() Function Alternative Certificate Chain Handling Certificate Validation Bypass  123176 6/11/2015 10 OpenSSL DTLS Application Data Buffering Invalid Free Remote Memory Corruption  123175 6/11/2015 7.8 OpenSSL signedData Message Unknown Hash Function Processing Infinte Loop Remote DoS  123174 6/11/2015 7.8 OpenSSL crypto/pkcs7/pk7_doit.c PKCS7_dataDecode() Function ASN.1-encoded PKCS#7 Blob Handling NULL Pointer Dereference Remote DoS  123173 6/11/2015 8.5 OpenSSL crypto/x509/x509_vfy.c X509_cmp_time() Function ASN1_TIME String Handling Out-of-bounds Read Issue  123172 6/11/2015 7.8 OpenSSL crypto/bn/bn_gf2m.c BN_GF2m_mod_inv() Function ECParameters Structure Binary Polynomial Field Parsing Infinite Loop Remote DoS  122875 6/2/2015 10 OpenSSL NewSessionTicket Ticket Re-use Double-free Remote Unspecified Issue  122733 5/26/2015 7.8 OpenSSL crypto/bn/random.c BN_rand() Function Off-by-one Buffer Overflow DoS  122331 5/19/2015 4 Transport Layer Security (TLS) Protocol DHE_EXPORT Ciphers Downgrade MitM (Logjam)  122984 5/19/2015 7.5 OpenSSL crypto/bn/bn_print.c BN_bn2hex() Function Off-by-one Buffer Overflow Weakness  119692 3/18/2015 7.8 OpenSSL Invalid Signature Algorithms Extension Renegotiation NULL Pointer Dereference Remote DoS  119760 3/16/2015 7.1 OpenSSL ssl/d1_lib.c dtls1_listen() Function SSL Object State Preservation DoS  119757 3/16/2015 7.8 OpenSSL SSLv2 CLIENT-MASTER-KEY Message Handling Assertion Remote DoS  119758 3/16/2015 7.1 OpenSSL ssl/s3_pkt.c ssl3_write_bytes() Function Multiblock Implementation DoS  119614 3/16/2015 7.8 OpenSSL Client Authentication DHE Ciphersuite Zero-length ClientKeyExchange Message Handling Remote DoS  119756 3/16/2015 7.1 OpenSSL PKCS#7 Missing Outer ContentInfo Handling NULL Pointer Dereference DoS  119759 3/16/2015 7.1 OpenSSL crypto/rsa/rsa_ameth.c rsa_item_verify() Function Invalid PSS Parameters Handling NULL Pointer Dereference DoS  119755 3/16/2015 9.3 OpenSSL crypto/asn1/tasn_dec.c ASN1_item_ex_d2i() Function ASN.1 Structure Reuse Memory Corruption  119761 3/16/2015 7.1 OpenSSL crypto/asn1/a_type.c ASN1_TYPE_cmp() Function Invalid Read DoS  119673 3/10/2015 2.6 OpenSSL s3_clnt.c ssl3_client_hello() Function Unseeded PRNG Handshake Completion Predictable Output  120058 3/3/2015 2.6 OpenSSL Malformed TLS Handshake False Start Data Remote MitM Disclosure Weakness  119328 3/2/2015 5.4 OpenSSL crypto/x509/x509_req.c X509_to_X509_REQ() Function Public Key Handling NULL Pointer Dereference DoS  118817 2/25/2015 10 OpenSSL crypto/ec/ec_asn1.c d2i_ECPrivateKey() Function Error Handling Use-after-free DoS  117855 1/19/2015 2.6 Secure Sockets Layer Version 3 (SSLv3) / Transport Layer Security (TLS) Protocols RC4 Cipher Key Invariance Weakness MitM Plaintext Disclosure (BAR-MITZVAH)  116791 1/8/2015 7.8 OpenSSL dtls1_buffer_record() Function DTLS Record Saturation Handling Memory Leak Remote DoS  116793 1/8/2015 7.8 OpenSSL dtls1_get_record DTLS Message Handling NULL Pointer Dereference Remote DoS  116790 1/8/2015 5.1 OpenSSL TLS DH Certificate Missing Certificate Verify Message Handling MitM Spoofing (SKIP-TLS)  116796 1/8/2015 5.1 OpenSSL Bignum Squaring Incorrect Result Weakness  116794 1/6/2015 4 OpenSSL RSA Temporary Key Handling EXPORT_RSA Ciphers Downgrade MitM (FREAK)  116792 1/5/2015 4.3 OpenSSL Signature Algorithm / Signature Encoding Modification Certificate Fingerprint Manipulation Weakness  116795 1/5/2015 5 OpenSSL Missing Server Key Exchange Message Handling ECDH Ciphersuite Downgrade Issue  116423 10/16/2014 7.8 OpenSSL s23_srvr.c ssl23_get_client_hello() Function SSLv3 Handshake Handling NULL Pointer Dereference Remote DoS 113377 10/15/2014 5 OpenSSL no-ssl3 Build Option SSL 3.0 Handshake Handling Weakness  113373 10/14/2014 7.8 OpenSSL DTLS SRTP Extension Parsing Code Handshake Message Handling Memory Leak Remote DoS  113374 10/14/2014 7.8 OpenSSL SSL/TLS/DTLS Server Failed Session Ticket Verification Handling Memory Leak Remote DoS  113251 10/13/2014 2.6 SSL 3.0 Protocol CBC-mode Ciphers Fallback MitM Remote Cleartext Information Disclosure (POODLE)  109892 8/6/2014 7.8 OpenSSL DTLS Handshake Messages Processing Memory Consumption Remote DoS  109893 8/6/2014 7.8 OpenSSL DTLS Packet Handling Double-free Remote DoS  109894 8/6/2014 5 OpenSSL OBJ_obj2txt Multiple Pretty Printing Functions Pretty Printing Output Remote Information Disclosure  109898 8/6/2014 7.1 OpenSSL SRP Ciphersuite NULL Pointer Dereference Remote DoS  109891 8/6/2014 7.8 OpenSSL Crafted DTLS Packet Handling Memory Leak Remote DoS  109897 8/6/2014 10 OpenSSL SRP Protocol Code Multiple Parameter Remote Buffer Overflow  109896 8/6/2014 2.6 OpenSSL SSL/TLS Server Code ClientHello Message Fragmentation Forced TLS Downgrade Weakness  109902 8/6/2014 9.3 OpenSSL ssl_parse_serverhello_tlsext Resumed Session EC Point Format Extension Handling Race Condition Use-after-free Issue  109895 8/6/2014 7.8 OpenSSL Anonymous (EC)DH Ciphersuite Crafted Handshake Messages NULL Pointer Dereference Remote DoS  107731 6/4/2014 7.8 OpenSSL TLS Client Anonymous ECDH Ciphersuite Unspecified Remote DoS  107730 6/4/2014 10 OpenSSL Invalid DTLS Fragment Handling Remote Buffer Overflow  107732 6/4/2014 7.8 OpenSSL ssl/d1_both.c dtls1_get_message_fragment() Function Invalid DTLS Handshake Handling Remote DoS  107729 6/3/2014 4 OpenSSL Crafted Handshake Weak Keying Material Rollback MitM Weakness  119743 5/6/2014 9.3 OpenSSL crypto/evp/encode.c EVP_DecodeUpdate() Function Base64 Decoding Integer Underflow  106531 4/30/2014 7.8 OpenSSL / LibReSSL ssl/s3_pkt.c do_ssl3_write() Function NULL Pointer Dereference Remote DoS  105763 4/11/2014 4 OpenSSL ssl/s3_pkt.c ssl3_read_bytes() Function Use-after-free Remote Content Injection  105465 4/7/2014 5 OpenSSL TLS Heartbeat Extension Packets Handling Out-of-bounds Read Remote Memory Disclosure (Heartbleed) 

47 NewVulns

10 CVSSv2 Score Max

10 Had Exploit

Public or PoC

Average CVSS 5.23

14 Had Private Exploit

Lets Talk Data

Vulnerability data Spreadsheet software Probably a browser

Putting Data to Use(without being a data scientist)

Data from public sources is limitedFFMPEG: CVE Details vs. VulnDB

CVE Details: 191VulnDB: 1,000+

DATA CAVEAT

“Fixes the following vulnerabilities [CVE LIST] …and more security issues that have no CVE number. Many of these issues can be exploited when a remote file is played back and a few are probable arbitrary code execution vulnerabilities.”

Vuln Spread:

…And multiple products by HP, Oracle (including Java), F-Secure, IBM, MySQL, Novell, OpenBSD, Intel, Juniper, Rapid7, nginx, Huawei, Trend Micro, Linux, Tableau, McAfee, F5, Cisco, Fortinet, Sophos, Python, Citrix, SUSE, Ubuntu, Debian, FreeBSD, RedHat…

Vuln Spread:

And also… OSX, Webkit, Firefox, OpenJDK, OpenOffice, StarOffice, Ubuntu, Gentoo, Oracle Solaris, SUSE, Slackware, BlackBerry products, Fedora, RedHat, Debian, Avaya products, PlayStation 3/4/Vita, Opera for Wii, multiple video games…

Vuln Spread:

Visio, PowerPoint, Adobe Photoshop/Flash/Illustrator, Webkit, iOS, OSX, Android, GIMP, Fedora, Debian, Ubuntu, Slackware, Red Hat, SUSE, Gentoo, Oracle Solaris, VMWare Server, and countless applications.

Vuln Spread:

Tivoli, Fedora, HP-UX, Ubuntu, NetIQ, Attachmate…

Vuln Spread:

Linux, Opera, Konqueror, HP, Sony & Logitech Google TVs…

Vuln Spread:

Library Vuln Count

Vulns Per Year

Releases Per Year

Average CVSS

90 10-11 3 5.4950 6 2 7.4328 3 2-3 6.65

100 12 5 4.72522 80 11 8.96539 98 4 7.07*2010-to present

*2009-to present

2015 Vulns

% total

29 32.2%0 0%0 0%4 4%

135 25.9%58 10.7%

The Numbers: Jan 2007-July 2015

Efficiency At What Cost?

Not just one library impacting many organizations

A single application may have as many as 100 different third party libraries implemented That is a whole lot of patching to keep up on for

both devs and customers

What should you measure library quality on? Count of vulnerabilities Frequency of update releases Average severity of vulns (CVSS or other) Existence of POC or Exploit

DEBATE

Yes!

Take Aways

Open source is secure because everyone can review it - more eyes

makes all bugs shallow.

Everyone *could* look at it, but they don’t.

Accountability for quality is deferred.

Code Quality

That means closed source is more secure because no one

can review it and it is supported by big enterprises,

right?

Bad code is just that, bad code.Bad code exists in Closed Source

software as well.

Code Quality

Vulnerability Management

Vulnerability management is the "cyclical practice of identifying, classifying, remediating, and mitigating vulnerabilities", especially in software and firmware. Vulnerability management is integral to computer security and network security.

Vulnerabilities can be discovered with a vulnerability scanner, which analyzes a computer system in search of known vulnerabilities, such as open ports, insecure software configuration, and susceptibility to malware. Unknown vulnerabilities, such as a zero-day attack may be found with fuzz testing, which can identify certain kinds of vulnerabilities, such as a buffer overflow exploit with relevant test cases. Such analyses can be facilitated by test automation. In addition, antivirus software capable of heuristic analysis may discover undocumented malware if it finds software behaving suspiciously (such as attempting to overwrite a system file).

Correcting vulnerabilities may variously involve the installation of a patch, a change in network security policy, reconfiguration of software (such as a firewall), or educating users about social engineering.

https://en.wikipedia.org/wiki/Vulnerability_management

So Vuln Mgmt is A NetSec Issue!

Cost to Fix Vulnerabilities

The National Institute of Standards and Technology (NIST) estimates that code fixes performed after release can result in 25+ times the cost of fixes performed during the design phase.

tl;dr: Pay me now or pay me later… with interest.

“

”

Fix vulnerabilities as early as is practical, resulting in fewer vulnerabilities to patch at the most expensive time - late in the development cycle.

THE GOAL OF VULNERABILITY MANAGEMENT

Easy, right?

Securityversus…

Performance

Usability

Functionality

Development cost & time

Secure Development Lifecycle

Training Requirements Design Implementation Verification Release Response

Vulnerabilities introduced

Vulnerabilities identified

Vulnerabilities identified

OSS Vulnerabilities identified

Vulnerability Management Process

Identify Issue1 Assess

Impact2 Dev & Test Fix3 Deploy

Fix4 Post Release5

Patch

Tuesday!

Incident Response

Identify Issue1 Assess

Impact2 Dev & Test Fix3 Public

Release w/ CVE

4 Post Release5

Identify Issue1 Assess

Impact2 Dev & Test Fix3 Release4 Post

Release5So you’re a software vendor…

Enterprise admin?

Your patch lifecycle starts

HERE

But wait!

The vulnerability

was in a third

party library!

Identify Issues

Internal Security Research Team, Consultants – pre-release vuln assessments External Security Researchers – post release incident response, bug bounties Third Party Libraries/OSS Disclosures – both pre and post release Automated Tools & Analysis Crash log analysis

Lots of vulnerabilities to manage Vulnerability Management

Identify Issue1

Assess Impact: Prioritization Matters

You have 150 vulnerabilities open with CVSS 7.5+ Your inbound new vulnerabilities average 15 dev tasks

per week, from both internal and external sources What do you fix first?

Highest CVSS Score? FIFO? LIFO? Externally known issues? Issues with Exploit Presence in Metasploit?

Intelligent prioritization reduces risk

Assess Impact2

Dev & Test Fix

“Just ship it, we can patch that later” is not cost effective, but becomes more likely the closer you get to release dates

Vulnerabilities are inevitable. Choose those that you fix pre-release and those you postpone to post-release carefully.

Don’t put off fixing the complicated vulnerabilities – they won’t get easier once the product is in customer hands

Sustainment planning is not just for post-release – you will have to patch vulnerabilities in perfectly functional code before RTM

Dev & Test Fix3

Now lets go write some code!secure^

Vulnerability Management in SDL

Define guiding Security principles

Define prioritization model and sustainment plan

Requirements Design Implementation Verification

Design for security and reduce attack surface

Evaluate vuln trends in libraries as part of selection criteria

Automated static analysis tools

Deprecate unsafe functions

Code scanning tools to monitor all third party libraries – know what you use and where

Automated static and dynamic analysis tools, fuzzing

Manual pen testing & attack surface review

Update 3rd party libraries regularly

Be Prepared

Analysis of vulnerability trends to predict future workload How many vulnerabilities are identified per month? What are their sources?

What are the vulnerability types? Is dev training indicated? How quickly is your vulnerability backlog growing (or shrinking)? What is your average Time To Fix? What early monitoring processes can you put in place to minimize

surprises? Can you identify low friction areas to diminish risk?

Network Admins

Ask potential software vendors about their SDL program and vulnerability trends

Monitor the third party libraries being used in software you deploy and press vendors for security fixes

Make it clear security is a priority

Case Study

Strong security team in rapidly growing enterprise software company

Attended my OSS talk with Jake Kouns at BlackHat 2014 Requested a copy of our slides for internal use

Shared both their own SIRT data and our data regarding security risk with Leadership

Case Study: VMWare

Already had mature incident response monitoring of 3rd party libraries in released products

Adding proactive evaluation and rating/approval of third party libraries in development phase

Case Study: VMWare

Evaluated and implemented code scanning tool for finding third party libraries in products

MOOSECON internal security conference session on 3rd party library vulnerabilities

Case Study: VMWare

Active testing of third party and OSS libraries along with native code in products

Partnering with dev teams to create proactive plans for routine patching cadence as part of dev lifecycle

Case Study: VMWare

ThanksJAKE KOUNSRISK BASED SECURITY

Discussion

Kymberlee Price Senior Director of Researcher Operations@kym_possible Bugcrowd