One OSINT Tool to Rule Them All by: Émilie St-Pierre BSidesLV Proving Ground, July 24th 2017

$whoami Émilie St-Pierre Security Analyst at Rapid7 Active in information security for 5 years Director at large for the SYN Shop hackerspace https://synshop.org Co-host of the weekly Greynoise podcast https://greynoi.se Twitter: @L4bF0x Since Defcon 21 SDR meetup at 6:30pm

How it all began A little under a year ago, I was working on one of my very first engagements. It was an external network penetration test, and this company had a single sign-on login page for internal access. It required 2 things: a company e-mail address and a password. I thought this was pretty straightforward: if I could find some valid company e-mail addresses, then I could try a password-guessing attack. Back then, I was still pretty green so I used a tool which was shipped with Kali Linux that is specifically made for harvesting e-mails: the Harvester. I ran the tool, entered the company’s domain, got some e-mail addresses back, also got a few from their website, made my list.... Until a few weeks later when I started reading this book by Peter Kim... There was at least 3 to 4 times the amount of e-mails on my original list. What if one of these had yielded internal access? What if there's a better tool? It got me thinking...

OSINT Tool Comparison Table Once I realized I needed to further the knowledge I had of the current tool landscape, I had an idea. What if I could compile a list of all the tools that search for publically available information and easily see the ones that will get me the kind of public information I need? Wouldn’t it be great to know which tool gets the best email lists? Which tool gets the most usernames? Which tools can get the most documents for metadata analysis? Or tools that have unique functions such as grabbing private API keys from repositories? This is where the OSINT tool comparison table comes in.

Define: OSINT Open Source Intelligence (OSINT) … is locating, and analyzing publically available sources of information … [with the] goal of producing current and relevant information that is valuable to either an attacker or competitor. http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#OSINT Remember when I was talking about grabbing that publicly available information? That’s called OSINT. OSINT is locating and analyzing publically available sources of information … [with the] goal of producing current and relevant information that is valuable to either an attacker or competitor. What's valuable to YOU. Don't need to be either to get value from OSINT data. Let me show you some examples of this

Valuable types of OSINT Usernames E-mails Technology in use Location data Corporate data Usernames: plan on password-guessing attacks E-mails: password guessing and phishing campaigns Tech in use: perhaps you can find a piece of software that has a vulnerability, find out more about their environment in preparation for the attack Location data: Now that you understand who uses OSINT, let’s get back to my project

Methodology Compiled a list of reputable, free and popular tools with a focus on organizational penetration testing: Default Kali Linux OSINT tools Tools listed in popular pentesting books Word-of-mouth OSINT tool lists (osintframework.com) For my methodology, I compiled a list of tools that focused on organizational penetration testing. By organizational, I mean tools which target companies or organizations, as opposed to individuals. The list includes Kali, tools encountered in books, word-of-mouth and OSINT tool lists like the OSINT framework. OSINTframework.com

Methodology Compared them against 3 benchmarks: Data variety Data quality Relevancy Data variety: How many types of data can be found using this tool? e-mail addresses, usernames, information on lawsuits, etc. Data quality: Is this data accurate? Is it complete? Think of my first example. Will I get back a few e-mail addresses, or 3 to 4 times more? Relevancy: Is the tool up-to-date? Is it using old resources like API’s that don’t work anymore?

Data Limitations Non-exhaustive list. Some tools contain some stand-alone tools. Some tools are hybrids that do more than OSINT. Data accuracy could be biased based on chosen sample (sample size = 42). I acknowledge there are limitations: Non-exhaustive list. New tools pop up every day, and they may not be on the list yet. Some tools contain other tools (Toolception) Some tools do more than just OSINT, I have found many to do different types of scans on top of OSINT. The fourth thing I want to bring up, I chose a small sample size, and results may differ for you depending on your target.   Which brings me to the results. (Drumroll please)

Results So using all of this data, I built MYSELF a data sheet that is really useful for engagements. Click:

Here it is! Of course I didn't make this table just for myself... On the left-hand side are the tool names, followed by columns which give the reader an overview with useful information. I know this is a small font but for those of you who have pulled up the table, you’ll be able to see… Starting with the first group of columns by categories, and talk about the categories.   At the top of the sheet I’ve added my information to contribute, etc. Pop a note in there. Help me, etc. I bet you’re asking yourself “Can you make this easy for me?”

https://bit.ly/osintcomparison I sure can. I want to emphasize how this chart isn’t just useful for penetration testers. Doing OSINT will help if you’re: Blue team Looking at competitors You’re an investor Or a lawyer

“So Émilie, which tool rules them all?” Disclaimer: These are the ones that work well for me, my customers, my engagements, etc. They may not work for you, etc.

My top picks Best e-mail lists: Recon-ng (URL) Most user-friendly: Spiderfoot Easiest metadata analysis: FOCA Is there one OSINT tool to rule them all? No, but there are some tools which shine in their own categories. WHY is this tool better than others? What did others not do? Even with that free version of FOCA, it pulls that data etc.

Thank you! Émilie St-Pierre @L4bF0x https://github.com/L4bF0x/osintco mparison OSINT Tool Comparison Table https://bit.ly/osintcomparison osintcomparison@gmail.com Positive! Check out my OSINT Tool Comparison Table. Let’s keep in touch Let’s collaborate I’m still developing this tool