1. One OSINT Tool to Rule Them All
by: Émilie St-Pierre
BSidesLV Proving Ground, July 24th 2017

2. $whoami Émilie St-Pierre Security Analyst at Rapid7
Active in information security for 5 years
Director at large for the SYN Shop hackerspace
Co-host of the weekly Greynoise podcast
Since Defcon 21
SDR meetup at 6:30pm

3. How it all began
A little under a year ago, I was working on one of my very first engagements. It was an external network penetration test, and this company had a single sign-on login page for internal access.
It required 2 things: a company address and a password. I thought this was pretty straightforward: if I could find some valid company addresses, then I could try a password-guessing attack.
Back then, I was still pretty green so I used a tool which was shipped with Kali Linux that is specifically made for harvesting s: the Harvester.
I ran the tool, entered the company’s domain, got some addresses back, also got a few from their website, made my list....
Until a few weeks later when I started reading this book by Peter Kim...
There was at least 3 to 4 times the amount of s on my original list. What if one of these had yielded internal access? What if there's a better tool?
It got me thinking...

4. OSINT Tool Comparison Table
Once I realized I needed to further the knowledge I had of the current tool landscape, I had an idea.
What if I could compile a list of all the tools that search for publically available information and easily see the ones that will get me the kind of public information I need?
Wouldn’t it be great to know which tool gets the best lists?
Which tool gets the most usernames?
Which tools can get the most documents for metadata analysis?
Or tools that have unique functions such as grabbing private API keys from repositories?
This is where the OSINT tool comparison table comes in.

5. Define: OSINT
Open Source Intelligence (OSINT) … is locating, and analyzing publically available sources of information … [with the] goal of producing current and relevant information that is valuable to either an attacker or competitor.
Remember when I was talking about grabbing that publicly available information? That’s called OSINT. OSINT is locating and analyzing publically available sources of information … [with the] goal of producing current and relevant information that is valuable to either an attacker or competitor.
What's valuable to YOU. Don't need to be either to get value from OSINT data.
Let me show you some examples of this

6. Valuable types of OSINT
Usernames s
Technology in use
Location data
Corporate data
Usernames: plan on password-guessing attacks
s: password guessing and phishing campaigns
Tech in use: perhaps you can find a piece of software that has a vulnerability, find out more about their environment in preparation for the attack
Location data:
Now that you understand who uses OSINT, let’s get back to my project

7. Methodology
Compiled a list of reputable, free and popular tools with a focus on organizational penetration testing:
Default Kali Linux OSINT tools
Tools listed in popular pentesting books
Word-of-mouth
OSINT tool lists (osintframework.com)
For my methodology, I compiled a list of tools that focused on organizational penetration testing. By organizational, I mean tools which target companies or organizations, as opposed to individuals.
The list includes Kali, tools encountered in books, word-of-mouth and OSINT tool lists like the OSINT framework. OSINTframework.com

8. Methodology Compared them against 3 benchmarks: Data variety
Data quality
Relevancy
Data variety: How many types of data can be found using this tool? addresses, usernames, information on lawsuits, etc.
Data quality: Is this data accurate? Is it complete? Think of my first example. Will I get back a few addresses, or 3 to 4 times more?
Relevancy: Is the tool up-to-date? Is it using old resources like API’s that don’t work anymore?

9. Data Limitations Non-exhaustive list.
Some tools contain some stand-alone tools.
Some tools are hybrids that do more than OSINT.
Data accuracy could be biased based on chosen sample (sample size = 42).
I acknowledge there are limitations:
Non-exhaustive list. New tools pop up every day, and they may not be on the list yet.
Some tools contain other tools (Toolception)
Some tools do more than just OSINT, I have found many to do different types of scans on top of OSINT.
The fourth thing I want to bring up, I chose a small sample size, and results may differ for you depending on your target.
Which brings me to the results. (Drumroll please)

10. Results
So using all of this data, I built MYSELF a data sheet that is really useful for engagements. Click:

11. Here it is!
Of course I didn't make this table just for myself...
On the left-hand side are the tool names, followed by columns which give the reader an overview with useful information.
I know this is a small font but for those of you who have pulled up the table, you’ll be able to see…
Starting with the first group of columns by categories, and talk about the categories.
At the top of the sheet I’ve added my information to contribute, etc. Pop a note in there.
Help me, etc.
I bet you’re asking yourself “Can you make this easy for me?”

12. https://bit.ly/osintcomparison
I sure can. I want to emphasize how this chart isn’t just useful for penetration testers.
Doing OSINT will help if you’re:
Blue team
Looking at competitors
You’re an investor
Or a lawyer

13. “So Émilie, which tool rules them all?”
Disclaimer: These are the ones that work well for me, my customers, my engagements, etc. They may not work for you, etc.

14. My top picks Best e-mail lists: Recon-ng (URL) Most user-friendly:
Spiderfoot
Easiest metadata analysis:
FOCA
Is there one OSINT tool to rule them all? No, but there are some tools which shine in their own categories.
WHY is this tool better than others?
What did others not do?
Even with that free version of FOCA, it pulls that data etc.

15. Thank you!
Émilie mparison OSINT Tool Comparison Table
Positive!
Check out my OSINT Tool Comparison Table.
Let’s keep in touch
Let’s collaborate
I’m still developing this tool