OSINT / Dox / Forensic Information Gathering Techniques

[CyberGod]

Information Gathering Techniques
Introduction
This tutorial will guide you through the steps of conducting and open-source intelligence gathering. We will cover various topics, and some are not going to be the easiest to understand. I tried to make the harder to understand sections more in-depth than ones that needed less explaining. Also I'll give you a little background info about this guide:
1. I Wrote This Myself, No C&P(MuddyBucket Better Be Proud XD)
2. This Took Me 2 Days to Make, So I Hope Everyone Enjoys
3. Enjoy Smile
Organizing Information
We will be using Dradis Framework to manage our information. Here's a tutorial on how to setup and use Dradis:
Step 1:
Download Dradis From Here: http://dradisframework.org/
Step 2:
Start Dradis Using This Command:
Spoiler(Hide)

Step 3:
Access the Dradis CP, Mine is on 127.0.0.1:3000
The First Time You Access it You'll Have to Setup a Password:
Spoiler(Hide)

Then You'll Sign-In.
Step 4:
Now Add a Node:
Spoiler(Hide)

Then the Rest is Pretty Easy to Figure Out.

To Stop The Server Press Ctrl+C or Close the Terminal.

Copying Websites Locally
For This We'll be Using a Program Called HTTrack. You Can Download it From Here:
https://www.httrack.com/
So Here is a Simple Tutorial on How to Use it:

https://www.httrack.com/html/step.html

Social Media
Social Media can be used to find individuals associated with the business. You can use that info for Social Engineering and Spear Phishing Campaigns. Different Social Media sites can serve different purposes, such as:

• Facebook - Tends to contain current events or situations.
• Twitter - The "pulse" of an employee
• LinkedIn - Contains technical information and work on various projects

Harvesting E-Mails
InfraGard and Sony are illustrations of how dangerous an email compromise can be. So to get emails we'll use a program called "The Harvester"
It's coded in Python and is designed to get emails for a specific domain. The code is located here:
https://theharvester.googlecode.com/svn-...rvester.py
This is the usage:
Code:
./theharvester.py -d microsoft.com -l 500 -b google
Here is an example usage:
Spoiler(Hide)

Information Gathering With Maltego
You can get Maltego from here:
https://www.paterva.com/web6/products/maltego.php

This tool can gather lots of different info. How you would scan a domain is like this:
1. Make a New Project
2. Add a Domain
3. Change Domain Name
4. Right-Click on Domain and Then Run "All Transforms"

This is an example can on Microsoft:
Spoiler(Hide)

The Power of Google
For this section you may just want to read my guide:

What is Google Dorks [ Find Vunerable sites + Dorks Database]

Hidden content

hellofhackers.com

That covers Google dorks which is what this section would cover. But this would be a little different, we could search for private files. Here is an example dork for files:
Code:
site:fbi.gov filetype:pdf

File Analysis
Files on websites can sometime contain info that can be used for further analysis. This information may include:

• Instructions for Accessing Systems
• Procedures, Training, and Human Resource Info

Also a file's metadata can contain some interesting info such as:

• Author and Modifier Usernames
• File Paths
• Software Version
• Printer Details
• Email Addresses
• Comments

Here are 2 tools to read metadata:
FOCA - Reads file metadata from a wide range of formats
Metagoofil - Another tool that extracts metadata from files on a customer website

Gather Information Using WhatWeb, HttpRecon, and SSL Scan
To use what web you can go here:
http://whatweb.net/
This is an example search response:
Spoiler(Hide)

Then you can also use HttpRecon for some more info:
https://w3dt.net/tools/httprecon
Here's an example search:
Spoiler(Hide)

Then we can use SSLScan to get some info about a servers SSL:
https://github.com/rbsec/sslscan
Here's an example response SSLScan may give:
Spoiler(Hide)

WhoIS and Finding Other Websites Hosted on a Webserver
You can find lots of useful info through a whois search on a domain. The only downside is that there are ways of protecting that info.
Here's an example of an unprotected whois:
http://pastebin.com/HPGEpE0K
Here's an example of a protected whois:
http://pastebin.com/tGzfcYAB

Now how to find other websites hosted on a webserver. To do this we'll use a site called SameIP
This is an example:
Spoiler(Hide)

The more sites on a server the more of a chance you have to find a vulnerability in it.

IP Geolocation
This can help you learn even more about your target. It is not always accurate though. Here's about how accurate they are:

• Accuracy rates on naming the city from an IP address vary between 50%-80%
• Determining the nation of an Internet user is 95%-99% percent accurate, for reasons that have to do with how IP addresses are allocated and registered
• Determining the physical location of the user, down to a city or ZIP code, is trickier and less accurate

Here's some info you may get from an IP geolocation:
Spoiler(Hide)

Websites for Geolocation:
http://whatismyipaddress.com/
http://www.ip2location.com/

Tools for Geolocation:
http://www.edge-security.com/soft/geoedge.py
WhatIsMyIPAddress.com

Finding Subdomains and Hidden Wed Directories
A simple way of finding hidden web dirs is by looking at the robots.txt file on a server.
Example: http://www.microsoft.com/robots.txt
There are no interesting hidden web dirs in that robots.txt file, but there is a redirect in it:
http://www.microsoft.com/imaginecup >> https://www.imaginecup.com/
If you can count that as "interesting"

Basically just look at anything that has "Disallow:" beside a directory/file.

Shodan
Shodan is a special type of search engine. You can find lots of info, like webcams:
Spoiler(Hide)

I'll just let everyone mess around with that.

HTTP and DNS Load Balance Detection
To do this we'll use a tool called LBD(Load Balance Detector). You can get it from here:
https://raw.githubusercontent.com/craig/...lbd/lbd.sh

Example Scan:
Spoiler(Hide)

As you can see Google uses DNS load balancing.

Web Firewall Detection
This will be done using a tool called wafw00f which is coded in Python. To get it run this command:
Code:
pip install wafw00f
Of course you'll need pip, just google get-pip.py.
Then this is the usage:
Code:
wafw00f This is an example: Spoiler(Hide) ...URL='http://www.example.com/']www.example.com)
[*]Local DNS server request authoritative DNS server for destination domain. (example.com)
[*]Local DNS server request IP address of destination host from authoritative DNS server. (X.X.X.X)
[*]Local DNS server returns IP address of destination host to source host.
[*]Source host connects to destination host.
[/LIST]

Important DNS Record Types
A Record - Also known as host record, this links a hostname to an IP address
AAAA Record - Returns an IPv6 address
NS Record - Authoritative nameserver for the zone
CNAME Record - Alias to another name, the DNS will try the new name
MX Record - Mail server for the zone

DNS Tools
Dig Command
Usage:
Code:
dig [website]

Host Command
Usage:
Code:
host [website]

There is also nslookup
Usage:
Code:
nslookup
Then when the console opens:
Code:
[website]

Conclusion
Hope that you have learned many things from this guide. If there's anything else you would like me to add HMU with a PM. That's all for now.