In the OSINT world, most places if you want to get hired to do the work, you usually have to have a clean criminal record check. So, if you have a crime in your background, you're not going to get hired. That's part of the job. It's not all sort of digital online, sometimes you have to pick up a phone and call this little old lady sitting in an archive in the middle of nowhere, United States, and she's so happy to help you. She will bend over backwards to find that one mention of that one guy from that one time 25 years ago. You would be surprised what can kind of hide in people's pasts that you can then use to conduct your investigations.
Introduction
MJ is a former investigative journalist with Vice who worked on numerous documentaries and feature stories. He later transitioned into intelligence analysis for defense contractors in Canada before founding Permanent Record Research with colleagues. Currently, he works in the nonprofit space focusing on anti-human trafficking and corporate fraud, while occasionally appearing on television for documentary and news work.
The Reality of OSINT as a Career
From a job perspective, the best way to think about it is OSINT is something you do within another job. You might be working in cyber security or information security where OSINT might be a part of that, looking for threats, hunting for different types of malware and bits of code spread across the web. You might be a journalist, you might be working in law enforcement, you might be working as an analyst for some corporation. OSINT is kind of interwoven into the work you do.
Is there a job that's specifically "OSINT professional"? It's starting to show up a bit more, but really it's something that's kind of interwoven and baked into other jobs. We use OSINT every day when conducting research to find information about an issue, a group, a target, or whatever. So OSINT isn't really a standalone job yet. It's something you'll be doing within a whole bunch of careers.
The Mindset Behind OSINT
Doing OSINT or doing investigations or intelligence gathering is not about the tools. It's about the mindset. Do you have the gum chewing ability? Do you have the interest in puzzle solving? That's what's going to make you good at this. The tools just make it easier. The tools are the things that make you a bit more efficient.
Tool No. 1: What's My Name
What's My Name (whatsmyname.app) was highlighted by Micah Hoffman and created by OSINT Combine. This username search tool is super cool. It essentially goes out and scrapes all of the different places where a username might be on the internet and gives you a list.
|
| What's My Name — Free Username Search Tool for OSINT Investigations |
The tool is free and runs on the cloud. It doesn't log anything. There's no list anywhere of what people are searching for. You can even Google it and it shows up. This has to be definitely one of the top tools. It's free, it will probably forever be free, and it works so well to show you all of the different places that username shows up. It also does a nice little Google search as well.
Real-World Application
This is used almost every investigation because of how well it works. For example, when working on a CBC documentary (Canadian Broadcasting Corporation), there was an effort to track down an individual who had committed heinous acts. He had a specific username online that he used all the time. By running searches against that username, multiple other social media pages as well as forums where he was participating under that username were found. This helped consolidate research into understanding who this person was, what his interests were, and when the NSFW filter was removed, more information was discovered that educated the investigation on his other interests outside of just committing crimes. You get a picture of who your target is.
Tool No. 2: Google Dorking Tools
Google dorking is the act of essentially manipulating Google searches to search specifically for file types, certain open ports, or documents that might exist, images, whatever.
DorkGPT
DorkGPT allows you to tell it what you want to search for and it will generate that Google dork for you. For example, if you want all documents from NASA, you can specify document types like Word or PDF, and it will produce the appropriate file type search parameters. You just copy it and open up a Google search, pop it in, and it will pull up all of the docs that show up.
|
| DorkGPT — Automated Google Dork Generator for OSINT Research |
Dork Search Pro
Dork Search Pro is a lot of fun, though it has a lot of ads, so make sure you have some sort of ad blocker going because it will try to redirect you to gambling websites and such. It has a bunch of cool different tools, with the most interesting one being the Google Dorking feature.
You can essentially punch in very specific things you're looking for. Maybe you're looking for public PDFs, Excel data, text or notes, presentations, spreadsheets, archives. You can look at server configurations if you're doing red teaming, threat hunting, bug bounties. This essentially will generate Google dorks for you that focus on these particular things. It really does the work for you versus having to keep a spreadsheet yourself.
One thing that's really useful is you can target a whole site and it will run a giant dork just kind of on its own. You can do a full system scan and it's going to essentially run a whole bunch of different types of dorks, look for the index of that site, give you any particular admin panels, anything like that. You can sort of cruise around and then pick what you want, what you're interested in, right, that you might find unique to your use case.
Important Legal Considerations
In the OSINT world, most places if you want to get hired to do the work that people in this field do, you usually have to have a clean criminal record check. So if you have a crime in your background, you're not going to get hired. That's kind of thing one for anyone who's maybe looking for work in this field, actual work that's taxed, not criminal work.
The second thing is really a lot of the work you end up doing has to go to court. So if it's obtained illegally or there's even a shred that this was obtained illegally, your client, whoever you're working for, is going to be really obviously upset at you, because all that evidence you collected is now officially garbage. There are a bunch of doctrines in the legal world where even if one piece of that data is stolen or from an illegal source, it poisons the rest of the evidence. So it all gets thrown out.
You just have to be really careful. When you start doing open source intelligence, the second you start kind of crossing into that gray illegal area, a judge could throw the entire case out and your client walks away super pissed off.
ODCrawler
ODCrawler searches open directories. These are all files that are just publicly available on the internet that sit in just open public repositories. You can punch in a keyword, maybe a target, maybe a company, and hit search. It will return downloads, books, open gateways of movies and documentaries. It's got everything that sort of has the keyword you're looking for, including MKV files, PDFs.
|
| ODCrawler — Open Directory Search Tool for Finding Publicly Available Files |
This is just searching open repositories of files. Obviously, very general examples will return broad results, but if you're searching for a specific target, something maybe more nuanced as part of an investigation, there might be documentation that mentions your target or mentions something you're looking into, a court case, whatever, that could exist in some public open repository somewhere.
Operational Security Warning
Operational security matters. You don't necessarily want to start clicking on links randomly in your browser. You want to use some sort of virtual machine or if you're using something like a remote desktop situation, you want to kind of container this as much as possible, just to prevent issues. While various repositories might be safe and legitimate, you want to be a bit careful about where you're getting your stuff from.
Tool No. 3: Kagi Browser
Kagi is a browser that's one of those paid browsers. When people hear "paid browser," they get a little hesitant because Google Chrome is free. But you have to remember that Google, in order to be able to provide you free searches, is selling you stuff. You become the product that they're monetizing.
Kagi doesn't do that. Kagi is really great because it's really cheap. It's like five bucks a month and you can get access to their browser. Kagi is like what Google was in 2004, before Google "shitified," before they kind of sold out to ad buys and all that stuff. Google was a really great search engine. It really revolutionized how we did searches online. Kagi is kind of a return to this.
Key Features
One thing that's really loved about Kagi is that you can set location specifics. So if you're doing OSINT research and you're looking at something in Argentina, you can tell Kagi to focus in on Argentinian based searches. Google can do this too, but one thing that Kagi does also, which is really neat, is you can classify the stuff you want to look at.
|
| Kagi Browser — Privacy-Focused Search Engine with Advanced OSINT Filtering Options |
If you just want to look at small web stuff—these are the websites that are a little small, or blogs and local papers and stuff like that—you can streamline your search to just look at that. So if you know, for example, you're looking at a target who owns a company in Argentina, local papers might have covered that, but a large news agency wouldn't. You can focus your search to small websites in Argentina and look there.
PDF and Specialized Searches
Another cool thing that you can do is you can search just directly for PDFs. You can punch in anything you want and it will find you all of the PDFs. You can look at things like Fediverse, forums, news, Usenet, and archives. There's the ability to search just directly forums alone. So if you're looking for something that's talked about a lot in forums, you can search that.
Benefits Over Traditional Search Engines
Kagi is a great little browser. The search results are a little different. You're not bombarded with AI. You're not bombarded with ads. You're not bombarded with sponsored posts or sponsored content. All that stuff's gone.
Kagi even has a free option. There's about 100 to 150 searches a month you can do on Kagi for free before they make you pay. The unlimited searches option, which provides around 5,000 searches, is for $5 a month. It's not that much. As you use it, you'll start to realize that you kind of like it more than Google. It does a better job of giving you the things that you want versus feeding you sponsored posts and AI trash.
Multiple Search Engine Strategy
Something like Kagi is great because when you're doing OSINT, a common rule in early OSINT intro stuff is you don't just want to use one search engine, because different search engines index stuff differently. So if you're using Google, that's awesome, but you also want to run your same searches in Bing, you want to run it in DuckDuckGo, you want to run it in Kagi, you want to run it in Yandex if you're operating in that part of the world. Different search engines will give you different results. So you always want to spread out your searches across multiple engines because they all work differently.
Kagi is definitely one of those game changers, especially when you start searching for those kind of smaller things like the forums, the academic papers, the small web stuff, and PDFs. There's some pretty good stuff you can find.
Tool No. 4: Ubicron
Moving on to the next tool, this one focuses on how you keep yourself and your investigation all contained so you can return to it. A new player in the game that's been explored recently and is really enjoyable to use is a company called Ubicron from Vortimo. Ubicron is built by the gentleman who built Maltego and subsequently sold it. This is his version of what we often see in tools like Hunchly or Forensic OSINT.
Ubicron is kind of this new kid on the block and it does a really cool job of how it classifies your searches. Currently, it's free and the free access is pretty robust. When you turn the recording on, it's going to look at all the stuff you've looked at. What's really liked about Ubicron is you get this little widget that shows up on your browser and you can take screen grabs. You can put in notes like "check this out later" or "research me," and you can leave notes for your staff or your associates, your partners, whatever.
|
| Ubicron — Investigation Documentation Tool with Screen Capture and Auto-Scroll |
Key Features
You can save the entire web page and it saves the page for you. You can even save it as a PDF. It'll even have page scroll functionality. So it's going to scroll the entire search for you. Imagine if you're using this potentially on social media or on a forum where you need to get all of the comments or the posts saved. You can start it and it will auto-scroll. So you're not even touching your mouse. It's auto-scrolling the search page and it's going to save all this as one large PDF.
If you have this for a YouTube channel or a forum where you're monitoring things, you can just scrape it all. When you return, you can see the documented question, look at it, see the link to it, open it up as a PDF and save it. This thing is giving you screen grabs, it's exporting names from there, and it all becomes searchable.
As you conduct investigations and as this thing soaks up data while you're investigating, it's also picking out all of those really cool selectors. It's going to look at a website and it's going to pull out the phone numbers. When you click on it, there's phone numbers there. You can grab a page and go back to Ubicron and click on page text. It pulls out all the text for you and it's going to start scraping and looking for any identifiable information.
Real-World Application
As it scrapes the page, it's also able to pull out specific names that it found in the document. It's read the first page of a court case, saved it, and then gone through and pulled out information. You can see there's names, there's even phone numbers that get mentioned. You can go through this and punch it in. As you're doing an investigation, you might bump into another phone number. You can punch it in and it's going to search those documents whether that phone number pops up in your research.
It's a really great tool for anyone who wants to back up their investigation as they're conducting it. This is the free version and it's very powerful on the free version. A shout out also goes to Forensic OSINT. These folks are terrific too. They build a similar tool. Forensic OSINT is a very similar tool that does great work. They do some cool stuff regarding YouTube videos and how they extract data from YouTube videos. If you're looking for something like that, check out Forensic OSINT as well.
Tool No. 5: Newspapers.com
This one's super simple but it's a favorite place to go. The cases that have been solved reading newspaper articles are numerous.
Real Case Example
There was a situation where there was the name of a woman and the search was for her son. Her son had some stuff that various authorities and agencies wanted to understand. But the last name of this young man was sort of a mystery and his mother's name, which was available, was changed because she remarried. So essentially there was very little information to work on. There was a woman's name and then the first name of the son.
After starting to go through social media and trying to track information, it became clear that this woman, this mother, was well aware that her son was in a bit of trouble. She had essentially deleted everything and there was very little, very difficult to track down anything about this young man. So the decision was made to pivot.
Using her first name as well as the suspected ex-husband's name at the time, those first names were entered along with the son's first name. So there were just three first names, but with some basic information about the date and the town they were from. When it was searched up in newspapers.com, it took to a local newspaper in British Columbia that had those three names in it. And it was a birth announcement.
|
| Newspapers.com — Historical Newspaper Archive for Finding Public Records in OSINT Research |
It all added up. It was her, it was the ex-husband, and it was him. And it had the last name. So it was possible to track down the person just by a birth announcement from a newspaper article from 1989 in Langley, British Columbia.
Why Newspapers Matter
You can find a lot of stuff about people in old newspaper articles because you're going to find people's old associates, old friends. When searching for individuals, it's a great way of searching in a place that doesn't really forget. Newspapers don't forget.
These stories will be filed. Every time anyone writes anything, they'll dig up these photos. Newspapers last forever. These are good places to go. You can often find old companies associated with an individual, old friends associated with an individual. When looking for someone and you don't know where they are, but you find a friend of theirs from 2006, they might know someone who might know someone, and then the investigation goes in that direction.
Newspapers.com archives a bunch of newspapers. The stuff you can find on here is surprising. It covers countries all over the world. Articles have been run in parts of North Africa, in Europe. It's all over the place.
Subscription Information
You can run a few basic searches for free, but you do eventually have to pay. The fee runs about $18.99 to $20 a month. So it's not a fortune to get a subscription to newspapers.com. It becomes one of those tools you start using more and more when you have it because you will just simply start searching up targets and you will find old articles about them sometimes. There have been plenty of cases where information about a target has been found using this service.
Beyond Digital Searches
That's not to say you don't go into local newspaper archives as well. There have been times where local papers in some small town have been called to ask if archives could be seen and they'll let you see them. Sometimes that's part of the job. It's not all digital online sleuthing. Sometimes you have to pick up a phone and call this little old lady sitting in an archive in the middle of nowhere United States. She hasn't talked to another human being for a week because no one comes to the archive, and she's so happy to help you. She will bend over backwards to find that one mention of that one guy from that one time 25 years ago.
Tool No. 6: Court Records - Judy Records and CanLII
Judy Records
For court cases and records for anything in the United States, Judy Records is recommended. This is all free. It has 760 million court records. You'd be surprised what you can find on targets in jury records. If you're Canadian, CanLII is your source for that. Obviously, South Africa, England, wherever you're from, France will have their own sort of public record search of court cases.
|
| Judy Records — Free US Court Records Database with 760 Million Records for OSINT Research |
Intelligence Value of Court Records
You would be surprised the intelligence you can gather from court cases. Here's an example: When doing an investigation into a construction firm that was engaging in some illegal activity, court records were found that they were being sued by an interior designer. The phone was picked up and the interior designer was called and they told everything. They gave phone numbers, they gave court records, they gave this giant dump of files because their civil lawsuit never went anywhere.
With that information, it was possible to pin down a whole bunch of staff members that worked for this construction firm. Then with that, more digital sleuthing could be done, searching them out in social media searches, breach data searches, and all of a sudden you have this picture of this company of what it looked like two or three years ago. Then with that, you can build out the rest of your investigation. Court records are solid gold.
The Importance of Historical Records
There's no such thing as a tool that's going to take you and give you the answer to every question. The tools will help you frame up your investigation and they just make it easier. But really at the end of the day, if you're looking for fraud, searching old archives, searching newspapers, searching court records is a great thing to do because oftentimes if somebody's committing fraud now, they may have committed fraud in the past and there could be records of that and they've simply moved on.
Honestly, people forget. There have been plenty of companies that have been investigated for clients where due diligence was needed on a company or investigation was needed because there was a suspicion of something illegal going on. Sure enough, you search an old newspaper and they, in 2014, broke a whole bunch of laws and were arrested for fraud. But nobody looked into it because it was an old newspaper. It didn't show up on any kind of basic social media search.
You would be surprised what can kind of hide in people's pasts that you can then use to conduct your investigations. Old newspaper articles oftentimes they're not googleable—they don't show up in Google's cache of data. So newspapers.com is that sort of storehouse for you as well as other newspaper archives.
Tool No. 7: OSINT Industries
Another tool that's used all the time is something called OSINT Industries. OSINT Industries is that tool much like Epio or other related tools. You feed it a phone number, you feed it a username, you feed it an email, and it's going to kind of go out into the internet and find information on that target.
Pricing and Professional Use
Prices range. It's about 20 bucks a month for boatloads of searches. So again, it's not a fortune. Professionally, 20 bucks a month is not a tough budget to swing. And honestly, this is probably one of those things that's used all the time.
What It Reveals
When you put in an email and search, it's going to pull up everything that they can find about that email. It shows you what's really cool—a timeline of when the accounts were created and last access. You can kind of zoom in on these and check it out. You can also see where data was breached. You can see various large data breaches that it's also sourcing information from.
|
| OSINT Industries — Comprehensive Email and Username Intelligence Tool for OSINT Investigations |
If you want, you can have it load a relationship graph to show you how it's all kind of talking to each other. Even if you just scroll down, there's pictures, old accounts, Etsy accounts, old Gravatar images, old Wix accounts. You can really see accounts from various years. It really builds out this kind of profile on an individual. It even tells you the stuff you signed up for—Spotify, Bitmoji, Facebook, various gaming accounts—all of this stuff shows up, even photos.
Cost-Benefit Analysis
This thing is incredibly powerful. When it boils down to the time you want to spend doing your work, you're going to sit there and say, "I'm going to search all this out myself and that's going to take me several hours or I could pay $20." There's that cost to benefit ratio of what else could you be doing with that time.
At 20 bucks a month, you can't go wrong. Really, those are the tools that are used sort of day-to-day in the OSINT world.
Honorable Mention: Maltego
Alongside OSINT Industries, a shout out goes to Maltego for two reasons. One, Maltego and Hunchly have now joined forces. If you are a Hunchly user and you prefer Hunchly over Ubicron or Forensic OSINT, Hunchly is now tied into Maltego, which is really great. Maltego has the searches, it has the graph.
Professional Use Case
Maltego is a tool that's used fairly often as a professional if there's a budget that could be used, because Maltego does a really good job of taking a lot of tools that are used, stuff like OSINT Industries, stuff like Dark Side or other breach data search engines, and it puts it all together into searches for you. They have a really great domain analysis tool. They have really great connections with different social media so that it pulls that data out and it charts it, which is really nice. The fact that it now ties in Hunchly is really cool because you can save your work and it gets hashed right there.
The other bit that's really cool is they've got some other cool tools like Monitor where you can put in a social media profile and it will automatically look at that profile by essentially tracking a social media profile. Or you can put in a website and it'll track changes or anything like that. So there's almost like a monitoring tool that's really liked.
When to Use Maltego
Maltego is used, but oftentimes an investigation sometimes stops and ends at a newspaper archive and a quick Facebook search. It really depends on how big investigations are. Where work happens or for clients that have been worked with, sometimes they want something super simple. Just need to find one person. A concerned adult needs to track down a missing person, and one hour later they're found.
Sometimes there's a situation where there are multiple companies believed to be conducting some sort of fraud, and there's a potential for some sort of illegal activity beyond the fraud, whether it's drugs or something else. Well, that's a whole bigger operation. That's a whole bigger investigation which would take sometimes months, sometimes years that you're trying to just collect as much data and information as you can to have collected and ready to use.
Maltego is great for those small operations because you can just get one little piece of information. It's great for those bigger things too because you can graph it all together and you build a picture of your investigation that you can easily share. But it's all based upon your money and your budget. Most of the tools shown today are free or dirt cheap, so you can do a lot of your investigating just using those.
Additional Tool: Dark Side
Dark Side is breach data. This needs to be caveated with a few things. The first key thing being is that if you're doing this work, your client or whomever you're working for, your employer is going to set the boundaries of the work you're doing. Some places will say breach data is not allowed to be used. They don't even want it touching the system because if things go to court, those things are discoverable. If it turns out that in the opposition, whoever that is, in their discovery that they find you were using breach data, they could petition the court and the judge to essentially throw that evidence out.
Legal Considerations
Breach data is not something that you can use all the time. Some jurisdictions allow it, no problem. Some jurisdictions don't. The big caveat on this is consult legal. Always consult legal, because you don't want to be the person who's doing all this work and found information and then all of a sudden you're getting fired because you screwed it up.
About Dark Side
On the flip side of that, Dark Side's great. They're a US-based firm and from a trust level, they're trusted implicitly. These guys are terrific. From a trust perspective, there are no concerns that they're bad actors or anything like that.
Dark Side is this tool. You can see how many records they have. You can even see the last hacks that have occurred and the breaches that they're pulling from. Essentially they've got two sections. You can search data breaches. They've also started developing info stealer logs as a new part of the business. Stealer logs, much like breach data, get in that kind of murky gray area. It really depends on your client, your jurisdiction and what you're doing.
What You Can Search
With data breaches, you can search all of these different tags or selectors. You can search for addresses, aliases and usernames, company names, cryptocurrency wallets, domain information, emails, IP information, general names like first names and last names, passwords, phone numbers, and user IDs. There's a whole series of things you can search for.
When you search an email, it's going to search all the compromised records. It will show you passwords (often blurred out), and as you scroll down, there's more. You can see there's often names attached to it. You can pull up addresses. There's always a more info button and that will allow you to figure out more information. You can see hash passwords, dates of birth, IP addresses that were logged in the data breach, salt keys that you could potentially utilize.
Filtering and Searching
The nice thing is you can stream this down. You can say you only want to search in specific countries or regions. This data is incredibly powerful because one, you have the potential to obviously cause harm with this if you're using it for bad reasons. But from an intelligence perspective or from an investigation perspective, you can find personal information, phone numbers and whatever.
You can search up IP addresses. You can right-click on information and search that IP. You can find essentially that someone has multiple accounts or usernames. You can keep searching. You can keep going. You can find that a particular email uses a specific username, but then there's another email that has that username as the front tag for the email address. It's possible this is the same person from different data breaches. It allows you to connect dots fairly quickly, but always operating under this caveat that sometimes this data is not always usable in a legal sense.
Speed and Efficiency
It becomes definitely a bit spooky when you start using it. But again, from the speed and efficiency of an investigation, Dark Side makes it very fast because you can find often information quite quickly.
International Privacy Laws
One thing to understand is for American viewers, you could Google search somebody and you can find their home address and their home phone number right on Google. There are so many services like Intelius and Spokeo that just aggregate all this data. Once you leave the United States though and you start doing searches on people outside of the states, Canada is a great example. Canadian privacy laws are incredibly strict. You can't just punch in a person's name and have services give you information about a Canadian because Canada doesn't release all that information in any public way. The United Kingdom is the same thing. France, Italy—these countries have really strict privacy laws that you just can't search up on Google.
Breach data often allows you to find that personal information because people enter it into various services that get breached and then all of a sudden there's a home address, phone number, all that. So this is oftentimes used to get around some of those more privacy controlled countries to find personal information. But again, it's a use with caution situation.
Management Advice
For anyone who's a little more professional and starting to build out maybe a team of people beneath them, like a team of analysts, if you're in a leadership position, don't give them access to it. You can't always trust they're not going to search up their old ex-girlfriend or ex-boyfriend. Anything related to breach data, try to keep it as tight and contained as possible in your environment, because the last thing you need is any publicity on the front page of a newspaper or before a judge saying that your teammates or you or whatever were doing all this illegal stuff.
Conclusion
At the end of the day, the work in OSINT is about the mindset of investigation and not just the tools all the time. A lot of times it's about mindset and how you use your brain to conduct the work. The tools shown are used day-to-day in the OSINT world, and most of them are free or dirt cheap, so you can do a lot of your investigating just using those.
Always stay on the right side of the law. For anyone starting out in this field, remember that in the OSINT world, most places if you want to get hired to do the work, you usually have to have a clean criminal record check. The work you end up doing often has to go to court, so if it's obtained illegally or there's even a shred that this was obtained illegally, your client is going to be upset because all that evidence you collected is now officially garbage. You just have to be really careful.
