Open Source Intelligence (OSINT) is about using openly available information to extract insights.
In this post, I’m going to give you a better look at how AI is enabling automated OSINT analysis of the Dark Web.
The Dark Web is an interesting beast. At one time, researchers have crawled and managed to find ~30,000 websites; however, in the next few months, over half of these websites have disappeared.
You might be curious as to what kind of crazy things can be found in the Dark Web. The majority is actually focused on a couple main things: illegal pornography, drugs and weapons. However, you can also find a lot of relevant cyber security information, such as sensitive information about zero-day exploits, stolen datasets with login information, and botnets for hire. Naturally, this knowledge may be used to prevent the same attacks. But what amounts of manpower could possibly be enough to scour the whole Dark Web on a regular basis keeping up with its changing nature, understand the different languages that are being used, and be on top of new developments? That’s where AI comes to the rescue.
There exist AI systems, right now, such as BlackWidow, that can crawl the Deep and Dark Web across different languages. Within a couple days of monitoring, such a system can collect years of relevant information in the areas of cyber security and fraud monitoring. It can infer relationships between authors and forums and detect trends for cybersecurity-related topics. The image below shows clearly how these trends can indicate what kind of attacks are incoming
