ChatGPT for OSINT? Integrating AI Chatbots in the Open Source Intelligence Cycle

From the advent of the radio to the first satellite in orbit, intelligence professionals have always been required to adapt quickly to new technologies. Adaptability might be the most important quality an intelligence analyst can possess, especially since Web 2.0, when user-generated content started dominating the internet. One of the most difficult tasks for intelligence analysts in particular is keeping up with all of the technological developments and new OSINT tools they can use to improve capabilities such as intelligence collection or threat analysis. With the pace of technological development appearing to accelerate, it might be easy to become complacent. However, current advancements in AI are so transformative that missing a beat could mean falling behind.  Although AI systems will not make intelligence analysts redundant anytime soon, they will transform many of the tasks analysts currently perform. The time-consuming tasks of an OSINT analyst, such as visualizing data and geolocating incidents, might soon be transformed completely. Analysts in the near future may simply ask their AI assistants to process data into specific diagrams, or to help them find the location where a video was recorded. For example, satellite imagery analysis is already being automated with generative AI.

In recent months, generative AI chatbots such as ChatGPT have been growing in popularity due to rapid developments in their underlying technology. These chatbots are large language models that use deep learning algorithms to generate responses that are unique to user input. This input is called a “prompt”, which can be a question or a request. Prompts allow the user to put AI chatbots “in the right frame of mind” in order to get the most desirable result. Prompt writing is one of the crucial new skills intelligence analysts should learn if they want to deploy these new tools effectively. This report will cover prompt writing and other methods intelligence analysts can use to get the most out of AI chatbots, before finally discussing the pros and cons of using AI chatbots for OSINT purposes. Firstly, though, it will follow the structure of the intelligence cycle to discuss the potential OSINT applications of AI chatbots such as ChatGPT at each stage.

Integrating AI chatbots with the Intelligence Cycle

Direction:

Every intelligence cycle begins with direction. During this first step, intelligence analysts need to identify who their audience/customer is and what their unique requirements are. The end user may have a specific question, or an area or group they are interested in. This step should provide analysts with a focus area and a list of priorities.

Large language models have the capability to transform many aspects of the direction stage. When it comes to OSINT, one example is “social listening”, a process that is used to determine public opinion on social media. Social listening allows analysts to perform sentiment analysis and spot trends towards social unrest, or more specific threats against actors or a brand’s reputation. For example, a spike in negative sentiment on social media in a specific geographic area could automatically alert a human analyst to take a closer look. The problem with current social listening software is that the tweet: “I just had a killer breakfast in New York City”, might contribute to a negative sentiment because it includes the negatively-labelled word “killer”, even though the emotional tone behind the text is positive. In the worst case, the tweet gets flagged and notifies an OSINT analyst at the New York City Police Department, who gets inundated with noise rather than intelligence. Large language models such as ChatGPT could significantly improve social listening software because these models are trained on unlabelled text.

AI chatbots can also be used as an OSINT tool to help analysts when they are not provided with clear direction. This often happens because the audience, whether that is a corporate client or a military general, does not know what their intelligence needs are. Whilst trained analysts might already have a better idea of their client’s intelligence needs and the threats they face, AI chatbots can still help them with formulating the questions they need to answer. An example of a prompt that can help guide direction is:

“What questions would the head of security of a tech company need answered by OSINT analysts to assess the risks of a trade mission to China?”

As discussed previously, AI chatbots can be used more effectively when prompts are specific and detailed about the desired context. A more detailed prompt could be:

“Create a list of questions that the US Secretary of State would need answers to, from the INR (Bureau of Intelligence and Research) and the CIA (Central Intelligence Agency), in order to be prepared for a meeting with the Foreign Minister of Russia to discuss the New START (Strategic Arms Reduction Treaty) agreement.”

Both of these previous prompts will cause a chatbot like ChatGPT to provide a list of relatively obvious questions. However, an additional prompt could make its responses more specific. For example:

“Provide more specific sub-questions under each one of the above questions.”

The results AI chatbots generate after these prompts are still not perfect. However, they do provide ideas that can help guide direction for intelligence gathering. Additionally, they can guide analysts to the appropriate intelligence collection methods. For example:

“Provide sub-sub-questions that are even more specific, and for each sub-sub-question, indicate the methods that can be used to collect the intelligence.”

Collection:

The methods by which information is collected may vary – whether it is automated data feeds, an intelligence analyst trawling through open-sources, or even a form of crowdsourcing. This stage follows from direction because all of these efforts should be guided, to avoid resources being spent on the collection of irrelevant information. 

Future AI chatbots might be able to automate many of the OSINT collection efforts. For example, an analyst that wants to collect information on future protests in France, could write the following prompt:

“Analyse recent Twitter posts related to protests in Paris. Focus on determining the level of organisation and potential for violence.”

Unfortunately, AI chatbots are still limited in terms of their training data and can therefore not analyse current events. They also do not currently have real-time access to Twitter, but this might change in the future. For the time being, AI chatbots can still help you learn the skills to build a scraper yourself: 

“Provide me with a six-week plan to learn Python for OSINT (Open Source Intelligence) automation and other OSINT tasks. Also, suggest free resources that can aid in my learning journey.”

ChatGPT can even do all of the coding for you:

“Can you help me by generating Python code for a tool that analyses recent Twitter posts? The tool should be able to gather tweets based on specified keywords or hashtags, analyse their sentiment, and provide insights such as the most common keywords, user mentions, and hashtags used. Additionally, it should be able to generate visualizations, such as word clouds or sentiment charts, to facilitate data analysis. Please write the Python code for this tool and include comments to explain the different functionalities. Also provide a README with instructions how to install and run the application on Windows 10.”

Processing:

Some intelligence cycles only consist of four stages and leave out the processing stage entirely. That is because processing looks different depending on the collection methods used previously. At this stage, all of the gathered data should be collated and validated to determine its accuracy and credibility. Additionally, the intelligence analyst should identify how much confidence can be placed in each piece of data and filter out misinformation, but also identify significant information that requires more attention.

AI chatbots such as ChatGPT can be used to assist OSINT processing by flagging information that they deem to be most important. This could enhance the capability of scrapers to quickly connect analysts with useful pieces of information. The following is an example of how Intelligence Fusion is experimenting with AI chatbots to help improve its threat intelligence data.

Analysis

After determining what is most important, information can be analysed, integrated into a larger picture and, finally, interpreted by the analyst during the fourth step. This is the analysis stage, where raw data is turned into actionable intelligence, by adding valuable insights, potential future scenarios, and a recommended course of action for the end user. 

In their current stage of development, AI chatbots such as ChatGPT are not equipped to answer the types of nuanced questions an intelligence analyst might need to answer using OSINT. Especially when it comes to the analysis stage of the intelligence cycle, human input and collaboration will remain crucial. That being said, AI chatbots can still be used as excellent writing tools that can aid analysts in terms of summarization and creative writing. The best results can often be achieved by only feeding an AI chatbot with the text it needs to improve. This prevents the message from being altered too much. For example:

“Summarise the following text in approximately 200 words, the tone should be analytical: [insert text].”

“Give me 3 counterarguments that dispute the following claim: [insert text].”

“Create a table using the following text, include the columns: incident type, date, time and impact rating: [insert text].”

“Rewrite the following sentence more plainly and clearly: [insert text].”

Dissemination

The final step is dissemination, which is when the analysis is shared with the appropriate audience. Getting the right information at the wrong time is as potentially damaging as the wrong information at the right time. However, generally speaking intelligence should be shared as soon as possible, so that the end user has enough time to make informed decisions in response to threats and disruptions. If all of the four steps have been carried out correctly, the end user will receive the relevant, accurate and actionable intelligence needed to act on what is critical. The cycle then repeats, when the audience/consumer gives new direction.

When it comes to this final stage of the intelligence cycle, AI chatbots could be able to ingest reports and determine who needs to see them based on the report’s alignment with that person’s position, schedule or current priorities. Future AI systems may even integrate a response capability to automatically react to the most pressing incidents. However, such a process of the future can only work with a human in the loop, who can ensure the consistency and reliability of both the raw data and the final product.

The development of AI is currently generating a high level of excitement in the intelligence sector. This blog post shows there is already great potential for AI chatbots to be integrated into an organisation’s intelligence cycle. However, OSINT analysts should use AI chatbots carefully and critically to maximise their advantages and mitigate their risks. The pitfalls of using AI chatbots such as ChatGPT for OSINT and intelligence analysis includes potential biases, false answers, bad reasoning, and limitations based on outdated or (intentionally) misleading training data. However, AI chatbots can also be beneficial by helping analysts challenge their biases, think outside the box, learn new skills, and most crucially, save time. In order to deploy AI chatbots most effectively, OSINT analysts need to learn how to work alongside them. One essential new skill analysts should learn is prompt writing. Being specific and detailed about the desired context, outcome, length, format and style, can bring AI chatbots in the right frame of mind to produce the results analysts desire. This skill will need to be honed over time, as AI chatbots are continuously developing and changing. In fact, the technology is developing so quickly that this blog post will soon be outdated. It is therefore important for OSINT analysts to keep up with new developments and stay flexible. Because their adaptability towards new technologies will ultimately determine their success, AI chatbots should become a part of their OSINT toolbox.

Since we were first set up in 2015, we’ve been challenging the way the intelligence space operates, embracing new technologies and building an innovative, highly visual and granular threat intelligence platform that clients across multiple different industries now rely on to protect their assets, operations and people.

To learn more about how Intelligence Fusion is changing the future of the threat intelligence and risk industry, book some time with a member of the team now.

Aaron is a Dutch-speaking intelligence professional.

After obtaining his degrees in International Relations (BA) and Political Science (MSc) at Leiden University, Aaron moved to the United Kingdom to pursue a career in the field of security and intelligence. Besides speaking Dutch, English and a fair bit of German, Aaron studied Mandarin and Spanish during extended periods of living and working in Asia and South America.

Aaron joined Intelligence Fusion in 2019 and worked as a Senior Intelligence Analyst for over 3 and a half years.

Aaron now works as a Senior Risk Consultant at Control Risks.