In 2018 we launched our new real-time monitoring platform (figure below) which visualises several types of information and makes our work much more timely and efficient. The platform displays information about reported incidents of conflict and the occurrence of natural hazards, and it displays the results of IDETECT, our natural language processing and supervised machine learning tool.
The Internal Displacement Event Tagging Extraction and Clustering Tool (IDETECT)
IDETECT works by automating many of the steps our monitoring team takes from identifying information on internal displacement and entering it in our database. First, IDETECT reads the world’s news, as well as UN and NGO reports, and filters it keeping only articles that are likely to relate to internal displacement. Next, it begins analysing the text and extracts key bits of information including:
-
type of displacement
-
cause (conflict, disasters or other)
-
location
-
number of people and
-
reporting units and terms (figure below)
Our team of monitoring experts reviews IDETECT’s outputs and validates the suggestions or corrects them. Whatever information is good, we store it in our database; what isn’t is flagged as erroneous or irrelevant. In both cases, IDETECT ‘learns’ from this process to improve its accuracy in the future.
Other sources of displacement data
At present, much of our data comes from pre-vetted and familiar partners, such as national and local governments, IOM, OCHA, UNHCR or civil society organisations (figure below).
In some cases, we have to infer displacement indirectly from proxy indicators and analysing nontraditional forms of data. In the past year, for example, we have begun analysing satellite imagery and anonymized, aggregated social media data to infer different patterns of displacement, particularly where no other data was available. We’re currently transforming our disaster displacement risk model into a tool to support real-time monitoring and potentially early warnings.