RELIGIOUS AND ETHNIC MINORITY (REM) HUMAN RIGHTS INFORMATION SYSTEM (HRIS) Considerations for the Design of a REM-Focused Monitoring and Early Warning System (EWS)

FEBRUARY 1, 2021 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International (MSI), A Tetra Tech Company for the MENA MELS Activity.

RELIGIOUS AND ETHNIC MINORITY (REM) HUMAN RIGHTS INFORMATION SYSTEM (HRIS) Considerations for the Design of a REM-Focused Monitoring and Early Warning System (EWS)

Contracted under GS00F280GA, Task Order 7200AA19M00017

Middle East & North Africa Monitoring, Evaluation, & Learning Services Activity

DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

i | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

CONTENTS ACRONYMS II

BACKGROUND AND PURPOSE 1 METHODOLOGY 1

FINDINGS 1 OVERVIEW OF EFFECTIVE EWS 1 TYPES OF EARLY WARNING SYSTEMS 3 METHODS AND PURPOSE 3 GEOGRAPHICAL LEVEL OF FOCUS 3 DECISION POINTS FOR CRAFTING AN EFFECTIVE EWS 4 DATA CONSIDERATIONS 4 ADAPTABILITY 7 SUSTAINABILITY 8 ADDITIONAL CONSIDERATIONS 9 WARNING AND RESPONSE 11

EWS GAPS 13

CONCLUSIONS AND RECOMMENDATIONS 15

ANNEX A: STATEMENT OF WORK 19

ANNEX B: LIST OF EARLY WARNING SYSTEMS REVIEWED 23

ANNEX C: LIST OF SELECTED INTERVIEWEES 24

ANNEX D: BIBLIOGRAPHY 25

ACRONYMS ACLED Armed Conflict Location and Event Data Project

CEWARN Conflict Early Warning and Response Mechanism

EWP Early Warning Project

EWS Early Warning System

FEWS NET Famine Early Warning Systems Network

GDP Gross Domestic Product

HRIS Human Rights Information System

HRW Human Rights Watch

MENA Middle East and North Africa

MSI Management Systems International

REM Religious and Ethnic Minority

USAID United States Agency for International Development

V-DEM Varieties of Democracy

1 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

BACKGROUND AND PURPOSE Management Systems International (MSI) is developing a human rights information system (HRIS) to enable the United States Agency for International Development’s (USAID) Religious and Ethnic Minority (REM) unit to access a wide range of relevant external and USAID data and information to support REM programming. REM unit priorities related to this work include monitoring the human rights of religious and ethnic minorities in the Middle East and North Africa (MENA) region. The unit is also considering developing an early warning system (EWS) that would track signs of rising tensions or risk of mass atrocity events across the region. This report provides a preliminary analysis of key considerations for the design of a REM-focused monitoring system and/or an EWS, specifically regarding data collection, validation, aggregation, and analysis.

METHODOLOGY

The EWS assessment team conducted a desk review of 19 EWSs and related systems. The team also conducted six informal qualitative interviews with subject matter experts and MSI’s REM HRIS dashboard development team to assess the attributes and effectiveness of existing EWSs.

Systems assessed include the following: the Famine Early Warning Systems Network (FEWS NET); the U.S. Commission on International Religious Freedom; the United States Holocaust Memorial Museum’s Early Warning Project (EWP); Humanitarian Data Exchange indexes of the United Nations Office for the Coordination of Humanitarian Affairs’ Centre for Humanitarian Data; the Armed Conflict Location and Event Data Project (ACLED); Arab Barometer; Varieties of Democracy (V-DEM); Intergovernmental Authority on Development Conflict Early Warning and Response Mechanism (CEWARN); and Fraym, among others.1

FINDINGS

OVERVIEW OF EFFECTIVE EWS

EWSs may vary in types and approach. This research is based on EWSs that are based on the idea that major events—including disasters, human rights violations, and violence—can be prevented, or their impacts mitigated, through warnings and information linked to development, security, political, and diplomatic responses. EWSs use a range of quantitative and qualitative analytical approaches to formulate scenarios, predictions, and response options for decision makers.2 Early responses to violence or human rights abuses are actions that occur, once a threat of potential violence is identified, to manage, mitigate, or prevent violent conflict. These actions can include investigations, accountability and application of the rule of law and/or local justice, monitoring, efforts to enhance/build social cohesion, civic education and

1 The full list includes: FEWS NET; U.S. Commission on International Religious Freedom (USCIRF); EWP; Amnesty International; Human Rights Watch (HRW); Uppsala University Violence Early Warning Systems; International Crisis Group Crisis Watch; Economic Community of West African States ECOWARN; Intergovernmental Authority on Development, CEWARN, United Nations Framework of Analysis for Atrocity Crimes; ACLED; Humanitarian Data Exchange (INFORM Global Crisis Severity Index, Risk Index, and Warning System); Arab Barometer; V-DEM; Fragile States Index; Global Peace Index; Horizon Intelligence; and Fraym. 2 Catherine Defontaine, “Setting up Early Warning and Response Systems to Prevent Violent Conflicts and Save Lives,” Development for Peace (blog), World Bank, February 15, 2019, https://bit.ly/3prBQ1M.

public engagement, mediation, dialogues, negotiations, and policies to address structural stressors for violence, preventive diplomacy, negotiations, or security responses.3

There are many debates on the purposes and structures of effective EWSs and the capability of these systems to prevent violence.4 At a high level, effective EWSs accomplish two critical goals: They enable the focus and prioritization of stretched resources on the areas most likely to experience violence, and they can effectively prevent or mitigate the risk of widespread human rights violations, including violence, if linked to response or program options.

Effective violence-tracking EWSs prioritize the risks most relevant to a specific type of conflict. These systems are designed around trade-offs to deliver specific and pertinent information in a time frame that enables action. To address these trade-offs, effective models cannot capture all aspects of conflict, but rather they seek to fill a specific need to provide decision makers with the data needed to act in ways that prevent or mitigate the impact of violence.5 For example, in East Africa, CEWARN uses multiple data sources and indicators to capture short- and longer-term risk, from immediate to quarterly and annual intervals.6 To be accurate and effective, early warning methods must be precise about what they seek to measure. For example, risk indicators that might be effective predictors of rising discontent with state services may not be the same as indicators that can effectively anticipate reemergence of intercommunal tensions.7

Effective systems rely on a mix of indicators that can be inputs to further expert analysis to produce findings that are more granular, nuanced, and adapted to specific needs. For example, FEWS NET (a well-known, USAID-funded food security warning service in use since the 1980s) uses numerous reliable data sets, including from the National Aeronautics and Space Administration and market price models, but it ultimately issues warnings after country experts review the data and agree on a range of risk for food insecurity in a country. In this way, an EWS can have sustainable and reliable data and be adaptable enough to produce actionable information for decision makers at USAID and in other governments.8

Building an EWS entails many design considerations, including (but not limited to): purpose, objectives, users, methodology (data collection and analysis), structures (owners of the data, partners, decision makers), geographic area and level, presentation (how information is communicated and how often), conflict sensitivity, accuracy, and timeliness.9 These considerations are interlinked, often as trade-offs. For example, a system can have an exhaustive number of indicators that give a more complete, contextual

3 Defontaine, “Setting up Early Warning and Response Systems.” 4 Organisation for Economic Co-operation and Development (OECD), Preventing Violence, War, and State Collapse: The Future of Conflict Early Warning and Response (Paris: OECD, 2009), https://www.oecd.org/dac/conflict-fragility-resilience/docs/preventing%20violence%20war%20and%20state%20collapse.pdf. 5 Multiple key informant interviews, including with FEWS NET, the United States Holocaust Memorial Museum, ACLED, and Fragile States Index. 6https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_prevention_tool.pdf. 7https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_prevention_tool.pdf. 8 Interview with FEWS NET. 9 United Nations Development Programme and Organization of American States, Practical Guide: Early Warning and Response Systems Design for Social Conflicts (Washington, DC: 2015), https://reliefweb.int/sites/reliefweb.int/files/resources/UNDP_RBLAC_Guia%20Alerta%20EN.pdf.

3 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

overview but will then have trade-offs related to cost and the resources needed to update, clean, and present data in a timely manner. This report discusses some of these trade-offs.

TYPES OF EARLY WARNING SYSTEMS

METHODS AND PURPOSE

EWS methods for data collection and aggregation related to catastrophic events vary among existing systems. Most systems combine information gained from detailed local or expert knowledge, broadly termed human intelligence, and multiple quantitative data sets that enable machine learning. EWS data collection and aggregation approaches differ, depending on their purpose:

• Monitoring and reporting: Data sets that report incidents of violence or human rights violations (ACLED, Human Rights Watch [HRW]) or conditions that may put populations at risk (United States Commission on International Religious Freedom’s Religious Freedom Report)

• Risk assessment: Data sets that highlight structural elements of a society that affect its likelihood of experiencing negative events or developments (Fragile States Index). These systems review various factors, such as political regime type, history of instability, connection to the global economy, and state-led discrimination, to create a range or score (usually high, medium, or low) of likelihood for future violence.10 Risk assessments provide an overall understanding of events but do not seek to predict when a large-scale violent event may take place.11

• Early warning: Systems such as FEWS NET and EWP that focus on the mid- and short-term factors that make violence likely. This approach includes attempts to identify the escalation of instability and whether it may tip into large-scale, sustained violence. It requires greater attention to dynamic elements such as increasing friction among political elites, changes in the military capacity of oppositional forces, highly contested upcoming elections, and the expansion of security forces.12

Predictive models require the most data points and the curation of predictive indicators. The EWP from the United States Holocaust Memorial Museum, for example, uses approximately 30 indicators (weighted based on past genocides) to predict future genocide events.

GEOGRAPHICAL LEVEL OF FOCUS

These systems (monitoring, risk assessment, and early warning) can be distinguished further by the geographical level at which they obtain and use data. Monitoring and reporting can use granular data to alert local leaders. Models that predict risk tend to require more data and include structural factors at the societal level. This modeling can work down to the substate level, where data coded by administrative units exist, but extrapolation to the hyper–local level is problematic. Even with sophisticated models and large amounts of data, data sets may not have high predictive value at the local level. For example, FEWS

10 Ernesto Verdeja, “Predicting Genocide and Mass Atrocities,” Genocide Studies and Prevention: An International Journal 9, no. 3 (2016): 13–32. 11 Verdeja, “Predicting Genocide and Mass Atrocities,” 13–32. 12 Verdeja, “Predicting Genocide and Mass Atrocities,” 13–32.

NET’s model can predict food insecurity with a high degree of certainty at the state level, but FEWS NET does not use it for the substate level. Figure 1 provides an overview of systems.

FIGURE 1: SYSTEMS OVERVIEW

DECISION POINTS FOR CRAFTING AN EFFECTIVE EWS

DATA CONSIDERATIONS

DATA COLLECTION The EWS models reviewed use a wide array of data collection methods that reflect the system’s intended goal. Designers of an EWS must understand the trade-offs of rapid versus granular, and accurate information and design data collection considerations to address the constraints.13 Highly manual data collection processes can yield more granular data and contextual understanding. In addition, incorporating inputs from civil society can be an important way to build trust and legitimacy, which can be useful to ease tensions or prevent escalation into violence.14 However, such processes present challenges to rapid data uploads, systematized collection, predictive modeling, and cost efficiency. Alternatively, more automated

13 Interview with ACLED pilot project for religious violence and interview with FEWS NET. 14 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_ prevention_ tool.pdf

Amnesty International

5 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

processes that pull from multiple, highly curated data sets may lose granularity and accuracy but can increase predictive machine learning capabilities with less effort and at lower cost. Even in highly automated, machine-learning-dependent models, however, human coders to aggregate, clean, and adapt data are critical to the EWS process.

Data collection methods also depend on data accuracy and availability, a critical consideration for REM populations in the MENA region. Broadly, researchers define data environments as either “rich” or “scarce.”15 Data-rich environments are those with multiple data points that are updated reliably and accurately on a near-real-time basis. Data-scarce environments involve data that are complex and difficult to attain by automated means. Areas that are data poor are open to bias. For example, more data are usually available from governments than from opposition; demographic data can be difficult to obtain from displaced populations, and there is a bias toward urban over rural data.16 Data-poor environments are often researched under large, expensive, academic or government-driven one-off research projects. Data collection in these environments and on these subject matters is hard to automate. As such, data may not be collected regularly and can become outdated.

The data collection approach ultimately depends on the end consumers of the EWS’ products. Local leaders and researchers may require highly granular data while state-level decision makers may require higher degrees of certainty and easily digestible overviews.

DATA ENTRY AND CLEANING PROCESSES Data entry and cleaning processes vary among EWSs. Models that focus on large data sets and/or use machine learning seek to automate data entry processes through coding and entry systems that pull various data sets into the machine learning model. Models that focus on data are difficult to categorize and require human researchers to manually code data into systems. For example, ACLED uses multiple news sources to verify the time, location, type, and perpetrator of a violent event. However, no automated approach exists to create a type of violent event because various news sources will present the violent event with varied language. News sources may variously describe an explosion as a detonation, bomb, explosion, mass catastrophe, and so on, so a researcher must categorize the event as an explosion and ensure that no duplicate reports of the same event are uploaded into the database.17 Trade-off considerations for data entry and cleaning processes are determined by the types of data used. For example, geographic information system data on the movement of overall populations may be easily entered and cleaned within a system. However, why that population is moving and the threats they face require manual data collection and cleaning.

DATA VALIDATION Data validation is the ability to ensure data are accurate and reliable to a degree of certainty that EWS users can rely on to make decisions. In each system reviewed, data validation occurs in two stages. The first is a careful curation of data sources, and the second is human review of the data. This human review process is necessary even in models based heavily on machine learning.18

15 FEWS NET researchers use these terms, confirmed in additional interviews. 16 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_ prevention_ tool.pdf. 17 Interview with ACLED. 18 Interview with researchers on the United States Holocaust Memorial Museum’s EWP, a predictive model based on machine learning.

Timeliness to accuracy and granularity is a notable trade-off in data validation. ACLED updates its event reporting weekly, including type of violent incident, location, time, and conflict actor/perpetrator. To be able to do so, ACLED accepts a larger possible degree of error than a less frequently updated model might accept. ACLED researchers accept that not all violent events are reported, and that each event cannot be reinvestigated after the fact. However, it is assumed that the curated data sources, primarily traditional media, make a best-effort attempt at accurate reporting. Researchers and managers review each piece of data manually to ensure the events are coded correctly before populating the system. HRW reporting represents an alternative model for context monitoring. Before issuing reports on events, HRW researchers carefully validate and triangulate multiple news sources, secondary sources, and eyewitness accounts. This data validation process results in reports that are more contextual and accurate but often months after the violent event has occurred.19

DATA AGGREGATION Data aggregation is the process of incorporating multiple data sets to build greater overall understanding of a specific context or build stronger predictive models. Data aggregation is particularly important for predictive models that require large amounts of data from indicators known to have high degrees of correlation with violent events in the past. Large data sets are required to isolate distinct variables to create highly predictive models. These data sets often reduce variables with multiple response options to binary outcomes—for example, the presence of violence versus the absence of violence, or the increase of child mortality versus the decrease.

For example, V-Dem measures democracy with 81 indicators under headings describing democracy as electoral, liberal, participatory, deliberative, and egalitarian. The V-Dem data set produces a range of democracies, from highly democratic to highly authoritarian, based on a highly structured guide for data aggregation.20 The lack of democracy is a strong predictor of genocide in the predictive model that the United States Holocaust Memorial Museum’s EWP uses.21 To facilitate its integration into the model, EWP data coders reduce those ranges into a binary variable indicating that a government is or is not democratic. The benefit of this approach is an increased degree of certainty about predictions, but large amounts of contextual data are lost.

The level at which a system seeks to inform a population (local, substate, or state) is an important consideration in data aggregation. A system that seeks to reach a local population needs specific and recent data that, while relevant, may not have sufficient amounts or types of data to be predictive. However, systems that seek to influence decision makers’ priorities at a state level may reduce the specificity of data to increase the degrees of certainty or predictability of an EWS. For example, FEWS NET provides detailed and highly relied-upon information at the national decision-maker level. FEWS NET raw data sets can be broken down to lower levels, for example, market prices in a particular village. However, when the data sets are combined to create an estimate of likelihood for food insecurity, researchers have a high degree of confidence for predictions at a state level but not at a substate level.22

19 Interview with HRW. 20 University of Gothenburg, V-Dem Institute, Quickstart Guide V9 (Gothenburg, Sweden: 2019), https://www.v-dem.net/files/59/V-Dem%20Startquide.pdf. 21 Interview with United States Holocaust Memorial Museum’s EWP. 22 Interview with FEWS NET.

7 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

DATA CATEGORIZATION AND SEARCHABILITY Data categorization can be thought of as consistent data coding. The ability to consistently code data enables data to be searched easily. For example, ACLED codes violent events by type (battles, explosions, violence against civilians, riots, protests, or strategic developments).23 Researchers carefully review news reports of violence and code within these categories. Events that fall outside these categories are not recorded. Data managers understand that more violence occurs within a country than is recorded in the ACLED database, but this data categorization increases the searchability.24 By contrast, qualitative sources that are text-heavy, such as Amnesty International reports on human rights issues, have less searchability because they are difficult to categorize.

DATA VISUALIZATION AND EASE OF USE Data visualization and ease of use supports end users’ ability to use the products of an EWS to design improved outcomes from avoiding or mitigating the impacts of violence. Effective models are designed, piloted, and tested with specific end users.25 For example, academics and researchers who build large modeling databases prefer data that is open source and easily aggregated through .csv files. However, this type of data may not be understood at the community level, where people are affected by violence. In addition, although highly quantified data enable sophisticated statistical modeling with high degrees of certainty, decision makers and policymakers may not easily understand such models. In that case, presenting data in ranges, percentages, or visualizations may make products more user-friendly. If local partners in the field are to use the information, statistical modeling and visualizations will not be as powerful as networks of trusted actors delivering information.26

Another example concerns heat maps. These present understandable visual location and time dynamics of multi-actor conflicts. Heat maps from organizations such as ACLED are useful to understand rapidly evolving dynamics and the presence of violence, a critical predictor of future violence. However, this visualization poses a trade-off: If the map tracks a number of factors rapidly, it is difficult to quickly research numerous details of the aftermath of a violent event or factors that may have led up to that event. For example, an individual reporting a bomb detonation in a market on the outskirts of Mogadishu on December 12, 2020, with Al Shabaab claiming responsibility, is easy to track and visualize along with other incidents for a heat map. However, the numerous enabling factors related to the recruitment of the individual detonating the bomb are harder to visualize. The more nuanced the data, the more difficult it is to code consistently and rapidly to build a heat map. As such, making data findings rapidly available, visual, and easy to digest entails losing some granularity.27

ADAPTABILITY

Each organization interviewed noted that (1) models must be adapted across countries to consider variations in conflict context, and (2) data points and sources must be adapted over time to include new contextual information. Adaptability is critical for local context, but it presents a trade-off because it affects

23 Armed Conflict Location and Event Data Project, “ACLED Introduces New Event Types and Subevent Types” ACLED (2019), https://acleddata.com/2019/03/14/acled-introduces-new-event-types-and-sub-event-types/. 24 Interview with ACLED. 25 Interview with FEWS NET. 26 Interview with FEWS NET. 27 J. Osorio and others, “Mapping Violent Presence of Armed Actors in Colombia,” Advances in Cartography and GIScience of the ICA, 1, no. 16 (2019).

longitudinal data sets and users’ ability to compare data over time.28 If current events cannot be compared with past events, machine models are not able to build models to prioritize events and compute a degree of threat.29 Systems that rely on expert judgments to issue a warning, such as HRW’s system, are more flexible than machine learning models such as EWP’s in this regard. Even with expert assessments, however, a design consideration is the ability to build in points of consistency throughout reports. Without consistency, such assessments are highly context-specific and difficult to compare over space or time.30

In addition, the ability to rapidly adapt relies on the availability of quality data. If there are years in which new data are not produced (for example, no new numbers for infant mortality in Yemen), then data must be moved forward from the previous year.31

Dealing with the occurrence of a surprising or rare event is an additional adaptability requirement for machine learning models. Models are designed on the assumption that past events can predict future events. Surprising events can impact machine learning models and need to be accounted for in data management processes. For example, changes in gross domestic product (GDP) year to year is a weighted indicator shown to increase the likelihood of violence. The spread of the coronavirus disease 2019 in 2020 resulted in global changes in GDP. Statistical models do not account for this type of rare event. The ability to adapt models, with consensus among researchers and coders, is necessary to support the model’s accuracy.

SUSTAINABILITY

Numerous one-off or short-lived monitoring systems, risk assessments, and early warning reports exist on the internet.32 The sustainability of an EWS is an often-undervalued design consideration.33 Sustainability is tied closely to the cost of building and maintaining the EWS, consistent funding, and the system’s ability to serve end users’ ongoing needs.34 The first step toward sustainability is to consider the needed institutional time and budgeting commitment. The second step is designing an EWS around the needs of users and clearly communicating the advantages and disadvantages of the approach.

ACLED’s EWS is a case study for understanding sustainability. The data sets, reports, and snapshot visualizations from ACLED are used widely by multiple governments, researchers, and civil society actors. ACLED made a funding transition recently. In the past, ACLED was funded primarily by the U.S. Department of State, which supported long-term and ongoing data collection. Because of growing concern about dependence on a single agency for data that multiple organizations use, the team at ACLED is currently working to implement a quasi–pay wall or asking for-profit or government entities to pay to access data. The shift is challenging because it is difficult to convince existing customers to pay for services

28 Interview with the United States Holocaust Memorial Museum’s EWP. 29 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_ prevention_ tool.pdf. 30 Interview with FEWS NET. 31 Interview with the United States Holocaust Memorial Museum’s EWP. 32 Interview with FEWS NET. 33 Interview with FEWS NET. 34 Interview with FEWS NET.

9 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

they are used to receiving at no cost.35 However, if the transition is successful, ACLED’s sustainability will not depend on USAID’s continued support.

ADDITIONAL CONSIDERATIONS

CULTURAL CONTEXT Both machine learning and human intelligence models require attention to cultural considerations to be effective, including prior (and ongoing) analysis of cultural context for making sense of patterns or warnings produced by the data and for responding to warnings appropriately. Models reviewed worked to identify and incorporate indicators on gender and ethnic identities in the data collection stage to support context analysis. Cultural context is important not only for situational awareness but also for the predictive power of a model. For example, in some countries, women’s freedom of movement is not a strong predictive indicator of escalating violence because it is restricted by cultural norms. In these contexts, males’ freedom of movement is a stronger predictor of violence.36 Analysis of cultural contexts is also important to understand options for action. For example, an understanding of social customs and local influencers is important to the design of effective and conflict-sensitive responses. Often, EWSs work with regional experts before issuing warnings or taking action to ensure contextual information is incorporated fully.37

Granular, culturally sensitive knowledge related to gender and age norms is lacking in most data sets because these nuances are difficult to capture fully and accurately in a timely and cost-efficient manner. These data are still highly relevant for programming and mitigation purposes. For example, incorporation of data on the so-called Islamic State of Iraq and Syria’s attitudes toward Yezidi women versus Shia or other minority women might have allowed earlier warning about the mass gender-based violence targeting Yezidi women.38

The capacity to read and scan traditional and social media in local languages is important to obtaining local contextual understanding. Data gathering methods that rely on English or nonlocal dialects risk creating a bias in databases toward incidents in large urban centers that house news outlets. Because many REM populations are rural and isolated, a lack of use of local language can misrepresent the context and miss early warning signs of violence or human rights abuses.

CONFLICT SENSITIVITY Systems that report on violence require conflict sensitivity and particularly an understanding of potential negative outcomes. Conflict drivers can come from both the state and local actors. For example, in recent conflicts in the Sahel, various actors incite violence between Dogon and Fulani leaders for various goals. Understanding local and state leaders’ motivations is important for understanding conflict. Monitoring systems such as HRW’s that rely on firsthand accounts of violence and have on-the-ground networks have established conflict sensitivity protocols. Generally, organizations have discussions about “do no harm” principles if it is shown their data can be used in a manner that can adversely affect populations on the ground because malign actors can target local networks. Organizations using sensitive firsthand accounts tend to follow the standards of the International Committee of the Red Cross’ protection work. These standards include anonymizing data, collecting data in ways that would not put anyone at risk (including

35 Interview with ACLED. 36 Interview with United States Holocaust Memorial Museum’s EWP. 37 Interviews with United States Holocaust Memorial Museum’s EWP, Fragile States Index, and FEWS NET. 38 United Nations Development Programme and Organization of American States, Practical Guide.

being sensitive to time, location, and language) and, in extreme cases, supporting witnesses to leave an area where they are likely to remain in danger or secure emergency legal and security assistance.39

National-level predictive systems usually aggregate multiple secondary sources of information and are designed for capital-based decision makers. The data used in these systems are often reported widely and not directly linked to individuals reporting violent incidents. As such, the predictions of violence are removed somewhat from the local context. For this reason, researchers from institutions with predictive risk assessment and early warning models, such as the Fragile States Index, report not having robust conflict sensitivity procedures in place.40

THE CHALLENGE OF RARE EVENTS Large-scale violence and genocide are rare events. Events that do not occur with frequency are difficult to model with high degrees of certainty, meaning it is difficult to predict when, where, or among whom a large-scale violent event will occur. The challenge of predicting rare events highlights a tension between data being highly accurate and the ability to provide programming in a timely manner to prevent the event from occurring. There is a trade-off of time and certainty with the occurrence of an event. For instance, a mass killing at a certain location and date is easier to predict a week before it occurs than a year before, but it is easier to prevent that event through programming options a year before it occurs than a week before.

Effective EWSs have clear and consistent communications with stakeholders on the degree of certainty with which they predict events. This communication is designed to address end users’ appetite for risk. At the local level, a low degree of certainty may be acceptable in return for the ability to act rapidly and save lives before a catastrophic event. There are downsides, however. For example, hurricane landings are difficult to predict. After numerous experts predict the landfall of a hurricane that does not arrive, homeowners may not listen to future warnings to evacuate an area. At the level of senior national decision makers, there is risk aversion along with fierce competition for resources and priorities. For this kind of audience, a range or ranking of risk may be more effective. For example, the Fragile States Index issues a ranking of countries from most to least likely to experience mass instability. In this way, governments can increase monitoring and support to countries most at risk.

COST CONSIDERATIONS Cost considerations are critical to the overall sustainability of an EWS. These are often rooted in trade-offs between the levels of granularity, automation, and staffing needed to fulfill the system’s purpose.

Set-up cost considerations include:

• The cost of assessing existing data and supplementing or augmenting existing data to build the desired model

• Curating data sources and aligning their models with the new EWS

• Initial coding and triangulation of data

39 International Committee of the Red Cross, Professional Standards for Protection Work (Geneva: 2020). 40 Interviews with researchers at ACLED, United States Holocaust Memorial Museum’s EWP, and Fragile States Index.

11 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

• Deciding on aggregation practices, including what data can be updated automatically, what data must be manually automated, and data cleaning processes

• Adapting data to contextual realities

• Expert validation

• Local stakeholder engagement/validation

• Setting warning thresholds and linking warnings to programming options

• Establishing dashboards and visualizations

Maintenance cost considerations include:

• Ongoing collection, uploading, cleaning, coding, and analysis of data

• Reviewing the data aggregation process

• Model adaptation with context changes

• Expertise for final validation of review

• Updating the model as curated sources of data change year to year

• Communication and coordination costs among stakeholders

WARNING AND RESPONSE

The goal of systems that monitor or predict violence is to reduce the occurrence or impact of future violence. If a system only reports and explains the context of violent events but does not issue warnings that enable and/or encourage action around those warnings, most experts and practitioners would not classify it as an EWS but rather as a monitoring system.41 The literature is clear that EWSs are important to preventing violence, but it does not specify when to issue a warning. Issuing an early warning is dependent on (1) to whom the system is designed to communicate a warning, (2) the risk tolerance of the entity receiving the warning, (3) the context of the country on which the warning reports, and (4) the cost of response to that warning. As such, the issuance of warnings is highly variable among EWSs and between countries monitored by EWSs. Ultimately, warnings are issued only after in-house oversight of data produced by researchers and EWS computations and a review by experts.42

Early warnings should be “timely, accurate, valid, reliable and verifiable,” but it is too high a bar to assume predictions can tell audiences exactly when and where violence will occur.43 It is a more reasonable

41 International Federation of Red Cross and Red Crescent Societies, Community Early Warning Systems: Guiding Principles (Geneva: 2012), https://www.ifrc.org/PageFiles/103323/1227800-IFRC-CEWS-Guiding-Principles-EN.pdf; key informant interviews with United States Holocaust Memorial Museum and FEWS NET. Note that with local EWSs, the principal of linking warnings to action is important for conflict sensitivity principals. 42 Interviews with FEWS NET, United States Holocaust Memorial Museum’s EWP, and HRW. 43 Verdeja, “Predicting Genocide and Mass Atrocities,” 13–32.

standard to say that EWSs monitor factors that make a violent event more likely to occur. Effective EWSs align monitoring capabilities that are timely and sufficiently accurate with organizations’ tolerance for the risk of being wrong to effectively inform policy and action.44 Organizations with a higher risk tolerance are more willing to act early based on incomplete information from warning systems, and organizations with lower risk tolerance will wait longer to have more complete information before acting.

To issue a warning, a system requires clear threshold definitions for various indicators. These are based on a level of acceptability to decision makers. These thresholds are based on detailed analysis of historical events to understand the level at which various indicators increase the probability of a violent event occurring. Then, various approaches to modeling, including scenario modeling and machine learning, produce a threshold (typically a color coding, compared ranking, percentage or likelihood) that is communicated to relevant stakeholders to inform actions. Scenario modeling can be used to understand the probability of an undesirable outcome and identify important windows of opportunity for action.

Trade-offs involved in being early depend on the model type. Machine learning models are not affected by risk aversion because they only weigh variables as they may result in violence. Human or expertise-based models are highly susceptible to being perceived as inaccurate or wrong about a prediction. Existing models tend to balance these trade-offs through blended approaches. Even models based on machine learning, such as the United States Holocaust Memorial Museum’s EWP, note that the data from prediction models should be a trigger to conduct further research, shift priorities, and mobilize resources, but they should not be the end determinant of action.45

Once a threshold is exceeded, warnings are issued and actions are adapted. In effective systems, these thresholds take cultural sensitivities into account and work in close coordination with local stakeholders, organizations that implement interventions, and decision makers. This process tends to require long negotiation times and iterative discussions because each actor’s level of tolerance for various events will vary. Ideally, these negotiations take place long before warnings are issued.

EWS data analysis is the beginning, not the end, of a system designed to warn populations and prevent violence. Once a warning is issued, teams need to be prepared to engage in additional rapid context analysis. Questions may include:

• How much attention has the country received? Have the resources affected the conflict context? Why or why not?

• What additional analysis is needed to understand the risk?

• What are other types of events that may serve as tipping points (coups, elections, protests, and so on)?

• Are there new events occurring that change original risk estimates (for example, coronavirus disease 2019, violence in Tigray, passage of a new, unpopular law)?

• Where are there unexpected results from data and reality?

44 Verdeja, “Predicting Genocide and Mass Atrocities,” 13–32. 45 Interview with FEWS NET.

13 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

• What partners may add to the understanding of or ability to act in each conflict context?

If an EWS is designed to support early action to mitigate or avoid violence, then the consensus of key stakeholders around programming and the ability to hold these actors accountable for acting once warned is critical. Alignment of actors who can rapidly implement impactful interventions will reinforce utility of an EWS.

Logistics and coordination can make or break an EWS. The ability to plan actions aligning with well-defined, user-friendly communication tools between actors is another critical EWS component. The importance of managing communications among stakeholders within an EWS is difficult to overstate. Clear communications between stakeholders with defined roles and responsibilities is essential for the ultimate efficacy of an EWS. The challenge of communication between different levels and types of stakeholders in an EWS is often overlooked, yet without effective lines of communication for local-to-central-to-local analysis or notification of decisions or outcomes, interest and motivation to participate in an EWS can wane and thus discredit or weaken the system.46

The need for an early-action trigger mechanism to facilitate decision-making and rapid humanitarian response was made clear by the 2010–2011 famine in Somalia. From August 2010, FEWS NET and the Food Security and Nutrition Analysis Unit at the Food and Agriculture Organization of the United Nations issued 16 warnings about the drought crisis in East Africa. However, a humanitarian response did not start in Somalia until after a famine was declared in 2011. Approximately 260,000 people died, half of them children under five. Subsequent investigations noted that warnings were timely and sufficient but not linked to partner strategies for early response.47 Since then, FEWS NET has invested in organizational structure, partnership models, and decision-making processes to align action with warnings.48

EWS GAPS There are numerous gaps in existing EWSs related to REM populations. These gaps reflect tensions between the needs for timeliness, deep contextual understanding, and supporting communities that may be affected by violence.

1. The lack of feedback loops with local communities is a gap reported in the literature and interviews. Local stakeholders, including subnational administrative bodies, need to at least be informed, even if for some reason they do not or cannot play a role in responses because of the localized nature of many rapidly emerging risks for violence and because these risks, to some extent, can be addressed directly at the local level. Engaging local-level stakeholders in regular occasions for feedback is also important to build trust (between communities but also toward authorities). However, nongovernmental organizations and local civil society organizations may be wary of sharing information without a clear understanding of how that information will be used

46 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_ prevention_tool.pdf. 47 Oxfam, From Early Warning to Early Action: What Can We Learn to Support Early Action to Mitigate Humanitarian Crises? (Nairobi, Kenya: 2017), https://oxfamilibrary.openrepository.com/bitstream/handle/10546/620345/dp-early-warning-early-action-somalia-021017-en.pdf;jsessionid=B5F730B34270A024B5F1E5520C32F16B?sequence=1. 48 Interview with FEWS NET.

and further opportunities for feedback.49 In addition, several programming options tied to existing databases are not community-based interventions. The further removed data and programming options are from a community, the more difficult it is for these options to be rapid in nature. Emerging systems are working to address this gap—see, for example, a pilot project in Guinea rooted in community-based responses.50

2. There is currently a gap in systems that rapidly monitor trends in violence specific to REM communities.51 One reason is that data on religion and ethnicity are often outdated, particularly when censuses are not conducted or they omit REM questions for political reasons. This gap makes analysis linking violent events to REM attributes of populations unreliable. More up-to-date, detailed, and quality data may be limited to one-off, large-scale research projects that are not easily replicated and updated. Within the next year, ACLED will produce data sets related to REM in a pilot phase. This new website will monitor different types of violence based on religion, but note that ACLED has concluded that there is not yet enough data to build predictive models.

3. Existing databases lack gender sensitivity and the ability to report the likelihood of the impacts of violence against women among various REM populations. Across EWSs, obtaining gender-sensitive data, especially around violence, is problematic. This challenge is not specific to REM, but it is clear in recent conflicts (such as the abuse of Yezidi women by the so-called Islamic State of Iraq and Syria) that understanding the risk of different types of violence toward women in various REM communities is critical to mitigating severe outcomes.

4. The ability to monitor speech and various types of communication is rapidly changing. For-profit companies build models that scrape social media platforms and can give insights to shifting sentiments. However, these companies rarely work with the contextual knowledge that supports a nuanced understanding of local communities.

49 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_ prevention_ tool.pdf. 50 Defontaine, “Setting up Early Warning and Response Systems.” 51 There are data sets that monitor minority violence (the Minority Rights Group International’s Peoples Under Threat database, for example, at https://peoplesunderthreat.org/data/), but the timing of the data updates on violence is unclear.

15 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

CONCLUSIONS AND RECOMMENDATIONS Table 1 presents key design considerations and recommendations. Researchers considered findings and gaps related to effective EWS along with several working assumptions about the USAID REM effort. These include that most data will need to be pulled from external sources, although some new data can be gathered from local monitors; budgets and staff time are limited; the ability to share data easily is critical; and USAID’s capacity to respond is not yet fully understood.

TABLE 1. KEY CONSIDERATIONS AND PRELIMINARY RECOMMENDATIONS FOR A REM EWS

DECISION POINT CONCLUSIONS / CONSIDERATIONS RECOMMENDATIONS

Purpose and Structure

● EWS effectiveness is based on the utility of the system to the end user and the goals of the system. In particular, systems differ in whether they are designed to monitor or predict certain types of violence in a given area and in a given time frame. The goals of the system determine the trade-offs posed for effort, cost, and quality of data.

● To be truly early in a response, higher degree of risk/lower degree of confidence that a violent may not occur must be acceptable.

● Effective systems are designed based on the realities of both the on-the-ground context and the organizational context of those managing EWSs and making program decisions. Careful consideration of factors—including country context, mandate of a mission, programming options, and resources available—is necessary to create effective EWSs.52 Piloting and testing requirements are substantial.

● A critical next step is a better understanding of the value of an EWS within USAID. The Agency should work from clear and detailed decisions regarding purposes and users.

● It is recommended that the EWS focus on monitoring situations where mass violence is likely to ensure sustainability of data and build out assumptions before attempting to build a predictive database.

● Given the complexity of these considerations, it is also recommended to launch pilot projects within limited geographic areas where there is mission buy-in. By doing so, the project can test assumptions, build internal processes, and grow critical on-the-ground networks.

o Build extra time and budget considerations into building the system, coding, testing, and validating data (especially if predictive).

52 United Nations Development Programme and Organization of American States, Practical Guide.

16 |USAID.GOV CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS

DECISION POINT CONCLUSIONS / CONSIDERATIONS RECOMMENDATIONS

● Relevant monitoring and data analysis are rapidly evolving. For example, ACLED launched a predictive model for violence this month (https://acleddata.com/conflict-pulse/) and will be launching monitoring capability for religious violence next year.

● A hybrid structure with both machine learning capabilities and human intelligence aspects is suggested, including a mix of data specialists to clean, aggregate, and create data dashboards and a team of experts for final determination of issuing warnings and creating programming decisions.

● Decisions about structure would benefit from comparative cost analysis.

Data ● EWS information and analysis capabilities are more effective when using existing data collection capabilities, infrastructure, and logistics, especially within government agencies. This makes the overall EWS easier to launch and maintain and increases the likelihood that stakeholders will use the new system.53

● An increase in indicators does not necessarily improve the reliability of a prediction/probability/likelihood.

● A great many data points for each variable are required for machine learning models to effectively isolate particular variables. An increase in variables may create more granular predictions, but it also increases the cost, complexity, and ability to visualize the data.

● Data-poor environments pose challenges to creating highly predictive and automated systems. Overall, REM

● Most REM data will come from existing data sources for reasons of cost, data accuracy, sustainability, and usability of data. The selection of data sources, or the curation process, needs careful consideration of biases, weaknesses, and data aggregation considerations. An additional consideration is the ability to consistently and sustainably aggregate data in a timely manner.

● The ability to link existing data sources and other EWSs with information from USAID programs may bolster the situational awareness, but the ability to gather and maintain such data is a significant undertaking.

53 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_prevention_tool.pdf.

17 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

DECISION POINT CONCLUSIONS / CONSIDERATIONS RECOMMENDATIONS

in MENA is a data-poor environment, with data that is dated, holds inaccuracies, is costly to validate, and holds temporal lags.

● Data points will vary based on country context and will need to be updated with new events; this adaptability is critical for an EWS to remain relevant. However, data that changes rapidly and often makes it difficult to build comparative data sets from it, and it is costly and time consuming to merge with complementary data.

● Effective EWSs are readily understood by a range of actors, because data are straightforward, visual, and somewhat easily updated. This ease of understanding may, however, entail a trade-off with highly contextualized data.

● Indicators must be chosen carefully based on the system’s end goal and hypotheses about how factors interact.

● Systems working with MENA REM populations must consider the cost and networks required to gather highly contextualized data and to gather enough accurate data to run predictive models.

● Determining the ultimate users of the system will enable USAID to make decisions about data presentation and communication.

Use and Response

● Warnings that are issued without action, especially from powerful organizations with funding, can inadvertently undermine trust, cooperation, and opportunities for future partnerships. However, failure to issue a warning may also undermine trust and cooperation.

● Factors that predict violence may not be the same as interventions that can prevent violence. For example, infant mortality may predict violence, but women’s health services may not be the most effective immediate intervention.

● A system that seeks to work with local populations will need to be comprehensive—working across sectors, understanding the context of certain localities to build

● A clear communication strategy about when warnings are issued and what action can be taken, long before a potential violent event occurs, must be developed as part of the EWS..

● For an EWS to be complete, warnings must be linked to responses. This suggests that the REM team needs to consider USAID’s capacity and commitment to respond before determining the utility of a predictive system.

● Contextual analysis, in addition to the warning, will be necessary for programming decisions and so must be built into the system.

● Related to the purpose recommendations, USAID should assess the potential interface

18 |USAID.GOV CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS

DECISION POINT CONCLUSIONS / CONSIDERATIONS RECOMMENDATIONS

responses that improve state-citizen feedback loops, supporting complaint mechanisms, enabling private sector engagement, and establishing platforms for dialogue at the local, substate, and state levels.54

● The ability to link data to programming is highly manual and requires design for partnerships and links of decision, data, and on-the-ground stakeholders. It goes beyond ensuring data can be visualized, easy to digest, and somewhat automated.

● It is challenging to measure the success of preventative systems, although it is not impossible to link programming strategies to positive or negative outcomes of interventions designed to prevent events.

between community monitoring and early warning and response mechanisms.

● It is suggested that the proposed system consider programming options along with or before finalizing design visualizations and data communications to best support decision-making processes.

● Careful consideration of monitoring and evaluation processes is required to demonstrate that an EWS is useful.

54 https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_prevention_tool.pdf.

19 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

ANNEX A: STATEMENT OF WORK

Desk Review / Assessment of Existing Prejudice/Discrimination, Violence, and Genocide Early Warning Systems (EWS)

Research Questions

Subtask 1 will assess the attributes and effectiveness of existing early warning systems55 based on the following key research questions:

1. What are the types and purposes of EWS models currently in use across program areas (e.g., conflict, genocide, famine, climate, etc.)?

2. What are the attributes of the most effective EWSs in terms of use, predictive capacity and sustainability? What trade-offs do EWSs face?

3. What warning signs dealing specifically with discrimination, violence and genocide affecting minority populations are most often tracked and by which EWSs or other databases/indices?

4. How are data communicated and used for EWSs dealing with minority populations? 5. What are the key gaps in existing systems regarding minority populations and what value might

a new system add? 6. What is the role of machine learning, digital literacy, and public information security concerns in

what is reasonable to be made public? Based on the findings from these questions, the Subtask 1 team will propose recommendations for design of a potential REM EWS.

Research Design

The Subtask 1 team will conduct a desk review and informal qualitative interviews with subject matter experts and MSI’s REM human rights dashboard development team to answer the research questions. The EWSs and databases/data sources that may feed into them include, but are not limited to:

● EW mechanisms: Famine Early Warning Systems (FEWS NET); U.S. Commission on International Religious Freedom (USCIRF); United State Holocaust Memorial Museum Early Warning Project; Amnesty International; Human Rights Watch; Uppsala University ViEWS; International Crisis Group Crisis Watch. The team will research additional organizations with EWS including the European Union and the United Nations.

55 An EWS is defined as a system that monitors, calculates or attempts to predict risk. In addition. EWS are designed to communicate risk though communication, building response capability, and initiating response protocols.

20 |USAID.GOV CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS

● Relevant data bases, sources and indices: Armed Conflict Location & Event Data Project (ACLED), Humanitarian Data Exchange (INFORM Severity Index); Arab Barometer, Varieties of Democracy (V-DEM), Fragile States Index, Global Peace Index, and Freedom House’s Freedom of the Net 2020.

● For profit services: Horizon Intelligence (Hozint), Fraym As a first step, the Subtask 1 team will set up and populate a working matrix/inventory of EWSs, indices and other relevant data sets to better understand how they operate and enable comparison of the following types of attributes, where data is available for the system:

1. Web address for accessing data 2. Cost, if any, to access 3. Purpose: alerting community, alerting policy makers, providing more general audiences with

data that they can use as needed; etc. 4. Data type(s)—description/list of key variables being measured (key concept definitions where

important) and covariates 5. Data format(s) 6. Data source(s) 7. Geographic units (what is the lowest level) 8. Countries covered 9. Relevant search terms 10. How data are cleaned, validated and entered 11. Whether/how system is updated (e.g. with new indicators) 12. Frequency of updates 13. How far back does the database go? 14. Measures to protect data sources, collectors, users 15. Users, how data are communicated and used 16. Elements of database/system that could be used for REM EWS; elements relevant to REM that

appear NOT to be tracked 17. Advantages of system 18. Limitations of system 19. Validity and reliability 20. If data are aggregated, how are they aggregated? Does aggregation present validity or other

issues? 21. Data visualization options 22. Cost of maintaining EWS/database

If publicly available data is insufficient to answer key research questions, the Subtask 1 team will seek to interview EWS, indices and database system administrators. It is possible that not all fields may be filled, however. Analysis of this review will entail developing a second matrix or other format to summarize the following:

21 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

1. What makes for an “effective” EWS? Aspects might include (but need not be limited to):

a. Data collection, validation, cleaning and entry processes b. Data precision, timeliness c. Data categorization and searchability d. Data visualization and ease of use e. Predictive capacity f. Adaptability g. Sustainability

2. What is the necessary set of indicators that would need to be included in a REM EWS? Each indicator should include:

a. a detailed definition, b. justification, c. link to REM programming options d. data source(s), e. limitations

3. What constitutes the threshold for a “warning”? How would data be analyzed to trigger a “warning” or initiation of a response? Compare scenario planning to monitoring-oriented system.

4. What tradeoffs do comparable EWS face? 5. What are critical cost considerations for a new EWS, including establishment and maintenance

costs? 6. What are the conflict sensitivity considerations that need to be incorporated in the system and

its uses? Answering the above questions will draw on internal analysis of the data collected, secondary studies (for example, by the UN), and selective KIIs of EWS users and administrators. Finally, the research team will present findings from these research questions to USAID for feedback, along with recommended key elements of a proposed REM EWS design, including discussion of its added value relative to other existing sources, and of feasibility with regard to timely gathering, cleaning, updating, analyzing data.

Key Deliverables

1. Matrix of EWS and relevant databases/indices 2. 3. Report responding to the research questions, with recommended key elements of a proposed

REM EWS and other considerations to consider regarding feasibility and need. 4. Establish critical design considerations for EWS, including data points/categories, sources, data

visualization.

Staffing

Team Leader: Will oversee the technical direction of the activity, including the approach and interview guides, literature review, interviews conducted, data analysis, and development of the Findings Report and Best Practices/Models Matrix.

22 |USAID.GOV CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS

Technical Advisor: Will support the team leader in providing subject matter expertise on Early Warning Systems in the development of the approach, interview guides, and document/literature review, and in data analysis and development of the reference guide.

Researcher: Will work closely with the team leader to conduct the literature review and support other tasks as needed.

Activities and Estimated Schedule

Timeframe Tasks Week 1 Nov 2-6

Finalize Scope of Work (SOW)

Week 2 Nov 9-13

Develop EWS and database matrix and background interview protocol Begin conducting interviews with in-house experts

Week 3 Nov 16-20

Begin conducting background interviews with external experts Document/literature review

Week 4 Nov 23-25

Continue interviews Complete document/literature review Complete EWS/database matrix

Week 5 Nov 30–Dec 4

Finish conducting interviews Analyze interview data Share matrix with REM, including interns Begin writing report

Week 6 Dec 7-11

Continue writing report

Week 7 Dec 14-18

Submit draft report for USAID review

Weeks 8-9 Dec 21-31

Receive USAID feedback

Weeks 10 Jan 4-8-

Present key findings and recommendations

Week 11 Jan 11-15

Submit final report

23 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

ANNEX B: LIST OF EARLY WARNING SYSTEMS REVIEWED • Famine Early Warning Systems Network (FEWS NET) • U.S. Commission on International Religious Freedom (USCIRF) • Early Warning Project (EWP) • Amnesty International • Human Rights Watch (HRW) • Uppsala University Violence Early Warning Systems • International Crisis Group Crisis Watch • Economic Community of West African States ECOWARN • Intergovernmental Authority on Development • CEWARN • United Nations Framework of Analysis for Atrocity Crimes • ACLED • Humanitarian Data Exchange (INFORM Global Crisis Severity Index, Risk Index, and Warning

System) • Arab Barometer • V-DEM • Fragile States Index • Global Peace Index • Horizon Intelligence • Fraym

24 |USAID.GOV CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS

ANNEX C: LIST OF SELECTED INTERVIEWEES The team conducted a select group of key informant interviews of current and former EWS managers to inform the desk review. The organizations include:

• Armed Conflict Location & Event Data Project (ACLED) • ECOWAS Early Warning and Response Network (ECOWARN) • Famine Early Warning Systems Network (FEWS NET) • Fragile States Index • Holocaust Museum Early Warning Project

25 | CONSIDERATIONS FOR THE DESIGN OF A REM-FOCUSED MONITORING AND EWS USAID.GOV

ANNEX D: BIBLIOGRAPHY

Armed Conflict Location and Event Data Project, “ACLED Introduces New Event Types and Subevent Types” ACLED (2019), https://acleddata.com/2019/03/14/acled-introduces-new-event-types-and-sub-event-types/.

Defontaine, Catherine,“Setting up Early Warning and Response Systems to Prevent Violent Conflicts and Save Lives,” Development for Peace (blog), World Bank, February 15, 2019, https://bit.ly/3prBQ1M.

International Federation of Red Cross and Red Crescent Societies, Community Early Warning Systems: Guiding Principles (Geneva: 2012), https://www.ifrc.org/PageFiles/103323/1227800-IFRC-CEWS-Guiding-Principles-EN.pdf.

Organisation for Economic Co-operation and Development (OECD), Preventing Violence, War, and State Collapse: The Future of Conflict Early Warning and Response (Paris: OECD, 2009), https://www.oecd.org/dac/conflict-fragility-resilience/docs/preventing%20violence%20war%20and%20state%20collapse.pdf.

Osorio, J. et al, “Mapping Violent Presence of Armed Actors in Colombia,” Advances in Cartography and GIScience of the ICA, 1, no. 16 (2019).

Oxfam, From Early Warning to Early Action: What Can We Learn to Support Early Action to Mitigate Humanitarian Crises? (Nairobi, Kenya: 2017), https://oxfamilibrary.openrepository.com/bitstream/handle/10546/620345/dp-early-warning-early-action-somalia-021017-en.pdf;jsessionid=B5F730B34270A024B5F1E5520C32F16B?sequence=1.

Palli, Sofia, Economic and Social Commission for Western Africa (ESCWA), Early Warning Systems as a Conflict Prevention Tool: Recommendations for the Arab Region (Beirut: 2017), https://www.unescwa.org/sites/www.unescwa.org/files/page_attachments/early_warning_systems_as_a_conflict_prevention_tool.pdf.

United Nations Development Programme and Organization of American States, Practical Guide: Early Warning and Response Systems Design for Social Conflicts (Washington, DC: 2015), https://reliefweb.int/sites/reliefweb.int/files/resources/UNDP_RBLAC_Guia%20Alerta%20EN.pdf.

University of Gothenburg, V-Dem Institute, Quickstart Guide V9 (Gothenburg, Sweden: 2019), https://www.v-dem.net/files/59/V-Dem%20Startquide.pdf.

Verdeja, Ernesto, “Predicting Genocide and Mass Atrocities,” Genocide Studies and Prevention: An International Journal 9, no. 3 (2016): 13–32.