Convinced that there was something of merit within existing databases concerned with large-scale state atrocity, I decided to try and evaluate what the different sources identified as the relevant cases. For this, I paired up with Ragnhild Nordas, beginning an effort we called "Bloodbath & Beyond".
The idea here was that there could really be no rigorous analysis of the topic if there was no list. This is afterall how the Correlates of War, UCDP, Nicholas Sambanis and James Fearon/David Laitin's projects got started.
Perusing journals we settled on 8 different sources that had been employed: Rudy Rummel, The Political Terror Scale (at 3 or above), the Political Instability Task Force - Genocide/Politicide listing, Jennifer Balint, Ben Valentino/Paul Huth, the UCDP One-sided violence data, Genocide Watch's list as well as some research by William Easterly. We identified all cases within each database and then attempted to figure out if the same cases were covered. The basic logic here was straightforward: if the different sources were covering the same topic in some manner, then there should be some overlap. If the sources were not covering the same topic, then there should be very little overlap. This is in the general direction of a cross-validation exercise.
What did we find? Well, before you look write down on a piece of paper what mass state-oriented killings existed between 1900-2010.
Essentially, we found that most cases only had one source associated with them (total 650 out of 897). A great many of these could be attributed to the fact that before 1950 there are only 5 sources out of all sources that were around but this is still a decent amount.
Numerous cases had two sources, providing some degree of corroboration (171 out of 897): the Canudus massacre in Brazil 1886-1897, the colonial massacres of the Herrero in South Africa 1900-1918, the Red and White terrors in Hungary 1919, the Transylvania conflict between 1919 and a variety of dates, the massacre of Jewish Refugees in Jordan 1920-1921/1929/1946, Turkey 1924-1927 against the Kurds, Germany from 1933-1945 against the Jews, Poles, Disabled, homosexuals, communists and Jehovah's Witnesses, the Dominican Republic against Dominco-Haitians in 1937, Former USSR against Meskhetians and Crimea Tartars from 1944-1968 as well as against Estonians in 1949, the Cultural Revolution in China from 1966-1975, Cambodia from 1968-1975, El Salvador 1979-1991, Nicaragua against the Miskito from 1981-1992, Yugoslavia against muslims as well as Croats between 1991-1995 and Thailand in 2003.
Fewer cases had three sources (38 out of 897). Croatia agains the Serbs, Jews, Gypsies between 1941-1945, Burma 1948-1962/87, Israel against the Palestinians between 1948-1955/1956/1967/1973, Sudan against Southern nationalists 1952-1972, Guinea-Bissau 1958-1984, Iraq against the Kurds 1961-1975, Rwanda against Tutsi ruling class 1962-1964 as well as 1990-1994, Nigeria between 1967-1970, India against the Naxalites 1968-1982, Cambodia 1970-1975, Philippines 1972-1976, Chile against Leftists 1976-1983, El Salvador against Leftists 1980-1989, Uganda 1980-1986, Iran against Kurds 1981-1992, India against the Sikhs 1984, Burundi 1988, Indonesia against the Auyu 1989-1992, Croatia 1993-1995 against Muslims/Serbs and Sudan 2003-present.
Even fewer cases had four sources (24 out of 897): China against the Kuomintang between 1949-1956, Indonesia against the communist/chinese between 1965-1967, Uganda against the Karamojong, Acholi, Lango, Catholic clergy and political opposition between 1971-1979, Ethiopia between 1974-1991, Indonesia 1975-1992, Argentina 1976-1980, Cambodia 1979, Iran against the bahai 1979-1984, Somalia 1988-1991, Kenya 1991-1994 and Rwanda 1994.
A handful had five (9 out of 897): Colombia 1948-1962, Angola/Portugal 1961-1962, Nigeria against the Igbo 1966-1970, Pakistan against Bengali nationalists 1971, Chile 1973-1976, Angola 1975-1994, Cambodia 1975-1979, Syria 1981-1982, Sudan 1983-1999/2002,
Three cases had six sources. None had more than this. Burundi 1972-1973, Afghanistan 1978-1992 and Bosnia 1992-1995.
So, what did/do we take away from this? Well, several things:
1) it is useful to create a list so that we can begin to discuss the cases as a community (please send me emails regarding your opinions about what should/should not be there as well as any sources you have regarding perpetrators, victims, onsets and terminations). This is something that I assumed the Political Instability Task Force or the Atrocity Prevention Board would do but whatever. Why wait for them. Let's get started.
2) The list provides a nice starting point. We have begun evaluating the cases (one at a time) to find whatever source material exists on who did what to whom as well as how confident we are about this information. Whatever one's opinion about Rudy Rummel's work in the Statistics of Democide, he did provide source material and one could (as we are beginning to do) go back and start to evaluate its quality.
3) 897 isn't a large number (compared to something like GDELT for example) but it ain't small either - especially with each case shrouded in violence, fear, smoke and often ev. If you want to help, let us know.
4) What do six sources converging actually mean? Dangzer back in the 1970s discovered that newspapers were likely to cover riots if there was a specific news organization present. We need to differentiate between our sources, getting down to where they got their information from to first gauge independence and then to gauge why the information was covered/provided. We are now trying to ascertain how each of the sources collected information and coded it. Not an easy task. Some are clearer than others.
5) The list does not completely fit our expectations regarding what we thought would be covered by the most sources but we were not off by that much. For example, there were no cases with three or more cases that we never heard of.
Note: For good overview of the field see Daniel Solomon's great review.