Utrecht University: Benefits of discriminatory algorithms unproven, drawbacks are clear
When Dutch authorities make a distinction in their databases between citizens according their geographic origins, it is rarely aimed at correcting inequalities. Gerwin van Schie demonstrates this in his dissertation. Instead, people with a migration background are disadvantaged and the use of data deepens the dividing lines in Dutch society.
In his dissertation ‘The Datafication of Race-Ethnicity’, for which he received his PhD from Utrecht University on 19 September 2022, Gerwin van Schie delves into the data systems of Dutch government bodies. He shows that from about 2005 the possibility of using algorithms to uncover patterns in large amounts of data (data mining) was enthusiastically embraced as a basis for policy.
The origin of citizens was also considered relevant here. Initially, it concerned the distinction between ‘native’ and ‘immigrant’ citizens, later citizens with a ‘Western’ or ‘non-Western migration background’. Incidentally, these are new names for the same categories. Until very recently, they played a role in, for example, calculating the risks of crime and fraud.
Skewed distribution of trust
As a result of ethnic profiling, groups that already have a harder time in Dutch society due to their historically rooted disadvantage are relatively more distrusted and checked, while more privileged groups are given the benefit of the doubt. One of the examples that Van Schie analyses is the ‘Liveability Barometer’ of the Ministry of the Interior, where a larger percentage of ‘immigrants’ was until very recently supposed to impair the quality of life in a neighbourhood.
Another example is the Crime Anticipation System of the Amsterdam police, in which it was assumed that incidents will occur more often in multicultural neighbourhoods. The tax authorities’ control mechanisms that led to a recent scandal in the Netherlands are another high-profile example where biased algorithms resulted in a skewed distribution of trust and mistrust across already more and less privileged groups.
Incongruities
Van Schie exposes some inconsistencies in the ethnic profiling practices of Dutch authorities. First, they were rolled out enthusiastically, even though it had never been shown that making a distinction between citizens according to their origins – however drastic and questionable in principle – led to more effective policy.
Second, colonial biases translate into algorithms that intuitively make strange distinctions. For example, people from Turkey have a non-Western migration background, but people from Indonesia have a Western background, and people from Suriname have a non-Western background. Van Schie further emphasizes that once a distinction is made according to origin, sooner or later it will be used in a stigmatizing way.
Effective and ethical data use
Lately, awareness about racist prejudice has grown in the Netherlands. Governments have gradually abandoned the practice of making a distinction according to origin in data mining. For instance, the ethnic distinction has been deleted in the Crime Anticipation System of the Amsterdam police since 2017, and since March 2022 the Liveability Barometer no longer distinguishes between identity characteristics.
The aforementioned scandal, where the Dutch tax authorities were widely criticized for their ethnic profiling, has led to a turnaround. Thanks to the Spinoza Prize that media scientist José van Dijck received from NWO in 2021, Van Schie is now conducting participatory research into effective and ethical use of data to shape policy. Based at the Utrecht Data School, he collaborates with many authorities and encounters great enthusiasm both from municipalities and ministries.