Do you live in a caravan? Were you a hairdresser until recently? Taxi driver, or construction worker perhaps? Then you run a greater chance of being labeled ‘suspicious’.
That is the logic behind the ‘fraud scorecard’, a controversial method of combating welfare fraud. This software that automatically profiles citizens on social assistance benefits – categorizes them into risk groups – was used by 158 Dutch municipalities for fifteen years.
All municipalities pulled the plug on this system in 2020, letter to parliament dated December 14, 2020. However, a derivative version of the fraud scorecard remained active in at least four Dutch municipalities. This is apparent from research by journalist collective Lighthouse Report, radio program argos (broadcasters VPRO and HUMAN) in collaboration with NRC† They invoked the Government Information (Public Access) Act (Wob). As a result of this investigation, the municipalities have recently discontinued their working methods. The fraud scorecard is still active in one other municipality, according to VNG. She won’t say which one.
The municipalities concerned – Nieuwegein, Houten, IJsselstein and Lopik – profiled their ‘social assistance volume’ (almost three thousand citizens on benefits) on the basis of a simple Excel file with fraud scores. Specific target groups were given the stamp of a risk group – potential fraudsters.
The scores can be found in one file. People in caravans, cleaning companies, ‘room residents’ and professions such as hairdressers, window cleaners, taxi drivers, construction workers and catering staff had an extra high chance of being checked.
Those who lived in a good neighborhood or had their own house ran less risk of inspections and were guided through the application procedure more quickly. The system was once designed with statistical data from 2001 and 2003, based on data that the inventors can no longer retrieve.
There are currently more than 400,000 people on social assistance benefits in the Netherlands. There is a high probability that those who applied for the benefit between 2004 and 2020 have been assessed by the fraud scorecard. The system was active in 158 Dutch municipalities. It assessed citizens on the basis of data obtained from intake interviews at the counter or supplied via Suwinet, an exchange database of government services.
Image damage
After a warning from a policy officer from Haarlem in 2019, the government realized that the fraud scorecard is not good. “Extremely careless” and there is no “algorithmic accountability† [de wetenschappelijke onderbouwing is wankel], write responsible officials of the Ministry of Social Affairs and Employment in 2020. They want to get rid of the fraud scorecard. Preferably quietly, because there is a threat of ‘image damage’.
The Association of Dutch Municipalities (VNG) advises all municipalities to disable the fraud scorecard by 2020 because the system violates citizens’ privacy rules. Research by Lighthouse Reports, Argos and NRC shows that at least four municipalities continued with a similar profiling system. This system is called DPS Matrix (‘Diagnosis, Planning and Control Method’) and is based on the same technique as the fraud scorecard.
The Wob request released documents showing that DPS Matrix was still working at ‘Work and Income Lekstroom’ in 2022. This is the service that arranges social assistance benefits at four Utrecht municipalities. It is not clear why the Utrecht municipalities went against the VNG advice.
The fraud scorecard in 2022 is full of old gut feelings and prejudices
Both the fraud scorecard and the derived DPS Matrix are based on historical data and literature research, according to conversations that Argos had with the designers. Stimulansz Foundation, a knowledge and advice organization for all Dutch municipalities, designed the underlying technology in 2003 and 2004. This was done with a subsidy from the Ministry of Social Affairs and Employment, with Mark Rutte as State Secretary. The municipality of Utrecht already used a predecessor of such software.
There seems to be no scientific basis for the system’s predictions. Whether fraud was more frequent or more serious in certain professional groups or high-risk neighborhoods is supported by outdated data. The system has never been validated – tested for truth – and the weightings (scores) have hardly been adjusted to actual results. Researchers from Erasmus University Rotterdam attempted in 2008 to check the system. That failed due to “lack of data,” according to the researchers.
This means that until 2022 assistance applications were still weighted on the basis of risk profiles and prejudices of twenty years old. Anyone who had a ‘suspicious’ business such as a cleaning company, or who worked as a taxi driver, construction worker or window cleaner, will receive a green or orange stamp in the Excel file: the difference between no fraud case and a possible fraud case.
For example, a caravan dweller received a standard 700 ‘penalty points’ – the limit for fraud was 983 points. According to the inventors, the system is designed in such a way that not one factor determines whether you get a visit from an inspector or not.
In 2022, the fraud scorecard and the DPS Matrix are full of old gut feelings and prejudices. For example, according to the system, a female hairdresser has a greater chance of fraud than a male hairdresser.
The fraud scorecard was conceived at a time when welfare was supposed to be a ‘trampoline’ that sent citizens back to work as quickly as possible – not a comfortable safety net. Automation and data collection had to help with this.
Both the Fraud Scorecard and the DPS Matrix served as an advisory tool. In the end, officials decided who got a check. That is not to say that the people the system identified as ‘suspicious’ actually committed fraud.
Sennay Ghebreab, computer scientist at the University of Amsterdam, looked at the technology behind the DPS Matrix. “If you use data to track down people, you risk creating a self-reinforcing system. Each time you focus the spotlight on a few groups that are increasingly highlighted. Because you don’t focus the spotlight on other places, where something might also be found. That gives you tunnel vision.”
A new approach
The four municipalities have recently stopped using the DPS Matrix, reports the responsible organization Werk en Inkomen Lekstroom. “We are looking for another instrument that meets the national guidelines on the use of algorithms.”
Pieter Omtzigt, one of the MPs who exposed the Allowances affair in which innocent citizens were wrongly regarded as fraudsters, does not look at the actions of the municipalities. “The worst part is that I am no longer afraid of this. We’ve had the Allowances affair and now we have another profiling system. Now not with the tax authorities, but with the social assistance.”
Minister Carola Schouten of Poverty Reduction and Participation (ChristenUnie) indicated this week that she wants to make the welfare rules more flexible. „The Participation Act [die de bijstand regelt] is out of balance,” the minister wrote in a letter to the House of Representatives. Her conclusion: there are too many strict rules, too strict sanctions and too little trust in citizens.
With the collaboration of Gabriel Geiger and Evaline Schot (Lighthouse Reports)
A version of this article also appeared in the newspaper of June 25, 2022
#warning #municipalities #continued #controversial #fraud #detection #method