Excessive depictions of violence, porn sites without an age check, Holocaust deniers and religious extremists, but also serious endangerment of young people through dangerous sexual practices or drug glorification and suicide instructions, there are masses of all of this on the Internet. Taking action against this is one of the tasks of the state media authorities. From March 2021 to April 2022, they submitted almost twice as many criminal complaints to the public prosecutor’s office as in the previous 12 months.
The extreme increase results from the introduction of a software called Kivi. The artificial word is made up of the abbreviation KI for artificial intelligence and vigilante for a vigilante. After the tool, which was developed by the State Media Authority of North Rhine-Westphalia together with the Berlin company Condat AG, was initially tested in North Rhine-Westphalia, it has been used nationwide by the state media authorities for a few weeks.
Word of the success has spread beyond the German national borders. Tobias Schmid, the director of the Düsseldorf media authority, was visiting Vienna. Kivi was one of the topics there. Because the Austrian media watchdogs are also interested in automated searches of websites, video platforms, social networks and messaging services. The Alpine republic is not the only country with which North Rhine-Westphalia is talking about exports.
“We are pleased about the great interest shown by our European colleagues, especially the Belgians and Austrians, with whom we are in exchange for the scope of Kivi. The use of this tool in Germany is already proving that effective media supervision can also be achieved at federal level with the right means,” says media watchdog Tobias Schmid. Kivi offers a technically extremely sophisticated help.
Barbara Banczyk is responsible for the AI project as a consultant in the supervision team at the State Media Authority in North Rhine-Westphalia. “In 2019, the question arose: How can we do our job, i.e. to search the web for potential legal violations, even better?” Hard stuff, because the content found is often not only disturbing, but also mentally taxing. But it wasn’t just about protecting employees, it was also about efficiency.
Like a search engine, the program searches for violations using keywords and links. Various techniques are used for this. Videos are decomposed using keyframe extraction. Texts in images are recognized by so-called Optical Character Regognition – OCR for short. Speech-to-text technology allows text to be recognized in audio files, and other routines are used to obtain additional information such as region identification, user numbers and other data. The tool can currently examine over 10,000 pages per day, but this performance can be increased if necessary.
In addition, the tool is capable of learning. New images or text examples can be fed in easily. The feedback from the media watchdogs who evaluate the locations is also included in the AI search. But there are also limits. Violations of journalistic due diligence – or the dissemination of fake news, such as by KenFM – cannot yet be detected via AI. “But we are also working on that,” says Barbara Banczyk.
The technology does the preliminary work. The subsequent examination is carried out by employees of the state media authorities. If this is the case, the violation can be forwarded to the investigating authorities. The media watchdogs are not only concerned with punishing perpetrators, but also with removing the incriminated content from the Internet.
Kivi means a considerable reduction in workload: criminal charges are prepared automatically. The tool later also automatically checks whether content for violations that have already been found can still be found online. The software also supports the automated documentation of further media law procedural steps such as hearing the person responsible for the violation.
The main part of the identified potential legal violations can be found in the area of pornography. A smaller, but not negligible part is depictions of violence and Holocaust denial and incitement to hatred. The detected violations are processed according to priority, i.e. the severity of the violation.
The work of the media guards is having an effect. However, one of them is that many perpetrators emigrate to non-European countries – or act anonymously. “In Germany you will hardly find a major provider of pornographic material that does not comply with the law,” says Banczyk.
But the destination country principle applies. The media authorities are not interested in where the website is located, but whether it is aimed at users in Germany. This also applies to the most recent proceedings by the media authorities against the large porn providers based abroad. “That’s no reason for us not to take care of it. We are tackling this and pursuing these providers because they are addressing the German market on a massive scale.” Thanks to Kivi, they too can now hide less easily.
