Silent Talking [043]

Silent Talking is an installation based on the biometric data algorithm currently trialed at the external European Union borders of Hungary, Greece and Latvia. This artificial intelligence “lie detector”, called iBorderCtrl, is designed to scan the micro-expressions of travelers seeking to enter the EU. Using an undisclosed micro-expressions data-set it scans the travelers’ faces seeking to confirm the verity of their intentions and personal background. The system is being developed at Manchester University as an off-shoot project of an earlier, and equally controversial human emotions recognition algorithm called “Silent Talker”.

There have been numerous forensic scholars’ and digital activists’ publications proving that iBorderCtrl use of AI emotional recognition algorithms is not only scientifically unacceptable (for example, due to at least 24% of cases resulting in false-positive response), but also unethical and unjustified. However, the software is poised to become an important tool at the external EU borders, particularly in those with high numbers of “illegal” migrants.

In Silent Talking a 8-channel electronics respond to a facial micro-expressions algorithm, which is modelled after the iBorderCtrl system (developed together with Yiannis Tsirikoglou). The sonic part is transmitted through a multi-speaker system and is generated using a machine- learning-based vocal rendition. This vocal part responds directly to the minuscule changes of the facial movement and the supposed emotional response, and reacts to verity deviations of the video. Thus, any deviation from the “standard” emotional response results in changes of speed, space and tuning of the vocal material. The sonic material is then distributed through an 8-speaker system, installed together with two video screens.

Supported by: Performing Arts Fund NL, The Creative Industries Fund NL, Stroom Den Haag, Hertz Lab and ZKM | Center for Art and Media Karlsruhe, BALTIC Centre for Contemporary Art, Gateshead.