Explanation-Aware Attacks against Machine Learning

Modern (deep) learning methods used to lack understandable reasoning in their decision process, making crucial decisions less trustworthy. Recent advances in "Explainable AI" (XAI) have turned the tables, enabling precise relevance attribution of input features for otherwise opaque models. Among many prospects in computer science, this progression has also raised expectations that these techniques can benefit defense against attacks on computer systems and even machine learning models themselves. However, so-called explanation-aware attacks allow an adversary to manipulate an ML model's decision and the output of XAI techniques simultaneously, questioning the applicability of ML in security-critical applications. This talk explores the prospects and limits of XAI in computer security, demonstrating where it can and cannot (yet) be used reliably.

Bio:

Christian Wressnegger is an Assistant Professor of computer science at the Karlsruhe Institute of Technology (KIT), heading the chair of "Artificial Intelligence & Security". Additionally, he is the spokesperson of the "KIT Graduate School Cyber Security" and co-director of the "KASTEL Security Research Labs," one of three competence centers for cyber security in Germany. He holds a Ph.D. from TU Braunschweig and has graduated from Graz University of Technology, where he majored in computer science. His research revolves around combining the machine learning and computer security.