Associate Perotto Professor Tom Goldstein receives $3.2 M DARPA Award for leading research to study the Security of Machine Learning Methods

A team of UMD researchers lead by Associate Professor Tom Goldstein, and including Assistant Professor John Dickerson, Assistant Professor Furong Huang, Professor David Jacobs, Professor Jonathan Katz, and Assistant Professor Abhinav Shrivastava, recently received a $3.2M award to study the security of machine learning methods. 

The awarded project, titled “Repelling Evasion and Poisoning Attacks: A Principled Way Forward,” is supported by DARPA's Guaranteeing AI Robustness against Deception (GARD) program.

“A number of new security threats to machine learning systems have recently emerged. This includes evasion attacks, in which small changes are made to the inputs of an ML system that enable the attacker to take control of the outputs, and poisoning attacks where the attacker makes subtle changes to the dataset with the goal of eliciting damaging behavior in models trained on that dataset,” explains Goldstein.

The UMD GARD team aims to develop new methods for making AI systems robust to unexpected inputs, including adversarially crafted and malicious content.

“Examples studied at Maryland include adversarial changes to audio clips that bypass copyright detection systems, adversarial clothing that makes the wearer invisible to object detectors and person tracking systems, and adversarial buy/sell orders that manipulate and confuse bots for high-frequency and algorithmic trading,” said Goldstein

The team targets the study of new training routines that harden neural nets against evasion and poisoning attacks, and will develop new theoretical frameworks for understanding these attacks.

The Guaranteeing AI Robustness against Deception (GARD) program was created by DARPA  to develop a new generation of defenses against adversarial deception attacks on ML models.

In addition to being an associate professor of Computer Science, Goldstein holds a joint appointment in UMIACS and an affiliate appointment in the Department of Electrical & Computer Engineering.

More about the GARD program- https://www.darpa.mil/program/guaranteeing-ai-robustness-against-deception