Improving Fairness and Trust in AI Used for College Admissions and Language Translation

Fully ingrained into our daily lives, artificial intelligence (AI) algorithms can help us shop online, calculate credit scores, navigate vehicles, and for those that run afoul of the law, offer judges criminal sentencing guidelines.

But as the use of AI increases exponentially, so does the concern that biased data can result in flawed decisions or prejudiced outcomes.

At the University of Maryland, two teams of researchers are helping to eliminate those biases by developing new algorithms and protocols that can improve the efficiency, reliability and trustworthiness of AI systems. Specifically, the Maryland faculty are working to improve fairness and accuracy in AI-based platforms used for college admissions and are rethinking traditional language translation systems to make them more user-friendly.

Their work is supported by a joint initiative called the NSF Program on Fairness in Artificial Intelligence in Collaboration with Amazon. Out of 11 proposals that were accepted this year, two are led by UMD faculty.

The program’s goals are to increase fairness, accountability and transparency in AI algorithms, and make them more accessible so that the benefits of AI are available to everyone. 

This includes machine learning algorithms—a subset of AI—used to rank applications for admittance to graduate school or allocate resources for faculty mentoring, teaching assistantships or coveted graduate fellowships.

“As these AI-based systems are increasingly used in higher education, we want to make sure they render representations that are accurate and fair, which will require developing models that are free of both human and machine biases,” says Furong Huang, an assistant professor of computer science who is leading one of the UMD teams.

That project, “Toward Fair Decision Making and Resource Allocation with Application to AI-Assisted Graduate Admission and Degree Completion,” received $625K from NSF with an additional $375K from Amazon.

A key part of the research, Huang says, is to develop dynamic fairness classifiers that allow the system to train on constantly evolving data and then make multiple decisions over an extended period. This requires feeding the AI system historical admissions data, as is normally done now, and consistently adding student-performance data, something that is not currently done on a regular basis. 

The researchers are also active in developing algorithms that can differentiate notions of fairness as it relates to resource allocation. This is important for quickly identifying resources—additional mentoring, interventions or increased financial aid—for at-risk students who may already be underrepresented in the STEM disciplines.

Collaborating with Huang are Min Wu and Dana Dachman-Soled, a professor and an associate professor, respectively, in the Department of Electrical and Computer Engineering.

The researchers note that while their initial work will focus on admissions to STEM graduate programs and resource allocation to students studying science, technology and math, the same algorithms could be applied to medical supply chains during a pandemic or natural disaster, employee hiring systems, or automated loan approval platforms.

A second UMD team led by Marine Carpuat, an associate professor of computer science, is focused on improving machine learning models used in language translation systems.

That project, “A Human-Centered Approach to Developing Accessible and Reliable Machine Translation,” is funded with $393K from NSF and $235K from Amazon.

The goal of her team’s research, Carpuat says, is to develop a human-centered design process that software engineers and others can use to build reliable machine translation platforms that are critical for high-stakes situations like an emergency hospital visit or legal proceeding.

“Low quality or incorrect outputs from machine translation systems are particularly detrimental for immigrants or people living and working in places where they do not know the dominant language,” says Carpuat. “This is a fairness issue, because these are people who may not have any other choice but to use machine translation to make important decisions in their daily lives. Yet they don’t have any way to assess whether the translations are correct or the risks that errors might pose.” 

To address this, Carpuat’s team will design systems that are more intuitive and interactive to help the user recognize and recover from translation errors that are common in many systems today.

Central to this approach is a machine translation bot that will quickly recognize when a user is having difficulty. The bot will flag imperfect translations, and then help the user to craft alternate inputs—phrasing their query in a different way, for example—resulting in better outcomes.

Carpuat’s team includes Ge Gao, an assistant professor in the iSchool, and Niloufar Salehi, an assistant professor in the School of Information at UC Berkeley.

The NSF/Amazon-funded project builds upon previous research by Carpuat and Gao that involved building multilingual translation systems that were cognizant of social sensitivities. 

Of the six researchers involved in the Fairness in AI projects, five have appointments in the University of Maryland Institute for Advanced Computer Studies (UMIACS), a multidisciplinary research powerhouse of faculty from nine departments and six schools and colleges on campus. 
“We’re tremendously encouraged that our faculty are active in advocating for fairness in AI and are developing new technologies to reduce biases on many levels,” says Mihai Pop, the director of UMIACS. “I’m particularly proud that the teams represent four different schools and colleges at two universities. This is interdisciplinary research at its best.”

—Story by Tom Ventsias

—Image by Maria Herd