Gender Bias in Online Language Translators: Visualization, Human Perception, and Bias/Accuracy Tradeoffs

Artificial intelligence(AI) systems that interact with humans, such as chatbots and language translators, have many useful applications. However, care must be put in addressing some concerns, such as the presence of bias, possible abusive language, and information leakage, that could hamper public trust in them. In this article, we focus on gender bias in online translators, as recognized and rated by a third-party assessment who does not have access to the training data, and we propose a visualization approach for such rating. We then conduct a survey of how users perceive bias in translators, whether they appreciate the proposed bias rating visualization, and how they may use it to reason about bias-accuracy trade-offs.