In debates over the future of artificial intelligence, many experts think of the new systems as coldly logical and objectively rational. But in a new study, researchers have demonstrated how machines can be reflections of us, their creators, in potentially problematic ways.
Common machine learning programs, when trained with ordinary human language available online, can acquire cultural biases embedded in the patterns of wording, the researchers found. These biases range from the morally neutral, like a preference for flowers over insects, to the objectionable views of race and gender.
Identifying and addressing possible bias in machine learning will be critically important as we increasingly turn to computers for processing the natural language humans use to communicate, for instance in doing online text searches, image categorization and automated translations.
“Questions about fairness and bias in machine learning are tremendously important for our society,” said researcher Arvind Narayanan, an assistant professor of computer science and an affiliated faculty member at the Center for Information Technology Policy (CITP) at Princeton University, as well as an affiliate scholar at Stanford Law School’s Center for Internet and Society. “We have a situation where these artificial intelligence systems may be perpetuating historical patterns of bias that we might find socially unacceptable and which we might be trying to move away from.”
The paper, “Semantics derived automatically from language corpora contain human-like biases,” published April 14 in Science. Its lead author is Aylin Caliskan, a postdoctoral research associate and a CITP fellow at Princeton; Joanna Bryson, a reader at University of Bath, and CITP affiliate, is a coauthor.
As a touchstone for documented human biases, the study turned to the Implicit Association Test, used in numerous social psychology studies since its development at the University of Washington in the late 1990s. The test measures response times (in milliseconds) by human subjects asked to pair word concepts displayed on a computer screen. Response times are far shorter, the Implicit Association Test has repeatedly shown, when subjects are asked to pair two concepts they find similar, versus two concepts they find dissimilar.
Take flower types, like “rose” and “daisy,” and insects like “ant” and “moth.” These words can be paired with pleasant concepts, like “caress” and “love,” or unpleasant notions, like “filth” and “ugly.” People more quickly associate the flower words with pleasant concepts, and the insect terms with unpleasant ideas.
The Princeton team devised an experiment with a program where it essentially functioned like a machine learning version of the Implicit Association Test. Called GloVe, and developed by Stanford University researchers, the popular, open-source program is of the sort that a startup machine learning company might use at the heart of its product. The GloVe algorithm can represent the co-occurrence statistics of words in, say, a 10-word window of text. Words that often appear near one another have a stronger association than those words that seldom do.
The Stanford researchers turned GloVe loose on a huge trawl of contents from the World Wide Web, containing 840 billion words. Within this large sample of written human culture, Narayanan and colleagues then examined sets of so-called target words, like “programmer, engineer, scientist” and “nurse, teacher, librarian” alongside two sets of attribute words, such as “man, male” and “woman, female,” looking for evidence of the kinds of biases humans can unwittingly possess.
The Latest on: Bias in artificial intelligence systems
via Google News
The Latest on: Bias in artificial intelligence systems
- Reports says attention to “pipeline” has not helped diversity crisis in artificial intelligence industry on April 17, 2019 at 1:41 pm
Women are severely underrepresented in the field of artificial ... AI systems to “use physical appearance as a proxy for character or interior states are deeply suspect,” and suggest that rather than ... […]
- One Way to Reduce Gender Bias in Performance Reviews on April 17, 2019 at 9:15 am
Even artificial intelligence algorithms prefer men ... unrelated to gender — changed its faculty teaching evaluation system from a 1-10 to a 1-6 scale. In total, we looked at 105,034 student ... […]
- Researchers say bias in AI a crisis, linked to lack of tech diversity on April 16, 2019 at 10:02 pm
... bias in artificial intelligence are urgent and the answers lie in diversifying tech workforces, researchers say. A yearlong look at the issue, which included poring through 150 previous studies, ... […]
- European Commission’s Ethics Guidelines on Artificial Intelligence on April 16, 2019 at 11:45 am
Justice: “Be Fair” – the systems should maintain freedom from bias, stigmatization and discrimination for individuals and minority groups. Positives and negatives resulting from artificial ... […]
- Unleashing the Value of Health Data in the Era of Artificial Intelligence on April 15, 2019 at 5:00 am
Implementing wholesale changes in a system that still retains many of its pre-Enlightenment era conventions is not going to be easy. But artificial intelligence will not fulfil its potential if its ... […]
- Can AI Save Us From Bias In The Diverse, Nonbinary Workforce Of The Future? on April 12, 2019 at 10:59 am
Consequently, startups and big companies have attempted to tap emerging technologies like artificial intelligence to improve ... “It would be possible to create biased systems. We wanted to make sure ... […]
- How Real Are ‘Ethical Artificial Intelligence’ Efforts by Tech Giants? on April 11, 2019 at 6:54 am
The moves are meant to address concerns over racial and gender bias emerging in facial recognition and other AI systems, as well as address ... CEO of the Allen Institute for Artificial Intelligence, ... […]
- Lessons for artificial intelligence from the study of natural stupidity on April 9, 2019 at 8:09 am
Artificial intelligence and machine learning systems are increasingly replacing human decision ... We examine three broad causes of bias—small and incomplete datasets, learning from the results of ... […]
- Artificial intelligence in Australia needs to get ethical, so we have a plan on April 4, 2019 at 10:53 pm
Nowhere is that more topical than in technolgies using artificial intelligence ... how data is used to develop AI whether an AI system is being used fairly in which situations should we continue ... […]
via Bing News