Chris Meyns

Society’s Bias Need Not Be Data’s Bias

What’s going on?

An academic workshop on how artificially intelligent communication technologies (read: technologies that work with speech, natural language processing, telecommunications, social media) relate to gender (as in: how someone performs and identifies in relation to categories such as woman, man, non-binary, gender queer).

Why is this happening?

Smart speakers. Chat bots. Voice-operated gadgets. So-called ‘smart’ technologies that use language are invading ever more aspects of people’s lives. Now you’d think that, because these are machines rather than biological organisms, they’d have nothing to with aspects of gender that structures so much of human societies. Well, think again. Google Translate liberally feeds us sexist stereotypes, and there’s a good chance that your digital assistant has plenty of feminine traits. Gender infuses these technologies. But why? Is it problematic? If so, what to do about it? People in this workshop want to answer those questions.

What did they find?

Individual talks

Professor Alison Adam provided us a long-term view on philosophical takes on artificial intelligence. In the past philosophers were mainly concerned with whether machines could ever be as smart as humans. Today, pushed by feminist critiques of technology, we’re much more aware of issues around algorithmic bias, such as in the outrage about Amazon’s “gloriously sexist” recruitment system.

Formal semanticist Dr Heather Burnett asked: Why are masculine pronouns (such as ‘he’, ‘his’, ‘him’) overused in both English and French, especially if only the latter is a grammatically gendered language? They argued that this is not because masculine pronouns are silently assumed as default. Instead, they noted, in patriarchal societies (those which exhibit a higher degree of relations of men dominating others), the gender category man is silently assumed as default, which in turn is reflected in pronoun use.

Dr Stefanie Ullmann argued that many collections of texts have a ‘linguistic gender gap’. That is to say, they have a fearth of feminine-marked words—not just pronouns such as ‘she’, or terms such as ‘woman’ or ‘girl’, but also occupational terms such as ‘businesswoman’ or ‘spokeswoman’. Such a gap is problematic, Ullman noted, because machine learning systems trained on such biased data sets may come to reflect these biases and feed them back into society.

Dr Dirk Hovy studied how syntax use—that is, the arrangement of words and phrases so as to create well-formed sentences—may often unwittingly be gendered. They suggest that for the cases they considered, much of training data consisted of texts written by men. Training on such biased data sets will affect how well natural language processing systems perform, for instance in parsing text or identifying parts of speech. More balanced, or even women-heavy input training data would improve these systems’ performance.

Computational socio-linguist Dr Dong Nguyen described how they built a model that would be able to automatically infer gender from the content of tweets, and developed a twitter-aligned competition to crowdsource user testing of that model. They noted that automatic gender inference could be useful in user profiling (for example by commercial companies), in social science, or to make natural language processing more robust.

Dr Ruth Page went beyond text in studying the #ugly and #uglyselfie tags on Instagram. They found no significant gender differences from looking at text in captions, comments or associated tags. But gender-stereotypical patterns did show up when analyzing the pictures themselves (strategies such as angle, gaze, showing or hiding one’s face), which in some cases could signal different associated mental health conditions.

Bigger picture

A major concern flagged in the discussions is that the data sets on which language-based artificially intelligent systems train is regularly gender biased. ‘Bias’ here can be understood as an inclination for or against (material from) a distinct subject, person or group, with potentially distortive results. Texts by men, references to men and male-, and even masculine pronouns were over-represented in the data sets. But is that really a problem in the data sets? Or does it just reflect language use in societies drenched in gender bias?

even if societies are biased, that’s no reason for technology to amplify such biases

Several participants agreed that even if societies are biased, that’s no reason for technology to amplify such biases, by reproducing them and feeding them back in. What can be done? Suggestions to help reduce bias in data floated at the event included: trim data sets to make them more gender balanced; compile gender-disaggregated data sets (Criado-Perez 2019); develop algorithms to soften or de-bias data (Bolukbasi et al. 2016); provide ‘data profiles’ for each set on how the data was collected, who collected it, and so on. While each of these proposals may come with its own challenges, doing nothing is guaranteed to make things worse.

What’s next?

The individual researchers will inevitably go on and do their own work. The research network will meet again in September 2019.

Formal bits