Latest news with #TimnitGebru
WIRED
23-04-2025
- Science
- WIRED
AI Is Spreading Old Stereotypes to New Languages and Cultures
Apr 23, 2025 12:31 PM Margaret Mitchell, an AI ethics researcher at Hugging Face, tells WIRED about a new dataset designed to test AI models for bias in multiple languages. Photo-Illustration:Margaret Mitchell is a pioneer when it comes to testing generative AI tools for bias. She founded the Ethical AI team at Google, alongside another well-known researcher, Timnit Gebru, before they were later both fired from the company. She now works as the AI ethics leader at Hugging Face, a software startup focused on open source tools. We spoke about a new dataset she helped create to test how AI models continue perpetuating stereotypes. Unlike most bias-mitigation efforts that prioritize English, this dataset is malleable, with human translations for testing a wider breadth of languages and cultures. You probably already know that AI often presents a flattened view of humans, but you might not realize how these issues can be made even more extreme when the outputs are no longer generated in English. My conversation with Mitchell has been edited for length and clarity. Reece Rogers: What is this new dataset, called SHADES, designed to do, and how did it come together? Margaret Mitchell: It's designed to help with evaluation and analysis, coming about from the BigScience project. About four years ago, there was this massive international effort, where researchers all over the world came together to train the first open large language model. By fully open, I mean the training data is open as well as the model. Hugging Face played a key role in keeping it moving forward and providing things like compute. Institutions all over the world were paying people as well while they worked on parts of this project. The model we put out was called Bloom, and it really was the dawn of this idea of 'open science.' We had a bunch of working groups to focus on different aspects, and one of the working groups that I was tangentially involved with was looking at evaluation. It turned out that doing societal impact evaluations well was massively complicated—more complicated than training the model. We had this idea of an evaluation dataset called SHADES, inspired by Gender Shades, where you could have things that are exactly comparable, except for the change in some characteristic. Gender Shades was looking at gender and skin tone. Our work looks at different kinds of bias types and swapping amongst some identity characteristics, like different genders or nations. There are a lot of resources in English and evaluations for English. While there are some multilingual resources relevant to bias, they're often based on machine translation as opposed to actual translations from people who speak the language, who are embedded in the culture, and who can understand the kind of biases at play. They can put together the most relevant translations for what we're trying to do. So much of the work around mitigating AI bias focuses just on English and stereotypes found in a few select cultures. Why is broadening this perspective to more languages and cultures important? These models are being deployed across languages and cultures, so mitigating English biases—even translated English biases—doesn't correspond to mitigating the biases that are relevant in the different cultures where these are being deployed. This means that you risk deploying a model that propagates really problematic stereotypes within a given region, because they are trained on these different languages. So, there's the training data. Then, there's the fine-tuning and evaluation. The training data might contain all kinds of really problematic stereotypes across countries, but then the bias mitigation techniques may only look at English. In particular, it tends to be North American– and US-centric. While you might reduce bias in some way for English users in the US, you've not done it throughout the world. You still risk amplifying really harmful views globally because you've only focused on English. Is generative AI introducing new stereotypes to different languages and cultures? That is part of what we're finding. The idea of blondes being stupid is not something that's found all over the world, but is found in a lot of the languages that we looked at. When you have all of the data in one shared latent space, then semantic concepts can get transferred across languages. You're risking propagating harmful stereotypes that other people hadn't even thought of. Is it true that AI models will sometimes justify stereotypes in their outputs by just making shit up? That was something that came out in our discussions of what we were finding. We were all sort of weirded out that some of the stereotypes were being justified by references to scientific literature that didn't exist. Outputs saying that, for example, science has shown genetic differences where it hasn't been shown, which is a basis of scientific racism. The AI outputs were putting forward these pseudo-scientific views, and then also using language that suggested academic writing or having academic support. It spoke about these things as if they're facts, when they're not factual at all. What were some of the biggest challenges when working on the SHADES dataset? One of the biggest challenges was around the linguistic differences. A really common approach for bias evaluation is to use English and make a sentence with a slot like: 'People from [ nation ] are untrustworthy.' Then, you flip in different nations. When you start putting in gender, now the rest of the sentence starts having to agree grammatically on gender. That's really been a limitation for bias evaluation, because if you want to do these contrastive swaps in other languages—which is super useful for measuring bias—you have to have the rest of the sentence changed. You need different translations where the whole sentence changes. How do you make templates where the whole sentence needs to agree in gender, in number, in plurality, and all these different kinds of things with the target of the stereotype? We had to come up with our own linguistic annotation in order to account for this. Luckily, there were a few people involved who were linguistic nerds. So, now you can do these contrastive statements across all of these languages, even the ones with the really hard agreement rules, because we've developed this novel, template-based approach for bias evaluation that's syntactically sensitive. Generative AI has been known to amplify stereotypes for a while now. With so much progress being made in other aspects of AI research, why are these kinds of extreme biases still prevalent? It's an issue that seems under-addressed. That's a pretty big question. There are a few different kinds of answers. One is cultural. I think within a lot of tech companies it's believed that it's not really that big of a problem. Or, if it is, it's a pretty simple fix. What will be prioritized, if anything is prioritized, are these simple approaches that can go wrong. We'll get superficial fixes for very basic things. If you say girls like pink, it recognizes that as a stereotype, because it's just the kind of thing that if you're thinking of prototypical stereotypes pops out at you, right? These very basic cases will be handled. It's a very simple, superficial approach where these more deeply embedded beliefs don't get addressed. It ends up being both a cultural issue and a technical issue of finding how to get at deeply ingrained biases that aren't expressing themselves in very clear language.
Arab News
07-03-2025
- Science
- Arab News
Women hold the key to tech innovation in the Arab world
International Women's Day is as good a time as any to reflect on the progress made and the challenges that lie ahead in achieving gender equality across all sectors. One area where encouraging strides have been made, although more are required, is the participation of women in science, technology, engineering and mathematics fields, particularly in artificial intelligence, machine learning, data science and advanced technologies. All over the world, including the Arab region, the demand for skilled professionals in AI, machine learning and data science is booming. Names like Fei-Fei Li, Timnit Gebru, Rana El-Kaliouby, Margaret Mitchell, Aishwarya Srinivasan, Daphne Koller and Chip Huyen bear testimony to the increasing contribution of female scientists and entrepreneurs. With civilization standing on the cusp of an industrial revolution driven by advanced technologies, it is vital for the momentum to not merely be maintained but accelerated. The inclusion of women in STEM fields, particularly in emerging technologies, is not a matter of equality for the sake of it, it is a necessity for innovation and economic growth. Research shows that teams that allow women's voices and ideas to be heard benefit from more varied perspectives, leading to faster problem-solving and more innovative solutions. In the realm of AI and machine learning, where algorithms can perpetuate biases and prejudices, the involvement of women is crucial to ensure that these technologies are developed with a broad and mature worldview. The field of AI presents plentiful opportunities for women in the Arab world. The global AI market is expected by some research firms to reach $1.8 trillion by 2030, with a compound annual growth rate of 36 to 38 percent. This explosive growth has resulted in a spike in demand for skilled professionals, creating a once-in-a-generation opportunity for women to prove themselves as leaders in this transformative field. The inclusion of women in STEM fields is not a matter of equality but a necessity for innovation and economic growth. Arnab Neil Sengupta Although the journey is far from complete, the Arab world has experienced a quiet transformation in recent years, with women increasingly pursuing STEM education and careers. Across the Gulf Cooperation Council bloc, new benchmarks are being set for gender parity in science and technology. Educational institutions across the Arab world are taking steps to attract and retain female talent in STEM fields. Saudi Arabia's Vision 2030 places a growing emphasis on encouraging women to pursue STEM programs. The reform strategy has created numerous scholarship opportunities and training programs aimed at narrowing the gender gap in these fields. The results so far are promising. King Abdullah University of Science and Technology has a female student body population of 39 percent in STEM programs, surpassing the global average. At KAUST, 47 percent of graduates in their AI academy program are women. Additionally, KAUST's entrepreneurship programs have trained more than 24,000 people, with an average female participation rate of 51 percent. Its MENA-based startup accelerator program, Taqadam, has a female founder rate of 49 percent, which is 'well above the global average.' In the UAE, 'the share of STEM enrolments from women … rose from 33 percent in 2018-19 to 41 percent in 2019-20,' according to Coursera's Global Skills Report. The fact that women make up 41 percent of UAE government university graduates in STEM is a compelling testament to the region's commitment to encouraging female talent in these critical fields. Despite the advancements in some Arab states, challenges persist in the rest of the Arab region. Cultural stereotypes, a lack of visible role models and systemic barriers still deter many young women from pursuing careers in STEM. Addressing these drawbacks requires a multipronged approach involving education, policy and private sector initiatives. Although the journey is far from complete, the Arab world has experienced a quiet transformation in recent years. Arnab Neil Sengupta To fully realize the potential of women in STEM and AI, efforts are needed at all levels of society. Action is required in five key areas. First is education reform, such as integrating AI and advanced technologies into school curricula from an early age. Secondly, mentorship programs can establish networks connecting experienced female STEM professionals with young aspirants. Third is the introduction of industry partnerships promoting collaboration between educational institutions and tech companies and providing internships and job opportunities. Fourth is the implementation of policy initiatives that support work-life balance and career progress. Finally, public awareness can be increased through the launch of campaigns that challenge stereotypes and showcase the achievements of women in STEM, particularly in AI and advanced technologies. The Arab world's private sector has a crucial role to play too. Companies have a responsibility to create work environments that support women's career advancement in STEM fields. This includes implementing flexible work policies, mentorship programs and transparent pathways for promotion to leadership roles. To sum up, the journey toward gender parity in STEM and AI is not just about numbers. It is about unlocking the full potential of half the world's population. By empowering women in STEM, nations are advancing gender equality at a minimum. On the macro level, they are encouraging innovation, spurring economic growth and shaping a future in which science and technology serves all humankind. The Arab world is well positioned to capitalize on the opportunities being created by the technological revolution. Countries like Saudi Arabia and the UAE are investing heavily in AI research and development, creating new ecosystems ideal for innovation. By encouraging more women to specialize in AI and related fields, these two countries can tap into a large pool of Arab talent to drive future economic growth and technological advancement. The Gulf states are leading by example, showing the Arab world — and indeed the global community — how investing in women's education and careers in STEM can transform societies and economies. Guided by the strategic foresight and bold initiatives of GCC leadership, the future of AI and advanced technologies in the Arab world is bright. There is no reason why Arab women cannot be equal participants in this transformative journey. * Arnab Neil Sengupta is a senior editor at Arab News. X: @arnabnsg
