UOC produces guide to raise awareness of AI's gender and diversity bias in the classroom

Generative AI is transforming how we teach and learn. AI chatbots such as ChatGPT, Gemini and Claude are now widely used by students to help with tasks such as finding new ideas, drafting texts and searching for information as part of their learning. Experts warn that this poses a challenge as, in addition to hindering critical thought and reflection, these tools often produce false information with gender and diversity biases.

Despite this, and as recognized by international organizations such as UNESCO, AI provides significant opportunities in education and for the creation of personalized learning environments. The EDUGENIA project was led by the researchers Milagros Sáinz, a member of the GenTIC research group at the Universitat Oberta de Catalunya's (UOC) UOC-TRÀNSIC research centre, and Marie-Monique Schaper, a Serra Húnter fellow and assistant professor in the GRIHO group at the University of Lleida. It provides a set of resources to tackle the challenges associated with the use of AI in the classroom, particularly from a gender perspective.

The result is an open-access guide that is the first of its kind in Spain. No guidance materials had previously addressed in depth the educational implications of using AI from an ethical standpoint or offered – as EDUGENIA does – practical examples for use in the classroom. The initiative was carried out over 12 months, between 2023 and 2024, with support of theInstitute of Women, an autonomous body under the Spanish Ministry of Equality.

Understanding the social impact of AI

"We risk becoming too reliant on these tools, leading to people making decisions based only on their answers. The unchecked use of AI can have very serious consequences. That's why it's so important for both students and teachers to develop a critical and informed understanding of how to use AI tools," said Sáinz. The EDUGENIA guide draws on principles of feminist pedagogy to counter the discrimination implicit in AI, due to its ultimately reflecting the cultural biases in society. 

According to Schaper, “the advent of AI requires us to redefine what teaching technology really entails. Computer science education has traditionally focused on technical skills and tools. However, there is growing consensus that it should also help students understand the social implications of these technologies and develop a critical approach to them. EDUGENIA provides tools to analyse algorithmic bias and encourages users to think carefully about who designs these technologies, what data they rely on and who stands to benefit or be adversely affected by them. This should encourage discussion of equality, diversity and social justice based on real-life situations."

At the root of EDUGENIA is the#ACTFEM project, created and promoted by the performing arts and community theatre organisation Plàudite Teatre. As part of #ACTFEM, the researchers worked with young women at risk of social exclusion on learning activities involving AI and other emerging technologies. Drawing on this experience and recognising the lack of available resources, Plàudite Teatre initiated a research projectin collaboration with GenTIC at the Universitat Oberta de Catalunya andGRIHO at the Universitat de Lleida to develop the Edugenia guide.

The EDUGENIA research was structured around three main strands: a narrative review of scientific articles on AI, education and gender; an analysis of European, Spanish and Catalan regulatory frameworks on AI and equality; and, finally, a study involving 36 secondary school teachers from Barcelona and Lleida, together with ten young women from #ACTFEM, to explore the needs and experiences involved in teaching AI from a gender perspective.

The topics examined at these practical sessions included the ethical implications of using AI-generated influencers, which perpetuate gender stereotypes, encourage objectification, glorify luxury and power, and promote an excessive disclosure of private life. Participants also worked on effective prompting, and bias and hallucination management strategies. The Gender Shades project, for example, showed that AI tools produced by major corporations such as IBM, Microsoft and Face++ identified male faces more accurately than female faces, as well as white people more accurately than Black people. These are just a few examples of how an uncritical use of AI can lead to discrimination in society.

The goal: to turn AI into an ally in the classroom

The EDUGENIA toolkit features recommendations, best practices and resources to mitigate AI bias. These include the fairness-aware machine-learning principles (Ferrara et al., 2024), which make it possible to identify discrimination based on gender, ethnic origin or socioeconomic status in AI chatbots. It also introduces artificial intelligence terms and concepts, provides an overview of the regulatory and legal framework governing AI to understand how it can be used ethically in the classroom, and includes case studies and guidance for its use as an educational tool.

The guide sets out activities that can be carried out at different educational levels, such as using an AI assistant to examine different jobs and learn about gender inequalities in a variety of professions; discussing how algorithms can reinforce gender bias in staff recruitment; or an activity introducing students to AI tools for use in physical education in which they must analyse data from a gender perspective and reflect on the ethical and educational aspects of these apps. In the field of philosophy, students are encouraged to use ChatGPT to analyse philosophy texts, discuss the ethical issues arising from AI and examine gender and social examples in the data used to train these technologies and how it can shape AI outputs. These are just a few of the activities included in the EDUGENIA guide, which will be widely distributed to schools, family associations and other educational organizations from next academic year. 

The authors hope that it will reach as many organizations as possible to encourage teaching staff to make critical and ethical use of AI in the classroom. "The key is to be aware of the risks involved, encourage a critical approach and create educational environments that guarantee inclusion, equity and gender equality," they said.

This research contributes to the UN 2030 Agenda's Sustainable Development Goals, specifically 4, Quality Education, 5 Gender equality, and 9, Industry, Innovation and Infrastructure. It is also aligned with the UOC's Education of the future, Culture for a critical society and Ethical and human-centred technologyresearch missions.

EDUGENIA guide:

• Full text of the guide (in Spanish)

• Authors:Milagros Sáinz Ibáñez, Marie-Monique Anastasia Schaper, Miruna Bivol, Rocío Segura Nebot, Beatriz Soledad López Pérez, María José Romano Serrano, Rosa María Gil Iranzo, Mercè Teixidó, Adela Rodríguez de Corti (illustrator)
• Publisher:Spanish Ministry of Equality, Institute of Women
• Year of publication: 2026

Transformative, impactful research

At the UOC, we see research as a strategic tool to advance towards a future society that is more critical, responsible and nonconformist. With this vision, we conduct applied research that's interdisciplinary and linked to the most important social, technological and educational challenges.

The UOC’s over 500 researchers and more than 50 research groups are working in five research units focusing on five missions: lifelong learning; ethical and human-centred technology; digital transition and sustainability; culture for a critical society, and digital health and planetary well-being.

The university's Hubbik platform fosters knowledge transfer and entrepreneurship in the UOC community.

More information: www.uoc.edu/en/research