A scientific study from Saudi Arabia has revealed that artificial intelligence systems struggle to comprehend Saudi culture, despite advancements in language processing. The study, published in an international journal by King Saud University through Springer Publishing, utilized a scientific scale consisting of 441 questions covering details of cultural life in the Kingdom.
The study included eight main areas, most notably food with 125 questions, entertainment with 95, crafts and professions with 60, language and communication with 42, social relationships with 34, clothing with 32, occasions with 30, and architecture with 23 questions, reflecting the diversity and breadth of Saudi culture.
Event Details
The questions were geographically distributed across the five regions of the Kingdom, including 76 questions for the central region, 84 for the south, 74 for the west, 67 for the east, and 54 for the north, in addition to 86 general questions reflecting shared culture. This distribution aimed to measure subtle differences between local environments.
The researchers tested six global AI systems, and the results showed that the best performance did not exceed 66% in understanding Saudi culture, while some systems dropped to 16%, indicating a clear weakness in understanding cultural context compared to linguistic capabilities.
Background & Context
The study indicates that the accuracy of responses increases in direct questions, reaching 81% in some cases, while it drops to about 52% in open-ended questions, and declines further in questions requiring multiple answers, potentially falling to 20%, especially on topics related to customs and regional differences.
Understanding varied by area, with crafts and professions recording the highest accuracy at 75%, followed by occasions at around 73%, while the systems struggled to comprehend social relationships and nuanced customs, reflecting a challenge in grasping deep social context.
Impact & Consequences
The study confirmed that this shortcoming is due to these systems relying on global training data that tends to favor non-local cultures, reducing their ability to represent the cultural specificity of Saudi society. These findings highlight the importance of developing AI systems that consider the local environment.
Additionally, existing systems need to be retrained with data that reflects cultural identity, especially as the use of these technologies expands in education, tourism, and digital services.
Regional Significance
This study serves as a call to develop AI technologies that align with local cultures in Arab countries. As reliance on these systems increases across various fields, it becomes essential for them to reflect the cultural and social diversity of the region.
In conclusion, these results underscore the urgent need to develop new strategies in the field of artificial intelligence that ensure a deeper understanding of local cultures, contributing to improved effectiveness of these systems in the future.