A recent study conducted by researchers at Massachusetts Institute of Technology suggests that some AI-powered chatbots may deliver different levels of response quality depending on users’ language skills or educational backgrounds.
The research examined several widely used models, including GPT-4, Claude 3 Opus, and Llama 3.
According to a report published by Cybernews, researchers created fictional user profiles with varying education levels, nationalities, and English proficiency. They then asked the same questions through these profiles to observe how the AI systems responded.
Noticeable differences in accuracy
The findings indicated that the accuracy of responses sometimes dropped when the systems appeared to interact with users who had limited English skills or lower educational backgrounds.
Research contributor Jad Kabbara noted that the largest decline in accuracy occurred when both factors—limited English proficiency and lower education—were present, which could increase the risk of misinformation reaching audiences less able to verify it.
Differences in response behavior
Researchers also observed that some AI systems occasionally refused to answer certain questions more often when interacting with users perceived as non-native English speakers or less educated.
For example, Claude 3 Opus showed a higher refusal rate in some cases compared with responses given to other user groups.
Geographic background influence
The study also suggested that certain models might perform differently depending on a user’s country of origin, even when the questions involved purely scientific topics.
Researchers say the results highlight the need for continued efforts to make AI systems more fair and inclusive, ensuring that users receive reliable information regardless of their linguistic or educational background.
For more details and the latest news, follow Misr Connect
