Exploring the Interaction of Language and ChatGPT

Ray Poynter, 28 June 2025

With Noriko Kishida (Japan), Tomoko Yoshida (Japan), Dangjaithawin (Orm) Anantachai (Thailand), and Seyo Adeoye (Nigeria), we have created a discussion and note relating to an experiment we ran earlier this year, examining the interaction between the use of ChatGPT and the languages used. You can read the complete note by clicking here.

Below is the Big Picture, along with a note about High and Low Context Languages.

If English is not your second language, should you use ChatGPT in English or in your first language? That is the question this paper addresses.

It has been established that in purely factual issues (such as answering exam questions), ChatGPT tends to be more accurate in English [see ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning, 2023, Viet Dec Lai et al]. What does this mean for users who are English language learners? Should they use their native tongue or English? Whilst purely factual matters might be more accurate when asked in English, what about other types of questions? Might asking in another language sometimes unlock different and potentially valuable insights?

In this informal note, the authors share the results of an experiment examining the impact of posing questions to ChatGPT in various languages. These questions were asked in the context of market research and insights.

The key findings were:

• When asked a question in English, the answers seemed longer and more analytical.
• Although the questions did not specify a location, asking them in different languages tended to highlight distinct cultural and social phenomena.
• Using a human translation of the question to Thai or Japanese and then using a machine to backtranslate the text did not generate the original question. This reminds us that it is often not possible to ask the same non-trivial question in two languages, thereby increasing the likelihood that ChatGPT will provide different answers to the same question asked in two languages.
• Asking the same question twice or regenerating the answer can generate differences, so some of the observed differences will be due to chance. In contrast, others will relate to the language being used.
• ChatGPT coped well with Thai and Japanese, but it struggled with Yorùbá in some cases.
• For speakers of a second language, there are potential benefits to asking the question in both English and in their first language.

High and Low Context Languages

The authors speculate that one reason for the differences between languages is whether they are high- or low-context. “A high-context communication … is one in which most of the information is already in the person, while very little is in the coded, explicit, transmitted part of the message. A low-context communication is just the opposite.” (Hall, E.T. (1976). Beyond Culture.) English is a low-context language. Communication is done with direct statements, and little shared knowledge is assumed. Thai and Yorùbá are classed as moderately high-context, and Japanese is high-context. High-context languages rely on more shared knowledge, context and even non-verbal clues. An answer provided in a high-context language may give more insight to a native speaker than a simple translation to English might suggest.

Additional note about Deep Research

Earlier this month, Tomoko Yoshida, Noriko Kishida and I were experimenting with Deep Research in Japan. One of the features of Deep Research is that OpenAI displays its ‘reasoning’ on the screen while it is iterating its way to an answer. When my colleagues asked questions in Japanese, the answers they received were in Japanese, but the ‘thinking’ that was displayed on the screen during the iteration process was in English.

Your Experience?

Please comment on this post to tell us about your experience with using different languages and LLMs.

Related