Beijing: Artificial intelligence (AI) models from Chinese tech giants Tencent Holdings and Alibaba Group Holding understand the Chinese language better than humans, according to a benchmark test measuring natural language processing.
On the Chinese Language Understanding Evaluation (CLUE) benchmark, which is a set of tasks to determine how well a machine can understand and respond to Chinese text in the same way that humans do, the two competing models scored Achieved high scores.
Since the benchmark was created three years ago by dozens of researchers, this is the first instance in which an AI model has outperformed humans on CLUE.
According to Monday's ranking on CLUE's website, Tencent's "Huanyuan AI Model" ranked first with a score of 86.918, followed by Alibaba's Alismind with a score of 86.685. Both performed better than the guys, who received a score of 86.678.
Chinese smartphone maker Oppo and food delivery giant Meituan each had an AI model that ranked fourth or fifth, respectively.
The cloud computing division of e-commerce giant Alibaba Cloud claimed in a Friday article that "the top scorers have never surpassed humans so far," despite the fact that the rankings change frequently.
The most recent results, it was added, show that "the Chinese language understanding of AI models has advanced to a new level."
China's big tech companies are working to advance their natural language processing (NLP) systems, which are used to power voice-activated virtual assistants from Alibaba like Tencent's Xiaowei and AliGenie, as well as other features like machine translation and spam detection.
Chinese Internet search giant Baidu claimed earlier this year that Du Xiaoxiao, a bot it uses, had written an article that, despite using incorrect Internet slang terms, referred to China's notoriously challenging national college admissions. He does better than most students in the exam, known locally as Gaokao.
Most AI models, according to some researchers, still have a long way to go before understanding the nuances of languages.
Researchers from Auburn University in the US state of Alabama and Adobe's research division discovered last year that some artificial intelligences (AIs) that outperform humans on some comprehension tasks were unable to recognize when words in a sentence were randomized. was changed from