SuperGLUE Benchmark Updated: Zhuiyi Technology Surpasses Facebook to Rank Third

2020-01-20

Recently, the latest ranking update of the authoritative dataset SuperGLUE in the field of natural language processing (NLP) has been released. Google's pre-training model T5 remains in the first place, while the AI Lab team of Chinese AI startup company Zhiyi Technology has surpassed Facebook AI and jumped to the third place on the list. It is worth noting that compared to large-scale research models like Google's T5, Zhiyi's RoBERTa-mtl-adv model, which made it to the list this time, also has strong commercial capabilities. The related technology has already been applied to Zhiyi Technology's AI digital employee product line, continuously empowering industries such as banking, insurance, securities, retail, real estate, energy, education, and the internet. SuperGLUE (SuperGeneral Language Understanding Evaluation) is one of the most challenging, authoritative, and valuable evaluation standards in the current NLP field. It was jointly launched by New York University, Washington University, DeepMind (a subsidiary of Google), and Facebook as the main initiators. It covers a wide range of NLP tasks that may be encountered in real life to the greatest extent, aiming to more realistically reflect the cognitive intelligence level that the cutting-edge NLP technology can achieve. In the process of providing intelligent interaction services to enterprises, Zhiyi Technology drives technology and applications mutually. It incorporates the technical means and experience summarized from actual commercial applications, such as multi-task learning, adversarial training, and knowledge distillation, into the participating models. As a result, it achieved the third place in the SuperGLUE evaluation dataset. Apart from the human-level reference, the current top-ranked model on the list is Google's T5 model. From an academic research perspective, the T5 model has reached the best level of "laboratory intelligence" currently. However, when considering commercial use, T5 requires a large amount of computing power and its model size is too large to be applied to actual business scenarios. It lacks practical commercial application value. On the other hand, Zhiyi Technology's RoBERTa-mtl-adv model achieves a reasonable model size and its performance is second only to Google's T5 through the use of multi-task learning, adversarial training, and knowledge distillation. Moreover, the related technology has been applied to Zhiyi Technology's AI digital employee product line. In terms of SuperGLUE ranking, it can be said that Zhiyi Technology's RoBERTa-mtl-adv model has the best performance and highest ranking among NLP models with actual implementation capabilities worldwide.

RECENT AI NEWS

RECENT AI TOOLS