Microsoft has released a private demo of its AI language model that it claims contains a massive 17 billion parameters, making it the largest publicly-known model of its class – about twice the size of Nvidia’s Megatron. Called the Turing Natural Language Generation (T-NLG), Microsoft’s model is said to not only outperform the other modern state-of-the-art AI training models for Natural Language Processing (NLP) on a variety of language modeling benchmarks, but also “excels when applied to numerous practical tasks, including summarization and question answering”, claimed the company.
The T-NLG demo, which has only been released to a small set of users within the academic community for initial testing and feedback, is said to show off the software’s capabilities for freeform generation, question answering and summarization. It is part of a larger initiative called Project Turing, which is Microsoft’s applied research unit that works to incorporate deep learning models for both text and image processing into the company’s hardware and software products, including Bing, Office and Xbox.
Like many of the other NLP models, T-NLG is also a Transformer-based generative language model, which means it can not only generate words to complete unfinished sentences, but also generate direct answers to questions and summarize entire documents. In fact, the company says that it is best suited to train a large centralized multi-task model, because “the bigger the model and the more diverse and comprehensive the pretraining data, the better it performs at generalizing to multiple downstream tasks even with fewer training examples”.
Explaining the benefits of its proprietary NLP technology, Microsoft said that its new technology can provide new opportunities for the company and its customers. According to Microsoft AI Research scientist, Corby Rosset, “T-NLG can enhance experiences with the Microsoft Office suite by offering writing assistance to authors and answering questions that readers may ask about a document. Furthermore, it paves the way for more fluent chatbots and digital assistants”.
You can check out Microsoft’s official Research blog to read all the heavy-duty technical details.
