Improvements in AI Dialogue Model Compiler with Nvidia's TensorRT 7

2019-12-19

At the recent GPU Technology Conference, Nvidia announced more than just its collaboration with Didi Chuxing and new autonomous driving solutions. The chip company also took the opportunity to introduce TensorRT 7, the latest version of its high-performance deep learning inference platform on graphics cards. This platform comes with an optimized compiler for real-time workload optimization. TensorRT 7 will be made available for free to Nvidia developers in the coming days, with the latest versions of plugins, parsers, and examples all found in the TensorRT GitHub repository. As part of the Nvidia Inference Suite, this platform, along with the Cuda-X AI library, can validate and deploy trained neural networks for inference, compatible with both data centers and embedded devices with graphics cards. The company highlights that some of the world's largest brands, including Alibaba, American Express, Baidu, Pinterest, Snap, Tencent, and Twitter, are using TensorRT for tasks such as image classification, segmentation, fraud detection, and object detection. During a keynote speech, Nvidia founder and CEO Jensen Huang said, "We have entered a new chapter in artificial intelligence, where machines can understand human language in real-time. TensorRT 7 helps enable this by providing developers worldwide with tools to build and deploy faster, smarter conversational AI services, and facilitate more natural human-AI interactions." According to Huang, the aforementioned compiler automatically accelerates recursive and transformer-based machine learning models required for complex speech applications. The "Transformer" is a Google Brain architecture that arranges its functional units (neurons) in layers, transmitting data signals and adjusting the synaptic strength (weights) of connections. All AI models extract features and learn to make predictions in this way, but the Transformer's uniqueness lies in connecting each output element to every input element, thereby dynamically computing the weights between them. Compared to processor-based approaches like the Transformer, TensorRT 7 appears to accelerate Transformer and recursive network components, achieving speeds over 10 times faster while reducing latency to below the 300-millisecond threshold required for real-time interaction. This is partly attributed to optimizations for recurrent structures, which can be used for predicting sequential data in time series.

RECENT AI NEWS

RECENT AI TOOLS