Providing the best customer experience that many large telecommunications service providers lack. Take the case of Jack, whose call was put on hold for 10 minutes, making him late for work. The third agent he spoke to, Gill, had difficulty understanding the brief explanation provided by the previous agent. So, she asked Jack some questions to clarify. With no colleagues around, Gill consulted multiple policy documents to resolve Jack's issue. After several resources, Gill found the necessary information, but unfortunately, Jack had already hung up.
Long wait times, complex service requests, and lack of personalized service are common issues faced by customers, leading to dissatisfaction and churn. To overcome these challenges, the telecommunications industry is turning to AI technology - specifically, conversational AI technology that utilizes speech, translation, and natural language processing (NLP) to facilitate human-like interactions.
This article explores why conversational AI systems are essential and why high-level transcription accuracy is crucial for optimal performance in downstream tasks. We explain the NVIDIA Riva speech recognition customization technology used by Quantiphi to improve transcription accuracy.
Accuracy of Conversational AI Systems
In telecommunications contact centers, highly accurate conversational AI systems are crucial for several reasons. Conversational AI systems can help customer service representatives extract valuable information from call interactions and make informed decisions, thereby improving service quality and customer experience.
One key component of conversational AI systems is Automatic Speech Recognition (ASR), also known as speech-to-text. Downstream tasks in telecommunications contact centers heavily rely on accurate transcriptions provided by ASR systems. These tasks cover a wide range of applications, such as:
- Customer insights
- Sentiment analysis
- Call classification
- Call transcription
Fast and accurate responses are crucial for efficient and effective customer service. This means that reducing the overall latency of various components, including Automatic Speech Recognition, is essential. By reducing the time required to complete tasks, contact center representatives can provide timely solutions, enhancing customer satisfaction and loyalty.
Additionally, accurate transcriptions with punctuation enhance readability. Clear, accurately punctuated transcriptions help representatives better understand customer issues, facilitating clear communication and problem-solving. In turn, this improves overall efficiency and effectiveness of customer interactions.
NVIDIA Riva Automatic Speech Recognition Pipeline
Speech recognition takes an audio stream as input, transcribes it, and outputs the transcribed text (Figure 1). First, the audio stream passes through an audio feature extractor and preprocessor, which removes noise and captures audio spectral features in a spectrogram or mel-spectrogram. Then, the acoustic model and language model work together to transcribe speech into text. Punctuation is added to the transcribed text to improve readability.
ASR Performance Evaluation Metrics
The performance of ASR systems can be measured using three metrics:
1. Accuracy: Accuracy is fundamental as it directly impacts the quality and reliability of transcriptions. By measuring accuracy using metrics such as Word Error Rate (WER), we can evaluate the system's ability to transcribe spoken language.
2. Latency: Latency is the time required to generate a transcript for a given audio. To maintain an engaging experience, captions should be delivered within a delay of a few hundred milliseconds.
3. Cost: The cost of developing and running transcription services on sufficient computational infrastructure is another important metric. While transcription-based on ASR is cheaper than human transcription, costs must be weighed against other factors.
In a contact center environment, transcription systems must have high accuracy, provide low-latency customer interactions, consider cost factors, and ensure cost-effective and feasible solutions for organizations. By optimizing these three metrics, transcription systems can effectively support contact center operations and enhance the delivery of customer service.
Methods to Improve ASR Accuracy
As shown in Figure 2, several techniques can be used to achieve optimal transcription accuracy for specific domains, with the simplest technique being word boosting. Automatic Speech Recognition word boosting involves passing a list of important, possibly domain-specific vocabulary to the model as additional input. This enables the ASR module to recognize these vocabulary words during the inference process.
In most cases, certain nouns (such as company or service names) are either not in the vocabulary or are frequently misrecognized by the Automatic Speech Recognition model. These nouns are added to the word boosting list. This strategy allows us to easily improve the recognition rate of specific vocabulary during requests.
Furthermore, Quantiphi's team:
- Retrained the language model on their own custom dataset to adapt the ASR engine to domain-specific terms and phrases.
- Fine-tuned the acoustic model to adapt the ASR engine to specific accents and noisy environments.
Customized Speech-Assisted Conversational AI System
One of the most significant challenges faced by telecommunications customer contact centers is the long time it takes to resolve complex queries. Agents often need to consult multiple stakeholders and internal policy documents to respond to complex queries.
Conversational AI systems provide relevant documents, insights, and suggestions, enabling contact center agents to expedite the resolution of customer queries.
The Quantiphi solution architecture for custom speech-assisted conversational AI pipelines includes:
1. Speech recognition pipeline: Creates transcripts by capturing spoken language and converting it into text.
2. Intent-slot model: Identifies user intent.
3. Semantic search pipeline: Retrieves answers to agent queries through a dialog manager.
Quantiphi has built a semantic search engine and a question-answering solution (Figure 3). It retrieves the most relevant documents for a given query and generates concise answers for telecom contact center agents.
ASR is also used in conjunction with the question-answering (QnA) system, as well as virtual agents and avatar-based chatbots. The accuracy of ASR transcriptions is crucial for the accuracy of responses generated by the retrieval-augmented generation (RAG) pipeline. Even slight variations in the transcription of a query can result in incorrect responses from the generation model.
Quantiphi's team experimented with off-the-shelf ASR models, which sometimes failed to transcribe proper nouns correctly. When ASR transcriptions are used in conjunction with the QnA pipeline, its quality is crucial, as shown in the following example:
Query: What is 5G?
ASR transcript: What is five G.
Generated response: Five G is the money you can earn in a month working at a factory.
Correct response: 5G is the next generation of wireless technology. It will be faster, more reliable, and more secure than 4G LTE.
To overcome these issues, we used word boosting, reverse text normalization, custom vocabulary, training language models, and fine-tuning acoustic models.
Word Boosting:
Words like mMTC and MEC are often transcribed incorrectly. We addressed this issue with word boosting. Consider the following example:
Before word boosting:
Multi-access edge computing, also known as MEG, is a network architecture that provides cloud computing capabilities and IT service environments at the network edge.
Mtc Fis is a service area that provides low-bandwidth connections with deep coverage.
After word boosting:
Multi-access edge computing, also known as MEC, is a network architecture that provides cloud computing capabilities and IT service environments at the network edge.
mMTC is a service area that provides low-bandwidth connections with deep coverage.
The before and after examples show how the response changes, even with slight differences in the representation of n-gram grammar. By reverse text normalization, the ASR model transcribes words like "5g" as "<>G," improving the performance of the QnA pipeline in the process.
Custom Vocabulary:
Most use cases typically have domain-specific words and associated jargon. To include these words in the ASR output, we added them to the vocabulary and retrained the ASR model. For more details, refer to the tutorial on how to use dictionary mapping for custom Riva ASR vocabulary and pronunciation.
Training N-gram Language Models:
The context present in QnA tasks often constitutes a good source of text corpus for training n-gram language models. Custom language models produce ASR outputs that are more likely to accept word sequences commonly found in the domain. We used NVIDIA NeMo scripts to train KenLM models and integrated them with the ASR model during the build.
Fine-tuning Acoustic Models:
To further improve ASR performance, we fine-tuned the acoustic model using small chunks (5-15 seconds) of audio data and their corresponding ground truth text. This helps the acoustic model pick up regional accents. We used Riva Jupyter notebooks and NeMo for fine-tuning. We further converted this checkpoint to Riva format using the nemo2riva tool and built it using the riva-build command.
Key Takeaways
Question-answering and insight extraction form the conversational solution that enables telecom customer service agents to provide personalized and efficient support. This improves customer satisfaction and reduces agent churn. To achieve highly accurate QnA and insight extraction solutions, high-precision transcription must be provided as input to the rest of the pipeline.
Quantiphi achieved maximum accuracy by customizing the speech recognition model using NVIDIA Riva ASR word boosting, reverse text normalization, custom vocabulary, training language models, and fine-tuning acoustic models. This is something off-the-shelf solutions cannot achieve.
What does this mean for Jack and Gill? Equipped with a customized speech-assisted conversational AI application for telecommunications, Gill can quickly scan the AI-generated summary of Jack's previous conversation. By the time Jack finishes asking his question, Gill's screen is already populated with the most relevant documents to address Jack's query. She quickly conveys the information to Jack. He decides to respond to the survey with positive feedback and still arrives at work on time.
Contact Quantiphi's experts to explore how conversational AI can profoundly enhance your organization's customer experience. If you're interested in delving deeper into the technical aspects of building agent-assisted solutions, join our webinar on supporting telecom contact center agents with multilingual speech AI for customized agent assistance.
Source: https://developer.nvidia.com/blog/enhancing-customer-experience-in-telecom-with-nvidia-customized-speech-ai/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.