Successful Attempt of Artificial Intelligence in Sign Language Translation Field

2023-05-25

The Barcelona Supercomputing Center (BSC-CNS) and the Polytechnic University of Catalonia (UPC) researchers have developed an automatic sign language translation tool that uses artificial intelligence to overcome common communication barriers for people with hearing difficulties. Despite advancements in voice recognition technologies such as Alexa and Siri, these applications still cannot recognize sign language, which is increasingly prevalent in everyday life. This poses a barrier for individuals who rely on sign language as their preferred mode of communication to interact with technology and access digital services designed primarily for spoken language. The development of this new open-source software is an important step towards enabling inclusive communication for everyone. To achieve this, researchers from BSC and UPC combined computer vision, natural language processing, and machine learning technologies to advance the research on automatic sign language translation. The vast number and variability of sign languages worldwide make automatic sign language translation a complex problem. The system is still in the experimental stage and utilizes a machine learning model called Transformers, which serves as the foundation for other artificial intelligence tools like ChatGPT. This model can convert entire sign language sentences in video format into spoken language in text format. Currently, the system focuses on American Sign Language (ASL), but it can adapt to any other language as long as all necessary available data exists, including a parallel corpus where each sign language sentence (in video format) has a corresponding oral language translation (in text format). "The newly developed tool is an extension of BSC and UPC's previous publication, How2Sign, which released the necessary data for training the model (over 80 hours of videos, including tutorials on cooking recipes or DIY tips translated by American Sign Language interpreters). With this data, the team developed a new open-source software capable of learning the mapping between videos and text," said Laia Tarrés, a researcher from BSC and UPC, who presented this research achievement during the celebration of Global Accessibility Awareness Day. The researchers state that this new work is a step in the right direction but emphasize that there is still significant room for improvement. Currently, these are only preliminary results and cannot provide specific applications for users. Their goal is to continue working on improving the tool to achieve real-world applications of accessible technology for people with hearing difficulties. The project has already been showcased at Fundación Telefónica in Madrid as part of the "Code and Algorithms" exhibition, which brings together various projects related to artificial intelligence, with BSC playing a significant role. The project will also be exhibited soon at the "Artificial Intelligence" exhibition at the Barcelona Contemporary Culture Center, which will open in October. "This open automatic sign language translation tool is a valuable contribution to the accessibility community. Its release represents an important step towards creating more inclusive and accessible technology for everyone," summarized Tarrés. Source: https://techxplore.com/news/2023-05-ai-tool-accessibility-technology-people.html

RECENT AI NEWS

RECENT AI TOOLS