On a big day of releases for Meta, the multinational technology company unveiled on Tuesday (August 22), its new AI model for speech and text translations, SeamlessM4T.

SeamlessM4T represents a monumental leap forward in the field of artificial intelligence, offering a single, all-encompassing model capable of automatic speech recognition, speech-to-text translation, speech-to-speech translation, text-to-text translation, and text-to-speech translation.

The development of a universal language translator has been fraught with challenges; traditional speech-to-speech and speech-to-text systems have struggled to accommodate the diversity of languages across the world.

SeamlessM4T draws on the collective insights and findings of past projects to deliver a multilingual and multimodal translation experience powered by a single model.

Meta, in a release on their blog site, said, “SeamlessM4T is the culmination of years of progress in Meta’s pursuit of a universal translator. Milestones such as the release of No Language Left Behind (NLLB), a text-to-text machine translation model,-and the debut of the Universal Speech Translator—a pioneering speech-to-speech translation system for languages lacking established writing systems—have all contributed to the development of SeamlessM4T.”

SeamlessM4T offers a unified approach that diminishes errors and delays inherent in separate models, thereby enhancing the efficiency and quality of translations. This technology brings languages together to enable more effective communication between speakers of different tongues.

The model supports nearly 100 languages for text translations and can output speech in 36 languages

SeamlessM4T is being made publicly available under a research license, inviting researchers and developers to contribute to its further development. Alongside this release, Meta is also sharing the metadata of SeamlessAlign, the most expansive open multimodal translation dataset ever created, encompassing a staggering 270,000 hours of speech and text alignments.

The Caribbean region is multilingual, only roughly six million out of the 40 million people in the region speak English. In the wider Latam region which consists of over 600 million people, close to double the population of the United States, only 10-11 million people speak English. For generations, language difficulties has been a barrier keeping the region from progressing faster.