Google turns to machine learning to advance real-world text translation TechCrunch

Google is upgrading its translation service with a new addition based on machine learning that will make it easier for users to translate text that appears in the real world, such as on storefronts, menus, documents, business cards and other elements. Instead of covering the original text with the translation, the new feature will intelligently overlay the translated text on top of the image, while reconstructing the pixels below with an AI-generated background to render the reading process more natural translation. .

“Often it’s that combination of the word and the context — like the background image — that really gives meaning to what you’re seeing,” explained Cathy Edwards, vice president and general manager of Google Search. , during a briefing ahead of today’s announcement. “You don’t want to translate text to cover that important context that can show through in the images,” she said.

Picture credits: Google

To make this process work, Google uses a machine learning technology known as Generative Adversarial Networks, otherwise known as GAN models – the same technology that powers the Magic Eraser feature to remove objects from photos taken on Google Pixel smartphones. This breakthrough will allow Google to merge translated text into even very complex images, making translation feel natural and seamless, according to the company. You should feel like you’re looking at the element or object itself with translated text, not an overlay obscuring the image.

The feature is another development that seems to indicate Google’s plans to invest more in creating new AR glasses, as the ability to translate real-world text could be a key selling point for such a device. The company noted that every month people use Google to translate text and images more than a billion times in more than 100 languages. It also started testing AR prototypes in public places this year with a handful of employees and trusted testers, he said.

Although there is a clear demand for better translation, it is not clear if users will prefer to use their smartphones for translations rather than special glasses. After all, Google’s first entry into the smart glasses space, Google Glass, ultimately failed as a consumer product.

Google didn’t talk about its long-term plans for the translation feature today, only noting that it will arrive later this year.

learn more about Google Search On 2022 on TechCrunch

Sherry J. Basler