Meta’s new AI bot can help devs build virtual worlds through voice commands

After sharing disappointing quarterly results earlier this month, Meta (formerly Facebook) chief executive, Mark Zuckerberg, detailed prototypes of artificial intelligence (AI) technologies for the metaverse yesterday. Zuckerberg also said that the company plans to build a universal speech translator, and a project to build a new virtual assistant for the metaverse.

The company, which rebranded from Facebook to Meta last year, is under pressure to prove that its vision for the metaverse is feasible right now. Earlier this month, lacklustre results from Facebook wiped over $200 billion from its market value in less than 24 hours, after the company said that iPhone maker Apple’s new policies had hurt its ability to drive revenues through online advertising.

First amongst Meta’s new AI tools is a system called Builder Bot. In a pre-recorded video, Zuckerberg demoed how the bot could be used to build virtual spaces through commands in simple language. The video details how developers can use the platform to build virtual spaces by giving commands like “let’s go to the beach”, and the system creates machine-generated art to create a 3D landscape that matches the command.

The exact abilities of the bot aren’t clear at the moment, but the video suggests that the bot can not only understand simple language, devs will also be able to point to specific locations in the virtual world and ask the bot to draw objects in those locations. For instance, Zuckerberg and his colleague ask for tables and trees to be put in particular locations by simply pointing to those locations.

Of course, Meta’s Builder Bot isn’t the first bot to have abilities like this. AI that can draw landscapes and imagery through simple voice commands has become a part of systems built by many others, including chip giant Nvidia and AI research firm OpenAI. Nvidia’s GauGAN2 also boasts similar abilities, as does OpenAI’s DALL-E AI.

Further, Zuckerberg said that MEta is working on a system called No Language Left Behind, which uses AI to provide “instantaneous” speech-to-speech translation across all languages, “even those that are mostly spoken”.

For the AI virtual assistant mentioned above, the company announced Project CAIRoke, which will power chatbots and assistants. “With this technology, people could one day have more fluid conversations with their virtual assistants,” the company said in a blog post.